-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Description
Describe the bug:
The Sites Tree File Format export produced by the exim add-on treats all request with bodies as if they are application/x-www-form-urlencoded
POST requests using the format key1=value1&key2=value2
. It attempts to parse them, strip values, and produce an output like key1=&key2=
.
This produces inconsistencies when the request body uses other formats, such as JSON or XML:
- Multi-line bodies are truncated and only the first line is preserved in the
data
field. - Bodies with
=
characters are misinterpreted. For example, an XML declaration like<?xml version="1.0"?>
becomes<?xml version=
after parsing. - The bodies of non-POST requests with bodies (e.g.
PUT
,PATCH
) are not included, since onlyPOST
requests go through the body-extraction logic.
Steps to reproduce the behavior:
Run the following automation plan, which uses a requestor
job to send different types of HTTP requests, and an export
job to export the site tree.
env:
contexts:
- name: Default Context
urls:
- https://httpbingo.org
jobs:
- type: requestor
requests:
- name: GET Request
url: https://httpbingo.org/get?key1=value1&key2=value2
method: GET
- name: POST Request With Form Data
url: https://httpbingo.org/post
method: POST
data: 'key1=value1&key2=value2'
headers:
- "Content-Type: application/x-www-form-urlencoded"
- name: PUT Request
url: https://httpbingo.org/put
method: PUT
data: '{"key1": "value1", "key2": "value2"}'
headers:
- 'Content-Type: application/json'
- name: POST Request With Multipart Form Data
url: https://httpbingo.org/post
method: POST
data: |
--boundary
Content-Disposition: form-data; name="key1"
value1
--boundary
Content-Disposition: form-data; name="key2"
value2
--boundary--
headers:
- "Content-Type: multipart/form-data; boundary=boundary"
- name: PATCH Request
url: https://httpbingo.org/patch
method: PATCH
data: '{"key1": "value1", "key2": "value2"}'
headers:
- 'Content-Type: application/json'
- name: POST Request With JSON Data
url: https://httpbingo.org/post
method: POST
data: '{"key1": "value1", "key2": "value2"}'
headers:
- 'Content-Type: application/json'
- name: POST Request With XML Data
url: https://httpbingo.org/post
method: POST
data: |
<?xml version="1.0"?>
<key1>value1</key1>
<key2>value2</key2>
headers:
- "User-Agent: test"
- 'Content-Type: application/xml'
- type: export
parameters:
type: yaml
source: sitestree
fileName: sitestree.yaml
Note: httpbingo.org
requires a User-Agent
header to respond correctly, but it's omitted here for brevity since the response is not relevant to this report.
The resulting sitestree.yaml
file should look like this:
- node: Sites
children:
- node: https://httpbingo.org
url: https://httpbingo.org
method: GET
children:
- node: "GET:get(key1,key2)"
url: https://httpbingo.org/get?key1=value1&key2=value2
method: GET
responseLength: 930
statusCode: 200
- node: PATCH:patch
url: https://httpbingo.org/patch
method: PATCH
responseLength: 1080
statusCode: 200
- node: POST:post()(<?xml version)
url: https://httpbingo.org/post
method: POST
responseLength: 1100
statusCode: 200
data: '%3C%3Fxml+version='
- node: "POST:post()(key1,key2)"
url: https://httpbingo.org/post
method: POST
responseLength: 1103
statusCode: 200
data: key1=&key2=
- node: POST:post()(multipart/form-data)
url: https://httpbingo.org/post
method: POST
responseLength: 1238
statusCode: 200
- node: "POST:post()({\"key1\": \"value1\", \"key2\": \"value2\"})"
url: https://httpbingo.org/post
method: POST
responseLength: 1078
statusCode: 200
data: '%7B%22key1%22%3A+%22value1%22%2C++%22key2%22%3A+%22value2%22%7D='
- node: PUT:put
url: https://httpbingo.org/put
method: PUT
responseLength: 1076
statusCode: 200
Note the following behavior:
Expected
GET Request
: no body, nodata
field.POST Request with Form Data
: keys are kept, values are stripped.POST Request with Multipart Form Data
: the body is present but intentionally left out in the export.
Unexpected
PUT Request
andPATCH Request
are missing thedata
field, even though they have bodies.POST Request with JSON Data
is mis-parsed (ends with an=
character.)POST Request with XML Data
is mis-parsed (only the first line is kept, and everything after the first=
is removed.)
Expected behavior:
- PUT and PATCH requests include a
data
field as long as they have bodies. - Request bodies are exported with minimal parsing to support non-form requests.
Software versions:
ZAP
Version: 2.16.1
Installed Add-ons: [...[id=exim, version=0.14.0]...]
Screenshots:
No response
Errors from the zap.log file:
No response
Additional context:
Omitting the values of each key in x-www-form-urlencoded
bodies seems reasonable, but this approach could lead to inconsistencies with JSON, XML, and possibly other formats. These would need to either:
- Contain the full body
- Or require parsing and sanitizing based on their formats, which (IMHO) adds unnecessary complexity
It might be worth considering keeping the entire body with minimal parsing, unless there's a strong reason to do otherwise.
Would you like to help fix this issue?
- Yes