Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for external parsers to bodyParser.json() #281

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 42 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,27 @@ specifies the number of bytes; if it is a string, the value is passed to the
[bytes](https://www.npmjs.com/package/bytes) library for parsing. Defaults
to `'100kb'`.

##### parser

The `parser` option is the function called against the request body to convert
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No description of the first argument. Also doesn't describe the mechanism of how it needs to be made to construct the syntax error during the recalling mechanism with the # inserted into the original string.

it to a Javascript object. If a `reviver` is supplied, it is supplied as the
second argument to this function.

```
parser(body, reviver) -> req.body
```

Defaults to `JSON.parse`.

##### reviver

The `reviver` option is passed directly to `JSON.parse` as the second
argument. You can find more information on this argument
You can find more information on this argument
[in the MDN documentation about JSON.parse](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse#Example.3A_Using_the_reviver_parameter).

##### strict

When set to `true`, will only accept arrays and objects; when `false` will
accept anything `JSON.parse` accepts. Defaults to `true`.
accept anything the `parser` accepts. Defaults to `true`.

##### type

Expand Down Expand Up @@ -159,6 +170,15 @@ The `verify` option, if supplied, is called as `verify(req, res, buf, encoding)`
where `buf` is a `Buffer` of the raw request body and `encoding` is the
encoding of the request. The parsing can be aborted by throwing an error.

##### parser

The `parser` option, if supplied, is used to transform the body of a request
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably describe the type and number of arguments it will be called with.

before being set to `req.body`.

```
parser(body) -> req.body
```

### bodyParser.text([options])

Returns middleware that parses all bodies as a string and only looks at
Expand Down Expand Up @@ -208,6 +228,15 @@ The `verify` option, if supplied, is called as `verify(req, res, buf, encoding)`
where `buf` is a `Buffer` of the raw request body and `encoding` is the
encoding of the request. The parsing can be aborted by throwing an error.

##### parser

The `parser` option, if supplied, is used to transform the body of a request
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably describe the type and number of arguments it will be called with.

before being set to `req.body`.

```
parser(body) -> req.body
```

### bodyParser.urlencoded([options])

Returns middleware that only parses `urlencoded` bodies and only looks at
Expand Down Expand Up @@ -274,6 +303,16 @@ The `verify` option, if supplied, is called as `verify(req, res, buf, encoding)`
where `buf` is a `Buffer` of the raw request body and `encoding` is the
encoding of the request. The parsing can be aborted by throwing an error.

##### parser

The `parser` option, if supplied, is used to in place of the default parser to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably mention the type of the argument and also that it is not called at all for zero length bodies (though, should it be always called because what if they want a zero length result to be something other than {}?). The other parsers seem to be called on zero length, so this seems odd not to.

convert the request body into a Javascript object. If this option is supplied,
both the `extended` and `parameterLimit` options are ignored.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the json parser, the reviver is not ignore but these are. Should json work like this and ignore the arguments that were originally going to the underlying parser that is now replaced, or should these be passed to the new urlencoded parser somehow or neither and a warning is a user gave these? Some thoughts.


```
parser(body) -> req.body
```

## Errors

The middlewares provided by this module create errors depending on the error
Expand Down
9 changes: 5 additions & 4 deletions lib/types/json.js
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ function json (options) {
var strict = opts.strict !== false
var type = opts.type || 'application/json'
var verify = opts.verify || false
var parser = opts.parser || JSON.parse

if (verify !== false && typeof verify !== 'function') {
throw new TypeError('option verify must be function')
Expand All @@ -80,13 +81,13 @@ function json (options) {

if (first !== '{' && first !== '[') {
debug('strict violation')
throw createStrictSyntaxError(body, first)
throw createStrictSyntaxError(parser, reviver, body, first)
}
}

try {
debug('parse json')
return JSON.parse(body, reviver)
return parser(body, reviver)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is right for custom parsers. Or at least it's not clear in the docs for the parser option that this is going to happen and how people should build their parser dunxtion to compensate.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option would be to remove the reviver param and rely on people using that parameter should change to this. I didn't do that as that would break BC, and I figure a project like this wouldn't want to break BC very often.

Maybe the right way is to make the docs more clear about this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I commented on the wrog line.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I don't think I can agree with you that the docs are not clear:

The reviver option is passed directly to the parser as the second argument.

If you can suggest an update to that text I'm all ears.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved my comment down. The new comment for this line is to clarify in the docs the arguments the custom parser will be called with.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the line from the reviver docs and add in the specific signature into the parser docs. The current parser section doesn't help someone build their own, I think it assumes someone is going to understand JSON.parse and copy that? If so, just write that down.

} catch (e) {
throw normalizeJsonSyntaxError(e, {
stack: e.stack
Expand Down Expand Up @@ -149,12 +150,12 @@ function json (options) {
* @private
*/

function createStrictSyntaxError (str, char) {
function createStrictSyntaxError (parser, reviver, str, char) {
var index = str.indexOf(char)
var partial = str.substring(0, index) + '#'

try {
JSON.parse(partial); /* istanbul ignore next */ throw new SyntaxError('strict violation')
parser(partial, reviver); /* istanbul ignore next */ throw new SyntaxError('strict violation')
} catch (e) {
return normalizeJsonSyntaxError(e, {
message: e.message.replace('#', char),
Expand Down
7 changes: 6 additions & 1 deletion lib/types/raw.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ function raw (options) {
: opts.limit
var type = opts.type || 'application/octet-stream'
var verify = opts.verify || false
var parser = opts.parser || defaultParser

if (verify !== false && typeof verify !== 'function') {
throw new TypeError('option verify must be function')
Expand All @@ -49,7 +50,7 @@ function raw (options) {
: type

function parse (buf) {
return buf
return parser(buf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need two levels of functions here. Just replace parse with a variable.

}

return function rawParser (req, res, next) {
Expand Down Expand Up @@ -87,6 +88,10 @@ function raw (options) {
}
}

function defaultParser (buf) {
return buf
}

/**
* Get the simple type checker.
*
Expand Down
7 changes: 6 additions & 1 deletion lib/types/text.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ function text (options) {
: opts.limit
var type = opts.type || 'text/plain'
var verify = opts.verify || false
var parser = opts.parser || defaultParser

if (verify !== false && typeof verify !== 'function') {
throw new TypeError('option verify must be function')
Expand All @@ -51,7 +52,7 @@ function text (options) {
: type

function parse (buf) {
return buf
return parser(buf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need two levels of functions here. Just replace parse with a variable.

}

return function textParser (req, res, next) {
Expand Down Expand Up @@ -92,6 +93,10 @@ function text (options) {
}
}

function defaultParser (buf) {
return buf
}

/**
* Get the charset of a request.
*
Expand Down
8 changes: 5 additions & 3 deletions lib/types/urlencoded.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ function urlencoded (options) {
var opts = options || {}

// notice because option default will flip in next major
if (opts.extended === undefined) {
if (opts.extended === undefined && opts.parser === undefined) {
deprecate('undefined extended: provide extended option')
}

Expand All @@ -61,9 +61,11 @@ function urlencoded (options) {
}

// create the appropriate query parser
var queryparse = extended
var parser = opts.parser || (
extended
? extendedparser(opts)
: simpleparser(opts)
)

// create the appropriate type checking function
var shouldParse = typeof type !== 'function'
Expand All @@ -72,7 +74,7 @@ function urlencoded (options) {

function parse (body) {
return body.length
? queryparse(body)
? parser(body)
: {}
}

Expand Down
12 changes: 12 additions & 0 deletions test/json.js
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,18 @@ describe('bodyParser.json()', function () {
.expect(200, '{"user":"tobi"}', done)
})

it('should use external parsers', function (done) {
request(createServer({
parser: function (body) {
return { foo: 'bar' }
}
}))
.post('/')
.set('Content-Type', 'application/json')
.send('{"str":')
.expect(200, '{"foo":"bar"}', done)
})

describe('when JSON is invalid', function () {
before(function () {
this.server = createServer()
Expand Down
10 changes: 10 additions & 0 deletions test/raw.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ describe('bodyParser.raw()', function () {
.expect(200, 'buf:746865207573657220697320746f6269', done)
})

it('should parse application/octet-stream with a custom parser', function (done) {
request(createServer({
parser: function (body) { return body.toString('utf8') }
}))
.post('/')
.set('Content-Type', 'application/octet-stream')
.send('the user is tobi')
.expect(200, '"the user is tobi"', done)
})

it('should 400 when invalid content-length', function (done) {
var rawParser = bodyParser.raw()
var server = createServer(function (req, res, next) {
Expand Down
10 changes: 10 additions & 0 deletions test/text.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ describe('bodyParser.text()', function () {
.expect(200, '"user is tobi"', done)
})

it('should parse text/plain with a custom parser', function (done) {
request(createServer({
parser: function (input) { return input.toUpperCase() }
}))
.post('/')
.set('Content-Type', 'text/plain')
.send('user is tobi')
.expect(200, '"USER IS TOBI"', done)
})

it('should 400 when invalid content-length', function (done) {
var textParser = bodyParser.text()
var server = createServer(function (req, res, next) {
Expand Down
10 changes: 10 additions & 0 deletions test/urlencoded.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ describe('bodyParser.urlencoded()', function () {
.expect(200, '{"user":"tobi"}', done)
})

it('should parse x-www-form-urlencoded with custom parser', function (done) {
request(createServer({
parser: function (input) { return input.toUpperCase() }
}))
.post('/')
.set('Content-Type', 'application/x-www-form-urlencoded')
.send('user=tobi')
.expect(200, '"USER=TOBI"', done)
})

it('should 400 when invalid content-length', function (done) {
var urlencodedParser = bodyParser.urlencoded()
var server = createServer(function (req, res, next) {
Expand Down