Export valid json from mongodb collection

Question

I am trying to export a valid json from the mongodb collection that I created using the node and instagram api. I have to miss something because it seems like it should be super easy. I read other posts and mongo documentation, especially about mongoexport.
My ultimate goal is to build a d3 map. I used the main mongoexport command from the documentation and returned the json file in the following format:

{'name':'dan'} {'name':'emma'} 

valid json will be:

 [{'name':'dan'}, {'name':'emma'}] 

I know that there are workarounds, even as simple as finding "$" in sublime text that goes to the end of each line and then can just add to whom. It would be great to know how to do this technically. One post suggested using JSON.parse, so I tried this with fs.readFile, but it returns an error:

 undefined:2 { "attribution" : null, "tags" : [], "location" : { "latitude" : 48.857141667, ^ SyntaxError: Unexpected token {. 

I just need to export the entire mongo collection to a json file that is valid (pass the http://jsonlint.com/ test successfully).

Any help would be really appreciated.

Answer 1

using themongoexport use --jsonArray

http://docs.mongodb.org/manual/reference/program/mongoexport/#cmdoption--jsonArray

Answer 2

As mentioned in the comments, parsing the mongodump result directly is probably not a good idea. Mongo does not give you any guarantee that the elements in the dump will be valid json (which is not the case).

But if you are so inclined to do this, you can use event-stream to read a line feed with line separators and a parser that allows your json to have single quotes instead of double quotes (i.e. hanson ).

Then your code might look something like this:

 var es = require('event-stream'), hanson = require('hanson'), in = process.stdin, out = process.stdout; in //read the input stream .pipe(es.split()) //split it on newline .pipe(es.map(function(data, cb) { if (data === '') { //necessary due to the last element produced by split cb(); } else { cb(null, hanson.parse(data)); //parse the line with hanson } })) .pipe(es.writeArray(function (err, array){ //convert resulting objects to array var strArray = JSON.stringify(array); out.write(strArray + '\n'); //write the resulting array to output stream })); 

If you have a stream of valid json objects (with double quotes), the same code will be reduced to:

 var es = require('event-stream'), hanson = require('hanson'), in = process.stdin, out = process.stdout; in .pipe(es.split()) .pipe(es.parse()) .pipe(es.writeArray(function (err, array){ var strArray = JSON.stringify(array); out.write(strArray + '\n'); })); 

Clarification of errors for clarity.