Mongoimport json file

Question

I have a json file consisting of about 2000 entries. Each entry that will correspond to a document in the mongo database will be formatted as follows:

{jobID:"2597401", account:"XXXXX", user:"YYYYY", pkgT:{"pgi/7.2-5":{libA:["libpgc.so"],flavor:["default"]}}, startEpoch:"1338497979", runTime:"1022", execType:"user:binary", exec:"/share/home/01482/XXXXX/appker/ranger/NPB3.3.1/NPB3.3-MPI/bin/ft.D.64", numNodes:"4", sha1:"5a79879235aa31b6a46e73b43879428e2a175db5", execEpoch:1336766742, execModify: new Date("Fri May 11 15:05:42 2012"), startTime: new Date("Thu May 31 15:59:39 2012"), numCores:"64", sizeT:{bss:"1881400168",text:"239574",data:"22504"}}, 

Each entry is on the same line in the JSON file, and single line breaks are at the end of each entry. Therefore, each line in the document begins with "{jobID:" ... I try to import them into the mongo database using the following command:

 mongoimport --db dbName --collection collectionName --file fileName.json 

However, I get the following error:

 Sat Mar 2 01:26:12 Assertion: 10340:Failure parsing JSON string near: ,execModif 0x10059f12b 0x100562d5c 0x100562e9c 0x10025eb98 0x10000e643 0x100010b60 0x10055c4cc 0x1000014b7 0x100001454 0 mongoimport 0x000000010059f12b _ZN5mongo15printStackTraceERSo + 43 1 mongoimport 0x0000000100562d5c _ZN5mongo11msgassertedEiPKc + 204 2 mongoimport 0x0000000100562e9c _ZN5mongo11msgassertedEiRKSs + 12 3 mongoimport 0x000000010025eb98 _ZN5mongo8fromjsonEPKcPi + 1576 4 mongoimport 0x000000010000e643 _ZN6Import8parseRowEPSiRN5mongo7BSONObjERi + 2739 5 mongoimport 0x0000000100010b60 _ZN6Import3runEv + 7376 6 mongoimport 0x000000010055c4cc _ZN5mongo4Tool4mainEiPPc + 5436 7 mongoimport 0x00000001000014b7 main + 55 8 mongoimport 0x0000000100001454 start + 52 Sat Mar 2 01:26:12 exception:BSON representation of supplied JSON is too large: Failure parsing JSON string near: ,execModif Sat Mar 2 01:26:12 Sat Mar 2 01:26:12 imported 0 objects Sat Mar 2 01:26:12 ERROR: encountered 1941 errors 

I do not know what's the problem. Can someone recommend a solution?

Answer 1

I managed to fix the error using the following query:

 mongoimport --db dbName --collection collectionName --file fileName.json --jsonArray 

Hope this helps someone.

Answer 2

try it,

 mongoimport --db dbName --collection collectionName <fileName.json 

Example

 mongoimport --db foo --collection myCollections < /Users/file.json connected to: *.*.*.* Sat Mar 2 15:01:08 imported 11 objects 

The problem is with your date format.

I used the same date-modified JSON as shown below and worked

 {jobID:"2597401", account:"XXXXX", user:"YYYYY", pkgT:{"pgi/7.2-5":{libA:["libpgc.so"],flavor:["default"]}}, startEpoch:"1338497979", runTime:"1022", execType:"user:binary", exec:"/share/home/01482/XXXXX/appker/ranger/NPB3.3.1/NPB3.3-MPI/bin/ft.D.64", numNodes:"4", sha1:"5a79879235aa31b6a46e73b43879428e2a175db5", execEpoch:1336766742, execModify:{"$date" : 1343779200000}, startTime:{"$date" : 1343779200000}, numCores:"64", sizeT:{bss:"1881400168",text:"239574",data:"22504"}} 

hope this helps

Answer 3

Using mongoimport you can achieve the same

 mongoimport --db test --collection user --drop --file ~/downloads/user.json 

Where,

 test - Database name user - collection name user.json - dataset file 

--drop collection if it already exists.

Answer 4

Prefixes:

 mongoimport -d dbName -c collectionName dataFile.js 

Answer 5

Your syntax looks completely correct:

 mongoimport --db dbName --collection collectionName --file fileName.json 

Make sure you are in the correct folder or provide the full path.

Answer 6

I used the command below to export DB

 mongodump --db database_name --collection collection_name 

and below the team worked for me to import DB

 mongorestore --db database_name path_to_bson_file 

Answer 7

Run the import command in another terminal. (not inside the mongo shell.)

 mongoimport --db test --collection user --drop --file ~/downloads/user.json 

Answer 8

In windows you can use Prompcmd cmd commands in Ubuntu you can use your terminal by entering the following command:

 mongoimport -d your_database_name -c your_collection_name /path_to_json_file/json_file_name.json 

then when you open your mongo shell, you will find to check the database_name when executing this command:

 show databases 

Answer 9

This command works where no collection is specified.

 mongoimport --db zips "\MongoDB 2.6 Standard\mongodb\zips.json" 

Mongo shell after executing a command

 connected to: 127.0.0.1 no collection specified! using filename 'zips' as collection. 2014-09-16T13:56:07.147-0400 check 9 29353 2014-09-16T13:56:07.148-0400 imported 29353 objects 

Answer 10

I tried something like this and it actually works:

 mongoimport --db dbName --file D:\KKK\NNN\100YWeatherSmall.data.json 

Answer 11

This works with me when the db with username and password

 mongoimport --db YOUR_DB --collection MyCollection --file /your_path/my_json_file.json -u my_user -p my_pass 

DB without user password, please delete -u my_user -p my_pass

My JSON Sample

 { "_id" : ObjectId("5d11c815eb946a412ecd677d"), "empid" : NumberInt(1), "name" : "Rahul" } { "_id" : ObjectId("5d11c815eb946a412ecd677e"), "empid" : NumberInt(2), "name" : "Rahul" } 

Answer 12

If you try to export this test collection:

 > db.test.find() { "_id" : ObjectId("5131c2bbfcb94ddb2549d501"), "field" : "Sat Mar 02 2013 13:13:31 GMT+0400"} {"_id" : ObjectId("5131c2d8fcb94ddb2549d502"), "field" : ISODate("2012-05-31T11:59:39Z")} 

with mongoexport (the first date created using Date(...) , and the second created using new Date(...) (if using ISODate(...) will be the same as on the second line)), then mongoexport output will look like this:

 { "_id" : { "$oid" : "5131c2bbfcb94ddb2549d501" }, "field" : "Sat Mar 02 2013 13:13:31 GMT+0400" } { "_id" : { "$oid" : "5131c2d8fcb94ddb2549d502" }, "field" : { "$date" : 1338465579000 } } 

So you have to use the same notation because strict JSON is not of type Date( <date> ) .

Also your JSON is invalid: all field names must be enclosed in double quotes, but mongoimport works fine without them.

You can find more information in the mongodb documentation here too .

Answer 13

Hey, I know this is not very convenient, but every time I need to import several jsons files into my mongo database, I make several copies, paste and run until I have enough !!! If you suffer the same way, I wrote a small batch script that will do this for me. INTRESTED?

https://github.com/aminjellali/batch/blob/master/mongoImporter.bat

 @echo off title = Mongo Data Base importing tool goto :main :import_collection echo importing %~2 set file_name=%~2 set removed_json=%file_name:.json=% mongoimport --db %~1 --collection %removed_json% --file %~2 goto :eof :loop_over_files_in_current_dir for /f %%c in ('dir /b *.json') do call :import_collection %~1 %%c goto :eof :main IF [%1]==[] ( ECHO FATAL ERROR: Please specify a data base name goto :eof ) ELSE ( ECHO @author amin.jellali ECHO @email [e-mail] echo starting import... call :loop_over_files_in_current_dir %~1 echo import done... echo hope you enjoyed me ) goto :eof 

Answer 14

Decision: -

 mongoimport --db databaseName --collection tableName --file filepath.json 

Example: -

Put your file in the admin folder: -

 C:\Users\admin\tourdb\places.json 

Run this command on your terminal: -

 mongoimport --db tourdb --collection places --file ~/tourdb/places.json 

Exit: -

 admin@admin-PC MINGW64 / $ mongoimport --db tourdb --collection places --file ~/tourdb/places.json 2019-08-26T14:30:09.350+0530 connected to: localhost 2019-08-26T14:30:09.447+0530 imported 10 documents 

For more link link

Answer 15

this should work inside mongoshell

 mongoimport -d databaseName -c collectionName --drop fileName.json 

if you did not run the shell in the correct directory, you must specify the file along with its path.