ExpressJS - How to handle concurrent requests? Requests seem to be blocking each other.

Question

I have the following code snippet

var express = require('express'); var routes = require('./routes'); var http = require('http'); ... app.get('/a',function(){ Card.findCards(function(err, result){ //Mongoose schema res.send(result); //Executes a query with 9000 records }) }); app.get('/b', function(req, res){ res.send("Hello World"); }); 

I find that when I am on localhost / a, it takes about 2.3 seconds to complete. This is not surprising, since it extracts quite a lot of data from the database. However, I found that if I get / b at boot time / b, b will not be displayed. It is as if calling / a blocked the call to / b.

Is this how the express should work? I have always worked on the assumption that individual routes are asynchronous as they accept callbacks, but it seems that express can only process one request at a time. Prior to calling res.end (), no other request is processed. Am I missing any configuration I need to do?

For reference, here is how I connect to mongoose

 mongoose.connect(dbConnectionString, {server:{poolSize:25}}); 

And this is my part of initializing the http server

 http.globalAent.maxSockets = 20; // or whatever http.createServer(app).listen(app.get('port'), function(){ console.log('Express server listening on port ' + app.get('port')); }); 

EDIT: Here is the code for the map model and its associated circuits + functions

 //Card.js var mongoose = require('mongoose') , Schema = mongoose.Schema; var CardSchema = new Schema({ _id : {type: String}, stores : [{ store: {type: Schema.Types.ObjectId, ref:'StoreModel', required: true} , points: {type: Number, required: true} }] }); exports.findCards = function(callback){ var query = Card.find({}, callback); } 

Answer 1

I have the same problem with a setting similar to yours. There are two problems, each of which has the same reason: Node has non-blocking I / O, but (as bbozo points out). CPU operations block it.

The first problem is your conversation in the mongoose. After the mongoose extracts documents from your collection, it converts them into mongoose objects. If you get 9000 records, he will do it 9000 times. The lines in question are in the mongoose query.js library; check the for loop for its completeMany function to find the appropriate lock operations.

The second problem occurs when Express builds the resulting JSON objects to send your response. The culprit is the res.json function in the Express response.js library. For a large response, the blocking nature of stringify will be noticeable.

I am not quite sure how to solve this problem. Perhaps you are probably trying to use the mongodb native library, not mongoose. You can also try installing the Express patch so that it uses JSON streaming calls instead of blocking. The markup of your request and response will also help, although I know that this is not very simple to implement.

Answer 2

I'll try:)

Afaik Node.js is not asynchronous in the general sense, it simply does not block in the sense that the connection does nothing, and another connection that does not block the connection does nothing,

Things that take up a lot of CPU time (for example, loading a lot of data) block the event loop, try to get the data in smaller fragments

Answer 3

You can set the lean option.

Documents returned from queries with the lean option turned on are simple javascript objects, not MongooseDocuments. They have no save method, getters / setters or other Mongoose magic.

Example:

 new Query().lean() // true new Query().lean(true) new Query().lean(false) Model.find().lean().exec(function (err, docs) { docs[0] instanceof mongoose.Document // false }); 

see mongoose documentation