How to ensure indexing of each object using Firebase FlashLight in a free instance of beta version of ElasticSearch

Question

Thanks to the FlashLight tutorial https://github.com/firebase/flashlight , it is somehow easy to do fulltextsearch with Firebase.

However, if you keep a free instance of ES, it is limited in terms of concurrency access, and when you start the node application, the following message appears in the log:

Failed to index firebase / xxx / -KHLhdwGplb3lHWjm8RS: Error: Parallel request limit exceeded. Please consider fulfilling your requests, or contact [e-mail]. for help [e-mail].

How to solve this?

Answer 1

If you have a ton of data to index, Flashlight will ask the ES to index each object on the fly without restricting access to resources. You must control / restrict access to this resource of the resource using Semaphore.

Install lib semaphore

npm i --save semaphore 

Edit the PathMonitor.js file and restrict access to the ES resource to 1

 PathMonitor.prototype = { _init: function () { this.sem = require('semaphore')(1); this.addMonitor = this.ref.on('child_added', this._process.bind(this, this._childAdded)); this.changeMonitor = this.ref.on('child_changed', this._process.bind(this, this._childChanged)); this.removeMonitor = this.ref.on('child_removed', this._process.bind(this, this._childRemoved)); }, ... _index: function (key, data, callback) { var that = this; that.sem.take(function () { that.esc.index({ index: that.index, type : that.type, id : key, body : data }, function (error, response) { that.sem.leave(); if (callback) { callback(error, response); } }.bind(that)); }); }, ... } 

This may not be necessary with a paid plan.