Background tasks in Meteor

Question

I am wondering if there is a way to implement background taxes, possibly with a pool of workers. Can you show me the direction I'm thinking of writing a package for this?

Answer 1

Update 2019

Before thinking about writing a package for anything, first look at whether there are packages that do what you need. In the Meteor world, this means that you need to look for packages "work / queue / task / employee management / planning" in the Atmosphere, and then npm for the same search conditions. You also need to more accurately define your requirements:

• Do you want persistence, or will the solution work in memory?
• Do you want to be able to distribute jobs on different machines?

Meteor specific

• The Job-collection is reliable (I used it in 2014 on production at startup), but is currently in maintenance mode. Allows you to schedule ongoing tasks to run anywhere (servers, clients).
• SteveJobs - actively supported by Max Savin, author of several powerful Meteor tools
• littledata: synced-cron - "A simple cron system for Meteor. It supports job synchronization between multiple processes."

Abandoned packages:

• artwells: queue - priorities, scheduling, logging, re-queuing. Queue supported by MongoDB. Last Code Accepted : 2015 Oct
• super basic cron packages: easycron . Last updated: December 2015
• differential: workers - generate headless working meteor processes for working on asynchronous tasks. Last Code Accepted : January 2015
• cron ( since 2015 )
• PowerQueue - abandoned since 2014 . The queue of asynchronous tasks, the use of gas resources, a retry failed. Supports nested queues. No planning . No tests, but a great demo . Not suitable for long work due to the use of recursive calls .

Npm Packages

Meteor has been able to use npm packages directly for several years now, so this question boils down to finding job / job / queue management packages in NPM. If you do not care about perseverance:

• Async "provides about 70 functions, which include the usual" functional "suspects ( map , reduce , filter , each ...), as well as some common patterns for an asynchronous control flow ( parallel , series , waterfall ...)"
• d3-queue - minimalistic, written by D3 author Mike Bostock

If you want to stay consistent, as Meteor already uses MongoDB, it might be beneficial to use a job-scheduling package with saving in MongoDb. The Agenda seems to be the most powerful and popular program , but, unfortunately, it has not been supported for months and has a significant lag in solving problems .

If you want to add a dependency supported by redis to your project, there are several options:

• bull is the most comprehensive Node job queue solution supported by Redis
• A bee is simple, fast, strong. Doesn't suffer from a memory leak that Bull demonstrates
• Kue - priority job queue for a node

Like MongoDB, Redis can also provide high availability (via Redis Sentinel), and if you want to distribute tasks between multiple work machines, you can send them all to a single Redis server .

Answer 2

There is a package based on Cron jobs that can be used to schedule tasks for specific intervals or dates. Here is the package: https://atmosphere.meteor.com/package/cron

And if you have to look into the source of this package, you will notice that they just use:

Meteor.setInterval( ... , delay );

So, if you save your tasks in a database, load them at intervals during startup, then you will probably be on the right track.

Answer 3

If you are looking for something that is typical of Meteor, I am happy to share that there is a new package called Steve Jobs. This facilitates the execution of background jobs, like calling a method.

It has all the standard functions that you expect, for example, starting a task only once, retrying failed tasks, etc. You can learn more about this on GitHub:

http://github.com/msavin/stevejobs

Answer 4

I assume that the correct support is included in their roadmap, but in the meantime I managed to start it using setInterval . See cron-tick package.