Geek Answers Handbook

Queue using table

I need to implement a queue using a table. The business requirement is to have one queue to which 5-10 boxes will be available in order to get the next job / job. There will be no more than 5,000 jobs per day. In addition, the job package must be unloaded at a time.

It’s just interesting what problem areas and problems I may encounter, since I have not done this before. If someone came across this / did it earlier, could you please indicate to me the implementation of the project / model or problems that need to be taken care of.

thanks

+4
source share
4 answers

There are many general-purpose queuing or messaging services. Even if you want to implement your own system, you can try to look at a few others. The first thing that comes to mind is JMS ( Java Message Service ) with implementations such as Apache ActiveMQ , OpenJMS or JBoss Messaging . Then you have many suggestions not related to open source.

The second thing that comes to mind is the Amazon Simple Queue Service . There are several products that implement the same interface as django-queue-service .

Good luck

+3
source

Problem areas:

  • Concurrency
  • Security
  • Speed
  • Encoding
  • Uniqueness

And of course this one:

  • Unclear specification of the problem domain

Happy coding!

+2
source

This does not sound too complicated; just specify a timestamp that you can sort whenever tasks are set. Depending on the database, this field may be autocomplete with the current timestamp.

When retrieving jobs, it is as simple as including SELECT and DELETE statements in a transaction. If you feel this is uncomfortable, something like this might do it:

UPDATE tblQueue SET mark = <unique application id> WHERE mark IS NULL ORDER BY timestamp ASC LIMIT 1 SELECT * FROM tblQueue WHERE mark = <unique app id> DELETE FROM tblQueue WHERE mark = <unique app id>

Using this setting, you can avoid transactions if they scare you.

Your definition of the party is somewhat fuzzy; if you just want me to be able to process 10 elements at a time, just change the LIMIT 1 clause of the first query to LIMIT 10.

If you mean that tasks can be grouped, you probably need a task queue and place sub-elements in another table (this is not a queue, just a regular table with a foreign key pointing to a task element).

+2
source

Thanks Vegard.

But the approach you proposed will lead to the loss of job requests in the event of a system malfunction / failures.

I was thinking of a Queue table with the following columns

  • RequestID / MessageID (primary key)
  • LockedBy (who is working on the request)
  • LockedTime (when the request is locked for processing)
  • RequestedTime (when the request is added to the queue)
  • CompletionTime (when the request is completed)
  • Status (pending / processed)
  • RequestMessage (serialized Java object in my case)
  • Requestor (who sent the request)

I can write a stored procedure [GetNextItemsInQueue] that returns a list of permissions, for example, 10 requests, sets a lock and a locked time. If Locked Time increases the specified limit, the record can be returned to the standby state.

Any problems with this?

0
source

More articles:


All Articles