Need help deciding what is the best solution for a tracking system (.NET)

Here is my first question, my name is Anna!

My problem: My client has several personal devices (black box with gps) to find people / car ... There are about 12,000 people / car using this device ... He sends his location to the specified IP / port ... I I can’t do anything on this side ...

My job? Devolope listener to catch all the data sent by devices and load it into the database using .NET ...

My idea: Window service using threads (maybe ThreadPool?). Thus, the service will catch all incoming messages, create a stream and put it in the database ...

Is this the best solution to the problem? I read about Message Queuing (MSMQ) here ... Do you think I should use this?

Anna

+4
source share
2 answers

The number of locations / time, as well as the method of transmitting location information (such as TCP, UDP, etc.) will help determine the optimal approach.

Some questions:

  • How often do 12000 devices send a message?
  • How are locations transmitted? UDP? TCP?
  • What reliability requirements do you have?

Of its sounds, having a service that can capture requests and just maintain a queue inside things to store in the database should work fine. I do not believe that MSMQ will be useful to you if you cannot change the direction of the transfer, and even then it may or may not be needed.


EDIT: Given the comments below, I would suggest something where you have requests to listen on a TCP listener to handle threadpool.

I would take a look at this tutorial on setting up a TCP listening server using a thread pool. The biggest potential problem that I see is the number of requests - from what you say, you will have about 400 requests per second. This would be a bit of a challenge without a good system. Threadpool will probably work better than trying to do its own threading, as you want to avoid the overhead of constantly creating new threads. You will definitely want to have a very small delay in the main processing cycle (for example, Sleep (0) or not sleep at all), since on average you will have one request per 2.5 ms. One Sleep tends to temporarily cut at least 14-15 ms, so you probably won't need to sleep in a loop.

I would say, however, that you may find that this does not work too well, since an untreated number of connections can be problematic. If you can somehow convert UDP packets to sent packets, this can improve performance (due to some reliability).

+3
source

The code below has not been tested or should only be considered evidence. It uses a thread pool for everything. I put everything in one file for an example only. You have to split everything into several files.

It is important not to allow all clients to save directly to the database, as this will starve the thread pool. Experiment with the "MaxThreads" constant to get a value that works with your db / server.

Also remember that I do not handle any exceptions at all. You need, for example, to handle a SocketException in the BeginRead, EndRead and TcpListener methods.

I tried to use the minimum number of thread synchronization locks, the code should be quite efficient. Bottle bottle is likely to be a database.

using System; using System.Collections.Generic; using System.Net; using System.Net.Sockets; using System.Threading; namespace FastListener { /// <summary> /// Example position class, replace with a real definition /// </summary> public class Position { public int X { get; set; } public int Y { get; set; } } /// <summary> /// Needed to be able to pass socket/buffer info /// between asynchronous requests. /// </summary> public struct ClientContext { public Socket socket; public byte[] buffer; } class Program { /// <summary> /// Positions received from mobile clients but not yet saved /// into the database. /// </summary> private readonly Queue<Position> _positions = new Queue<Position>(); /// <summary> /// Number of threads currently saving stuff to the database. /// </summary> private int _poolThreads; /// <summary> /// Maximum number of threads that can save info to the database. /// </summary> private const int MaxThreads = 10; static void Main(string[] args) { new Program().Start(); } private void Start() { TcpListener listener = new TcpListener(IPAddress.Any, 1343); listener.Start(50); listener.BeginAcceptSocket(OnAccept, listener); } // Listener got a new connection private void OnAccept(IAsyncResult ar) { TcpListener listener = (TcpListener) ar.AsyncState; // It very important to start listening ASAP // since you'll have a lot of incoming connections. listener.BeginAcceptSocket(OnAccept, listener); // I recommend that you create a buffer pool to improve performance byte[] buffer = new byte[400]; // Now accept the socket. Socket socket = listener.EndAcceptSocket(ar); StartRead(new ClientContext {buffer = buffer, socket = socket}); } private void StartRead(ClientContext context) { // start reading from the client. context.socket.BeginReceive(context.buffer, 0, 400, SocketFlags.None, OnReceive, context); } // Stuff from a client. private void OnReceive(IAsyncResult ar) { ClientContext context = (ClientContext) ar.AsyncState; int bytesRead = context.socket.EndReceive(ar); if (bytesRead == 0) { // Put the buffer back in the pool context.socket.Close(); return; } // convert bytes to position. // i'll just fake that here. Position pos = new Position(); // Either handle the request directly if (_poolThreads < MaxThreads) ThreadPool.QueueUserWorkItem(SaveToDatabase, pos); else { // Or enqueue it to let a already active // thread handle it when done with the previous position lock (_positions) _positions.Enqueue(pos); } // Don't forget to read from the client again StartRead(context); } // will save stuff to the database. private void SaveToDatabase(object state) { // Could use Interlocked.Increment, but not really vital if // one more more extra threads are saving to the db. ++_poolThreads; Position position = (Position) state; while (true) { // IMPLEMENT DB SAVE LOGIC HERE. // check if another position is in the queue. lock (_positions) { if (_positions.Count > 0) position = _positions.Dequeue(); else break; // jump out of the loop } } --_poolThreads; } } } 
+2
source

All Articles