An easy way to limit HttpClient requests

Question

I am using HTTPClient in System.Net.Http to make requests against the API. The API is limited to 10 requests per second.

My code is something like this:

List<Task> tasks = new List<Task>(); items..Select(i => tasks.Add(ProcessItem(i)); try { await Task.WhenAll(taskList.ToArray()); } catch (Exception ex) { } 

The ProcessItem method performs several actions, but always calls the API using the following: await SendRequestAsync(..blah) . It looks like this:

 private async Task<Response> SendRequestAsync(HttpRequestMessage request, CancellationToken token) { token.ThrowIfCancellationRequested(); var response = await HttpClient .SendAsync(request: request, cancellationToken: token).ConfigureAwait(continueOnCapturedContext: false); token.ThrowIfCancellationRequested(); return await Response.BuildResponse(response); } 

Initially, the code worked fine, but when I started using Task.WhenAll, I started receiving "Exceed Limit Values" messages from the API. How can I limit the speed at which requests are made?

It is worth noting that ProcessItem can execute between 1-4 API calls depending on the element.

Answer 1

The API is limited to 10 requests per second.

Then just ask the code to make a package of 10 requests, ensuring that they take at least one second:

 Items[] items = ...; int index = 0; while (index < items.Length) { var timer = Task.Delay(TimeSpan.FromSeconds(1.2)); // ".2" to make sure var tasks = items.Skip(index).Take(10).Select(i => ProcessItemsAsync(i)); var tasksAndTimer = tasks.Concat(new[] { timer }); await Task.WhenAll(tasksAndTimer); index += 10; } 

Update

My ProcessItems method makes 1-4 API calls depending on the element.

In this case, batch processing is not a suitable solution. You need to limit the asynchronous method to a specific number, which implies SemaphoreSlim . The tricky part is that you want to resolve more calls over time.

I have not tried this code, but the general idea I came up with is to have a periodic function that frees up a semaphore up to 10 times. So something like this:

 private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(10); private async Task<Response> ThrottledSendRequestAsync(HttpRequestMessage request, CancellationToken token) { await _semaphore.WaitAsync(token); return await SendRequestAsync(request, token); } private async Task PeriodicallyReleaseAsync(Task stop) { while (true) { var timer = Task.Delay(TimeSpan.FromSeconds(1.2)); if (await Task.WhenAny(timer, stop) == stop) return; // Release the semaphore at most 10 times. for (int i = 0; i != 10; ++i) { try { _semaphore.Release(); } catch (SemaphoreFullException) { break; } } } } 

Using:

 // Start the periodic task, with a signal that we can use to stop it. var stop = new TaskCompletionSource<object>(); var periodicTask = PeriodicallyReleaseAsync(stop.Task); // Wait for all item processing. await Task.WhenAll(taskList); // Stop the periodic task. stop.SetResult(null); await periodicTask; 

Answer 2

UPDATED RESPONSE

My ProcessItems method makes 1-4 API calls depending on the element. So with a batch size of 10, I am still surpassing the speed limit.

You need to embed a calendar window in SendRequestAsync. A queue containing timestamps for each request is a suitable data structure. You delete records with a time stamp of more than 10 seconds. As it happens, there is an implementation as an answer to a similar question about SO.

ORIGINAL RESPONSE

May be useful to others

One easy way to handle this is to execute your requests in groups of 10, run them at the same time, and then wait until a total of 10 seconds have elapsed (if it hasn't already). This will lead you to a speed limit if the request package can complete in 10 seconds, but will be less optimal if the request package takes longer. Take a look at the .Batch () extension method in MoreLinq . The code will look something like

 foreach (var taskList in tasks.Batch(10)) { Stopwatch sw = Stopwatch.StartNew(); // From System.Diagnostics await Task.WhenAll(taskList.ToArray()); if (sw.Elapsed.TotalSeconds < 10.0) { // Calculate how long you still have to wait and sleep that long // You might want to wait 10.5 or 11 seconds just in case the rate // limiting on the other side isn't perfectly implemented } } 

Answer 3

The answer is similar to this .

Instead of using the task list and WhenAll use Parallel.ForEach and use ParallelOptions to limit the number of simultaneous tasks to 10 and make sure that each of them takes at least 1 second:

 Parallel.ForEach( items, new ParallelOptions { MaxDegreeOfParallelism = 10 }, async item => { ProcessItems(item); await Task.Delay(1000); } ); 

Or, if you want to make sure that each element takes as long as 1 second:

 Parallel.ForEach( searches, new ParallelOptions { MaxDegreeOfParallelism = 10 }, async item => { var watch = new Stopwatch(); watch.Start(); ProcessItems(item); watch.Stop(); if (watch.ElapsedMilliseconds < 1000) await Task.Delay((int)(1000 - watch.ElapsedMilliseconds)); } ); 

Or:

 Parallel.ForEach( searches, new ParallelOptions { MaxDegreeOfParallelism = 10 }, async item => { await Task.WhenAll( Task.Delay(1000), Task.Run(() => { ProcessItems(item); }) ); } );