Running several async / wait functions at once and processing them separately

Question

How do you run multiple HttpClient.GetAsync() requests at HttpClient.GetAsync() and process them each as soon as their responses are returned? At first I tried:

 var response1 = await client.GetAsync("http://example.com/"); var response2 = await client.GetAsync("http://stackoverflow.com/"); HandleExample(response1); HandleStackoverflow(response2); 

But of course, he is still consistent. So, I tried to run them both at once:

 var task1 = client.GetAsync("http://example.com/"); var task2 = client.GetAsync("http://stackoverflow.com/"); HandleExample(await task1); HandleStackoverflow(await task2); 

Now the tasks start simultaneously, which is good, but, of course, the code should still wait one after another.

I want to be able to process the answer "example.com" as soon as it appears, and the answer "stackoverflow.com" as soon as it appears.

I could put two tasks in the array to use Task.WaitAny() in a loop, check which one is finished, and call the appropriate handler, but then ... how is it better than regular regular callbacks? Or is this not really the intended use case for async / wait? If not, how to use HttpClient.GetAsync() with callbacks?

To clarify - the behavior I need is something like this pseudo code:

 client.GetAsyncWithCallback("http://example.com/", HandleExample); client.GetAsyncWithCallback("http://stackoverflow.com/", HandleStackoverflow); 

Answer 1

You can use ContinueWith and WhenAll to wait for one new Task , task1 and task2 will be executed in parallel

 var task1 = client.GetAsync("http://example.com/") .ContinueWith(t => HandleExample(t.Result)); var task2 = client.GetAsync("http://stackoverflow.com/") .ContinueWith(t => HandleStackoverflow(t.Result)); var results = await Task.WhenAll(new[] { task1, task2 }); 

Answer 2

You can use a method that will reorder them as they are completed. This is a good trick described by Jon Skeet and Stephen Toub and is also supported by my AsyncEx library .

All three implementations are very similar. Accepting my own implementation:

 /// <summary> /// Creates a new array of tasks which complete in order. /// </summary> /// <typeparam name="T">The type of the results of the tasks.</typeparam> /// <param name="tasks">The tasks to order by completion.</param> public static Task<T>[] OrderByCompletion<T>(this IEnumerable<Task<T>> tasks) { // This is a combination of Jon Skeet approach and Stephen Toub approach: // http://msmvps.com/blogs/jon_skeet/archive/2012/01/16/eduasync-part-19-ordering-by-completion-ahead-of-time.aspx // http://blogs.msdn.com/b/pfxteam/archive/2012/08/02/processing-tasks-as-they-complete.aspx // Reify the source task sequence. var taskArray = tasks.ToArray(); // Allocate a TCS array and an array of the resulting tasks. var numTasks = taskArray.Length; var tcs = new TaskCompletionSource<T>[numTasks]; var ret = new Task<T>[numTasks]; // As each task completes, complete the next tcs. int lastIndex = -1; Action<Task<T>> continuation = task => { var index = Interlocked.Increment(ref lastIndex); tcs[index].TryCompleteFromCompletedTask(task); }; // Fill out the arrays and attach the continuations. for (int i = 0; i != numTasks; ++i) { tcs[i] = new TaskCompletionSource<T>(); ret[i] = tcs[i].Task; taskArray[i].ContinueWith(continuation, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default); } return ret; } 

Then you can use it as such:

 var tasks = new[] { client.GetAsync("http://example.com/"), client.GetAsync("http://stackoverflow.com/"), }; var orderedTasks = tasks.OrderByCompletion(); foreach (var task in orderedTasks) { var response = await task; HandleResponse(response); } 

---

Another approach is to use a TPL data stream ; at the end of each task, send its operations to the ActionBlock<T> , something like this:

 var block = new ActionBlock<string>(HandleResponse); var tasks = new[] { client.GetAsync("http://example.com/"), client.GetAsync("http://stackoverflow.com/"), }; foreach (var task in tasks) { task.ContinueWith(t => { if (t.IsFaulted) ((IDataflowBlock)block).Fault(t.Exception.InnerException); else block.Post(t.Result); }); } 

---

Any of the above answers will work fine. If the rest of your code uses / can use the TPL data stream, then you may prefer this solution.

Answer 3

Declare the async function and pass the callback to:

 void async GetAndHandleAsync(string url, Action<HttpResponseMessage> callback) { var result = await client.GetAsync(url); callback(result); } 

And then just name it a few times:

 GetAndHandleAsync("http://example.com/", HandleExample); GetAndHandleAsync("http://stackoverflow.com/", HandleStackoverflow);