Parallel loading of Blob Throw 404 Unsuccessful request with interruptions

Question

I have a very simple service,

public class AzureService : IAzureService { private readonly CloudBlobContainer _container; public AzureService(ISettings settings) { var storageAccount = CloudStorageAccount.Parse(settings.BlobConnectionString); var blobClient = storageAccount.CreateCloudBlobClient(); _container = blobClient.GetContainerReference(settings.BlobContainerName); } public Task UploadBlobAsync(string fileName, Stream stream) { var blob = _container.GetBlockBlobReference(fileName); return blob.UploadFromStreamAsync(stream); } public Task DeleteBlobAsync(string fileName) { var blob = _container.GetBlockBlobReference(fileName); return blob.DeleteAsync(); } } 

This method is called from

  public Task SaveAllAsync(Dictionary<string, Stream> images) { var tasks = new List<Task>(); foreach (var image in images) { var fileName = image.Key; var stream = image.Value; var task = _azureService.UploadBlobAsync(fileName, stream); tasks.Add(task); } return Task.WhenAll(tasks); } 

My stream is HttpPostedFileBase.InputStream . Sometimes this works, and sometimes I get The remote server returned an error: (400) Bad Request. . If I put a breakpoint, it also works.

Answer 1

I had the same problem, I tried to load 20 + images into 1 strike, single-threaded jobs, multithreading using await Task.WhenAll with the error "Remote server answered error: (400)" Bad request ".

• see RequestInformation inside Microsoft.WindowsAzure.Storage.StorageException , which is called from Async [xxx] methods for more details.
  

• First, RequestInformation said something about the MD5 problem with the error code "Md5Mismatch", buy my intuition, said otherwise, because one thread works like a charm, and then .. I found it ... DefaultRequestOptions.ParallelOperationThreadCount on CloudBlobClient object and problem sovled .

• MSDN BlobRequestOptions Members

  private CloudBlobContainer ConnectToImageContainer() { var credentials = new StorageCredentials(AccountName, ImagesContainerKey); var account = new CloudStorageAccount(credentials, useHttps: true); var client = account.CreateCloudBlobClient(); client.DefaultRequestOptions.ParallelOperationThreadCount = 64; // max value client.DefaultRequestOptions.SingleBlobUploadThresholdInBytes = 67108864; // max value var container = client.GetContainerReference(ImagesContainerName); return container; } 

Answer 2

The behavior you describe is very similar to a threading problem (i.e. if you leave code that works fine because it is efficiently single-threaded at the time), which leads to incomplete or incorrect data being sent to the Azure Storage API.

Your definition of "var image" can lead to unpredictability in a multi-threaded environment (if you use ReSharper, it will highlight this variable and suggest changing the code, since it is potentially dangerous).

Read this SO post to understand a little more and how you could better implement your code.

ID and close foreach