How to wait for a large file to load?

Question

I have an application that successfully uses synchronous methods to upload files (NSData initWithContentsOfURL and NSURLConnection sendSynchronousRequest), but now I need to support large files. This means that I need to beat the stream to disk. Although streaming to disk and becoming asynchronous should be completely orthogonal concepts, the Apple API forces me to go asynchronously for the stream.

To be clear, I have been instructed to increase file downloads rather than rebuild the entire application for more asynchronous use. I have no resources. But I admit that the approaches that depend on the reorganization are reliable and good.

So, if I do this:

NSURLConnection* connection = [ [ NSURLConnection alloc ] initWithRequest: request delegate: self startImmediately: YES ]; 

.. In the end, I did didReceiveResponse and didReceiveData called myself. Excellent. But if I try to do this:

 NSURLConnection* connection = [ [ NSURLConnection alloc ] initWithRequest: request delegate: self startImmediately: YES ]; while( !self.downloadComplete ) [ NSThread sleepForTimeInterval: .25 ]; 

... didReceiveResponse and didReceiveData are never called. And I understood why. Oddly enough, asynchronous loading happens in the same main thread that I use. Therefore, when I sleep the main thread, I also dream that I am doing work. Anyway, I tried several different ways to achieve what I want here, including telling NSURLConnection to use another NSOperationQueue and even do dispatch_async to create the connection and run it manually (I don't see how this can't work - I shouldn't have done this is correct), but nothing works. Edit: What I didn’t do right is understanding how start loops work, and that you need to run them manually in secondary threads.

What is the best way to wait for a file to load?

Change 3, working code : The following code really works, but let me know if there is a better way.

Code is executed in the source thread, which establishes a connection and waits for the download to complete:

 dispatch_queue_t downloadQueue = dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0 ); dispatch_async(downloadQueue, ^{ self.connection = [ [ NSURLConnection alloc ] initWithRequest: request delegate: self startImmediately: YES ]; [ [ NSRunLoop currentRunLoop ] run ]; }); while( !self.downloadComplete ) [ NSThread sleepForTimeInterval: .25 ]; 

Code is executed in a new thread that responds to connection events:

 -(void)connection:(NSURLConnection*) connection didReceiveData:(NSData *)data { NSUInteger remainingBytes = [ data length ]; while( remainingBytes > 0 ) { NSUInteger bytesWritten = [ self.fileWritingStream write: [ data bytes ] maxLength: remainingBytes ]; if( bytesWritten == -1 /*error*/ ) { self.downloadComplete = YES; self.successful = NO; NSLog( @"Stream error: %@", self.fileWritingStream.streamError ); [ connection cancel ]; return; } remainingBytes -= bytesWritten; } } -(void)connection:(NSURLConnection*) connection didFailWithError:(NSError *)error { self.downloadComplete = YES; [ self.fileWritingStream close ]; self.successful = NO; } - (void)connectionDidFinishLoading:(NSURLConnection *)connection { self.downloadComplete = YES; [ self.fileWritingStream close ]; self.successful = YES; } 

Answer 1

... didReceiveResponse and didReceiveData are never called. And I found out why. Oddly enough, asynchronous loading occurs in the same main thread that I use. It does not create a new thread. So when I sleep the main thread, I also dream that I am doing work.

That's right. The connection is controlled by a start cycle; if you sleep a thread, the start cycle stops and this prevents your connection from doing its job.

So don’t do anything special. Let the application sit there when starting the launch cycle. A small spinner may appear on the screen to entertain the user. Talk about your business if you can. If at all possible, let the user continue to use the application. Your delegate method will be called when the connection is complete, and then you can do what you need to do with the data.

When you move the code to a background thread, you will again need a run loop to connect this connection. Thus, you will begin to create a launch cycle, plan your connection, and then just return. The run loop will continue to work, and your delegation method will be called again when the connection completes. If the thread is completed, you can stop the execution loop and let the thread exit. That is all that is needed.

Example:. We put this in specific terms. Let's say you want to make several connections one at a time. Paste the URL into the mutable array. Create a method called (for example) startNextConnection that does the following:

• grabs the url from the array (removes it in the process)

• creates a URL request

• NSURLConnection starts

• return

Also, complete the required NSURLConnectionDelegate methods, especially connectionDidFinishLoading: Ask this method to do the following:

• write the data somewhere (write it to a file, transfer it to another stream for parsing, whatever)

• call startNextConnection

• return

If there were no errors, this would be enough to get data for all the URLs in your list. (Of course, you want startNextConnection to be smart enough to just return when the list is empty.) But errors occur, so you have to think about how to deal with them. If the connection fails, do you want to stop the whole process? If so, just try the connection:didFailWithError: method to do something suitable, but don't call it startNextConnection . Do you want to go to the next URL in the list if there is an error? Then do ...didFailWithError: call startNextRequest .

Alternative: If you really want to maintain the consistent structure of your synchronous code, so that you have something like:

 [self downloadURLs]; [self waitForDownloadsToFinish]; [self processData]; ... 

then you have to load in another thread so that you can block the current thread. If you need it, configure the download stream with a start loop. Then create a connection using -initWithRequest:delegate:startImmediately: as you did, but pass NO in the last parameter. Use -scheduleInRunLoop:forMode: to add a connection to the load stream startup loop, and then start the connection using the -start method. This gives you the ability to sleep the current thread. Ask the procedure for completing delegation of the connection to set a flag in your example, for example, the self.downloadComplete flag.

Answer 2

I hesitate to give this answer because others are true that you really need to structure the application around the asynchronous model. Nonetheless:

 NSURLConnection* connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:NO]; NSString* myPrivateMode = @"com.yourcompany.yourapp.DownloadMode"; [connection scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:myPrivateMode]; [connection start]; while (!self.downloadComplete) [[NSRunLoop currentRunLoop] runMode:myPrivateMode beforeDate:[NSDate distantFuture]]; 

Do not do this in the main thread. Your application is just as likely to be terminated to block the main thread as to load an oversized file into memory.

By the way, given that you are loading the file instead of memory, you should go from NSURLConnection to NSURLDownload .

Answer 3

I think your sleepForInterval blocking NSURLConnection activity -

The loop thread processing is not performed while the thread is blocked.

From the NSThread documentation .

---

I think you might have to rethink how you configure the downloadComplete variable. Think about using the connectionDidFinishLoading:connection delegate method to determine when the download will complete, and not your loop + sleep?

 - (void)connectionDidFinishLoading:(NSURLConnection *)connection { self.downloadComplete = YES; // release the connection, and the data object [connection release]; [receivedData release]; } 

In the NSURLConnection manual .

You can use the connection:connection didFailWithError:error delegate method to make sure that you are dealing with situations where the download is not completed.