AVAssetReaderOutput copyNextSampleBuffer freezes when encoding video from audio on a device

Question

After implementing a solution for encoding video (with sound) in this issue Encoding video using AVAssetWriter - CRASHES , I found that the code works correctly in iPhone Simulator, Unfortunately, some videos cannot encode their sound while working on a real iPhone 5 ( and other devices).

For example, videos created from WWy 2011 RosyWriter sample code ( https://developer.apple.com/library/IOS/samplecode/RosyWriter/Introduction/Intro.html ) are not fully encoded because the function -[AVAssetReaderOutput copyNextSampleBuffer] never not returning.

Video buffers arrive correctly, but as soon as it tries to copy the first audio file CMSampleBufferRef, the call hangs. When I try to do this on videos that come from other sources, such as those recorded in the iOS camera app, the sound is imported correctly.

This thread, https://groups.google.com/forum/#!topic/coreaudio-api/F4cqCu99nUI , notes the copyNextSampleBuffer function, which copyNextSampleBuffer when used with AudioQueues, and suggests saving operations on a single thread. I tried to keep everything in a separate thread, in the main thread, but no luck.

Has anyone else experienced this and got a possible solution?

EDIT: It seems that the videos created from RosyWriter change their tracks relative to the video from the camera’s own application, that is, the audio stream as stream 0, and the video stream as stream 1.

 Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 60 kb/s Metadata: creation_time : 2013-10-28 16:13:05 handler_name : Core Media Data Handler Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080, 8716 kb/s, 28.99 fps, 29.97 tbr, 600 tbn, 1200 tbc Metadata: rotate : 90 creation_time : 2013-10-28 16:13:05 handler_name : Core Media Data Handler 

Not sure if this matters to AVAssetReader.

Answer 1

This is an AVFoundation bug in iOS7. Now it is fixed in iOS8.

Answer 2

I was still experiencing this problem on iOS 9.3.2, and all that resolved it was that when I called -[AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks] AVAssetReaderAudioMixOutput* was initially set with parameters, not nil .

Example:

 NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey, [NSNumber numberWithFloat:44100.0], AVSampleRateKey, [NSNumber numberWithInt:16], AVLinearPCMBitDepthKey, [NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved, [NSNumber numberWithBool:NO], AVLinearPCMIsFloatKey, [NSNumber numberWithBool:NO], AVLinearPCMIsBigEndianKey, nil]; // create an AVAssetReaderOutput for the audio tracks NSArray* audioTracks = [asset tracksWithMediaType:AVMediaTypeAudio]; AVAssetReaderAudioMixOutput* _audioReaderOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:audioTracks audioSettings:outputSettings]; 

This prevented subsequent calls -[AVAssetReaderOutput copyNextSampleBuffer] from freezing when they otherwise did.

Answer 3

Check the time range on the audio tracks using something like this:

 NSLog(@"audioTrack timeRange: %lld, %lld", audioTrack.timeRange.start.value, audioTrack.timeRange.duration.value); 

Empty time ranges (0, 0) can cause CopyNextSampleBuffer () to freeze.

Answer 4

This is a hardware problem. I tested it on a simulator, iPhone5 and iPhone 6. The problem occurs 100% with certain video files, but ONLY on iPhone 5. I can’t speak for iPhone 5S. iOS8 will NOT fix this problem. Fix may be limiting your iPhone 5 users from certain functions. Another job is for the user to select a video from his camera roll, thereby forcing him to special compression on it - and baptizing this video into a format that the iPhone 5 hardware can work with.