AVPlayer Real-time Stream Level HLS Indicator (FFT Data Display)

Question

I am using AVPlayer for a radio AVPlayer using HTTP streaming. Now I want to implement a level meter for this audio stream. The best thing is a level meter showing different frequencies, but a simple left / right solution will be a great starting point.

I found some examples using AVAudioPlayer . But I can not find a solution to get the necessary information AVPlayer .

Can anyone think of a solution to my problem?

EDIT I want to create something like this (but nicer)

EDIT II

It has been proposed to use MTAudioProcessingTap to obtain the original audio data. Examples that I could find using the array [[[_player currentItem] asset] tracks] , which, in my case, is an empty array. Another suggestion was to use [[_player currentItem] audioMix] , which is null for me.

EDIT III

After many years, a solution still does not exist. I really made progress, so I share it.

During installation, I add an observer with a key to the playerItem element:

 [[[self player] currentItem] addObserver:self forKeyPath:@"tracks" options:kNilOptions context:NULL]; ////////////////////////////////////////////////////// - (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)changecontext:(void *)context if ([keyPath isEqualToString:@"tracks"] && [[object tracks] count] > 0) { for (AVPlayerItemTrack *itemTrack in [object tracks]) { AVAssetTrack *track = [itemTrack assetTrack]; if ([[track mediaType] isEqualToString:AVMediaTypeAudio]) { [self addAudioProcessingTap:track]; break; } } } - (void)addAudioProcessingTap:(AVAssetTrack *)track { MTAudioProcessingTapRef tap; MTAudioProcessingTapCallbacks callbacks; callbacks.version = kMTAudioProcessingTapCallbacksVersion_0; callbacks.clientInfo = (__bridge void *)(self); callbacks.init = init; callbacks.prepare = prepare; callbacks.process = process; callbacks.unprepare = unprepare; callbacks.finalize = finalise; // more tap setup... AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix]; AVMutableAudioMixInputParameters *inputParams = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:audioTrack]; [inputParams setAudioTapProcessor:tap]; [audioMix setInputParameters:@[inputParams]]; [[[self player] currentItem] setAudioMix:audioMix]; } 

So far so good. All this works, I can find the desired track and configure the input parameters for Params and audioMix, etc. But unfortunately, the only callback that is called is the init callback. None of the others will shoot at any moment.

I tried different (types) of streaming sources, one of which is the official Apple HLS stream: http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8

Answer 1

Unfortunately, using the HLS stream with AVFoundation does not give you any control over the audio tracks. I ran into the same problem trying to disable the HLS stream, which turned out to be impossible.

The only way you could read the audio data is to press AVAudioSession .

EDIT

You can access AVAudioSession as follows:

 [AVAudioSession sharedInstance] 

Here is the documentation for AVAudioSession

Answer 2

Sound measurement using AVPlayer looks like a problem that is still ongoing. However, I believe that a solution can be achieved by combining AVPlayer with AVAudioRecorder .

While the two classes have seemingly conflicting goals, there is work that allows AVAudioRecorder to access the AVPlayer audio output.

Player / Recorder

As described in this answer , AVPlayer audio recording is possible if you gain access to changing the audio route using kAudioSessionProperty_AudioRouteChange .

Please note that sound recording must be started later in order to access the audio rerouting. Use the response of the linked stack as a link - it contains more detailed information and the necessary code.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~

Once you access the AVPlayer audio track and record, the measurement is relatively simple.

Audio levels

In response to a question about the stack regarding measuring the microphone input, I describe the steps required to access the sound level measurements. Using AVAudioRecorder to control volume changes is more complex than you might think, so I included the GitHub project, which acts as a template for monitoring the sound changes during recording.

~~~~~~~~~~~~~~~~~~~~~~~~~~~ Pay attention ~~~~~~~~~~~~~~~~~~~~~~~ ~~~~

This combination during the HLS streaming stream is not what I tested. This answer is strictly theoretical in nature, so it may well be clear that both classes will work fully.