How do I highlight text backlight with audio for a karaoke-like app on an iPhone?

Question

Say, for example, I had an application that would show fragments of lyrics, 3 or 4 lines at a time. Now I want to run the sound file to start playing, and then the time for highlighting the lyrics with the sound itself. It looks like a bouncing ball that bounces from one word to another.

The only thing I can think of is that I will need to create metadata to go along with each sound file so that I can start a timer that works next to the audio file. Or, maybe the audio player in iOS can set its own start timer, and I could call the highlighted word to change at certain points?

Is any idea the best way to do this? I would think that it could be taxation on the processor, if I have to check every second if the word should change, but I do not know how I could call it otherwise.

Answer 1

Not the best answer I know, but a very similar solution was discussed in a WWDC 2011 video using Core Animation.

Find a video called Core Animation Essentials (there should be 421 video on iTunes U for WWDC 2011). An example should be about 29 minutes. Basically, the scenario was that there was a ball bouncing over the lyrics that were played for the song. The project talks about how to revive time and position over words using an interesting idea ...

Although it will take a little time to develop, implement the project with NSTimer and your song, starting from the same time. Implement the project so that every time you click on the screen, the offset from the last time interval is inserted into the NSMutableSArray, which is then written to the file. Now start the project and make timestamps for the words by clicking down when each word is pronounced. (This assumes that you already know what songs you are going to implement in advance, and the speed of singing is not too intense). Ok ... now you have your metadata.

I would recommend trying a bouncing ball first, because the implementation is already described, and I see a couple of problems with what you are trying to implement. First of all, (maybe I'm wrong), but I don’t think that UIlabel / NSString has any methods of selecting substrings. This means that you may have to make a shortcut for each individual word, which can become really tedious ... So check out this video and hopefully you can do something. Good luck

Answer 2

I would go with the metadata option, since I don’t know how easy it would be to, say, recognize when a singer changes words. (e.g. screamo, sting, etc.)

Note that you can actually assign a function in the main thread; you don’t need to check if the word has changed. You can use:

[self performSelector:@selector(changeWord) withObject:self afterDelay:metaData.nextChangeTime] 

I'm not quite sure about memory issues, but I use it to play recorded midi files in my application. I am just planning a metadata-based changeWord function.

Regarding the creation of metadata, I'm not sure which method you could use other than brute force. There must be some prerecorded metadata; after all, there are other karaoke apps.