tvOS: playing audio
Playing MIDI on tvOS
According to Apple’s App Programming Guide for tvOS, AVFoundation is supported in tvOS.
Groovy!
Let’s use AVMIDIPlayer to play a file!
Set up
AVMIDIPlayer’s init func wants two URLs: one to a MIDI file and another to a SoundFont or DLS file.
I created a standard MIDI file in Sibelius. The SoundFont I used is the one distributed with MusesCore.
Here is how I created the player.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
func createAVMIDIPlayer() { guard let midiFileURL = NSBundle.mainBundle().URLForResource("sibeliusGMajor", withExtension: "mid") else { fatalError("\"sibeliusGMajor.mid\" file not found.") } guard let bankURL = NSBundle.mainBundle().URLForResource("GeneralUser GS MuseScore v1.442", withExtension: "sf2") else { fatalError("\"GeneralUser GS MuseScore v1.442.sf2\" file not found.") } do { try self.midiPlayer = AVMIDIPlayer(contentsOfURL: midiFileURL, soundBankURL: bankURL) print("created midi player with sound bank url \(bankURL)") } catch let error as NSError { print("Error \(error.localizedDescription)") } self.midiPlayer.prepareToPlay() } |
And then I played it by calling the cunningly named play() function. I use a completion handler to reset the playback position.
1 2 3 4 5 6 |
dispatch_async(dispatch_queue_create("com.rockhoppertech.Play",nil), { self.midiPlayer.play { () -> Void in print("finished playing") self.midiPlayer.currentPosition = 0 } }) |
This is pretty much what I do in iOS and Cocoa and it works.
The result
In the tvOS simulator, it works almost OK. You hear sine waves – just like in iOS. When you run it on an actual device, you should hear the instruments.
In tvOS, not so much. I hear two notes with a loaded instrument, and then blammo.
Crash.
Removing the dispatch_async call yields the same result.
Here is the love letter from the debugger.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
created midi player with sound bank url file:///var/mobile/Containers/Bundle/Application/60C6893D-4472-4D61-9718-605EF41C2545/EarTrainer.app/GeneralUser%20GS%20MuseScore%20v1.442.sf2 (lldb) bt * thread #13: tid = 0x5ddc8, 0x000000018302a62c AudioToolbox`bool DoProcessMono<2, 2, false>(ProcessParams const&, unsigned int*, float*, float*, double*, float*, float*, double, float (*) [4]) + 84, name = 'AURemoteIO::IOThread', stop reason = EXC_BAD_ACCESS (code=2, address=0x103103ffc) * frame #0: 0x000000018302a62c AudioToolbox`bool DoProcessMono<2, 2, false>(ProcessParams const&, unsigned int*, float*, float*, double*, float*, float*, double, float (*) [4]) + 84 frame #1: 0x0000000183027d14 AudioToolbox`VoiceZone::Process(unsigned long long, AudioBufferList**, unsigned int, unsigned int) + 1876 frame #2: 0x00000001830141f4 AudioToolbox`non-virtual thunk to SamplerNote::Render(unsigned long long, unsigned int, AudioBufferList**, unsigned int) + 168 frame #3: 0x0000000183031b40 AudioToolbox`SynthGroupElement::Render(long long, unsigned int, AUScope&) + 508 frame #4: 0x000000018302f92c AudioToolbox`AUInstrumentBase::Render(unsigned int&, AudioTimeStamp const&, unsigned int) + 432 frame #5: 0x0000000182f4fe0c AudioToolbox`SamplerBase::Render(unsigned int&, AudioTimeStamp const&, unsigned int) + 68 frame #6: 0x0000000182f99edc AudioToolbox`AUBase::DoRenderBus(unsigned int&, AudioTimeStamp const&, unsigned int, AUOutputElement*, unsigned int, AudioBufferList&) + 216 frame #7: 0x0000000182f986e4 AudioToolbox`AUBase::DoRender(unsigned int&, AudioTimeStamp const&, unsigned int, unsigned int, AudioBufferList&) + 604 frame #8: 0x0000000182fc20cc AudioToolbox`AUMethodRender(void*, unsigned int*, AudioTimeStamp const*, unsigned int, unsigned int, AudioBufferList*) + 44 frame #9: 0x0000000182f9d35c AudioToolbox`AUInputElement::PullInput(unsigned int&, AudioTimeStamp const&, unsigned int, unsigned int) + 148 frame #10: 0x0000000183071738 AudioToolbox`AUDynamicsProcessor::Render(unsigned int&, AudioTimeStamp const&, unsigned int) + 592 frame #11: 0x0000000182f99edc AudioToolbox`AUBase::DoRenderBus(unsigned int&, AudioTimeStamp const&, unsigned int, AUOutputElement*, unsigned int, AudioBufferList&) + 216 frame #12: 0x0000000182f986e4 AudioToolbox`AUBase::DoRender(unsigned int&, AudioTimeStamp const&, unsigned int, unsigned int, AudioBufferList&) + 604 frame #13: 0x0000000182fc20cc AudioToolbox`AUMethodRender(void*, unsigned int*, AudioTimeStamp const*, unsigned int, unsigned int, AudioBufferList*) + 44 frame #14: 0x0000000182f9d35c AudioToolbox`AUInputElement::PullInput(unsigned int&, AudioTimeStamp const&, unsigned int, unsigned int) + 148 frame #15: 0x0000000182f901c4 AudioToolbox`AUInputFormatConverter2::InputProc(OpaqueAudioConverter*, unsigned int*, AudioBufferList*, AudioStreamPacketDescription**, void*) + 216 frame #16: 0x0000000182ee05b8 AudioToolbox`AudioConverterChain::CallInputProc(unsigned int) + 332 frame #17: 0x0000000182ee02a0 AudioToolbox`AudioConverterChain::FillBufferFromInputProc(unsigned int*, CABufferList*) + 148 frame #18: 0x0000000182ebc04c AudioToolbox`BufferedAudioConverter::GetInputBytes(unsigned int, unsigned int&, CABufferList const*&) + 180 frame #19: 0x0000000182eeadf8 AudioToolbox`Resampler2Wrapper::RenderOutput(CABufferList*, unsigned int, unsigned int&) + 196 frame #20: 0x0000000182ebbed4 AudioToolbox`BufferedAudioConverter::FillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 444 frame #21: 0x0000000182ee0020 AudioToolbox`AudioConverterChain::RenderOutput(CABufferList*, unsigned int, unsigned int&, AudioStreamPacketDescription*) + 116 frame #22: 0x0000000182ebbed4 AudioToolbox`BufferedAudioConverter::FillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 444 frame #23: 0x0000000182e94918 AudioToolbox`AudioConverterFillComplexBuffer + 340 frame #24: 0x0000000182f8fe84 AudioToolbox`AUInputFormatConverter2::PullAndConvertInput(AudioTimeStamp const&, unsigned int&, AudioBufferList&, AudioStreamPacketDescription*, bool&) + 120 frame #25: 0x0000000182f8f918 AudioToolbox`AUConverterBase::RenderBus(unsigned int&, AudioTimeStamp const&, unsigned int, unsigned int) + 268 frame #26: 0x0000000182f99edc AudioToolbox`AUBase::DoRenderBus(unsigned int&, AudioTimeStamp const&, unsigned int, AUOutputElement*, unsigned int, AudioBufferList&) + 216 frame #27: 0x0000000182f986e4 AudioToolbox`AUBase::DoRender(unsigned int&, AudioTimeStamp const&, unsigned int, unsigned int, AudioBufferList&) + 604 frame #28: 0x0000000182f7e240 AudioToolbox`AURemoteIO::PerformIO(unsigned int, unsigned int, unsigned int, AudioTimeStamp const&, AudioTimeStamp const&, AudioBufferList const*, AudioBufferList*, int&) + 488 frame #29: 0x0000000182f7f1d0 AudioToolbox`AURIOCallbackReceiver_PerformIO + 560 frame #30: 0x0000000182f75574 AudioToolbox`_XPerformIO + 104 frame #31: 0x0000000182ea1054 AudioToolbox`mshMIGPerform + 248 frame #32: 0x0000000182f4820c AudioToolbox`MSHMIGDispatchMessage + 36 frame #33: 0x0000000182f7e584 AudioToolbox`AURemoteIO::IOThread::Run() + 136 frame #34: 0x0000000182f8209c AudioToolbox`AURemoteIO::IOThread::Entry(void*) + 12 frame #35: 0x0000000182e8bf00 AudioToolbox`CAPThread::Entry(CAPThread*) + 124 frame #36: 0x000000019534fb28 libsystem_pthread.dylib`_pthread_body + 156 frame #37: 0x000000019534fa8c libsystem_pthread.dylib`_pthread_start + 156 |
By the way, I also tried MusicPlayer and Core Audio since the guide says AudioToolbox and AudioUnit are supported too. Same result. The stack trace shows that error occurs deep in the bowels of AudioToolbox.
Summary
Well, crap. It would be nice to be able to play MIDI files; I’d like to port my ear training app to tvOS. But without playback, it would be reaaalllly hard to recognize and identify the intervals, chords, and scales playing.
Update
I spent far too much time on this. I looked at my code over and over again. I was certain that I was doing it correctly (but there is always that doubt every developer deals with), so why was it crashing?
It wasn’t my code.
It was the MusesScore sound font. Out of desperation, I tried another sound font and the sun started shining , the birds sang, and my cat’s litter box was magically clean. Oh, and it played on the AppleTV.
Resources
App Programming Guide for tvOS
If you need a cable to connect your MacBook to your AppleTV, I bought this one.
Well it never gets easy in the ios development game 🙂
Thanks for another great post by the way.
Ed