MIDIMetaEvent and that damn tuple
Swift and AudioToolbox pains
Using C APIs in Swift is still a pain in the neck – even in Swift 4.
Here is one of the problems with using the C AudioToolbox API.
Introduction
The problem is using C APIs in Swift where an array is imported to Swift as a tuple.
For example using the AudioToolbox framework.
(I’ve highlighted the offending line in each of the following examples)
In Objective-C
1 2 3 4 5 6 7 8 9 |
typedef struct MIDIMetaEvent { UInt8 metaEventType; UInt8 unused1; UInt8 unused2; UInt8 unused3; UInt32 dataLength; UInt8 data[1]; } MIDIMetaEvent; |
This comes into Swift as:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
public struct MIDIMetaEvent { public var metaEventType: UInt8 public var unused1: UInt8 public var unused2: UInt8 public var unused3: UInt8 public var dataLength: UInt32 public var data: (UInt8) public init() public init(metaEventType: UInt8, unused1: UInt8, unused2: UInt8, unused3: UInt8, dataLength: UInt32, data: (UInt8)) } |
This problem exists also for MIDIRawData which is used for Sysex data – aways variable – and MusicEventUserData. You can still use MusicEventUserData in some cases; I use it as a loop marker.
1 2 3 4 5 6 7 8 9 10 11 12 |
public struct MIDIRawData { public var length: UInt32 public var data: (UInt8) public init() public init(length: UInt32, data: (UInt8)) } public struct MusicEventUserData { public var length: UInt32 public var data: (UInt8) public init() public init(length: UInt32, data: (UInt8)) } |
Getting back to MIDI Meta events, there are several types of MIDI Meta events. Text events are one category. The data field needs ASCII text that can be any length.
So, try to create a MIDIMetaEvent for the track name.
1 2 |
var metaEvent = MIDIMetaEvent() metaEvent.metaEventType = 3 // track or sequence name according to the MIDI file spec. |
Ok so far.
Now we need to place text into the data tuple.
1 2 3 |
let text = "Piano" let data = [UInt8](text.utf8) metaEvent.dataLength = UInt32(data.count) |
Now what? How do you allocate the tuple? I have an array with the data.
metaEvent.data = what?
I could do this:
1 2 3 4 5 6 7 8 9 |
withUnsafeMutablePointer(to: &metaEvent.data, { ptr in for i in 0 ..< data.count { ptr[i] = data[i] } }) // I"m guessing that this copies the data let status = MusicTrackNewMetaEvent(track, 0, &metaEvent) |
But that is copying into metaEvent.data which doesn’t have any memory allocated. So as is common with pointer problems, it works sometimes and sometimes it doesn’t.
So, how to allocate memory for the tuple?
I’ve spent far too much time trying different things out.
I filed a Radar. Helpful as always, Radar told me it’s a duplicate. So, I’m not alone, but thanks, Radar for allowing us to see outstanding reports. /s
Finally, I punted and used one of my Technical Support Incidents with Apple Developer Technical Support. After all, last year I asked them a Core MIDI question in Swift (including code examples and the phrase “How can you do this in Swift?) and they responded with an example in Objective-C. So, I was prepared for another quality response.
Here it is:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Hello Gene, Thank you for contacting Apple Developer Technical Support (DTS). Our engineers have reviewed your request and have determined that you are experiencing a known issue for which there is no known workaround at this time. If you have already filed a bug report for this issue, thank you. If you have not filed a bug report for this issue we encourage you to do so using Apple Bug Reporter: https://developer.apple.com/bug-reporting/ While a Technical Support Incident (TSI) was initially debited from your Apple Developer Program account for this request, we have assigned a replacement incident back to your account. Best Regards, Developer Technical Support Apple Worldwide Developer Relations |
tl;dr yeah we know. Shut up and go away.
btw, I didn’t receive a “replacement incident”. I’m disinclined to use them again anyway.
Summary
Many C structures in the AudioToolbox contain arrays that are imported into Swift as tuples. This makes then unusable.
Doctor, Doctor, it hurts when I do this!
Then don’t do this!
Try something like that:
NOTE: It MUST FIT to the layout of the original struct definition of MIDIMetaEvent !!!
Then actually use THAT NEW structure in your allocation blocks and callbacks…
i.e.:
You may then iterate the stuff of the tuple with something like that:
or even write to that struct memory to provide data…
PLEASE NOTE:
This is a HACK! But it actually works. And it is the only way that makes any sense.
Why? The reason obviously is (guessing), that SWIFT does not give us direct access to the data in memory, but copies that memory into the structs we provide for the function calls. So it is always on our side to provide the correct sizes!
Otherwise we actually would pump shit into the memory of the user devices!!!
Please note:
The code above is tested SWIFT 3 Code with Xcode Version 8.3.2 .
You may actually just copy and past that to several projects from Gene at GitHub. ^^
Hi Gene!
Did you actually notice the fundamental difference to your code examples?
It is just your code. I learned so much from you.
So this way we would be on the save, secure side with our code, but, I guess, that it is nevertheless a quite risky operation.
Why?
Because this way one possibly would be able to exploit users data in memory (because you actually are able to read memory areas with that, which may NOT be intended for our eyes ^^)
.. just took notice, that the editor did screw up my code.
So i uploaded a SWIFT file with the complete extension definitions to my server here:
http://digitster.com/MIDIRawDataExtension.swift
.. updated the file above to support easy (and save) writing of raw Midi data with 3 class examples ready to use. It simplifies the write and read access to few lines of code…
Just discovered, that even popular, already released SWIFT MIDI frameworks actually excessively write junk into the memory of the users devices with these wrong pointer operations on uninitialized data structs in memory ^^
By The way: I am coming from C/C++. These Apple programming languages are just awful! And with each release they will become more terrible. You need XXX times longer to get something useful out of it…
Swift as a language is decent. The churn is settling down now, but I’m disappointed that ABI compatibility has been put off after Swift 3.
Apple’s API “bindings” to Swift have glaring problems. This damn tuple problem is one of them.
AUGraph is going to be deprecated in favor of the AVFoundation classes.
It would be nice if they created a usable MIDI replacement in AVFoundation instead of making us use the Core MIDI C API.
I don’t see that on the horizon though. (I have indeed filed Radars. Everyone should – squeaky wheel…)
I do not have any problem with a deprecation of AUGraph, Because it actually is quite superfluos with the available functions of the AV stuff now.. (it’s kind of wrapped now)
Although, CoreMIDI on the other side is essential to music apps. They surely won’t deprecate that for some certain reasons. Maybe they will dorrect that tuple stuff or at least show us the correct way of using that… ^^
We can just ignore a stupid AVAudioSequencer class, if we have access to CoreMIDI.
Hi Gene,
Did you ever work out a solid solution to this?
Also, any thoughts on how to use this for setting Key signature? I saw it used for a signature with sharp notes in your Swift3MIDI project, but for signatures containing flats, that requires a negative number (https://www.csie.ntu.edu.tw/~r92092/ref/midi/#meta_event), which of course causes an error when placed into a UInt8 variable.
Thanks in advance.
Neil
No, I gave up. I use Objective-C++ for meta events. I’ve checked each new Apple release and nothing is being updated. They pretty much responded to my Radars with “too bad”. But if you can, go ahead and file one. Maybe if someone else mentioned it, they’d maybe take notice.
ps National Chiao Tung University? Back in the 80s, Phil Winsor and I did a lot of planning for their music studios.
Thanks for the update. I gathered that you’d likely have a fresher post on this, but thought I’d ask.
Personally, I ended up using the suggestion made in this Stack Overflow question (which I know you also contributed to), but it’s far from ideal having to use such an elaborate workaround. Also doesn’t seem to work properly for setting key signatures, for some reason (it messes up how the audio sounds – very strange).
NCTU just happened to be in the top results when looking at MIDI meta specs, but clearly they learned from the best!