A few months ago, I was contacted and asked to help a developer who was having trouble porting their music app to the BlackBerry 10 platform. The developer wanted a sound to be played whenever the screen was touched, but found that the sound did not start playing until 150 ms after touching the screen and they felt this was too long. They wanted to know how to get the delay below 100 ms before proceeding further.
I set out to build an app that would help to recreate the issue. After a little work, I had a sample app that achieved a latency ranging from 60 ms to 120 ms. The higher range came into play only when I tried to tap the screen continuously and only from time to time. Obtaining some code from a QNX audio team sample late in the process helped immensely. To understand just how fast this is, 60 ms is the theoretical minimum latency possible according to the QNX audio team with the current platform (based on the minimum buffer fragments and number of buffers at 48K sampling rate). They are working on reducing the latency in future OS releases.
The initial sample was posted on GitHub earlier this year, but it didn’t support the BlackBerry Q10 so I thought I should add that support before publishing a blog post. Since then, I have added multi-touch support for up to five fingers to allow users to play chords and I fixed a bug in the blending algorithm that only revealed itself when multi-touch support was added.
If you have a good ear, and figure out where to press the screen to get certain notes, you can have a lot of fun with this app, especially if you experiment with the square wave generation as you will be reminded of some of the classic video games when you try out various tones. Here’s a video I recorded of one of my sessions to hear what the app sounds like before you build it yourself:
You’re probably wondering how you can use this sample to add low latency audio to your app?
- If you are writing a native app, you can just add the audiopcm.c and .h files to your project and integrate the calls required from the BPS event loop in main.c into your BPS event loop. Make sure all external references are in place and away you go.
- If you are writing a BlackBerry 10 Cascades app, you can do something similar but you want to trigger the tones based on onTouch events with a touchID of 0. Because Cascades does not have multi-touch support yet, chords won’t work right unless you utilize a trick in my next sample app on a workaround that enables multi-touch support in a Cascades app. Stay tuned.
Functions you need to know about:
extern int initToneGenerator();
- This is the function you call to start the play thread. This should be called during app startup.
extern void cleanupToneGenerator();
- This is the function you call to stop the play thread and cleanup resources. This should be called during app shutdown.
extern Tone *addTone(double frequency, double intensity);
- This is the function you call to add a tone to the queue for playback. This is called in the sample app when a touch press / move is detected and no tone is currently playing for the tone calculated from the position.
- The parameters you specify are frequency and intensity. The sample app calculates the frequency based on a grid defined in the screen space. Intensity is a value from 0 to 1 and in the sample app is calculated from a range of values generated by touch pressure. A tone pointer is returned by this call that you can use for calling updateTone() later.
extern void updateTone(Tone *tone, double intensity);
- This is the function you call to update a tone to the queue for playback. This is called in the sample app when a touch press / move is detected and the calculated tone is the same as the last added tone for a specific touch point.
- The parameters are an existing Tone pointer and an updated intensity. This allows you to vary the intensity based on how the touch pressure changes while you play a note.
extern void endTone(Tone *tone);
- This is the function you call to stop the current tone. This is called by the sample app when a touch up event is detected.
What’s next with this code? Well, the play thread I have in this sample app could be extended to support playing sampled sounds as well as generated tones. A Qt class with equivalent or extended functionality for easier integration into Cascades apps is also planned.
I recently met one developer who integrated the audio code into a Cascades app for guitar tuning. If you use this code in your app or just having some fun trying to play some tunes with the app, please add a comment to this post and let us know about it.
To access the sample app source code on GitHub in our Core-Native-Community-Samples repo, go to: https://github.com/blackberry/Core-Native-Community-Samples/tree/master/ToneGenerator .
Keep on Toning … 🙂