DEVELOPERS BLOG

Diary of a Unity 3D Newbie – Bluetooth Low Energy Plugins

Diary of a 3D Newbie_2_1

If you have been following my earlier blog posts, you know that I tend to explore areas like NFC and Bluetooth Low Energy from a BlackBerry 10 developer’s perspective, usually in conjunction with my colleague Martin Woolley.

So you may be wondering what this Unity 3D thing is all about, and why I’m rooting around gaming.  Well, Unity 3D is a phenomenal framework that allows you to develop amazing games on multiple platforms, including BlackBerry 10. And what’s more, it contains a very flexible plugin framework to allow you to extend your game’s capabilities down into native capabilities of BlackBerry 10!

Because of this framework, I saw an opportunity to add the capability to a Unity 3D game to interact with a Bluetooth Smart device. Imagine influencing gameplay through data streamed from a heart rate monitor worn by the player; perhaps zombies could become more frenetic as your heart rate increases! This capability goes by the name of “haptics.” Hold that thought for a while!

I’ve already touched on this in a previous post but to I’m going to explain in a lot more detail how I implemented a Bluetooth Plugin for Unity to allow game objects to access live heart rate monitor (HRM) data worn by a player.

If you’ve never encountered Bluetooth Low Energy technology before, you’ve got a little bit of homework to do. Take a look at these documents:

Now that you’ve read about how a Bluetooth LE Heart Rate Monitor can be integrated into a standard native BlackBerry 10 application, it should be clear how we can do this in Unity 3D. It’s just a case of embedding the Bluetooth functionality in a native shared library and plumbing it into the Unity 3D framework. I’ll skip over the Bluetooth specific details in this article since these are already well documented in the links above; I’ll focus instead on the Unity 3D bits.

The Plugin Shared Library

Let’s start with a little background. Unity 3D plugins are basically shared libraries, such as libBtHrmPlugin.so (this is the name of my Bluetooth Low Energy Heart Rate Monitor Plugin) that expose a set of “C” functions something like this:

Diary of a 3D Newbie_2_2

The first thing to notice is that the plugin shared library should expose functions using “C” linkages. If you expose these using “C++” linkage conventions then external names get “mangled” to allow accommodation of “C++” class qualifiers and other annotations.  I’ve exposed five functions:

  • initialiseBtLeImpl(): This function would be called by the Unity application to initialize the Bluetooth Low Energy environment. Essentially doing things like ensuring Bluetooth is on, establishing callbacks for Bluetooth events and other housekeeping.
  • scanForHrmDevices(): Before we can use a particular HRM device, we perform a scan for candidate Bluetooth LE devices. There may be many devices that offer the HRM service so we need to enumerate them and pass the list of the ones that have been found back to the Unity application – more on how we actually do this later. Scanning for devices could be a time consuming activity so this task is handled by a separate thread that is started as part of this functions execution. When we return from this function back to the calling Unity application, the thread will still be running in the background searching for devices.
  • startMonitoringImpl(const char *address): Once the Unity application has determined the number, names and addresses of the available Bluetooth HRM devices, it needs to select one to connect to and monitor. That’s what this function does. After successfully returning to the Unity application, Heart Rate Monitor events will be delivered to it. We’ll talk about the mechanics of how this happens later.
  • stopMonitoringImpl(const char *address): You’ve guessed it. The Unity application will call this function to stop receiving Heart Rate Monitor events.
  • terminateBtLeImpl(): This is the reverse of the initialize step where we tidy things up after the Unity application using the services of this library has finished.

These functions allow the Unity application to instruct the plugin to perform specific tasks, and they all pass back an integer return code to indicate whether they were called successfully or not. Functions such as initialiseBtLeImpl and terminateBtLeImpl are quite simple and a return code is all they need to return. However, you’re probably wondering how the plugin communicates information other than simple return codes back to the Unity application. If you take another look at the code snippet above, you’ll see a declaration of a function called UnitySendMessage(); this should give you a clue.

UnitySendMessage() is a function that is exposed by the Unity 3D framework itself. This is the way that data is passed back asynchronously to Unity from our shared library plugin. Let’s examine the parameters that this function takes:

  • const char *obj: This a pointer to a NULL terminated string that identifies a “Game Object” within the Unity application.
  • const char *method: This is a pointer to a NULL terminated string that identifies a method on the “Game Object” that we want to call.
  • const char *msg: This is a pointer to a NULL terminated string that represents the data that will be passed to the above method on the aforesaid “Game Object.”

You can see that we can only pass a single string back into the Unity application. This means that if we want to pass back a complex object, such as a list of discovered Heart Rate Monitor devices with their names and addresses, we’d better find some way of encoding that information in a single string. Sounds like a job for JSON, which is exactly the approach taken in the plugin. Let’s solidify that with a concrete example.

Diary of a 3D Newbie_2_3

The code fragment above comes from the plugin itself. You can see that it covers two cases:

  • The plugin has successfully scanned and detected a number of HRM devices. It’s encoded these into a JSON string ( json_for_unity ) and passes that as the single parameter to a function called RequestScanForHrmDevicesSucceeded on the Game Object called “BtHrmPlugin” .
  • In the case that there has been an error in the plugin while scanning for HRM devices, an error message is constructed as a JSON object with an error ID and an error text string. This is passed as the single string parameter to a function called RequestScanForHrmDevicesFailed on the same Game Object, BtHrmPlugin”.

Some of you may have noticed that UnitySendMessage returns pointers to the Unity 3D framework. Isn’t that unsafe? In the above fragment it returns a pointer to a char array containing the JSON string and this becomes invalid after exiting the block after returning from UnitySendMessage. This is a good question and what seems to happen is that Unity copies the data rather than pass around pointers, so it’s safe to do it this way.

In essence, that’s all there is to the plugin. The key to constructing a Unity application and plugin as a functioning pair is to identify these functions, which represent events that can be emitted from the plugin asynchronously. Let’s just have a look at the ones that I’ve used:

  • RequestScanForHrmDevicesSucceeded: This indicates that the plugin has successfully completed a scan for Bluetooth HRM devices and these are returned encoded in a JSON string.
  • RequestScanForHrmDevicesFailed: This indicates that the plugin has failed in some way during its scan for Bluetooth HRM devices. An error ID and a description of the error are returned encoded in a JSON string.
  • RequestServiceDisconnected: This indicates that the plugin has been notified that the HRM device has been disconnected from the plugin. This normally occurs as the result of a request from the Unity Application but the reason is returned as a JSON encoded string.
  • BtHrmNotification: This indicates that the HRM device has notified the plugin of a heart rate measurement. The measurement data can contain many data points and is described in detail in the Bluetooth Special Interest Group (SIG) Heart Rate Profile specification. In essence, it contains the current measured heart rate in beats per minute, with optional details on energy expended, skin contact indication and other data points dependent on the device’s implementation.
  • BtHrmMessage: This indicates that the plugin is sending a message to the Unity application. It’s simply a text string, but the application can chose to display it in a suitable way. I typically use it to show progress of the plugin’s processing and also simple debugging information.

Let’s now just a have a quick look at the structure of the project in Momentics.

Diary of a 3D Newbie_2_4

This project was created using the standard BlackBerry shared library template.

I use “.c” and “.h” file types so that the compiler recognizes these as “C” files and not “C++” ones. Do you recall the discussion above on “C” versus “C++”?

The shared library itself is created in the standard locations and called libBtHrmPlugin.so.

I’ve also added a Scripts folder. The reason for this is that the shared library needs to be copied onto the Unity 3D framework project where I build my Unity application, and having to do this manually became tedious, so I added some scripts to make the process a little easier.

The Unity 3D Application

Now that we understand the way the plugin is architected, and the API that has been designed to interact with it in terms of functions it exposes and events that it communicates back to the Unity 3D framework, let’s look at other side of the API: how to interact with this plugin from a Unity 3D game.

Unity 3D applications typically use C# (using the cross-platform Mono Framework) or JavaScript to express game logic. I used C# in this example rather than JavaScript for no other reason than I quite like C#. C# has a very nice ability to represent events. It allows applications to subscribe to these events and be notified when they occur, and it sits nicely with the model of a Unity 3D application which is driven by external events such as button clicks, timers, network events,or even events that percolate up from our Bluetooth LE HRM plugin!

This means that we need to implement a “layer” beneath the Unity application which can map both calls made by the application onto the plugin and asynchronous events from the plugin onto C# events that can be subscribed to by the application. We do that in a C# class called BtHrmPlugin. Here are the relevant elements of the definition of this class focusing on the one method used to enable the application to initialise the plugin’s Bluetooth environment:

Diary of a 3D Newbie_2_5

This class needs to link to the external functions in the plugin that we described in the previous section. C# uses the statement DllImport to achieve this. It has this peculiar name since C# originated on Windows and a DLL is a Dynamic Link Library, which is the Windows analogue to a shared library on BlackBerry 10. Notice that the name of the library to import is “BtHrmPlugin” rather than “libBtHrmPlugin.so”. Mono will try a variety of permutations of names as it tries to locate the underlying library and this gives an element of platform independence to naming of the plugin libraries.

In this case the function that is exposed to the Unity 3D game application is called InitialiseBtLe() and this results in a call to the corresponding function in the plugin library called initialiseBtLeImpl(). Notice the #if … #else … #endif construct. This ensures that the call to the plugin is made only if the application is running on a BlackBerry 10 device and we’re not running the application inside the Unity editor. If we are running in the Unity editor we just return a good return code.

Now, how do we deal with C# events? When the plugin makes an asynchronous call to UnitySendMessage this needs to be mapped onto a C# event that the application can subscribe to. Here’s how it’s done; I’ve highlighted the key points to help you see what’s going on.

Diary of a 3D Newbie_2_6

A successful scan for Bluetooth LE devices will result in the plugin calling the C# method in our class called RequestScanForHrmDevicesSucceeded. First we check to see if there are any subscriptions in place for this type of event, and if so, we parse the JSON string provided by the plugin and create a new instance of the class ScanForHrmDevicesEventArgs which is used as a container for the list of devices discovered and their attributes. Then we raise the ScanForHrmDevicesSuccessfulEvent event we’ve defined and it’s sent to the subscribers.

Here’s how part of the Unity game logic would register for this event in C#:

Diary of a 3D Newbie_2_7

The successful scan resulting in a list of Bluetooth LE heart rate monitor devices end up being processed asynchronously in this function (ScanSuccessful) in the game logic. All the other functions and events supported by this plugin work in exactly the same way, the pattern is identical.

It’s worth looking at how these elements are arranged in a simple Unity 3D project. In order to test it, I wrote a simple Unity 3D sample application. I’m only a newbie when it comes to being able to design sophisticated and elegant user interfaces, so the application is a very functional one using OnGUI() to demonstrate how the plugin is functions. Here’s the layout of the project:

Diary of a 3D Newbie_2_8

This is where you must place the shared library plugin libBtHrmPlugin.so. It won’t work if it is placed anywhere else. Now you can see why I had some useful scripts in my Momentics plugin project to be able to copy a new version of the plugin to this location whenever I’d re-built it.

Diary of a 3D Newbie_2_9

I placed the C# BtHrmPlugin.cs file containing the C# class BtHrmPlugin in the folder above outlined in RED. There are no constraints on where you can place this. You’ll notice that there is also, outlined in GREEN, a Game Object also called “BtHrmPlugin”. This is important. In order to attach the C# plugin script to the game play scene you need to attach the script to a Game Object. Here we’re closing the loop on a point I made earlier. Do you recall that I noted that the first parameter on the UnitySendMessage() function identified a Game Object that the message should be sent to? Well, here’s confirmation of that! The plugin send its messages to a Game Object called “BtHrmPlugin”.

Diary of a 3D Newbie_2_10

So, we’ve come full circle regarding the plugin and how it works. My simple test application is in the file BtHrmGuiHandler.cs as shown below.

Diary of a 3D Newbie_2_11

Below is a snapshot of the application running using a real heart rate monitor! The large number is my instantaneous heart rate while testing this application. You can see that I happened to be using a “Zephyr” Heart Monitor, which I had already paired to my Z30, although any others that support the Bluetooth SIG Heart Rate Profile will also work.

The buttons at the top of the screen control the ability to start and stop monitoring through the plugin, and the area at the bottom of the screen represents a log showing the operation of the plugin and the data that it’s receiving from the heart rate monitor.

Diary of a 3D Newbie_2_12

Summary

I hope that you’ve found this description of my journey through this simple Unity 3D plugin useful. The plugin itself and the associated simple Unity application are available on GitHub.

It would be nice to see a real game exploiting Bluetooth Low Energy accessories. Watch this space!

Additional Resources

About jcmurray2012