DEVELOPERS BLOG

Beta Test your Apps with the BlackBerry Beta Zone – A Case Study by S4BB Limited

The following guest post was written by Patrick Kosiol, BlackBerry Developer Elite and CEO of S4BB Limited


Testing apps properly before publishing a release on BlackBerry World is crucial for the success of a mobile app. We at S4BB Limited have been developing BlackBerry apps for over 9 years now, and from the onset we have been running beta test programs of our apps.

Running such beta tests has always been a messy business: First you had to set up a “Beta Test” mailing list, then ask people to subscribe to that mailing list so you can communicate with them about your beta test. That means you had to “acquire” beta testers from somewhere (e.g., through your website, existing user base, related forums like CrackBerry, etc.) and convince them to give you their personal information like their email address so you can send them a download link. That was very time-consuming and absolutely inefficient.

Furthermore, if you wanted to run a beta test for a BlackBerry 10 app, you would have had to send your users a release-build BAR file and they would have to sideload it onto their phone. There are so many steps in between where things can go wrong, your users get frustrated and they didn’t even start the beta app yet. Luckily, there is a solution for this now: The official BlackBerry Beta Zone.

Recently, BlackBerry opened up their BlackBerry Beta Zone to a private beta test community, and we at S4BB Limited have been lucky enough to participate in it. Today, I would like to talk about our experience with the BlackBerry Beta Zone, which I hope will help all of you get to know this incredible resource and encourage you to utilize it to build better apps that put a smile on your users’ faces.

What is the BlackBerry Beta Zone?

The BlackBerry Beta Zone is both a mobile app and a platform to manage your beta tests through a web panel.

Two Beta Tests to Rule the Zone

While we have many paid apps at S4BB Limited, in terms of user numbers, our free apps are obviously the most popular ones worldwide. We just ran beta tests for two such apps in the BlackBerry Beta Zone. The first app is Easy Smiley Pack, a fun application that gives you access to 1,020+ smileys, stickers, emoji, flags, ASCII art and more. The second app is Radio Shuffle, an app for listening to a random radio station from the top 500 radio stations worldwide. Its unique shuffle feature and beautiful user interface offer a new way of experiencing radio on your BlackBerry 10 phone.

Beta Tests for Easy Smiley Pack and Radio Shuffle showing in the BlackBerry Beta Zone app on a BlackBerry Z10.

Beta Tests for Easy Smiley Pack and Radio Shuffle showing in the BlackBerry Beta Zone app on a BlackBerry Z10.

There are many reasons you might want to run a beta test. Some technical reasons could be testing new features or styles or finding out if any bugs occur on certain devices in specific countries with certain language settings. You might also want to consider economical reasons for running a beta test: Creating an outstanding app that is loved by thousands of users requires hard work and many hours, weeks and months of dedicated development and design. You probably have a good idea what you want to do with your app, but you can never know exactly what the user wants.

We at S4BB Limited have always believed it’s the best approach to get an early beta (or even alpha) version into the hands of users as early as possible. Of course, it’s good to test an app with your mom (moms are awesome testers!), but only real-world users will be able to tell you whether they would be using your app and/or would even be willing to spend money for it. Furthermore, building up a user-base for your app even before the official launch helps a lot in landing a successful app launch!

This is what it looks like when two beta tests are running in the BlackBerry Beta Zone web panel:

s4bb2

Advantages of BlackBerry Beta Zone for Developers

There are many advantages to using an integrated beta testing solution like the BlackBerry Beta Zone. For us at S4BB Limited, the following points are the most important:

Direct access to thousands of experienced beta testers

Thousands of users have already downloaded the BlackBerry Beta Zone app. Hence, they are very likely to have experience with beta testing a “not quite finished” app and are willing to try new things even if they aren’t perfect.

Receive bug reports through integrated discussion boards

Receiving bug reports for an app beta is a perfect way to find out which parts of an app don’t perform as expected. Keep in mind that there are hundreds of network operators in hundreds of countries that sell BlackBerry devices. Many of them have their own languages, and unless you are a large international conglomerate, you won’t be able to test your app with all such combinations successfully.

Get direct feedback through developer-initiated surveys

Having a unique app idea, developing great features and creating an outstanding design is not all it takes to make an app successful. You probably have a good idea of what you will add in future versions of your app, but before you invest a lot of time (and therefore money) into that, why not run a survey? Asking your users is free, and most of them are very willing to tell you what they want and what they do not want.

s4bb3

Spur users’ creativity through discussion boards and replying directly in a timely manner

Again, listening to users should be your #1 priority. While surveys are a great way to learn what people think about your ideas for your app, the integrated discussion boards of the BlackBerry Beta Zone offer a perfect playground for your users’ thoughts – unrestricted and open to all testers. Users who care about your app, whether they hate it or love it, are often willing share their thoughts and ideas about what your app should be. Listen to those folks, reply to their feedback and enter into a conversation with your users to make them happy. They will reward you.

In my view, the BlackBerry Beta Zone is the most comprehensive platform to beta test your app before the official launch. It provides a platform to manage multiple beta tests, connect with your beta testers directly, gather aggregated information about your app and the way it’s being used, and therefore get a quality app out of it.

The two apps used as an example in this post have now graduated from their beta tests and are available for the general public now. You can get them on BlackBerry World™:

About donnyhalliwell