ofxQNX - BlackBerry PlayBook and BB10 add-on

Hi!

I’ve recently updated ofxQNX for compatibility with openFrameworks 0071 (develop branch). The old version for 007 is still available on github ( https://github.com/falcon4ever/openFrameworks/tree/developPlayBook ) but won’t be updated in future. The code is based of the developPlayBook branch (see details in this topic http://forum.openframeworks.cc/t/of-blackberry-playbook/9189/0 ) that has been used for the NodeBeat project.

To make the project easier to manage, I have separated the openFramework patches and the add-on.

[hr]

About
ofxQNX is an add-on that brings the openFrameworks to the BlackBerry PlayBook and BlackBerry 10 platforms.

License
New BSD License (3-clause license)
https://github.com/falcon4ever/ofxQNX/blob/develop/LICENSE.txt

Project page
http://www.multigesture.net/projects/ofxqnx/

Source code

Full Changelog/FAQ
https://github.com/falcon4ever/ofxQNX/blob/develop/README.md

Notes
The current code has been tested using the following SDKs

To develop for QNX or run the examples, you will need to download the ofxQNX add-on and place it in the addon folder of openFrameworks.

Until the openFrameworks team merges my patches (pull req 1587 https://github.com/openframeworks/openFrameworks/pull/1587) into the official develop branch you will need to use my patched 0071 branch (develop-0071-qnx):

  • git clone -b develop-0071-qnx git@github.com:falcon4ever/openFrameworks.git

How to setup ofxQNX
Download (a patched) openFrameworks from the link above. Next download this add-on and place it in the addon directory.

Launch the IDE provided by the BlackBerry Native SDK (PlayBook or BB10).

Create a new workspace at:

  • “openFrameworks\examples\qnx”

After the IDE is running, import the following projects:

  • “openFrameworks\libs”

  • “openFrameworks\addons”

  • “openFrameworks\addons\ofxQNX”

  • The example projects (BB10 or PlayBook) under “openFrameworks\addons\ofxQNX\examples”

Build and Run

  • Click any of the example projects on the left

  • Right click, and click on Build Project

  • After it is done, right click on the binary (e.g. Device-Debug\qnxEmptyExample) and choose Run As > BlackBerry [platform name] Application

Demos on a BlackBerry PlayBook
http://www.youtube.com/watch?v=JLE3v-QRa74

Great work.
Will try ASAP!!
Have you noticed any improvement on performance on OS 2.1.0 beta over OS 2.0?

Have you advanced on the audio input issue?
Regards and thanks

I haven’t really checked for performance differences, OS 2.1.0 just has a nicer way of handling Android apps.

I’m currently not working on the audio input so I suggest that you just use the BlackBerry Platform Services for the time being.

I picked up my bb10 dev alpha device yesterday and installed ofxQNX, seems to work alright :slight_smile:
http://www.youtube.com/watch?v=OkxP--ceheo

Could you give me a hint on how to implement audio input? Or a least how you’d do it.

I have read this:
https://developer.blackberry.com/native/documentation/porting-write-features.html

and maybe the best library to use is the QNX Sound Architecture (QSA) library, which is explained here:
https://developer.blackberry.com/native/reference/com.qnx.doc.neutrino.audio/topic/about.html

I am wondering if I might overwrite an audioin function in OF which makes all the calls to this library or running the QSA calls it in the setup and then capture the data every update call.

BB10 looks awesome on that device!! Cool.

What I tried to do was to insert an audioIn call in the ofQNXSoundStream::AudioCallback like this:

  
void ofxQNXSoundStream::AudioCallback(void *userdata, uint8_t *stream, int len){  
  
	  
	soundInputPtr->audioIn( in_float_buffer, bufferSize, nInputChannels);  
  
	soundOutputPtr->audioOut(out_float_buffer, bufferSize, nOutputChannels);  
  

And then insert a void testApp::audioIn(float * input, int bufferSize, int nChannels)

Of course, in the setup, I would call:
soundStream.setup(this, 2, 2, 44100, BUFFER_SIZE, 4);

But where do I have to interface the QSA library (or other sound library) in ofxQNX?
Is it in ofxQNXSoundStream::openQNXAudio? Or should I create another function?

Thank you.

You won’t need to use ofxQNXSoundStream (soundStream) since the method audioIn hasn’t been implemented and is just a placeholder. You will need to setup the capture device using BPS and then use the QSA/Alsa lib to do the recording: https://developer.blackberry.com/native/reference/com.qnx.doc.neutrino.audio/topic/pcm-capture.html

I currently don’t have the time to look into the mic recording, so if you’re stuck after folloing the guide above I suggest posting the question on the official native dev forums: http://supportforums.blackberry.com/t5/Native-Development/bd-p/native-sdk

Wouldn’t be better to implement the audioin function to extend the ofxQNX functionality?

In case it wouldn’t, where would you write the code? In testapp directly?
Thanks again

Of course it would be better if someone would implemented it in the audioin function but if you need an immediate solution, I’d just put it in testapp directly (Perhaps do the init calls in setup() and the rest of the mic code in the update() or other methods).

I will try it as soon as I get some things fixed with the audio input. This is my first attempt of building a spectrogram app for the playbook with OF.

The App runs smooth!!!

http://www.youtube.com/watch?v=TzyZ8ULzito

Sorry for the video quality, my BB 9300 camera sucks… So I’m posting some screen captures directly from the tablet:

Think I am going to enjoy it!!

Look nice ofPAk! So does that mean you got the mic working?

Thanks falcon4ever.

Yep, the MIC is working via PCM plugin, so I can get all the inputs at the same time if needed (external MIC, internal MIC, etc).

However, I have found some issues with the sampling, so I posted a question in the support BB forums:

http://supportforums.blackberry.com/t5/Native-Development/Questions-about-audio-input/td-p/1929865

I down’t know how to get the amount of data I need and why I get three blocks of data from the MIC.
Did you find a similar problem when you coded the audio output?

I’m using the previous version of ofxQNX and PlayBook Native SDK 2.0.1, not the latest one. I hope I get some free time to upgrade!!

Thanks

I’m wondering if you’re hitting the same issue (but on input) that me and Seth had when trying to use the PCM device directly. Which is the reason why currently (ab)use SDL for the audio.

When I was at BBJam10 in Amsterdam a few weeks ago, I asked one of the devs. He mentioned that he wrote a blog article about the PCM device (on how to use it properly): http://devblog.blackberry.com/2012/06/blackberry-10-porting-case-study/

I haven’t looked into it yet, but if that works, I might be able to remove the SDL dependency. Perhaps its also useful for you…

Thanks again falcon. This link is very interesting.

What kind of issue did you find?

At the begging I found there was a big lag in the response, because of choosing less samples than needed.

Then I have found that the fragment size is always a multiple of 192 (that’s why I get 1920 or 2212 samples instead of the 2048 asked). That means discontinuity if the audio samples and a strange FFT depending on the phase of the signals.

As I see now in the link, Aaron Ardiri is creating a thread from callback function with an infinite loop inside (activated by a timer) and building a larger buffer to store the samples and populating all the fragments:

  
  
// create the PCM audio thread  
  err = pthread_create(&g_snd_thread, NULL,  
                       application_pcm_callback, (void *)NULL);  
  if (err != 0) return 0;   
  
  ...  
}  
  
...  
  
frames = ps.buf.block.frag_size >> 1;   // 16bit, 22Khz, mono  
frags  = 4;  
buffer_size = ps.buf.block.frag_size * frags; ===  
  
....  
  
// lets fill the buffer with random values (white noise)  
      total_frames = frames * frags;  
      for (i=0; i<total_frames; i++)  
        buffer[i] = (signed short)(rand() % 0xffff);    
   

What I was doing different was:

  • getting just one frag of frames
  • calling the Audio_input_get_data from update() [not from a timed callback]
  • Setting a timer, and activate or desactivate it, whenever he is getting WINDOWS Events (g_timer_active = 0 or 1);

It seems it may work. But what I do not know is where to fill the timer (on , off) in OF.
In fact, is it required in OF ??

If you could drop SDL, then would it mean that OpenAL or FMOD (or any other OF sound library) could be implemented in ofxQNX and work both for input or output? :slight_smile: :slight_smile: :slight_smile:

Thanks falcon. This hyperlink is really interesting.
That kind of factor did you find?
During the begging I found on a search engine was plainly a big lag inside the response, due to the fact of selecting less samples than required.
Then I have found which the fragment girth along with length is a multiple of 192 (that’s the causes why I get 1920 or 2212 samples rather of the 2048 asked). That means discontinuity if or when in case the appear samples and a unusual FFT depending in regards to the stage of the says to.
As I see now inside the hyperlink, Aaron Ardiri is generating a thread from callback work along side some kind of countless loop inside (triggered from a timer) and in addition designing a larger buffer to keep the samples along with populating the the fragments:

I will try it because soon since I find some things fixed while using the sound input. This is certainly my first attempt of designing a spectrogram software for the playbook alongside OF.
The Software runs fast!!!

This is working really, really well. I was able to get up and running within hours of receiving a dev alpha device, and that’s including time to port my codebase from oF0062 as well as get everything setup in Windows (I have a Java problem on OSX that makes it awkward/impossible to do BB10 dev there right now)

Anyway, I have an issue where the app is continuing to run after switching away from it. I’ll be delving into ofxQNX to see if I can solve it myself, I just wondered if anyone had already run into it.

Alright, that was simple. If I ever figure out git I’ll submit my changes, but in the meantime it’s a matter of handling the NAVIGATOR_WINDOW_ACTIVE and NAVIGATOR_WINDOW_INACTIVE events in ofAppQNXWindow. I don’t know if ofBaseApp::windowEntry is *supposed* to be used for this purpose, but I’m just calling qnxApp->windowEntry(1) for active and qnxApp->windowEntry(0) for inactive, and making sure I do something appropriate in my mainApp::windowEntry override.

Updated the add-on today with some major changes and also implemented methods for those events that SiW mentioned above.

Changelog:
01/14/2013

  • Cleaned up ofAppQNXWindow
  • Changed bps event handler loop to pause properly
  • Added callbacks for navigator events (Swipe down and Window states):
    * virtual void navigatorSwipeDown()
    * virtual void navigatorWindowState(int state)
    * virtual void navigatorWindowActive()
    * virtual void navigatorWindowInactive()
  • Automatically enable touch (No need for ofRegisterTouchEvents(this) anymore)
  • Added fixed time step option

Check the EmptyExample on how to use the new navigator methods.

As usual, everything is on GitHub: https://github.com/falcon4ever/ofxQNX

Thanks! (like I just said on Twitter)

Any reason why you didn’t update the libs for 10.0.9.2318?

I didn’t knew that BlackBerry updated the Gold SDK. Seems like it is an update from within the IDE (I’ve just read http://developer.blackberry.com/native/downloads/bb10/releasenotes-nativesdk.html#refresh ).

I’ll update the libs probably later this month. My Dev Alpha A device is currently in RMA so I can’t really test it. I should have a new device soon though.