GUI wish list

[Chris OShea]

following on from this thread I have started working on a GUI addon for OFW.

Features I want to include:
- buttons
- slider
- check box
- rotating dial
- number box (think max msp)
- radio list group
- file menu
- colour picker
- timeline

With the interface colour and style being loaded from XML. Thats the plan anyway.

Anyone think of anything else they would like to see?

What is the recommended naming conventions for addons from ofw 0.05 onwards, I was thinking ofControlGui for this one.

[tim]

Hi Chris,

Looks like a solid list to me.
Definitely like the innitiative!

One question/remark though:
Most of the features you mention are quite straightforward as to how they would function. But I guess a 'timeline' can mean lots of different things, as they come in all kinds of shapes and loaded with different functionalities.

Some 'inspirational' timeline tools:
(that might already come in handy as they both send out their values over OSC!)

Iannix - 'an OSC poly-temporal meta-sequencer' (from La Kitchen)
http://iannix.la-kitchen.fr/

TimelinerSA - 'a multipurpose Timeline' (from the VVVV camp)
http://vvvv.org/tiki-index.php?page=TimelinerSA&highlight=timeliner

Cheers,
-Tim

[Chris OShea]

I started drafting out the structure of how to create a generic gui class & objects and realised this will quickly get complex. I'd been discussing with Andreas some of the features & structure issues, and we thought it best to start this on the forum & get everyone involved in the decisions from the beginning.

Here was all I started, it really is nothing, but gives an idea:
http://www.chrisoshea.org/storage/gui.zip

It would be great to hear your thoughts on the following...

-- class structure -------------------------------------------

My first thought of structure for the classes
- manager (ofControlGui)
^-- button (controlGuiButton)
^-- slider (controlGuiSlider)
etc etc

It was pointed out that for others to add new object types, they would have to modify the manager class quite a bit. Instead I should use a uiObject type and then button/slider/checkbox inherits this uiObject.

Because the number of ui objects are unknown, I am using std::vector to hold each object.

Any thoughts on the class structure?

-- input events --------------------------------------------------------

We can't inherit the mouse events from testApp, correct? So within each testApp mouseDown etc you would have to put:
guiClass.mouseDown(x,y);

-- gui events --------------------------------------------------------

When a user clicks a button, what should the action be? Should a generic function in testApp be called, then use switch statements to do something with that action. What about creating event listeners like in flash for each object & definining our own function?

-- colours / style sheets ---------------------------------------------

Should each individual button & slider be colour changable, or one general stylesheet gui?

What other properties do I need?
foreGround color
background color
border color
mouse over color
mouse pressed color

-- typeface --------------------------------------------------------

Should we have a ofTrueTypeFont within the gui manager and then a seperate function to load a font, or pass a ofTrueTypeFont from the testApp by reference?

I have gone for the second option in ofControlGui with this:
void init(ofTrueTypeFont & _type);

However perhaps the first is better

-- tabs / panels ---------------------------------------------------

Gui objects should be able to be grouped, then hidden/shown as needed to create multi layout interfaces.

In controlP5 for processing these are Tabs I believe. Andreas mentioned in his code he has Panels.

I can imagine for each object you could set the panel it belongs to:
guiClass.createPanel("panel1", "Layers"); // id, label, size & position etc
button.setPanel("panel1")

Any thoughts on these?

-- xml ---------------------------------------------------

In theory when the new ofXml is out with 0.05, you could build the whole user interface from an xml file, with that file storing a list of all objects, positions & events.

For now I will use xml for a style sheet, with loadStyleFromXml("red.xml") and saveStyleToXml() functions.


Any of your thoughts on the above most welcome,
Thanks

[hahakid]

I've uploaded what I'm currently using here:
http://www.nanikawa.com/temp/ParameterUI.zip

It looks like this:
http://flickr.com/photos/hahakid/2262339861/

I've been meaning to convert it to OF and post it, but unfortunately I suffer from chronic lack of time.

Also, the latest version of the code is not incorporated in my clean "starter template", but in a client project, so the .zip only contains the UI code and will not compile as is, I'm just posting as a reference as me and Chris were discussing pros and cons of how to approach it yesterday.

I think the way I went about it got a bit messy, so hopefully we'll be able to come up with a better way together, I'm very keen on doing some programming on this as this is a big deficiency in my projects at the moment.

So, onto roughly how it works.

The last thing I do before defining main() is this:

Code:

AParameterUI* parameterUI;

#include "uiDefinition.h"
#include "ParticleUiDefinition.h"

int main( int argc, char** argv )

That way, for instance "ParticleUIDefinition.h" can define functions that reference objects we have defined earlier, an example would be this callback for a float3 sider

Code:

void changedSwayingAxis( float _newVal, int arg1, int arg2, int arg3  )
{
	if( arg3 == 0 ) { interactiveWowScene.particleSystem->swayingAxis.x = _newVal; }
	else if ( arg3 == 1 ) { interactiveWowScene.particleSystem->swayingAxis.y = _newVal; }
	else if ( arg3 == 2 ) { interactiveWowScene.particleSystem->swayingAxis.z = _newVal; }	
}

Referencing interactiveWowScene which was defined earlier in the main.cpp file.

This is the part that I find the messiest and if you look at ParticleUIDefinition.h you'll see how much code it takes to make the interface you see in the screenshot I linked to earlier.

Everything inherits from a base object AParameterUIObjectBase and panels have a:

Code:

vector< AParameterUIObjectBase* > uiObjects; 

Adding for instance a checkbox is done like this (from ParticleUIDefinition.h)

Code:

	AParameterUIObjectCheckbox*	tmpSwayingCheckBox = tmpMovementPanel->addCheckbox( "Do Swaying",  interactiveWowScene.particleSystem->doSwaying );

	tmpSwayingCheckBox->changedParameterCallback = &doMovementToggleFunction;

	tmpSwayingCheckBox->funcPointerSet = true;

	tmpSwayingCheckBox->p_boolVal = &interactiveWowScene.particleSystem->doSwaying;

   
The first line gets a reference to to the checkbox (and internally the Panel adds it to it's uiObjects list), giving it the start value. ( this is redundant since I implemented pointers to the values themselves.)

The second line sets a function pointer to call with the new value when clicked.
The third line could be eliminated by checking (changedParameterCallback != NULL)

The fourth line gives the checkbox object a pointer to the bool it is supposed to be checking, this is so that it will update it's internal case in case something else changes the bool's value.


This probably doesn't make a lot of sense as is, but I just wanted to throw something into the discussion, I think it's an extremely important component for people like us to have.

/A

[theo]

Nice - hahakid.

I saw that image from another thread and I was going to post it here. Pretty slick UI!

Also I just stumbled on this today - it is a short text on user interface design from this guy who built a massive multiplayer game from scratch on his own. The rest of his work is pretty incredible but there is some nice stuff he talks about in this text:

Might be good for ideas - http://www.quelsolaar.com/technology/seduce.html

Anyway I will chime in on this topic

-- class structure -------------------------------------------

It was pointed out that for others to add new object types, they would have to modify the manager class quite a bit. Instead I should use a uiObject type and then button/slider/checkbox inherits this uiObject.

A base ui object is definitely the way to go.

Personally I would keep functionality quite minimal but give it as much information as possible about things like mouse click location, drag time, drag direction etc etc

You could have some basic functions (boundingBox hit etc) that could be easily overridden for more complex behaviors by people extending the base object.

If the base object is structured simply it will allow for a nice range of gui tools.

I learned this the hard way with Laser Tag - it took me a couple of tries to build a base brush class that would be useful but would not limit the brushes that were extending it.

Any thoughts on the class structure?

I Would have to think about it more but something like:

[guiengine /manager]
     
   [gui settings]
    - xmlSettings saver/loader
    - (later on) save load entire gui from xml

   [gui appearance]
   - theme chooser
   - type size etc
   - overall effects - overall gui transparency etc

   [gui communication]
   -event callbacks
   - (later on) networked events (with networked gui objects)

   [gui objects]
   - base gui object
      - simpler slider
      - simple button
      - key assignment object (eg letter 'c' does capture)
      - (later on) advanced controls
      - (later on) networked gui objects

Now that I look of it I am not sure if that is a overall class structure or a wishlist - but I would imagine something like that. I think an important decision would be whether the manager owns the gui objects or whether we create them and then register them with the gui manager class.

-- input events --------------------------------------------------------

We can't inherit the mouse events from testApp, correct? So within each testApp mouseDown etc you would have to put:
guiClass.mouseDown(x,y);

This is one of my biggest frustrations - obviously it would be great to register mouse and keyboard events with the of core - so that you don't have to fill the testApp mousePressed, mouseReleased, key pressed etc with all the relevent gui events.

If you do go this way (I am not sure how you would avoid it at the moment) one thing I like to do with keyPressed is first send the keypress to my gui and then return true or false if the key is mapped to something - that way you can avoid duplicate key press events

Code:

   
testApp:keyPressed(int key){

    if( !gui.keyPressed(key) ){
         
       //gui is not using this key - 
       // put you misc testApp key events here
 
    }
}

-- gui events --------------------------------------------------------

When a user clicks a button, what should the action be? Should a generic function in testApp be called, then use switch statements to do something with that action. What about creating event listeners like in flash for each object & definining our own function?

I think callbacks are the way to go - my gui class for laser tag didn't have callbacks and it was pretty miserable -  I had to ask my gui for every setting whether it had changed its value - then if it had I would request its new value. Uggg. But callbacks can be tricky and could make the code quite complicated - I would cede this to someone who knows more on this subject.

-- colours / style sheets ---------------------------------------------

Should each individual button & slider be colour changable, or one general stylesheet gui?

What other properties do I need?

Thats a tough one.

The objects could have abstract properties like strong, normal, highlight,  etc that define their importance, mode etc  and then the stylesheet could just interpret these with a color - font weight etc.

I would go maybe go with general stylesheets that defines the whole gui  and then people can tweak the stylesheet if they want to change the appearance.

For color I would do rgba rather than rgb - as it might be quite useful down the road.

-- typeface --------------------------------------------------------

Should we have a ofTrueTypeFont within the gui manager and then a seperate function to load a font, or pass a ofTrueTypeFont from the testApp by reference?

I have gone for the second option in ofControlGui with this:
void init(ofTrueTypeFont & _type);

However perhaps the first is better

I am not sure - I might imagine you would need to have more than one typeface, especially as OF loads the typeface in at the requested size - so you would need two ofTrueTypeFont objects to have two different sizes of the same typeface.

One thing I have found is that type in gui's can really slow things down especially on laptops with limited graphics hardware. This can be as much as 10-15 fps when drawing a lot of type to the screen. It is so much that in Laser Tag all non dynamic type is actually ofImages and the gui's for some of my other installations have an option to just show the text of the selected control.

-- tabs / panels ---------------------------------------------------

Gui objects should be able to be grouped, then hidden/shown as needed to create multi layout interfaces.

In controlP5 for processing these are Tabs I believe. Andreas mentioned in his code he has Panels.

I can imagine for each object you could set the panel it belongs to:
guiClass.createPanel("panel1", "Layers"); // id, label, size & position etc
button.setPanel("panel1")

Any thoughts on these?

I imagine that could work fine with specifying relative coords to the panel rather than absolute screen coords for each gui. I think that could be a really nice feature and of course collapsable, grouped, all that stuff would be really useful. You could reposition the panels and then their position could be saved to xml so they remember how you had them.

-- xml ---------------------------------------------------

In theory when the new ofXml is out with 0.05, you could build the whole user interface from an xml file, with that file storing a list of all objects, positions & events.

For now I will use xml for a style sheet, with loadStyleFromXml("red.xml") and saveStyleToXml() functions.

Sounds good!

Anyway I don't think I am an expert on this subject at all - so maybe someone else has better advice.

Theo


Any of your thoughts on the above most welcome,
Thanks

[zach]

great thread! thought I'd jump in on this:

This is one of my biggest frustrations - obviously it would be great to register mouse and keyboard events with the of core - so that you don't have to fill the testApp mousePressed, mouseReleased, key pressed etc with all the relevent gui events.

this is a frustration for me too --- we don't want to make the code too complicated, but in my ideal world there'd be some sort of solution for this.

for example, a way to create objects that extend a base class ofEventReceiver and register themselves with OF, so that OF calls them one by one to give them  mouseEvents, etc.  

ie

Code:

class blah : public ofEventReceiver {

   blah() {  ofRegisterEventReceive(this);
   }

};

at the OF level, you could have a std::vector of  (ofEventReceiver *) that grows with all calls to ofRegisterEventReceive.

then when events happen at the OF level, they get passed to everyone one by one (this is where return false or true might come in handy for objects "owing" the event).

anyway it's just a thought, and it could come in handy for example, if this isn't at the OF level, to have some sort of system like this in the gui management level...

take care!
zach

[zach]

got to this site completely randomly, but noticed a few gui samples - could be helpful to take a look at :

http://www.robthebloke.org/opengl_programming.html

- zach

ps : his pointers tutorialhas a pretty awesome code example for anialiating your computer and a corresponding hand drawn image of what that looks like.

[hahakid]

haha, that tutorial is awesome, very informative too!

/A

[Pierre Proske]

Just a query - doesn't the random memory map trace look a little similar to the sort of output your drawing software makes Zach?? That caricatured squiggle of a human looks strangely familiar.

Ghost in the Machine!? Eller hur?

[hahakid]

So, I figured I'd had a few too many to work on client work tonight anyway and started the process of converting the rudimentary UI I had to OF.

I'm a bit stuck with regards to actually setting variables in the "testApp" (or "OFSAptr"?). This might have a lot to do with how I went about it in the past, I'd have a .h file that defined some functions that accessed objects that were available from main.cpp, but in OF the basic app is an instance of a class, so it's a bit different.

Anyway I'm uploading what I've got so far here:

http://www.nanikawa.com/temp/ParameterUIOfConversion.zip

in case anyone is interested. Only in XCode project format I'm afraid, but it should just be a matter of adding overwriting the files in a copy of "advancedGraphicsExample" and adding the extra files to the project.

Hit "p" on your keyboard to bring up the UI.

What I was stuck on was setting the "spin" variable in the testApp. A few more hours with OF and I'm sure the way to go about it will reveal itself.

I'm sure this will make a lot more sense to me tomorrow :)

Oh, this was very much a quick and dirty conversion in an hour, so it's still using whatever dependencies it had in my own libraries, which I've copied over, but the idea is to move this over to OF so there is something available until a better solution comes along.

/A

[Chris OShea]

Hi Andreas

(i'm in the workshop now)

You can't access variables directly in testApp, you have to pass the parent into your class.

You would need something like this:

in AParameterUI.h

Code:

class testApp;

class AParameterUI	
 public:
	void setParent(testApp * _parent);
 private:
        testApp * parent;
}

and in AParameterUI.cpp

Code:

#include "testApp.h"

void AParameterUI::setParent(testApp * _parent){
 parent = parent;
}

then you can do parent->spin =

make sure on .h you have 'class testapp' at the top and in the .cpp have the include, as otherwise you get a recursively included file.

Do you mind if I take some elements from your Ui and integrate them into this Ui addon, to keep them using ofw specific drawing & type?

[Chris OShea]

I've just tried building your example. It won't work at the moment as AParameterUIObjectBase.h includes glut.h

1>AParameterUIPanel.cpp
1>c:\openframeworks\v0.04\app\work\andreasui\src\parameterui\AParameterUIObjectBase.h(26) : fatal error C1083: Cannot open include file: 'GL/glut.h': No such file or directory

[hahakid]

Hi Chris, that's an easy one since OF uses GLUT already, I just replaced my code for including OpenGL and GLUT with the line:

Code:

#include "ofConstants.h"

I've uploaded the src folder here: http://www.nanikawa.com/temp/ParameterUIConversion_src.zip

You are more than welcome to make it "more OF", the only thing I wouldn't want is for us to double our efforts, so if I find time to work on this in the short term I won't touch the drawing code.

The best way to go about setting the vars is something I'd love to get some more brains on as it should be as un-intrusive to the rest of the app as possible.

Also as I mentioned sometimes you need to call functions, so you can do stuff like:

Code:

glClearColor( _newVal, _newVal, _newVal, 1.0f );

There is code in there as well to get a pointer to a value so the UI can update if the var is changed from somewhere else.


/A

[Chris OShea]

Hi Andreas

When trying to build the new files I get lots of link problems:

1>AParameterUI.cpp
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(112) : error C2011: 'fd_set' : 'struct' type redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(54) : see declaration of 'fd_set'
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(147) : warning C4005: 'FD_SET' : macro redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(88) : see previous definition of 'FD_SET'
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(156) : error C2011: 'timeval' : 'struct' type redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(97) : see declaration of 'timeval'
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(212) : error C2011: 'hostent' : 'struct' type redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(153) : see declaration of 'hostent'
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(225) : error C2011: 'netent' : 'struct' type redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(166) : see declaration of 'netent'
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(232) : error C2011: 'servent' : 'struct' type redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(173) : see declaration of 'servent'
1>C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock2.h(244) : error C2011: 'protoent' : 'struct' type redefinition
1>        C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\winsock.h(185) : see declaration of 'protoent'

[hahakid]

Hmm, I've only tried it on the mac, I can give it a go on the PC tomorrow.

It might have something to do with me including:

Code:

#include <windows.h>    // included in all Windows apps
#include <winuser.h>    // Windows constants

in AParameterUI.cpp and some other files in the Utils folder.