Optimization with ofImage

I’m working on a swarming program. I started out just drawing circles as representations of objects on the screen. When I’m using ofCircle, I can have 3000 objects on the screen at once and still get good performance. I’m wanting to have the swarm actually look like a swarm of birds, so I tried loading and placing images on the screen, which is really making the program slow down. My images are only 20x20px, but they do have transparency. I am sampling from three different images, and just calling draw on the images to get them on the screen. I’m only able to get abotu 150 images before it starts really slowing down.

Any optimization techniques or gotchas that I should be aware of? I’m pretty new to openframeworks.

Thanks,
Denny

I’ve noticed if you’re resizing your images at all, it can really slow down performance. So if you’re just doing a simple load and draw(x, y) with no w and h specified, I’m not really sure.

I would say that if you’ve only got 3 separate images that you’re using… maybe load each one only once, and then pass a pointer to the items in your swarm. That way, you’re not replicating the same image in memory. If you’re already doing that… I don’t know how you could make it better :S I’m very interested to hear what you end up doing though!

Something else you may want to do is run it through a performance tool such as mallocDebug to see if you have any memory leaks or anything like that. Although chances are if you had any memory leaks, it’d be crashing after a short amount of time.

Good luck!

Yeah, I noticed that if I tried resizing the images it could only handle about twenty images, so I ditched that. I’m still trying to wrap my head around memory management, so that’s certainly a possible culprit. Although it hasn’t crashed.

Thanks for your input.

Hmm, well if you are new to C++, pointers are often one of the hardest things to wrap your head around… Relating to your problem, consider something like this…

  
  
class testApp{  
private:  
  ofImage* images[3]; // an array of pointers to ofImage objects  
  
  void makeNewSwarmObj();  
  
public:  
  void setup();  
  void exit();  
};  
  

  
  
void testApp::setup(){  
  // dynamically allocate the ofImage objects (ie. allocate some memory for the objects and make the pointers point to them)  
  
  this->images[0] = new ofImage();  
  this->images[0]->loadImage("image1.jpg");  
  
  this->images[1] = new ofImage();  
  this->images[1]->loadImage("image2.jpg");  
  
  this->images[2] = new ofImage();  
  this->images[2]->loadImage("image3.jpg");  
}  
  
void testApp::exit(){  
  // make sure to clean up dynamic memory  
  for(int i=0; i < 3; i++){  
    delete this->images[i];  
  }  
}  
  
void testApp::makeNewSwarmObj(){  
  SwarmObj* newSwarmObj = new SwarmObj();  
  
  // select random image index between 0-2  
  int imgType = (int)ofRandom(0, 3);  
  
  // passing in only the address of the ofImage object  
  newSwarmObj->setImage(this->images[imgType]);  
}  
  

  
  
class SwarmObj{  
private:  
  ofImage* image; // a pointer - just an address where the object can be found in memory  
  
public:  
  SwarmObj();  
  
  // sets our local pointer to the address of one of the three ofImage objects allocated in our setup() method  
  void setImage(ofImage* _image){ this->image = _image; };  
};  
  

I know it’s a really stripped down example and leaves a lot of detail out… I just wanted to illustrate the parts that use pointers to avoid having to reload the same image more than once.

The alternative/bad approach (loading the image more than once) is probably something like this…

  
  
class testApp{  
private:  
  string imageFileNames[3]; // we are just storing our image's filenames  
  
  void makeNewSwarmObj();  
  
public:  
  void setup();  
};  
  

  
  
void testApp::setup(){  
  this->imageFileNames[0] = "image1.jpg";  
  this->imageFileNames[1] = "image2.jpg";  
  this->imageFileNames[2] = "image3.jpg";  
}  
  
void testApp::makeNewSwarmObj(){  
  SwarmObj* newSwarmObj = new SwarmObj();  
  
  // select random image index between 0-2  
  int imgType = (int)ofRandom(0, 3);  
  
  // we are passing just the name of the file (the image hasn't even been loaded yet!)  
  newSwarmObj->setImage(this->imageFileNames[imgType]);  
}  
  

  
  
class SwarmObj{  
private:  
  ofImage image; // this is an ofImage object - there will be a seperate one allocated for every single SwarmObj - ie. nothing is recycled - we are essentially copying the same ofImage object 3000 times in memory  
  
public:  
  SwarmObj();  
  
  // now we are loading it... separately for every instance!  
  void setImage(string imageFileName){ this->image.loadImage(imageFileName); };  
};  
  

Sorry for the long winded explanation/illustration… hopefully you can get the idea of what I’m trying to show :S

This will help…
http://forum.openframeworks.cc/t/using-a-single-texture-for-multiple-objects/858/0
Its a long thread but beneficial.

I appreciate this explanation. The more I see pointers in use the closer I am to wrapping my head around them. It looks like I’m not using them in my current iteration of the code. I’ll try fixing that.

Based on the thread that a link was posted to, it sounds like I have some other potential optimizations for drawing those images. I’ll try to wrap my head around those next.

Thanks for the help!

You should be able to draw thousands of images without much overhead.
I think you will get quite good results if you take pl0ng’s approach.

If you want some help optimizing your code - the best thing would be to zip up your project and post it to this thread.

Theo

wow, the thread ding is linking to is a blast from the past :stuck_out_tongue:

While all the concepts in that thread remain true, the upcoming version of openFrameworks (soon to be released I think) will have most of the required changes in the core. The bottleneck was enabling & binding the texture and then disabling & unbinding for every particle draw. So for maximum performance you will be able to:

  
  
myImage.getTextureReference().bind(); // bind the texture of myImage  
for(int i=0; i<numParticles; i++) quickly draw all of the particles   
myImage.getTextureReference().unbind(); // unbind the texture of myImage  
  

One thing I’m curious about with this is the best approach if you’re randomly choosing from multiple images. Would it be best to randomly select which particles get which image first, then cycle through all particles related to each image individually using the bind/unbind code? Am I understanding this correctly?

So in pseudo code, this would be preferred:

  
  
image1Particles = // array of randomly selected particles  
image2Particles = // array of randomly selected particles  
image3Particles = // array of randomly selected particles  
  
bindImage1  
for each image1Particle  
  drawImage1Particles  
unbindImage1  
  
bindImage2  
for each image2Particle  
  drawImage2Particles  
unbindImage2  
  
bindImage3  
for each image3Particel  
  drawImage3Particles  
unbindImage3  
  

And this would not be preferred:

  
  
images = // array of images  
particles = // array of particles  
  
for each particle  
  randomlySelectImage  
  bindImage  
  drawImage  
  unbindImage  
  

Hi Yea, the first method would give better results if you have a *lot* of particles (e.g. 15K particles goes from 22 fps -> 35 fps by binding/unbinding only once). As number of particles decrease, the less critical this overhead becomes. Best to try and see if it makes a difference in your case.

You will not always be able to do it this way if your particles have their own texture choosing code based on some logic. But if you do just want to have loads of particles, some with textureA, some with textureB and some with textureC then you should be able to do this.

Also bear in mind that you can’t just bind, then do loads of myImage.draw(), then unbind. That defeats the whole point because ofImage.draw() does the bind and unbind. You need to use your own draw method, something like:

  
  
glPushMatrix();  
      glTranslatef(x, y, 0);  
      glScalef(scaleX, scaleY, 1);  
      glBegin(GL_QUADS);  
         glTexCoord2f(0,0);         glVertex3i(-0.5,-0.5, 0);  
         glTexCoord2f(1,0);         glVertex3i(0.5, -0.5, 0);  
         glTexCoord2f(1,1);         glVertex3i(0.5, 0.5, 0);  
         glTexCoord2f(0,1);         glVertex3i(-0.5, 0.5, 0);  
      glEnd();  
      glPopMatrix();   
  

Note: for the above (texture coordinates) to work, you need to be using GL_TEXTURE_2D. ofTexture defaults to using GL_TEXTURE_RECT_ARB if it can, so you need to prevent that by calling ofDisableArbTex() before loading your images (you can ofEnableArbTex() again after loading the images if you want to enable TEXTURE_RECT for any textures you create afterwards, otherwise you can leave it disabled).

Hope that helps!
[/code]

BTW, using pointers made all of the difference in the world.

@memo

Thanks for the clarification. I’ll give that a shot. Looks like a good way to start getting my hands wet with OpenGL.

hi, I was wondering what the drawing routine would look like for plong0’s code?

I am trying to load a bunch of images and running into performance problems:

.h

  
	  
	ofImage* 	images[5];  
  

loading the images in setup()

  
	  
   for(int i = 0; i < 5; i++){  
		this->images[i] = new ofImage();  
		this->images[i]->loadImage(DIR.getPath(i));  
   }  

draw():

  
	for(int i = 0; i < objList.size(); i++){  
			images[objList[i].resourceIndex]->draw(objList[i].x + xoff, objList[i].y + yoff);  
		  
	}  
	  

where objList[i].resourceIndex just returns an integer index for which an image in the images array (eg to display the 3rd image, the resourceIndex is 3).

I figure that is sort of a confusing way to do it, and that the resourceIndex could probably be a pointer – would that be faster?

currently I am getting about 33fps for 12 images onscreen which is pretty low, but I am unsure if it is because of my code or because of my slow GMA950. The images are the 300x300 example images provided with the ofDirList example

any help would be much appreciated !

edit: I fixed up my code a bit, but still getting the same performance

Hi Nikhan,

if what you want is to load 5 different images, then use them to draw lots of copies of them, here is what I would do:

-in your testApp, load the 5 images:

  
  
// in your class declaration  
vector<ofImage*> assets;  
  
// in your testApp::setup  
int nImages = DIR.listDir("images");  
  
for(int i=0;i<nImages;i++){  
  ofImage* image = new ofImage();  
  image->loadImage( DIR.getPath(i) );  
  assets.push_back(image);  
}  
  

-create a new class, e.g. ‘MyImage’ and assign it one of the loaded images. Just make sure, when exiting your program, that all your MyImage instances are deleted before you delete the loaded images.

  
  
// in your class declaration, the constructor will look like this:  
  MyImage(ofImage* img);  
  
// and add a draw function  
  void draw();  
  
// and you will have a property containing a pointer to the image  
  ofImage* image;  
  
// and some coords or something, to make drawing easier  
  float x, y;  
  
// in your MyImage constructor:  
MyImage::MyImage(ofImage* img){  
  image = img;  
  x = ofRandom(0, ofGetWidth());  
  y = ofRandom(0, ofGetHeight());  
}  
  
// and the draw function:  
void MyImage::draw(){  
  image->draw(x, y);  
}  
  

-in your testApp, you can now create as many MyImage’s as you want and then include them in your testApp::draw() function:

  
  
// in your testApp class declaration  
  vector<MyImage*> images;  
  
// in testApp::setup(), after you loaded the 5 image assets:  
  for(int i=0;i<300;i++){  
    int rnd = (int) floor( ofRandom(0, assets.size()) );  
    images.push_back( new MyImage( assets[rnd] ) );  
  }  
  
// in testApp::draw()  
  for(int i=0;i<images.size();i++){  
    images[i]->draw();  
  }  
  

Note that I wrote this without being able to check the syntax, so there might be a few errors in it. Also, the way I select a random asset (‘int rnd=…’) might be improved on if there is a handier function for it.

I hope this will increase your performance a bit. The reason this might work, is that you only load each image once and pass its reference to any of the MyImage objects you create. They don’t load themselves and drawing can be very fast this way. I had no problems drawing 500+ ants that each was 400x400 pixels (but scaled down) this way.

Hope I made sense,

Droozle

thanks a lot droozle! that helped a lot.

Now I can display about 10 times as many images as I did before (127 images @ 33fps or so). My video card definitely isn’t the greatest, but it’s a good challenge to work with.

thanks again!

Also consider what memo wrote a few posts back: whenever you call the ofImage.draw() function, OpenGL will bind the texture, draw it and then unbind it again. This binding and unbinding comes at a cost and has a bad (but often small) effect on performance. You could optimize your code further by using memo’s drawing code.

Isn’t OpenFrameworks great? I have been playing with it for only 2 weeks now, coming from a Macromedia Director background, and it feels like ditching my Commodore Amiga and buying the latest Apple powerhouse instead. Read: it opens up a world of possibilities. Thanks to Zach, Theo and the lot for their awesome work! I hope I will be able to contribute some code once I feel accomplished enough to do so.

Paul (Droozle)

Hi,

I really have some problems to understand this, could somebody zip up any example ?, it would be very helpfully, thanks.

Sem Aser
http://www.estudioaser.com.mx