Preloading images- Memory allocation error?

I’m loading a list of URL as images in a vector using ofxThreadImageLoader
The problem is when I’m trying to upload more than 12 images --I get a memory alloc error. How can this be? I have 16GB RAM and 8GB GPU --the images are JPGs of around 2MB each, even uncompressed in memory won’t be that much, isn’t. I already loaded some CVS and text files and usually in memory/screen there are six layers with images, that’s it – In the Visual Studio Diagnostic Tools the overall used memory doesn’t go over 600MB. Thanks a lot for any help!

ofxThreadedImageLoader loader;

vector<ofImage> mempic;
mempic.clear();
mempic.reserve(2000);

// loads to memory the final image list - the problem is when numpix is greater than 12 or so.
	for (size_t i = 0; i < numpix; i++) {
		mempic[i].clear();
		loader.loadFromDisk(mempic[i], mempicfiles[i]);
	}
1 Like

do you still have this problem?

Hi @roymacdonald !
Yes, the preload was the way to go to avoid the lags but now I have this problem with the number of images and memory. With smaller files I can load a larger number of images but I’m hitting a limit nevertheless.

Thanks for any hint!

1 Like

I see. so where are you declaring your images? this is important because it could mean it is either the stack or the heap. Check this ofBook Chapter

2 Likes

Thanks, that was enlightening. In any case a vector declaration should create the variable in the heap, isn’t? If I move the declaration of the preloading variable like this in ofApp.h :

vector<ofImage> mempic

And then in WImage.ccp:

void Wimage::newImage() {
	extern size_t ix;
//	extern vector<ofImage> mempic;

	if (mempic[ix].isAllocated()) {
		pic.clear();
		pic.loadData(mempic[ix]);
		ix++;
		if (ix >= mempic.size()) ix = 0;
	}	
}

How can I see the ofApp variable here? --I can’t use extern because the variable is now inside ofApp.h-- What I am missing?

Thanks a lot!

1 Like

dont put it in ofApp.h just declare it as a class variable of Wimage
About where vector stores its stuff it depends, but by default it should be on the heap.
https://www.quora.com/C-programming-language-Does-std-vector-use-stack-or-heap-memory-and-why

A good rule of thumb is that if you call the keyword new you’ll allocate on the heap. Although, it is now a bit of a forbiden keyword, because it implies that you need to manually code to delete what ever you created. std::shared_ptr is a really good way to go as you’ll have all the benefits of using the new keyword while not having to care about deleting.
Said so, a vector<shared_ptr< ofImage>> would be a good choice, an you’ll make sure that it will be allocated in the heap. to add elements to the vector you could do

vector<shared_ptr< ofImage>> mempic;
	for (size_t i = 0; i < numpix; i++) {
		mempic.push_back(make_shared<ofImage>());
		loader.loadFromDisk(*mempic.back(), mempicfiles[i]);
	}

taking this a bit further you can make the vector a shared_ptr and have it shared between ofApp and Wimage.

1 Like

I put vector<shared_ptr< ofImage>> mempic; in the class definition in WImage.h
now mempic can’t be seen in a class function in ofApp nor have the usual ofImage functions --like isAllocated ( I receive an error: error C2039: 'isAllocated': is not a member of 'std::shared_ptr<ofImage>'"

Also

void Wimage::newImage() {
	extern size_t ix;
//	extern vector<ofImage> mempic;

	if (mempic[ix].isAllocated()) {
		pic.clear();
		pic.loadData(mempic[ix]);
		ix++;
		if (ix >= mempic.size()) ix = 0;
	}	
}

pic.loadData can’t load the images stored in mempic[n] because is not an ofTexture --again because is a type shared_ptr< ofImage>

in ofApp mempic is undefined (if I just put it in Wimage.h how can it be visible in ofApp?)

**Edit: The most intense work with mempic happens in ofApp (mempic stores the preload of images that occurs on boot and also other loads that happen in other functions in ofApp). In WImage the class newImage only gets one pic from the preloaded images (mempic) and assigns it to itself (pic). **

So maybe I should declare mempic in ofApp and then use vector<shared_ptr< ofImage>> mempic; in WImage? I tried that but then I don’tn know how to express this newImage function under the new shared_ptr sintaxis.

Thanks a lot, really helpful!

Hi, so when using a shared_ptr, you have to use “pointer” syntax, which means that instead of using the dot to access elements or functions you use -> so it would be mempic[i]->isAllocated(), etc.

Then, what is the purpose of the WImage class?, If most of the manipulation of its data is been done in ofApp why not keep it there?

Hi Roy, thanks a lot. WImage is really a class for the layers that show the preloaded images. Every layer have some attributes related to size, crop, fadein/outs, speed of movement and fades, blend modes, etc, so I think to create a class that could have all the related methods of presentation. The program preloads some CSVs that contains a list of all the images (more than 14000) and related keywords and then proceed to preload a few that will be showed for some time using the WImage layers (using the keywords as criteria). The layers shown in the screen could be from 1 to many (this is something I want to push, to show as many images I can at the same time in some of the presentation modes). After some time a new batch of images is loaded and the whole process is repeated again.

I modified the function like bellow but now I’m receiving an error of “vector subscript out of range”.

void Wimage::newImage() {

	extern size_t ix;
	vector<shared_ptr< ofImage>> mempic;

	if (mempic[ix]->isAllocated()) { // here is the error
		pic.clear();
		pic.loadData(*mempic[ix]);
		ix++;
		if (ix >= mempic.size()) ix = 0;
	}	
}

A question, why this won’t work using “extern”?

Thanks a lot and sorry for the continuous bother, I hope after this things will get easier.

I understand, you are trying to access an element in the vector that is not there.
I wouldn’t use the extern keyword as it implies that you’ll have a globally defined variable which I personally don’t like having. it probably does not work because you are linking it incorrectly or it is not declared with the same types.

1 Like

Thanks a lot. I thinkI’m closer now. Last question: How can I access the image data in mempic?

if (mempic[ix]->isAllocated()) {
		pic.clear();
		pic.loadData(mempic[ix]->getPixels()); // this won't work
		ix++;
		if (ix >= mempic.size()) ix = 0;
	}

I’m getting this error:Access violation reading location 0xDDDDDDCD.

what is pic?

The following

mempic[ix]->getPixels()

returns an ofPixels reference. In which you can access the pixel data. ofPixel class has a lot of handy functions for doing so. If you want to access the raw data (which is a pointer to the unsigned char array that stores the image data) you can call

unsigned char* data = mempic[ix]->getPixels().getData();

be careful when accessing that data array so you don’t go out of bounds.
Use mempic[ix]->getPixels().size() to get the number of elements that the data array holds.

This might be useful

pic is the local image of the class (the one that is shown). It only gets a picture from all the preloaded images on mempic from time to time, every time we call this function newImage().

After trying the snippet it says that an ofTexture can’t use an unsigned char. I changed pic to be an ofImage and it doesn’t have a loadData function.

Just to be on the same page: I have a mempic vector of images that is preloaded in ofApp with some images (I’m hitting a memory limit after less than 15 images of 2MB, that’s the reason we are redoing this using pointers)
The class WImage is a class that have an ofTexture pic variable that will take one of the images from this mempic preloaded images. How can I take one of the pictures inside mempic from the WImage class?

Also, Calling
vector<shared_ptr< ofImage>> mempic;

again in the WImage class (besides in ofApp.h) is not a double declaration? How can I access the first mempic declared in ofApp without destroying its content?

So you have a bit of a mess, with your class design.

If you have declared in ofApp.h
vector<shared_ptr< ofImage>> mempic;
leave it there.
Then in your WImage class declare

shared_ptr< ofImage> pic;

then in a function in ofApp resize vector<shared_ptr< ofImage>> mempic to have the number of elements you want it to have and initialize the shared_ptr

mempic. resize(NUMBER_OF_IMAGES);
for(size_t i =0; i < mempic.size(); i++){
mempic[i] = make_shared<ofImage>();
}

set the Wimage pic in a call in ofApp.

wimage.pic = mempic[some index ];

At this point the image is not loaded.

Then load the mempic images, either by threading or not.
The pic element in your WImage class is the same as the one in the vector in ofApp.

The problem here is that every WImage layer calls newImage() every time its alpha value gets to 0. Every layer (ther are a few at the same time on screen) calls this at different times (fades are random in speed) so I would think that taking a new image from the mempic (stack of images preloaded in ofApp that are triggered and reloaded in other ofApp events) should be done inside the Wimage class, not from ofApp.

ofApp in this case preloads in mempic a stack, periodically, with different images every time and then the WImage object will take one of whatever is stored in mempic. I could do wimage.pic = mempic[ix] from ofApp but the idea is to let the layers do this independently, whenever they need to change its own image.

Please let me know if I’m wrong and I should study your solution.
Thanks so much for your help.

I see. you can then, instead of making an vector<shared_ptr<ofImage>> make a shared_ptr<vector<ofImage>>, both in ofApp and Wimage. initialize it in ofApp and then share it with the WImages when you create the WImage instances. This way you’ll share the vector and it would be quite the same as your initial idea of having it declared as extern

How can I do this? This was my second question, how can I see a vector from ofApp inside a class (not sending it in a class call but more like a global variable, in the heap)?

Thanks a lot, I think I’ll do it with this or throw the towel and work with a limited set of preloaded images. This is driving me nuts. Thanks for your patience!

declare in your WImage class

shared_ptr<vector<ofImage>> images;

in ofApp.h

shared_ptr<vector<ofImage>> images;

vector<WImage> wimages; //I am just guessing that you have it like this.

in ofApp::setup()

images = make_shared<vector<ofImage>>();

then, were ever you create a new WImage, which I guess is somewhere in ofApp

wimages.push_back(WImage());
wimages.back().images = images;// this will make the shared pointer to be the same one in ofApp and WImage. Because of this kind of use is why it is called shared.