Wriitng to JSON efficiently

I am trying to write ofVec2f points from a Kinect to a JSON File, and then read them in on a different sketch, reformat them, and draw them the same way as the original. I can write out the JSON file successfully, but am wondering what a more efficient way of doing it is. Currently in my Update I am just checking to see if my JSON element is less then 5000 and when it hits 5000 I save it to the JSON and clear the JSON element. I am doing it like this:

This solution is laggy as pushing all 5000 vector elements to the JSON at once is a lot. I was wondering if any of you could help me get this to be more efficient. I have tried different values other then 5000 and they seem to all be laggy.

Additionally, I am planning on using WebSockets to send the JSON file to a different computer in a different place, but don’t know much about them yet. I have heard that I can do some of this computation on the WebSocket side, and am open to trying to figure out how to make more efficient from this standpoint.

is your json object ever cleared, e.g. js.clear(), if not, it’s just going to keep getting bigger and every ofSaveJson() call is going to get slower and slower.

Ahh okay so if I call is js.Clear will it clear the entire file, or just the Json element that I am using to write out with?

In the past for really fast loading/saving I have created my own formats and saved as binary data via ofBuffer.

You can save a vector of ofVec2f essentially as a single stream of memory with ofBufferToFile() and then load it back in via ofBufferFromFile().

        ofBuffer myBuffer;            
        myBuffer.append(&myVec2fVector[0], myVec2fVector.size() * sizeof(ofVec2f));
        ofBufferToFile("myKinectData.bin", myBuffer, true);

I think this should work, its from memory - you might need to make it an array of a pure data structure to get it to save / load nice.

ie:

struct vert{ 
float x; 
float y;
};

vector <vert> verts; 

For more complex data structures we’ve used ofxCereal which can save to a variety of formats including binary. https://github.com/hanasaan/ofxCereal

1 Like

very interesting.

I tried to apply this to saving all dmx levels but got this error:

void ofxDmx::saveAllLevels(string _filePath){
    //std::vector<unsigned char> levels;
    ofBuffer myBuffer;            
    myBuffer.append(&levels[0], levels.size() * sizeof(unsigned char));
    ofBufferToFile(_filePath, myBuffer, true);
}
void ofxDmx::loadAllLevels(string _filePath){
    ofBuffer myBuffer = ofBufferFromFile(_filePath);
    levels = myBuffer.getData();
}

I am using OF0.10.1 (not nightly build)

I got a bit further but now need to work out how to cast new_levels in to the dmx levels vector

void ofxDmx::saveAllLevels(string _filePath){
    //std::vector<unsigned char> levels;
    ofBuffer myBuffer;  
    myBuffer.set((char*) &levels, sizeof(levels));  
    ofBufferToFile(_filePath, myBuffer, true);
}
void ofxDmx::loadAllLevels(string _filePath){
    ofBuffer myBuffer = ofBufferFromFile(_filePath);
    unsigned char *new_levels = (unsigned char*)myBuffer.getData();
}