Load to virtual filesystem with Emscripten

Finally I managed to add files to the virtual filesystem and read those files with OF into an ofBuffer.
Loading images works with that method in this example: https://import.handmadeproductions.de/
https://github.com/Jonathhhan/ofEmscriptenExamples/tree/main/emscriptenImport

Maybe it would make sense to put all the filesystem stuff into one class (something like ofxEmscriptenFilesysten?). Or at least simplify those methods a little. For now almost everything that is Emscripten filesystem related is written in ofApp.cpp.

This is how I read a(n image) file into the filesystem:

void ofApp::bang_3onMousePressed(bool & e){ 
	EM_ASM(
	var input = document.createElement('input');
	input.type = 'file';
	input.accept = '.apng, .avif, .gif, .jpg, .jpeg, .jpe, .jfif, .png, .svg, .webp, .bmp, .ico, .tif, .tiff';
	input.onchange = function(e) {

		// getting a hold of the file reference
		var file = e.target.files[0]; 

		// setting up the reader
		var reader = new FileReader();
		reader.readAsArrayBuffer(file);

		// here we tell the reader what to do when it's done reading...
		reader.onload = function() {
			var arrayBuffer = reader.result;
			var uint8View = new Uint8Array(arrayBuffer);
			
			FS.createDataFile("/data/", "data", uint8View, true, true);
			Module.loadImage();
		}
	};
	input.click();
	);
}

And this is how I read a file with OF from the filesystem:

void ofApp::loadImageX() {
	FILE * pFile;
	long lSize;
	char * buffer;
	size_t result;

	pFile = fopen("data/data","r");
	
	if (pFile==NULL) {fputs ("File error",stderr); ::exit (1);}

	// obtain file size:
	fseek (pFile , 0 , SEEK_END);
	lSize = ftell (pFile);
	rewind (pFile);

	// allocate memory to contain the whole file:
	buffer = (char*) malloc (sizeof(char)*lSize);
	if (buffer == NULL) {fputs ("Memory error",stderr); ::exit (2);}

	// copy the file into the buffer:
	result = fread (buffer, 1, lSize, pFile);
	if (result != lSize) {fputs ("Reading error",stderr); ::exit (3);}

	/* the whole file is now loaded in the memory buffer. */

	// terminate
	fclose (pFile);
	EM_ASM(
		FS.unlink("/data/data");
	);
	ofBuffer buffer1(buffer, lSize);
	image.load(buffer1);
	ofLog(OF_LOG_NOTICE, "Image buffer size: " + ofToString(lSize));
	free (buffer);
}

I am sure it can be optimized, but it works so far…
Edit: Reading for example text files this way works too.

And if I can add the file to the Emscripten IndexedDB database, then I can maybe load it directly into OF like a preloaded file…

The IDBFS file system implements the FS.syncfs() interface, which when called will persist any operations to an IndexedDB instance.

This is provided to overcome the limitation that browsers do not offer synchronous APIs for persistent storage, and so (by default) all writes exist only temporarily in-memory.

https://emscripten.org/docs/api_reference/Filesystem-API.html

Okay, I think i got it, and indeed FS.syncfs() was the solution.
It works now like the normal OF filesystem. Just load something with ...load("something"); and it works. Also loading audio files with ofxPd into Pure Data works.
Here is a good filesystem example (Emscripten without OF): https://github.com/emscripten-core/emscripten/blob/main/tests/fs/test_idbfs_sync.c

And here is my example (in this case for loading an image):

This is called for selecting the file with a file browser:

void ofApp::bang_3onMousePressed(bool & e) { 
	EM_ASM(
	var input = document.createElement('input');
	input.type = 'file';
	input.accept = '.apng, .avif, .bmp, .gif, .ico, .jfif, .jpe, .jpeg, .jpg, .png, .svg, .tif, .tiff, .webp';
	input.onchange = function(e) {

		// getting a hold of the file reference
		var file = e.target.files[0]; 

		// setting up the reader
		var reader = new FileReader();
		reader.readAsArrayBuffer(file);

		// here we tell the reader what to do when it's done reading...
		reader.onload = function() {
			var arrayBuffer = reader.result;
			var uint8View = new Uint8Array(arrayBuffer);	
			FS.createDataFile("/data/", "data", uint8View, true, true);
			FS.syncfs(true, function (err) {
				Module.loadImage();
				assert(!err);
        		});	
		}
	};
	input.click();
	);
}

And this is called to read the file from the virtual filesystem into OF and unload it from the filesystem:

void ofApp::loadImageX() {
	image.load("data");
	EM_ASM(FS.unlink("/data/data"));
}

And since I dont really know what I am doing, maybe somebody can tell me how to simplify my code. I would really appreciate it. :slight_smile:

And it should also be possible to download files generated with OF from the virtual filesystem in a similar way. :wink:

I haven’t been able to look at any of this. sorry. I will do as soon as possible

1 Like

@roymacdonald no problem. I think my last comment is a good solution (could be optimized perhaps, but its fine for now). Its also independant from any library.

Next step would be to find out how to download Emscripten generated files. I guess the Java script File System Access API is good for that:

Here is an example: Text Editor
And here is another example for loading and saving files (with Pure Data):
https://purrdata.glitch.me/
https://github.com/cuinjune/purr-data

Downloading works now, too. Just not with the save dialog (I can open the save dialog, but not sure how to pass the blob there). Regular download works with this code (as an example - got it basically from the wasm purr-data):

void ofApp::bang_4onMousePressed(bool & e){
	EM_ASM(
	var found = false;
	for (const file of FS.readdir("/data/pd/")){
		if ("record.mid" == file) {
		found = true;
        	}        
	}
	if(found){
        	var content = FS.readFile("/data/pd/record.mid");  
        	console.log(content);      
		var today = new Date();
		var time = today.getFullYear() + "_" + (today.getMonth() + 1 ) + "_" + today.getDate() + "_" + today.getHours() + "_" + today.getMinutes() + "_" + today.getSeconds();
		var a = document.createElement('a');
		a.download = "markovRemix-" + time + ".mid";
		var blob = new Blob(
		[content],
		{
		type: "text/plain;charset=utf-8"
		}
		);
		a.href = URL.createObjectURL(blob);
		a.style.display = "none";
		document.body.appendChild(a);
		a.click();
		document.body.removeChild(a);
		URL.revokeObjectURL(a.href);
	} else {
	alert("Please play a markov chain before download!")
	}
	);
}

I implemented it here, so you can record the markov chains, and download them (after pressing stop) as a midifile:
https://midifilemarkov.handmadeproductions.de/

Edit: Haha, the save as dialog was just a browser setting…