Emscripten and CORS policy

Jona · June 15, 2023, 1:47am

I wonder if there is a way to solve this.
I actually have a (AI) model online: model.json · Jona0123456789/clip-text-vit-f16-tfjs at main
And want to include it in my script, like it is done here:
let modelPath = quantizedSelectEl.value === "no" ? 'https://huggingface.co/rocca/openai-clip-js/resolve/main/clip-text-vit-32-float32-int32.onnx' : 'https://huggingface.co/rocca/openai-clip-js/resolve/main/clip-text-vit-32-uint8.onnx';
openai-clip-js/onnx-text-demo.html at main · josephrocca/openai-clip-js · GitHub
But it seems not to be possible with Emscripten (or I just do not know how to handle it…). Same with pictures etc…

This is the error:
Access to fetch at 'https://huggingface.co/Jona0123456789/clip-text-vit-f16-tfjs/blob/main/model.json' from origin 'http://localhost:6931' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled. Failed to load resource: net::ERR_FAILED

Does someone has an idea how to solve those CORS issues with Emscripten?
Loading the model from the data folder works, but those files are quite too large for including them online: ofEmscriptenExamples/emscriptenCLIP2/clip.js at main · Jonathhhan/ofEmscriptenExamples · GitHub

Similar problem with those images: GitHub - EliSchwartz/imagenet-sample-images: 1000 images, one per image-net class. For easy visualization/exploration of classes.
I would like to show them in the app, if they belong to the promt. But it only works locally for now…

Jona · June 15, 2023, 5:00am

I found a solution for loading the model. I need to use (replace blob with resolve):
'https://huggingface.co/Jona0123456789/clip-text-vit-f16-tfjs/resolve/main/model.json';

Jona · June 16, 2023, 4:28am

Here it works for loading the CLIP model and images from the web (it takes quite long to load the model, because it is 128MB big):
https://clip.handmadeproductions.de/

github.com

Jonathhhan/ofEmscriptenExamples/blob/main/emscriptenCLIP3/clip.js

interferenceText__deps = ['$GL'];

var textModel;
var imageModel;
var tokenizer;
var textTensor;
var imageTensor = [];
var dict = {};
const promises = [];

function loadTextCLIP() {
        var modelPath = 'https://huggingface.co/Jona0123456789/clip-text-vit-f16-tfjs/resolve/main/model.json';
        tf.loadGraphModel(modelPath).then(mod => {
        	tokenizer = new tok();
        	textModel = mod;
		console.log("Text Model loaded.");
		// loadImageCLIP();
		fetch('imageTensors.json').then((response) => response.json()).then((json) => imageTensor = json);
		Module.textModelLoaded();
	});

This file has been truncated. show original

For images, .crossOrigin needs to be set before the .src:

var image1 = new Image();
image1.crossOrigin = "Anonymous";
image1.src = "some.jpg";