Using emojis to map the lightness in a video

Hi, I was following a YouTube tutorial to make an ASCII “filter” type of effect on video and was wondering if I could use emojis instead of ASCII characters.

I tried a few things but the most I’ve gotten to is little crossed squares showing up (the kind when a symbol is missing/not supported) and that is by using different fonts, not the Apple one as stated below in the code. (picture attached below)

This is the code I have so far;

void ofApp::setup(){
    ofBackground(0, 0, 0);
    camWidth = 640;
    camHeight = 480;
    
    vidGrabber.setVerbose(true);
    vidGrabber.setup(camWidth, camHeight);
    ofTrueTypeFontSettings settings("Apple Color Emoji.ttc", 10);
    settings.antialiased = true;
    settings.dpi = 72;
    settings.direction = OF_TTF_LEFT_TO_RIGHT;
    settings.addRanges(ofAlphabet::Emoji);
    myFont.load(settings);
 emojiCharacters = string("😀 😃 😄 😁 😆 😅 😂 🤣 ☺️ 😊 😇 🙂 🙃 😉 😌 😍 🥰 😘 😗 😙 😚 😋 😛 😝 😜 🤪 🤨 🧐 🤓 😎 🤩 🥳 😏 😒 😞 😔 😟 😕 🙁 ☹️ 😣 😖 😫 😩 🥺 😢 😭 😤 😠 😡 🤬 🤯 😳 🥵 🥶 😱 😨 😰 😥 😓 🤗 🤔 🤭 🤫 🤥 😶 😐 😑 😬 🙄 😯 😦 😧 😮 😲 🥱 😴 🤤 😪 😵 🤐 🥴 🤢 🤮 🤧 😷 🤒 🤕 🤑 🤠 😈 👿 👹 👺 🤡 💩 👻 💀 ☠️ 👽 👾 🤖 🎃 😺 😸 😹 😻 😼 😽 🙀 😿 😾");
    ofEnableAlphaBlending();
void ofApp::draw(){
 vidGrabber.draw(0,0);
    ofPixelsRef pixelsRef = vidGrabber.getPixels();
    
   
    
    for (int i = 0; i < camWidth; i+=7) {
        for (int j = 0; j< camHeight; j+= 9){
            float lightness = pixelsRef.getColor(i,j).getLightness();
            
            int character=powf(ofMap(lightness, 0, 255, 0, 1), 2.5)*emojiCharacters.size();
           
            myFont.drawString(ofToString(emojiCharacters[character]), i, j);
            
        }
    }
}

7F95D8D1-DBB1-43F0-BF54-43BB804F8584

Hello @jtuli

I see the same behavior. I found this.
If I remove the dpi setting and change the fontsize to 15, I can print a single character with
myFont.drawString("☺️", 100, 400);
Since emojis are not a single character unlike ascii characters,
e.g. myFont.drawString("\342\230\272", 100, 418); gives the same emoji,
addressing a single character (emojiCharacters[character]) will not show anything.

If I change emojiCharacters to be a vector<string> and add all the characters like this

emojiCharacters = {"😀","😃","😄","😁","😆","😅","😂","🤣","☺️","😊","😇","🙂","🙃","😉","😌","😍","🥰","😘","😗","😙","😚","😋","😛","😝","😜","🤪","🤨","🧐","🤓","😎","🤩","🥳","😏","😒","😞","😔","😟","😕","🙁","☹️","😣","😖","😫","😩","🥺","😢","😭","😤","😠","😡","🤬","🤯","😳","🥵","🥶","😱","😨","😰","😥","😓","🤗","🤔","🤭","🤫","🤥","😶","😐","😑","😬","🙄","😯","😦","😧","😮","😲","🥱","😴","🤤","😪","😵","🤐","🥴","🤢","🤮","🤧","😷","🤒","🤕","🤑","🤠","😈","👿","👹","👺","🤡","💩","👻","💀","☠️","👽","👾","🤖","🎃","😺","😸","😹","😻","😼","😽","🙀","😿","😾"};

and change the steps in x/y directions for the change in emoji size it works.

PS: I used emojione-mac.ttc and set the size of x/y to 18.
Unfortunately, I don’t know how to show the color of the emoji ;(

I see what you mean! I definitely missed that. Thank you so much for your reply! :slight_smile:
Yes, I do not know how to do that either. :confused:
Hopefully I can find an add-on which supports those(?) not sure!

Hey @jtuli,

I just found ofxMixedFont.
Maybe that does the trick!
I will try and let you know :wink:

Okay, so it seems the addon ofxMixedFont and openFrameworks 0.9.4 work as promised in the addon readme.
I am on macOS Catalina so I had some trouble with the old openFrameworks and my camera,
but the emojis are in color :wink:
(The emojis all look the same because I replaced the float lightness with a constant value.)

Unfortunately I couldn’t get the addon to work with the latest version of openFrameworks.
I might try another day…

1 Like

Hey @jtuli,

I got the addon working under oF 0.11
Please check out the post here to see where to get it.

1 Like

Oh wow, thank you so much! I actually did end up trying ofxMixedFont with oF 0.11 and it wasn’t working, as you mentioned above.
But I am glad you got it to work now! Excited to try it out!
Thanks again! :slight_smile: