mapping video to a simple rgb led matrix

Just getting started in OF and for a first project to focus on I want to try mapping video to a simple RGB led matrix being controlled by an arudino.

On the OF side I would need map the video to a virtual representation of the rgb led matrix and then send serial commands to the arduino to update the display in realtime (i.e. set specific rgb leds to specific colors).

The first step is mapping video to the simple grid, what is the best way to do this?

Hi intellijel,

I did a project a few months ago mapping DV footage to RGB LEDs in realtime, but in a freeform arrangement rather than in a grid:

http://forum.openframeworks.cc/t/please-remember-me-when-i’m-gone/1409/0

The main issues are the amount of data, and finding a good chip to talk to the LEDs (or a good LED matrix that’s already available). You can use firmata for talking between ofw and arduino, but it’s slower and not specialized for high-bandwidth data like this.

A good chip for talking to LEDs is the TLC5940, but you need to make sure your LEDs are common anode. There’s already an Arduino library available for this chip that makes it super easy to use.

Regarding the bandwidth, do some envelope calculations: a 8x8 grid is 64 LEDs, or 192 independent channels. A TLC5940 is 16 channels, so you’d need 12. If you’re assigning each LED at 30 fps, that’s 5,760 bytes/sec, or 57,600 baud at 8n1. Serial on the 168 is decent at 57,600 so you should be fine. More than 8x8 RGB LEDs, or at a faster framerate, and you’ll need to use a different layout.

I hope some of this is helpful!

Hey Kyle,

Thanks for the tips!

On the hardware side I am totally fine. I have developed a modular matrix, firmware and communication protocol with adequate frame rate capabilities.

I am not looking to make a video screen, I just see translating video to the grid as a good test of the capabilties to do crude animations.

What I really want to develop eventually is alternatives to the video input and create unique and intuitive methods of interacting with the grid in real time (perhaps with tactile devices).

cheers,
Danjel

Nice! Sounds like you’re more than half finished, if the hardware, firmware, and communications are in place :slight_smile:

All you would really need to do in ofw is sample portions of the video using ofVideoPlayer’s getPixels() and use ofSerial to send the pixels in whatever format you’ve already worked out. If the grid is the same aspect as the video, you could even load ofVideoPlayer’s pixels into an ofImage, resize it to the dimensions of the grid, and just use the pixels directly without worrying about averaging.

I’ve built a pretty robust ofxLights add-on that includes a class that lets you overlay your grid of lights to a section of the screen, and then have them update based on what ever’s drawn underneath.

The whole add-on is based on two main components - the “light displays” (which interface drawing to the hardware or screen), and the “light sequences” (which control the colours that will be used by the displays).

It’s all setup to be pretty flexible and easy to extend… I can send you a copy if you’d like.

Nice – if it’s small enough you can also just post it here as an attachment.

I kind of confused, though, what’s exactly were you originally asking about, then? It sounds like you have all the hardware, firmware and software worked out already…?

[quote author=“kylemcdonald”]
I kind of confused, though, what’s exactly were you originally asking about, then? It sounds like you have all the hardware, firmware and software worked out already…?[/quote]

Not exactly. I can easily talk to the grid of lights with serial commands like (x,y,r,g,b) for position and color. This would then be interpreted by the arduino and drive the appropriate rgb led. I have also made other functions internal to the arduino for illuminating columns, rows, whole face, sore/recall pattern etc. But these are all basic ways to use the grid.

The most important (and innovative) layer to all this is in creating the intuitive tools that will generate those x,y,r,g,b values.

Like say for example I wanted a point to appear to move zig zag on the screen and morph bewteen colors? I could write static code to do this or I could pre-program a series of patterns and sequence them but it is way more interesting to have a set of controls with a UI (combo of software and hardware) that can be used to animate this in realtime or record motion or link to triggers like audio.

Imagine joysticks, sliders, color pickers etc.

Another approach is to have a camera and approximate the motion and graphics you want in some other way (use VDMX? hold different colored light sources up to a camera etc?) and then have OF intelligently interpret these into the appropriate xyrgb values to be sent to the arduino.

This is all an experiment:P

basically many many companies make rgb led matrix products and this is not what I am trying to do. I am trying to develop applications and controls for them to make them more interesting or useful for visual performance/installation.

Ok… I will post the add-on either later this evening, or sometime over the weekend. Just don’t have time to package it up properly right now (since a good example would almost definitely be needed to get it setup properly).