Mapping live video input to ardunio


I am looking for a way to map live video feed to a 2x2 RGB matrix, which would be used to determine the needed angles for some ardunio motors. I was really exited to stumble upon this community- the video grabber example looked really promising, but I have no experience with ardunio/hardware design so I wanted to make sure that this idea was feasible.

How should I get started:

  1. Mapping the video from an external camera to an RGB matrix (would it just be using the videograbber example?)
  2. Sending the arduino the required angles of the motors in real time

Thank you!

I have some experience with Arduino and other micro controllers, though unrelated to openFrameworks.

Micro controllers usually work with compiled C code that is uploaded, and repeated until some condition is met or ad absurdum. Nowadays, there’s also MicroPython, where the micro controller simply interprets Python files that have been uploaded to it.

If you want much more dynamic behavior, you’d probably need a computer - for instance a Raspberry Pi 4 (which is not a micro controller) - to manage information and pass it back and fourth, or in this case run openFrameworks that communicates with the Arduino.

One way would be to get a Raspberry Pi 4 that would run the video grabber example and ofArduino, which would communicate with the micro controller. To be clear, the Raspberry Pi is not needed if you plan to use some other computer that can run openFrameworks!
I guess you’d connect your camera and Arduino to the Raspberry Pi. The micro controller usually gets connected via USB and the camera depends on the type.
The motors and RGB LED matrix could be connected to and controlled by the Arduino.

If you want something smaller than, cheaper, more powerful, and with Wifi and Bluetooth capabilities, you could substitute the Arduino with an ESP32 or Teensy. I’d check first though if ofArduino supports those! Generally speaking, both are fully Arduino IDE compatible.

Addendum: it is possible to drive hardware from a Raspberry Pi directly using the GPIO ports. ofxGPIO can help - it also has a specific example for servo motors.

@Coltraiin could you elaborate on how your video feed should translate to moving motors? There are various methods of obtaining and analysing video data but how you would map it to motors would be up to you.

1 Like

Hello, I saw at this site a lot of information about video mapping, so maybe you find an answer to your question there