Cross-compile OF 0.10 for Raspberry Pi Armv6 Guide

#1

Hi all, I wanted to share my research into getting cross-compiling working for Raspberry Pi with OF 10.1 - in the hopes we can make this accessible to intermediate users (myself included).

As a recap - cross-compiling allows an Pi Arm6 / 7 application to be compiled on a host virtual Linux machine instead of the Pi - the binary / app then copied back across to the Pi for testing, which greatly speeds up the development process (something which can be painfully slow otherwise).

I’m keeping a personal guide for working with the Pi over here:

And cross-compiling guide here: of-cross-compiled-guide.md

What I’ve seen is that there are broadly three documented methods for cross-compiling OpenFrameworks each encompassing the steps:

  1. Mount or build an up-to-date root filesystem
  2. Update mounted root filesystem symlinks / paths
  3. Download and build GCC cross compiler toolchain

1) Debian with Samba

Summary: compiles via network drive, but can’t get past SMB / NFS permissions

System: Debian 9 w. Parallels and macOS

This method mounts the root filesystem via SMB, and also compiles directly onto the network attached Pi, and is first outlined by @jvcleave in the guide for 0.9.0 here:

0.9.0 Jessie Guide

With an updated gist build_cross_gc.sh for 0.10 here:

build_cross_gc.sh needs some repeat runs before completing - and removing caches helps (ie. rm ./build-gcc/gmp/config.cache ), after which I’m stuck at permission write errors for the SMB drive during compile.

This is documented in the original thread and also amended in the SMB guide (where root is mounted at /rootfs ), but no such luck. I’ve also tried with NFS and am starting to think it could be an issue with using Parallels over VirtualBox.

2) ofNode 0.10 Branch

Summary: working, but difficult root setup and addons use

System: Ubuntu 18.4 w. Parallels and macOS

This is the CMake ofNode project last updated for 0.9.8 with Raspbian Jessie, with @avilleret maintaining a 0.10 Stretch branch over here;

And install guide here: how_to_create_sysroot.md

I was unable to create sysroot this way so instead reused RPI_ROOT from the previous guide. Once available, its relatively straight forward generating makefiles and I’ve had success compiling example apps with Arm6.

Addons and addons dependencies however must be defined in the CMakeLists.txt, and I’ve had difficulty getting ofxOMXPlayer v2 and others to work.

3) Continuous Integration / Travis

Summary: works, but with some jiggery pokery

System: Debian 9 w. Parallels and macOS

This is a method outlined by @arturo in the original 0.9.8 thread and reuses Travis CI scripts from the main OF branch to build both cross-compiler and raspbian root.

The main issue I found was that against the autogenerated raspbian root (created during install.sh), the build is passing - but the resultant apps / binaries are non functional.

OF compiles extremely quickly, but this seems to be because the outputted app is missing its internals;

./colorExample: symbol lookup error: ./colorExample: undefined symbol: glTexEnvf

My solution has been to again re-use the the RPI_ROOT from the second method alongside ENV variables - with success compiling examples for ofxOMXPlayer, ofxOMXCamera and ofxGPIO etc. I also opted to mount the root as a USB clone rather than over network, which speeds up compiling.

I also made a fork of OF here containing:

  • buildAllRPiExamples.sh - updated to have failsafes against non-example folders and files; a summary of what is passing and what is not; and loop to clean and rebuild failing examples w. chmod 777
  • build_crosscompiler_arm6.sh - for now just a WIP reference for stages outlined in the guide

OpenFrameworks 0.10.1 Cross-Compile Guide Armv6

Tested macOS Mojave with Debian 9 via Parallels Desktop, Raspberry Pi 3B+ with Raspbian Stretch Lite, OpenFrameworks 0.10.1

Prerequisites

  • Linux Debian 9 installation or vitual machine
  • USB stick (can be smaller than SD card, but large enough for filesystem)

From the Pi

1️⃣ Install Raspbian Stretch

2️⃣ SSH in and update everything:

sudo apt update && sudo apt upgrade
sudo raspi-config

# >>> expand filesystem
# >>> update raspi-config
# >>> enable GPIO, camera etc 

3️⃣ Install all OF dependencies:

cd

# Get OF armv6 from downloads page...

wget https://openframeworks.cc/versions/v0.10.1/of_v0.10.1_linuxarmv6l_release.tar.gz 
tar -xvzf of_v0.10.1_linuxarmv6l_release.tar.gz
cd of_v0.10.1_linuxarmv6l_release/scripts/linux/debian
sudo ./install_dependencies.sh && sudo ./install_codecs.sh

4️⃣ Install any additional addon dependencies (optional):

# ofxOMXPlayer dependencies:

cd ~/of_v0.10.1_linuxarmv6l_release/addons
git clone https://github.com/jvcleave/ofxOMXPlayer.git
cd ofxOMXPlayer 
sudo ./install_depends.sh

5️⃣ Attach USB stick and copy entire filesystem:


# Install rpi-clone to bin

git clone https://github.com/billw2/rpi-clone.git 
cd rpi-clone
sudo cp rpi-clone rpi-clone-setup /usr/local/sbin
cd ../
rm -rf rpi-clone

# Find out name of USB stick

df -h

# Clone it

sudo rpi-clone sdX

You now have a backup USB…

From Linux

1️⃣ Attach the USB and mount it


# View disks

df -h

# USB root is the larger partition

mkdir /media/Data
sudo mount /dev/sdX2 /media/Data/

2️⃣ Create a non-destructive copy of the root

cd ~
mkdir RPI_ROOT
cd RPI_ROOT

# Create symlinks

ln -s /media/Data/etc/ etc
ln -s /media/Data/lib/ lib
ln -s /media/Data/opt/ opt

# Copy entire /usr directory

cp -Rv /media/Data/usr/ usr

3️⃣ Update hardcoded links to relative links in usr folder (see comment)

cd usr/lib/arm-linux-gnueabihf

rm libanl.so libBrokenLocale.so libcidn.so libcrypt.so libdbus-1.so libdl.so libexpat.so libglib-2.0.so liblzma.so libm.so libmnl.so libnsl.so libnss_compat.so libnss_dns.so libnss_files.so libnss_hesiod.so libnss_nisplus.so libnss_nis.so libpcre.so  libresolv.so librt.so libthread_db.so libusb-1.0.so libutil.so libuuid.so libz.so

ln -s ../../../lib/arm-linux-gnueabihf/libanl.so.1  libanl.so        
ln -s ../../../lib/arm-linux-gnueabihf/libBrokenLocale.so.1  libBrokenLocale.so      
ln -s ../../../lib/arm-linux-gnueabihf/libcidn.so.1  libcidn.so        
ln -s ../../../lib/arm-linux-gnueabihf/libcrypt.so.1  libcrypt.so       
ln -s ../../../lib/arm-linux-gnueabihf/libdbus-1.so.3.14.15 libdbus-1.so       
ln -s ../../../lib/arm-linux-gnueabihf/libdl.so.2  libdl.so        
ln -s ../../../lib/arm-linux-gnueabihf/libexpat.so.1.6.2  libexpat.so       
ln -s ../../../lib/arm-linux-gnueabihf/libglib-2.0.so.0  libglib-2.0.so       
ln -s ../../../lib/arm-linux-gnueabihf/liblzma.so.5.2.2  liblzma.so        
ln -s ../../../lib/arm-linux-gnueabihf/libm.so.6  libm.so        
ln -s ../../../lib/arm-linux-gnueabihf/libmnl.so.0.2.0  libmnl.so        
ln -s ../../../lib/arm-linux-gnueabihf/libnsl.so.1  libnsl.so        
ln -s ../../../lib/arm-linux-gnueabihf/libnss_compat.so.2  libnss_compat.so      
ln -s ../../../lib/arm-linux-gnueabihf/libnss_dns.so.2  libnss_dns.so       
ln -s ../../../lib/arm-linux-gnueabihf/libnss_files.so.2  libnss_files.so      
ln -s ../../../lib/arm-linux-gnueabihf/libnss_hesiod.so.2  libnss_hesiod.so      
ln -s ../../../lib/arm-linux-gnueabihf/libnss_nisplus.so.2  libnss_nisplus.so      
ln -s ../../../lib/arm-linux-gnueabihf/libnss_nis.so.2  libnss_nis.so       
ln -s ../../../lib/arm-linux-gnueabihf/libpcre.so.3  libpcre.so        
ln -s ../../../lib/arm-linux-gnueabihf/libresolv.so.2  libresolv.so       
ln -s ../../../lib/arm-linux-gnueabihf/librt.so.1  librt.so       
ln -s ../../../lib/arm-linux-gnueabihf/libthread_db.so.1  libthread_db.so      
ln -s ../../../lib/arm-linux-gnueabihf/libusb-1.0.so.0.1.0  libusb-1.0.so       
ln -s ../../../lib/arm-linux-gnueabihf/libutil.so.1  libutil.so        
ln -s ../../../lib/arm-linux-gnueabihf/libuuid.so.1.3.0  libuuid.so        
ln -s ../../../lib/arm-linux-gnueabihf/libz.so.1.2.8  libz.so

4️⃣ Clone OpenFrameworks from Github, using one of two options:

# A) my fork with additional scripts

git clone -b cross-compile-pi --recursive git@github.com:autr/openFrameworks.git

# B) current release

git clone -b patched-release --recursive git@github.com:openframeworks/openFrameworks.git

5️⃣ Install dependencies & toolchain

# Dependencies

sudo openFrameworks/scripts/linux/download_libs.sh
sudo openFrameworks/scripts/linux/debian/install_dependencies.sh
sudo openFrameworks/scripts/linux/debian/install_codecs.sh

# CI Toolchain

sudo openFrameworks/scripts/ci/linuxarmv6l/install.sh

6️⃣ Update bash profile with ENV variables taken from install.sh (taking care to update RPI_ROOT)

# Edit .profile

sudo nano ~/.profile

# Add these

export OF_ROOT=/home/username/openFrameworks # Optional
export GCC_PREFIX=arm-linux-gnueabihf
export GST_VERSION=1.0
export RPI_ROOT=/home/username/RPI_ROOT # Updated to RPI_ROOT
export TOOLCHAIN_ROOT=${OF_ROOT}/scripts/ci/$TARGET/rpi_toolchain
export PLATFORM_OS=Linux
export PLATFORM_ARCH=armv6l
export PKG_CONFIG_LIBDIR=${RPI_ROOT}/usr/lib/pkgconfig:${RPI_ROOT}/usr/lib/${GCC_PREFIX}/pkgconfig:${RPI_ROOT}/usr/share/pkgconfig
export CXX="ccache ${TOOLCHAIN_ROOT}/bin/${GCC_PREFIX}-g++"
export CC="ccache ${TOOLCHAIN_ROOT}/bin/${GCC_PREFIX}-gcc"
export AR=${TOOLCHAIN_ROOT}/bin/${GCC_PREFIX}-ar
export LD=${TOOLCHAIN_ROOT}/bin/${GCC_PREFIX}-ld

# Reload .profile

. ~/.profile

# Check its reloaded

echo $RPI_ROOT

7️⃣ Compile openFrameworks

sudo openFrameworks/scripts/linux/compileOF.sh

8️⃣ Grab examples from commonly used addons (optional)

cd $OF_ROOT/addons

# For video playback...

git clone https://github.com/jvcleave/ofxOMXPlayer.git
mkdir ../examples/ofxOMXPlayer
cp -R ofxOMXPlayer/example* ../examples/ofxOMXPlayer/

# For the GPIO...

git clone https://github.com/kashimAstro/ofxGPIO.git
mkdir ../examples/ofxGPIO
cp -R ofxGPIO/example* ../examples/ofxGPIO/

# For the CSI camera...

git clone https://github.com/jvcleave/ofxOMXCamera.git
mkdir ../examples/ofxOMXCamera
cp -R ofxOMXCamera/example* ../examples/ofxOMXCamera/

# For awesome projection mapping...

git clone https://github.com/kr15h/ofxPiMapper.git
mkdir ../examples/ofxPiMapper
cp -R ofxPiMapper/example* ../examples/ofxPiMapper/

9️⃣ Try compiling all of the examples

sudo chmod -R 777 $OF_ROOT
cd $OF_ROOT/scripts/linux/buildAllRPIExamples.sh

If using my fork, script will copy template Makefile for each, loop back around and try to clean and rebuild failing examples, and finally copy each compiled app to an of-rpi-examples folder. You can then copy these examples in one go for testing:

scp $OF_ROOT/of-rpi-examples pi@raspberrypi.local:~/

:keycap_ten: Test an app from the Pi

ssh pi@raspberrypi.local
./of-rpi-examples/3d/3DPrimitivesExample

Done! :crossed_fingers:

Next Steps

Next up I would like to try the same method in mind to create an Arm7 and headless Arch guide. Re. Travis CI scripts it would be great to work out the built raspbian is completing correctly, to avoid the RPI_ROOT generation step. From there perhaps too a customisable stage of multistep.conf for addon dependencies…

There is also talk of an official 64bit OS coming to Pi soon, and in the mean time some experimental 64bit OSes, so it may be interesting to try building binaries for these:

Cheers!

5 Likes
Error cross-compiling OF 0.10 for Stretch with Vagrant and Debian
#3

Hey!

I was following your guide to try to do the same but for Raspbian Buster, using a Debian 10 VM.

I had to modify few things from your guide and here are my comments:

:three:
Most of hardcoded links to remove do not exists. For most libraries I have .a and not a .so actually.
Though I didn’t know what to do so I didn’t change anything in here.

:five: :
Need to fix the install_dependencies.sh script:

Replace libsndfile-dev by libsndfile1-dev.
Replace libgles1-mesa-dev by libglvnd-dev

Also need to fix some thing before running the CI Toolchain install.sh script

Modify file /openFrameworks/scripts/ci/linuxarmv6l/multistrap.conf
Replace stretch by buster everywhere in the file.

Also fix the dependencies, replace libsndfile-dev by libsndfile1-dev.
Replace libgles1-mesa-dev by libglvnd-dev

Needed to modify /usr/sbin/multistrap, as shown here: https://github.com/volumio/Build/issues/348#issuecomment-462271607

After that I could go to the end of the guide without problem but when I try to compile something on the Pi I get following error:

make[2]: ccache: Command not found

Any idea of what I missed?

#4

Hey @Meach , thanks for the updates. Are you able to compile OF directly on Buster? I haven’t had a chance to try out Buster or Pi4 yet, but hope to soon. For ccache I’d hazard just installing it with sudo apt ?

#5

If you’re in a rush then it might be best to keep working with Stretch until OF is updated for the Pi4 and latest version of Raspbian. The differences are greater because although the Pi3 was able to run a 64bit OS, Stretch was limited to being a 32bit, and the alternatives were experimental (ie. Pi64 and Gentoo).

I would keep an eye on the Pi4 / Buster thread and also eshkrab’s Rpi4 branch here. It’s exciting because the OF build for Pi on Buster seems to basically become a Linux64 / Debian build, which should unlock the potential of the Pi3s and not just 4s.

So for cross-compiling you’d want to first confirm having a working install of Buster with OF on the device, and then recreate cross compiler steps with changes to reflect this pi4/buster/debian/linux64 build :slight_smile:

#6

I don’t use a Pi4, I actually use a Pi 3 A+. I managed to compile OF on it, even run some example and compiled my project.

Also I cannot use Stretch for my project as the version of Poco is not correct and I have to use Buster instead (see this post).

I will try again your guide later on but I think it was just failing at the very end. I just couldn’t compile any examples from the VM, Always got the error " No rule to make target 'ReleaseABI'

#7

Yes - but making a cross compiler on Buster will be different than on Stretch regardless if you’re on a 3 or 4. If you’re compiling OF on Buster then that’s good news! What make target did you use?

#8

Yes it compiles on the RPi and in your guide I can also compile in the VM (step :seven:).

Where it fails is in step :nine:. Here is what I get when compiling one of the example:

>>3DPrimitivesExample
HOST_OS=Linux
detected cross compiling 1
checking pkg-config libraries:   cairo zlib gstreamer-app-1.0 gstreamer-1.0 gstreamer-video-1.0 gstreamer-base-1.0 libudev freetype2 fontconfig sndfile openal libcurl glfw3 rtaudio gtk+-3.0 libmpg123 
with PKG_CONFIG_LIBDIR=/home/pi/RPI_ROOT/usr/lib/pkgconfig:/home/pi/RPI_ROOT/usr/lib/arm-linux-gnueabihf/pkgconfig:/home/pi/RPI_ROOT/usr/share/pkgconfig
Compiling OF library for Release
make[1]: Entering directory '/home/pi/openFrameworks/libs/openFrameworksCompiled/project'
HOST_OS=Linux
detected cross compiling 1
checking pkg-config libraries:   cairo zlib gstreamer-app-1.0 gstreamer-1.0 gstreamer-video-1.0 gstreamer-base-1.0 libudev freetype2 fontconfig sndfile openal libcurl glfw3 rtaudio gtk+-3.0 libmpg123 
with PKG_CONFIG_LIBDIR=/home/pi/RPI_ROOT/usr/lib/pkgconfig:/home/pi/RPI_ROOT/usr/lib/arm-linux-gnueabihf/pkgconfig:/home/pi/RPI_ROOT/usr/share/pkgconfig
HOST_OS=Linux
detected cross compiling 1
checking pkg-config libraries:   cairo zlib gstreamer-app-1.0 gstreamer-1.0 gstreamer-video-1.0 gstreamer-base-1.0 libudev freetype2 fontconfig sndfile openal libcurl glfw3 rtaudio gtk+-3.0 libmpg123 
with PKG_CONFIG_LIBDIR=/home/pi/RPI_ROOT/usr/lib/pkgconfig:/home/pi/RPI_ROOT/usr/lib/arm-linux-gnueabihf/pkgconfig:/home/pi/RPI_ROOT/usr/share/pkgconfig
HOST_OS=Linux
detected cross compiling 1
checking pkg-config libraries:   cairo zlib gstreamer-app-1.0 gstreamer-1.0 gstreamer-video-1.0 gstreamer-base-1.0 libudev freetype2 fontconfig sndfile openal libcurl glfw3 rtaudio gtk+-3.0 libmpg123 
with PKG_CONFIG_LIBDIR=/home/pi/RPI_ROOT/usr/lib/pkgconfig:/home/pi/RPI_ROOT/usr/lib/arm-linux-gnueabihf/pkgconfig:/home/pi/RPI_ROOT/usr/share/pkgconfig
Done!
make[1]: Leaving directory '/home/pi/openFrameworks/libs/openFrameworksCompiled/project'


Compiling 3DPrimitivesExample for Release
make[1]: Entering directory '/home/pi/openFrameworks/examples/3d/3DPrimitivesExample'
make[1]: *** No rule to make target 'ReleaseABI'.  Stop.
make[1]: Leaving directory '/home/pi/openFrameworks/examples/3d/3DPrimitivesExample'
make: *** [/home/pi/openFrameworks/libs/openFrameworksCompiled/project/makefileCommon/compile.project.mk:129: Release] Error 2
Error compiling :  3d/3DPrimitivesExample