This Solved the problem.
– set GL version to 3.2. as hamoid suggested –
then the code below does what is expected.
mask.setImageType(OF_IMAGE_COLOR_ALPHA);
mask.getTexture().setSwizzle(GL_TEXTURE_SWIZZLE_A, GL_RED);
imgFront.getTexture().setAlphaMask(mask.getTexture());
The shader example from roymacdonald also has GL set to 3.2, (though there is a test in the example and uses different shaders depending upon GL Version)