RGB Image Processing Strange Behaviour

I am doing RGB pixel manipulation, however when I increase the severity (of Colour Blindness), some pixels behave really strange, such as some turning bright yellow, or bright red (the Colour Blindness they select shouldnt let them see these colours at all).

I read into the OF book and it mentions int overflow, I tried to restrict the values churned out but still the strange pixels come up. Here is an example showing two colour changes which shouldn’t happen:

The code pulls in each pixel and then converts it using a special 4D array which will then give the correct colour blindness pixel.

Does anyone know what this problem is? Is it Int overflow or something else?

Thanks if anyone can help!