Anything that changes the original sound (like adding 'color' to it) is a form of distortion. However, we do not refer to it as distortion because when a device only colors (=adds small changes to) the original sound the distortion is very low.
Simple harmonic distortion occurs when an audio processing device adds low level signals of harmonically related frequencies to the original (input) signal. If for instance the frequency of the input signal is 1kHz and the device adds a small amount (say 1% of the output) of a 2kHz signal onto the original (signal), which is twice the original frequency, we can then say that the device produces 1% of second (even) harmonic distortion.
If the device adds a 3kHz signal (x3 the input signal frequency) to the original then the output signal will contain third (odd) harmonic distortion products, and so on.
So when for instance a speaker is connected to a hi-fi monitor the sound coming out of it may be clean and transparent but this amp/speaker system still 'colors' (= distorts very, very slightly) the signal.
This is because:
a) Most amps, no matter how clean they are rated to be, produce a very low level of distortion (coloration).
b) Speakers distort (color) the sound; this is why different speakers have different 'tones'.
c) The way an amp interacts with the speaker generates a certain amount of distortion (coloration).
d) The level of any distortion (coloration) in audio systems due to a)b)c) , no matter how low, is frequency dependent. This affects the phase relationships between different signals.
e) Many audio systems sound 'dirtier' in signals that contain different frequencies due to intermodulation distortion (IM).
When a combination of harmonically related frequency signals (=cords, a few instruments together etc) is processed through an audio system, intermodulation distortion is produced which is far more detectable by the human ear than distortion in single frequency signals. This is because not all new frequency signals generated by the intermodulation process are harmonically related (i.e. in 'tune') with the original signals. As a result this distortion colors the sound more intensely.
The IM effect is particularly strong in the color amp so the drive controls must be set at much lower levels if only mild coloration (and not distortion) is desired in mixed frequency signals.
f) Transient intermodulation distortion, (TIM) occurs during bursts of sound, like fast transitions as well as beats in music or things like a shotgun effect in sound projects and it is an indication how the time duration and level of one signal transient affects time durations, harmonic coloration/distortion and levels of other signals when they are mixed.
TIM is very prominent and highly discussed in power amps, many of them employing moderate to high levels of negative feedback and drive speakers. The color amp project does not belong to this category.
Low levels of distortion/ coloration is difficult to detect by ear in single notes, even if it is as high as 5% especially in guitars, vocals and other single frequency sound sources. In the case of the electric guitar, a certain amp/loudspeaker combination may generate up to 10% distortion and can still sound clean but with an extra 'warmth' and crunch added to the sound texture.
Tube microphone pre-amps are an interesting case too, they are producing low levels of harmonic distortion which color the human voice to (subjectively speaking) sound deep, sharp and warm.
Finally, it is worth mentioning (even though not necessarily related to the 'color amp ') that the environment where the sound is produced. for example the acoustic parameters of the room, also colors the original signal and affects the frequency response of a system.