Because our only access to color qualities is through their appearance, Byrne & Hilbert's insistence on a strict distinction between apparent colors and real colors leaves them without a principled way of determining when, if ever, we see colors as they really are.
