With perfect pitch you hear absulote 'tone colours'. With modulations you have to hear relative relations between keys. You could focus on the differces between the two keys in terms of notes. But when the 'new note' appears the modulation hasn't happened yet. The modulation happens the moment the new tonic has established. So this is an issue of note dominance, not hearing absolute 'tone colours'.