This is getting pretty interesting! I'll weigh in as someone who has perfect pitch but who admires much more the superb musicianship of my friends who don't all have it. Perfect pitch is simply a memory for sounds. Like anything else we wish to remember-- music, colors, phone numbers, names-- some people seem to do it brilliantly and very specifically without effort, including details or timbres, some do it extremely well with some extra effort, maybe not as much detail or get confused by change of timbre, some get in the ballpark with practice, and others aren't very successful no matter what they do. As with other memory uses, there are tactics for improvement that work for many people, perhaps not all, and which some people don't need because they somehow just remember things.
Sometimes I find the skill to be useful, sometimes not. I don't usually find it annoying. I know some people with perfect pitch who claim to border on hysteria when the telephone dial-tone "clashes" with the train whistle going by. I have observed this reaction to be personality-driven, not pitch-sense driven. Because how we perceive sound has everything--or a great deal--to do with our expectations. If we expect the fundamental tone of the dishwasher to blend with the upper partials of a hairdryer, we're going to be aggravated. If we expect something very tonal and classical sounding and hear instead two clarinets playing a parallel passage a major second apart, we are apt to think, "Holy cow, what are they playing???" Whereas if the clarinet thing is introduced as a contemporary piece showing, I don't know, the difficulty of life and the despair of the human condition, we might hear it and say, "Yeah, that's pretty poignant, I get it." Or if the clarinet thing is explained as two beginners who've learned the passage but oops, one is playing in the wrong key, we might think, "Aw, how cute, they're little beginners." If you see what I mean.
"Wrong" and "out of tune" are how we perceive sounds that don't meet our expectations, sometimes because of the above reasons, or perhaps those sounds are outside the tuning systems of our particular culture or instrument. The first few times I heard Indian classical music, I almost crawled out of my skin. They use so many more notes than we do, and I just couldn't catch on to it. My ear is still not developed enough to understand a 26(?)-note scale, but it doesn't bother me anymore because I expect it to sound as it does. Even instruments that we're familiar with can have a different tuning tradition/expectation: string players (when they're not playing with piano) tune to acoustic intervals when possible, rather than equal temperament. So an unaccompanied violinist playing an arpeggiated G major chord, starting on a G that matches the nearby tuned piano, will NOT play a B that also matches the piano. He will play a lowered major third, ideally 14 cents lower, but in any case, measurable lower than the piano. And it will be right, and it will be obviously different from the piano, which is also right. Who's out of tune? Nobody. Unless they're playing Mozart together! Then the expectation is definitely that the violinist must adjust his pitch! even though, alone, he was right in line with acoustic-based expectations.
Another thought, since I'm blah-ing on and on... the pitch thing is sometimes nice to have, as I've said, but because of other tuning systems (different instruments, a cappella singing, performing medieval and other early music), which present all these different tuning situations, having the memory for specific sounds isn't worth a whole lot without good TASTE in tuning. If piano is your primary or only instrument, you're pretty much off the hook with the taste thing, I reckon. But if a person with excellent pitch recognition also sings or plays a tune-able instrument, and still only creates tones that exactly match those of a piano, s/he is not exercising good pitch judgment and is actually going to be out of the system -- OUT OF TUNE-- most of the time. And if this person does have the judgment to know when to use which "versions" of pitches, that still doesn't guarantee the technical skill necessary to perfectly produce those pitches either vocally or on another instrument. I guess I just think that there are so many things that can make a musician great. I think the pitch thing is a helpful one, but not a guarantee-- and there are so many amazing people without it.
Topic: perfect pitch