I think playing by ear is a gift; but a gift that many people are able to have. For some, it is easier; for others, it takes effort. I do not think it is "bullshit"; after all, preserved music was first preserved by ear. (Even in cultures where musical notation existed, learning by ear was a big part. For example, in medieval europe many of the troubadours and trouveres played by ear. In China from ancient times up to the early 1900s, the street musicians learned to play by ear.)
Also, some people can play very sophisticated pieces by ear. A co-winner of the 2009 Cliburn Competition had to learn all his pieces by ear; he was blind.