For starters, if you haven't already done so, I recommend the movie "32 Short Films about Glenn Gould."
The reason why I consider Gould a genius is because when I listened to his recordings for the first time, I had - you see - a sort of revelation. Whether it's his Bach, Mozart, Brahms, or that one unforgettable Chopin CD, his take on familiar music is often astonishingly unorthodox and brings out harmonies and lines that I had never heard before. If you didn't feel this sudden rush of "revelation" or "euphoria" when you listened to him, then there's nothing I can do to convince you of his "genius." Music is, I suppose, subjective.
For instance, take his interpretation of Chopin's Piano Sonata no. 3. Of course, we all know about Gould's legendary disdain for Frederic Francois. For someone who is used to orthodox Chopin (Dinu Lipatti, for instance), hearing Gould's trademark semi-staccato treatment of the 4th movement was absolutely astonishing. Or take his infamous performance of the Brahms piano concerto no. 1, with Leonard Bernstein conducting. It was so outrageous and different ("tempi"-wise) that Bernstein had to make a speech before the concert giving a little "disclaimer" to the audience. Again, that performance, to me, was absolutely fresh and exciting.
Finally, you should listen to his 1982 radio interview with Tim Page in which he discusses, among other things, just exactly what the heck he means by "tempi." This interview was made just weeks before his death in 1982 and can be found on the 2-CD set "A State of Wonder" that is widely available.
In short, Glenn Gould, for me, is not merely a pianist but a completely fascinating and multi-faceted personality. One cannot criticize his mannerisms and his humming without understanding how they fit in with and are inseparable from his larger persona.