The 1981 is the most cogent all-round performance. It's a perfectly planned, unified interpretation with a consistent pulse throughout all the variations. Despite it's calculated nature, the playing always sounds fresh to my ears, and of course Gould does lots of cutesy and eccentric effects as he is always prone to do. It's quite incredible that one man could come up with such a perfect interpretation of this work. Interestingly, I find that the 50 year old Gould is more technically solid than in any of the other recordings (I'm not joking - if you've listened to the '55 recording as many times as I have you'll notice him blurring certain passages and speeding up and then slowing down in places to compensate for a too fast tempo.)
I think that Salzburg is the next best performance. Gould claimed to hate live performance. I think it's because he wanted people to think of him as a calculating intellectual rather than an unabashed romantic...but in this recording you get to hear the latter. He's totally uninhibited and spontaneous...a really gorgeous performance.
The '55 is probably the most famous Bach recording ever made, and with good reason - nothing like it had been heard before and it sent shockwaves through the musical world. I actually prefer Gould's romantic playing of the Aria in this recording to all the others (even though it's been recently adopted as the background music for Hannibal Lecter's dinner in the recent Silent of the Lambs Movie

). This recording is quite exciting, but in my mind Salzburg is more romantic and 81 is just a much better performance overall.
'54 is not great by Gould's standards. He didn't have as firm a conception of the piece as he did when he recorded it for Columbia officially the following year.
In the 1960s Gould recorded the Aria and Canonic Variations in a CBC video (which can be viewed on YouTube. The playing is technically accurate, but totally boring and monotonous. The Aria is downright obnoxious. It seems as if Gould really didn't want to record it at that time! Sadly, I'd rather hear Gould playing these variations than the Bach wannabes (Hewitt, Barenboim, Perahia etc.)
To summarize:
1. 1981 (best Goldberg's Recording ever, and one of the greatest interpretations of the recording age)
2. Salzburg
3. 1955
4. 1954
5. 1960s Canonic Variations for the CBC