Well there is an obvious amount of insignifance when it comes to comparisons like this (I'm not assaulting the thread, btw, but am warning that it will no doubt happen) but I like 'em both about the same. I love Brahms, it seems, though I haven't heard much beyond his symphonies, Requiem, concerti (check out the string concerti!), piano sonatas, and a few assorted piano pieces (I know I'm missing out big time on the chamber works).
I haven't heard Arrau's recording, but I'm sure it is superb. I would recommend Rubinstein's, for both concerti, and also it would be interesting to get Gould's. I actually love Gould's--indeed it was the first I heard--not to say I understand it entirely but I think it sounds good at that pace (which is, if you don't know, (in)famously slow, though to be honest in reality it isn't as slow as some reviewers might have you believe), as Gould--and Bernstein for that matter, though he was somewhat opposed to the performance--was able to bring out certain aspects about the music one might not normally hear. Actually, I felt a little disappointed at first when I heard a few other recordings, including Rubinstein's, because their tempi seemed so fast as to overlook some of the more exciting aspects of the piece. But I understand that it really isn't the greatest idea to play it like Gould unless you are, in fact, Gould...so for everyone else there are recordings like Rubinstein and Arrau!
Oh, and I understand that Gilels is widely regarded as a "standard" mark for the concerti, but regretfully I haven't heard the disc myself.