I played it about two years ago. The first and third sections (outer, in your words) are harder, imo, because it is very hard to balance the two hands and the outer sections are to be played much more sustained and intimate and because they run all over the keyboard it is very hard to play in that manner. But it is a very beautiful piece and it will definitely satisfy you when you've learned it.
Btw, I don't know if you listen to Gould a lot with Brahms, but make sure you don't imitate his interpretations. In the middle section, Brahms marks a lot of the notes staccato, but Gould plays it very legato, so... (But he is still a wonderful pianist, my favorite in fact). Just beware of that.
Hope that helps!
BW,
marijn