Lots of good questions and answers in this thread. To add a few points:
The latency caused by signal traveling down a cable is comparable to the speed of light going over some reasonable distance. It can be enough to mess up some digital communication (e.g., long USB cables, apparently), but it is nowhere near what is required to cause audible latency, which is measured relative to the time it takes sound to travel reasonable distances. Latency long enough to be felt and heard is caused by buffering, typically. MIDI cables can be limited in length by impedance-type issues but they can go at least 50 feet with no problems.
The notion that Ivory and other sampled pianos require powerful computers is false. This was once true to some degree but not any more. Two things happened: computers got faster, and software pianos got smart streaming/buffering technology. Any core 2 desktop and indeed laptop is more than enough to power any modern piano. They store the first few seconds or so of all notes in RAM and then only read from disk if needed, which is not all that often . PianoTeq requires a more powerful computer (though is less hard drive intensive) but computer power is not an issue there either. The limitation, if there is any, will be windows and the driver associated with your audio interface (poor buffering properties). If you have a good ASIO driver or a mac, you will be fine with any reasonable computer. I have a first generation core 2 duo underclocked and undervolted and it is more than a match for all software pianos I've used.
Regarding the question about TRS, I have a friend with an AvantGrand that tells me TRS does not work when using a software piano. I don't know why or if he has explored all the possibilities. So you might figure out a better way.
Regarding Ivory and Galaxy, I played Ivory 1 and Galaxy 2, and currently use Vintage D. Vintage D is my favorite but the main Galaxy Steinway is also very good. Ivory 1 can't compete but I understand Ivory 2 is a big improvement and compares favorably to Galaxy if you can stomach the price. By the way you can get the Galaxy pianos individually in download version if you want. The Galaxy people are not real good at advertizing in my experience. The videos on the galaxy website give you a good idea of the capabilities and they have a couple of good video demos. Their audio clips are much less impressive, considering how good their pianos are. Here's a video on youtube
https://www.youtube.com/watch?v=Fgl1C_drg4MIt has both Galaxy Steinways. Vintage D is a bit more intimate than this song and player require, so in some ways this example favors the main Steinway.
Here's another video comparing a bunch of sampled pianos. Quite interesting, though the use of the MIDI file prevents us finding out how good the playability features of each piano are (more on this in a minute).
https://www.youtube.com/watch?v=cmlcvM5IPdoI talked with the author and asked if he could get just one piano he tested, which would it be. He said Vintage D.
I'll add my vote against PianoTeq as well. There is a company that knows how to advertize and distribute it's product. And it has a wonderfully designed interface. Everything is great about it except one thing: the sound. It is never great and can be downright unpleasant, quite a step down from the native AG sounds. There's a lot of misinformation spread by PianoTeq users about sampled pianos (many PT users have no experience with sampled pianos and therefore do not know what they are missing), suggesting that sampled pianos have playability issues or are lifeless in some way. A few minutes playing one product and then the other will easily show that PianoTeq is completely dominated by any modern sampled piano. To be fair, there are many sampled pianos and only a few have the necessary features to be really playable (partial pedal, sympathetic resonance, release samples, repedalling, etc.). Basically Galaxy, Ivory, and Garritan qualify as good, modern sampled pianos. Maybe Imperfect Samples. Early marketing by PianoTeq attacked the playability of early sampled pianos, and many sampled pianos do have a problem there, but it's not an issue with the current generation.
A point about the AvantGrand to be aware of: if you intend to use the line-in features and play a software piano through the onboard speakers you may have an issue with the noise gate that cuts off the sound when it gets quiet. It's an unfortunate Yamaha design decision that has disappointed many who would like to use software pianos. Basically if you hit a note and let it decay all the way, near the end it will flutter in and out as the volume goes below and above the noise gate threshold.
To answer your main question: I expect that Ivory II or Galaxy would be a significant step up in realism, detail, and beauty over the onboard piano sounds of the AvantGrand. However, you would best appreciate it through nice speakers or headphones. In some ways the AG is more than the sum of its parts as the speakers and samples are designed for each other, so playing a software piano through them may not be as much of an improvement as expected. Through headphones the story would be different and Ivory would be a very clear winner.
I have not played Ivory 2 unfortunately, but I have a good friend who has both Ivory 2 and Galaxy 2 (and an AvantGrand) and overall he thinks Ivory 2 has an edge over Galaxy 2. Take that for what it's worth. Since you bought an N3 the price difference between the two products may not be very important to you, so you might want to go with Ivory.