Actually it is true that the only thing that matters is sound.
The good question is, how do you know what it is supposed to sound like? Sometimes, the composer will incorporate that into how it is notated. Sometimes he won't. The problem for me here is partly that the whole idea of thinking that a "composer's intentions" are evident in every detail is to reduce every composer to the same homogenous material.
Essentially we are taking an aesthetic that developed with very stringent 20th century composers like Schoenberg and Bartók, and evolved through the ultra-controlling composers of the second half of the century, and applying it to music of the past.
Well composers of the past didn't write like that, and Rachmaninoff didn't write like that. If you don't believe me, listen to his recording of the Etude in a minor from op.39. The way he plays it is simply not notated in the music. I reject the question, does one play it as notated, or as the composer plays it. Instead, the very fact that he notated it one way, and played it another, means it is strong enough to support multiple points of view.
And in this case, the notation did not inform the sound. That's why the only thing that matters is sound itself. How you get at it, is another matter.
Or take the case of Beethoven's sonata op.27 no.2. Andras Schiff argued very persuasively in recent Guardian lectures that the standard, traditional interpretation of the first movement of this sonata is, in his words, "wrong." He very logically illustrates through Beethoven's manuscript, and through other source material (linking this movement with "Don Giovanni") that Beethoven intended the tempo to be almost twice as fast as most have played it.
In a way I agree with him; I agree with his logic. But I disagree with his judgment. Are the great recordings of Friedman, Schnabel, Paderewski, and so on, all wrong? Should we throw them away, and refuse to listen? Are they guilty of rejecting the "composer's intentions" and of immorality?
No, because they found another sound, one which it turns out, was not justified by a logical reading of a score, but by a shared cultural association, that communicated music of great power - even if it wasn't the music that was written on the page.
I refuse to believe that just because Schiff plays this movement so much faster, and can logically prove that is the "composer's intention," that makes his performance inherently better.
As you can probably guess this kind of thinking has a lot of implications. But if you disagree with me, than you must agree that those great artists are less great than is commonly recognized, because they sought a sound and a feeling which the composer did not actually write down.
Walter Ramsey