MIDI-editing in the piano-recording process???

[johnlewisgrant]

I raised this question at a few other sites, as well.  I'm interested in what the classical music community thinks:

Now that Bos/Yamaha and Steinway have perfected the MIDI-piano (modern equivalent to the old player piano), why aren't these used in making and producing solo piano recordings?

I've read that even classical music performances routinely involve quite a lot of splicing together of short sections of a longer piece.

So wouldn't that sort of thing be made much easier, and much more precise, at least in the context of solo-piano performances, with a MIDI record of the performance?

Inotherwords, instead of performing at a regular Steinway or Bos or CFX (etc.), the pianist performs at a MIDI-rigged Steinway or Bos or Yamaha CFX. Then the MIDI record of that performance is edited and the performance spat out (replayed) by the MIDI-equipped instrument while being recorded again in the studio. No pianist, this time, of course; because the MIDI-piano player is providing the "performance" in front of the microphones, etc., In fact, we don't want the pianist playing, not only because he/she might make a mistake, but more important, because the pianist thought the first take was really good, perhaps a "once-in-a-lifetime" performance. We've preserved that performance, but can now re-record it with the desired corrections.

The pianist would, under this scenario, still listen to the end-product through loudspeakers or headphones, as usual, and make assessments, which I'm guessing is what normally happens in the studio. But any necessary changes to the performance would, as before, be made by MIDI-editing, as opposed to wave file splicing and re-takes of relevant sections, etc... Of course, re-takes by the pianist could be done, too. They're not necessarily ruled out. But those retakes would also become part of the "MIDI-record," so to speak, and if they are used, they are spliced in, not using audio splicing, but using MIDI-editing.

(Putative) Advantages:

1) If the pianist and engineer really liked a specific take, but just one or two notes got out of tune, the take would not be lost. Rather, the offending notes could be retuned and the MIDI-piano could spit out exactly the same once-in-a-lifetime performance for the recording microphones, now with the out-of-tune note corrected.

2) Alternative microphone placements can be tested and the results recorded, without losing a superior or "once-in-a-lifetime" take.

3) The recording process could, conceivably, end up being faster and cheaper than the old approach which, I think, consists of doing multiple takes and a lot of splicing.

(Putative) Disadvantages:

1) Midi-pianos don't in fact spit out exactly what is fed into them. The technology has not advanced that far.

2) 127 velocity layers is not enough. (I know, the best MIDI machines work in increments of, I think, over one thousand velocity layers. So there's the counter-argument.)

3) Midi-editing takes the life out of the music.

4) Your not hearing the pianist, really; once you MIDI-edit his/her performance your listening to a MIDI-editor.

5) The pianos are too expensive.

There are other arguments and considerations. But, on the face of it, I tend to think the arguments FOR are stronger than the arguments AGAINST.

Are MIDI-pianos (my terminology) being used or contemplated in the piano recording buz?

[ronde_des_sylphes]

Assuming I'm not missing the general tenor of this post, I think that, to a certain extent, much of this is rendered redundant by two considerations.

Firstly, if your take is really good but there was one (or two!) missed notes, splicing a correction from another, less optimal, take really isn't that hard to engineer.

Secondly, and arguably more critically, if you have access to autotune software such as Melodyne, you can often correct individual problematical notes, including returning. I don't have a huge amount of experience with it, but it has worked wonders in cases where I've needed an emergency editing tool for stuff I want to release and don't have the capacity to do retakes.
 

[Bob]

Have they cut down on lag?  I experimented a little with MIDI and a digital piano years ago, but the lag became an issue.  Notes would clump up, and I realized what I was hearing in real time from headphones was not exactly what I was playing on the keyboard. 

[johnlewisgrant]

Have they cut down on lag?  I experimented a little with MIDI and a digital piano years ago, but the lag became an issue.  Notes would clump up, and I realized what I was hearing in real time from headphones was not exactly what I was playing on the keyboard.

So that's "disadvantage" 1: not accurate enough.   I was at a free piano maintenance course at RCT (Royal Conservatory of Toronto) many, many years ago.  The piano tech teaching the course took us next door to Koerner Hall, where in addition to their Hamburg Steinway for concerts they kept a Yamaha concert grand with Disklavier.

He gave us a demo, plus answered questions about the technology.  He claimed, and this is several years ago--so things have improved I'm sure, that the replay of the player was "dead-on accurate."

Otherwise, obviously the whole issue is moot.  No serious jazz or classical artist would accept anything less.

[klavieronin]

I remember an experiment someone did online a few years ago where they asked people to guess various audio formats (MP3, WAV, AIFF, ACC, etc.) but didn't tell anybody which was which. The results were that while most people could hear a difference, no one was able to distinguish the the higher quality WAV format form the highly compressed MP3 format. Meaning that MP3s sound just as good as WAVs as long as you don't realise you're listening to an MP3.

My guess is that if you recorded a midi performance and told people, many would say it sounds mechanical. If you did it and didn't tell anybody, no one would notice. I think people generally believe their listening skills are a lot better than they actually are.

That's my guess anyway. For recordings it doesn't seem like such a bad idea. For live performance on the other hand…

[johnlewisgrant]

I remember an experiment someone did online a few years ago where they asked people to guess various audio formats (MP3, WAV, AIFF, ACC, etc.) but didn't tell anybody which was which. The results were that while most people could hear a difference, no one was able to distinguish the the higher quality WAV format form the highly compressed MP3 format. Meaning that MP3s sound just as good as WAVs as long as you don't realise you're listening to an MP3.

My guess is that if you recorded a midi performance and told people, many would say it sounds mechanical. If you did it and didn't tell anybody, no one would notice. I think people generally believe their listening skills are a lot better than they actually are.

That's my guess anyway. For recordings it doesn't seem like such a bad idea. For live performance on the other hand…

That's certainly the case, even with "fake" pianos, ie digital keyboards, equipped with convincing piano samples.

But I'm talking REAL pianos, of course, equipped with piano player mechanisms that I'm told are, at this point in the technology, dead-on accurate, and that are invisible to the pianist.  Invisible in two ways: they can't be seen; what is more, they have no discernable affect on key weight, on touch, generally speaking.  Drop and let-off are unaffected. 

You can't tell you're playing a piano that's hooked up to one.

Piano rolls exist which, it is claimed, are the product of Rachmaninoff himself at the piano.  This suggests a modern day equivalent, but with the huge benefit of MIDI, which allows the performance to be altered in small or big ways.   Hence, the opportunity to take near perfect performances, preserve them for recording with microphones, etc., but with little boo-boos fixed, if that's what the pianist wants.  Pollini's much disputed recording of the Chopin Etudes comes to mind. 

[johnlewisgrant]

I've raised this issue, which I think is pretty sensitive for many people, over at Piano World.  It was suggested there, and at Gearslutz, an English site, that the technology IS most likely starting to take hold in recording studios. 

Accuracy of the replay is the first issue.  Then the question is the enormous power of MIDI editing.  Audio splicing and countless retakes is the current norm in studio recordings of classical music, and has been since wax, 78s, 33s, and cds.

But MIDI-editing is a much more powerful editing tool.  It removes us one step further from the idea of a recording as the perfect live performance overheard, and overheard by someone sitting in the best seat in the house.

And we move further away from the idea of technical/aesthetic prowess.  MIDI-editing allows not only the correction of the odd missed note; it invites the wholesale re-creation of the original performance.  So the meaning of "performance" is changed beyond recognition. 

Assuming that Steinway (Spirio) and Yamaha-Bosendorfer (Disklavier) have conquered the accuracy-of-replay issue, and I suspect that for all intents and purposes, they have, huge changes are in the cards for recorded classical piano music.  Live performance will, of course, remain the gold standard: a completely different kettle of fish, so to speak.  Studio recordings, on the other hand, will become something very, very different, as Gould seemed to predict, 50 years ago. 

The myth that studio recordings are just re-creations of the live performance will be completely exposed.  But more than that, the studio recording may have to be acknowledged as what it has always been: a representation of the recording artist's concept of a piece, as opposed to his or her actual physical performance of the piece.

I'm accustomed to the shock of hearing a pianist live who sounds NOTHING like his or her recorded performances.  So perhaps this is nothing new.   On the other hand, I continue to marvel at the technical wizardry of pianists like Yuja Wang (to cite one obvious example), all the more so, in this new era; because her technical perfection is even more apparent at her live concerts than it is in her recordings.  Not that technical perfection is the be all and end all of music.  It isn't.  Which is, perhaps, where the musical "concept" of a piece "in the mind," as it were, comes into play.

[klavieronin]

I think this kind of thing would be of much more interest to composers than performers. A good performer probably doesn't need it that much but giving a composer the ability to tweak a performance to match their vision of the work seems like it would be very appealing.

[lostinidlewonder]

When playing live the exact values for volume and velocity of notes go into many many decimal places (ad infinitum) but a midi recordings can only round the exact value. I am yet to record on a hybrid grand and have it play back exactly what I did, although they can be close depending on what you play. This loss of information is not desirable because the soul of someones playing rests within these tiny differences.

A second thought is that I really dislike the editing process of recordings and the idea that all mistakes or inaccuracies need to be improved, masked or removed. Some of the most beautiful things in this world are not totally uniform, totally coloring within the lines at all times is quite sterile. I am not distracted by a subtle wrong note or something else done that is not totally the norm nor do I search for a total absence of these and thus judge that I am going through a heightened listening experience.

[johnlewisgrant]

When playing live the exact values for volume and velocity of notes go into many many decimal places (ad infinitum) but a midi recordings can only round the exact value. I am yet to record on a hybrid grand and have it play back exactly what I did, although they can be close depending on what you play. This loss of information is not desirable because the soul of someones playing rests within these tiny differences.

A second thought is that I really dislike the editing process of recordings and the idea that all mistakes or inaccuracies need to be improved, masked or removed. Some of the most beautiful things in this world are not totally uniform, totally coloring within the lines at all times is quite sterile. I am not distracted by a subtle wrong note or something else done that is not totally the norm nor do I search for a total absence of these and thus judge that I am going through a heightened listening experience.

On the first point, the accuracy of replay is claimed (by Yamaha for its Disklavier player piano, and by Steinway for its Spirios player piano) to be flawless.  That's something that could easily be tested independently.  Let's allow that there may be a little inaccuracy in the replay.  Certainly the listener (as opposed to the pianist) would never discern that from a recording!  (But, yes, if true it might deter some pianists from using Disklavier to correct errors in the recording studio.)

About the recording studio, it seems to me that what happens in a live recording is very often the opposite of what goes on in the recording studio.  In unedited live recordings the microphones record the pianist expressing a musical idea with his or her "raw fingers," so to speak, and nothing more.

In the recording studio, the pianist has recourse to retakes and to computer audio splicing, and to who-knows-what else, to express his or her musical conception.  This is a lot more than raw fingers.  The result is that we get what the pianist would like the music to sound like, as opposed to what the pianist is physically able to express at the keyboard.

For these reasons, it doesn't matter to me, personally, whether or not a MIDI equipped Disklavier (or Spirio-Steinway) is used in the recording studio.  The measure of a "good" studio performance for me personally isn't how it was done, because I can never really know that for sure; instead, it's the end result, how it sounds to me, which of course involves many, many variables of musical expression.

It's curious that the more a performance is edited, the less of a "performance" we get. That's why, I think, extensive MIDI editing can very quickly put a performance on a slippery slope to a "conceptualization," that is, a "performance" that was not, for the most part, achieved using raw fingers.  That's why pure MIDI interpretations of Bach, Mozart or Beethoven, whether they are replayed by a Disklavier (which is a real piano with a MIDI controlled player piano system hidden inside the piano) or by a digital piano, are not "performances," in the ordinary sense of the word.  I think everyone understands that.  The fact that you often can't differentiate between pure MIDI and raw-fingers-performance is a different matter.  That issue speaks to the effectiveness of the conceptualization, a product of good equipment and a feel for how to achieve a "real" sounding performance without ever touching the keyboard.  Again, this is what makes MIDI-editing in the process of studio recording worrying.  It's not that the end result is not a "performance" or that it's not "music," it's just that the end result is not MADE the way we think it is.

[j_tour]

I must be missing something here.

Isn't the main idea really some putative advances in what parameters can be controlled via the old MIDI standard?

To me, that's firmly in the camp of what can be done in the studio, and has already been done using tape and in the past twenty-thirty years, hard drives.  Tape or ProTools/<insert DAW>.

But I don't hear the OP claiming this is any kind of a substitute for performance. 

I suppose you could assemble a "performance" out of various bits, but you can do that with a razor blade and 2-inch tape.  I wouldn't be surprised if Glenn Gould didn't spend some time trying to fiddle with things like that, but such results are probably lost, and for the better.

I don't quite see why the MIDI-fitted instruments are essential for playback in real time to tape/hard drive, once the data has been captured, but I suppose it could simplify things and would be kind of a neat entertainment for the tape ops and the engineers. They can be there at four or five in the morning, and recapture the performance as though they were catching a live take.  Obviously, the decisions would have been made, but it would be pretty neat.

Actually, I do see the point, but I admit I'm kind of skeptical that the playback through one of the acoustic two-way models is all that much better than using a multi-gig sampled piano/<insert favorite softsynth>.  Actually, it's the same thing, right?  After all, whatever package YamaSteinKorg puts on it, they're not using pneumatic pistons to recreate the sound.  Yes, I have seen pretty closely a Disklavier in the basement of a music dealer whose daughter I was friends with in high school:  it was pretty neat, you know, put in, I no longer remember what media there was, but like Gershwin or even Tatum.  So, what's new?  What's the difference?

You can re-capture the distorting effects of live recording, using microphones, room tone, and all that, which is kind of neat:  and everyone's used to the traditional recorded sound of an acoustic instrument.

I'm a bit underwhelmed, but I agree that it's a studio option to be aware of, and it will be interesting to see if the manufacturers' claims turn out to be "as advertised on the tin."

[themeandvariation]

When playing live the exact values for volume and velocity of notes go into many many decimal places (ad infinitum) but a midi recordings can only round the exact value....This loss of information is not desirable because the soul of someones playing rests within these tiny differences.

This is exactly my reservation to this approach.

On another note, then, the quantizing of rhythm??? are the ethics ('of individuality') easier to see in this context?
Ah, forget it.  I hear that computers can write like Bach now.. (to the uninitiated, of course)
I guess making art should be convenient. It's just for consumers.    Warhol was right.

[lostinidlewonder]

On the first point, the accuracy of replay is claimed (by Yamaha for its Disklavier player piano, and by Steinway for its Spirios player piano) to be flawless. 

Yeah I don't really believe this claim myself though I'm still open to the fact that they will get closer and closer. Replaying midi's through a real piano has improved the sound (as opposed to playing the midi through a synthesizer) a great deal, one of the largest component improved being the effect the pedal has on sustained notes, this is something that synthesizers struggle to emulate appropriately. Nevertheless I can tell a midi recording because of perfect accuracy and eveness in larger edited sections. If you are talking about a single note within a phrase then this is more difficult to pinpoint however if an entire recording is reduced to midi and juxtaposed with the raw audio file there will be a noticeable difference and in my opinion the midi will remove a lot of the "soul" of the persons playing due to the estimated values of the playing recorded within the midi data file.

Certainly the listener (as opposed to the pianist) would never discern that from a recording!

Yes well someone who has never heard you play before logically would struggle to know especially if the editing was a very small % of the original. Though if I hear someone play with their fingers and then listen to a midi recording playing that back, there is always a difference that I notice.

In the recording studio, the pianist has recourse to retakes and to computer audio splicing, and to who-knows-what else, to express his or her musical conception.  This is a lot more than raw fingers.  The result is that we get what the pianist would like the music to sound like, as opposed to what the pianist is physically able to express at the keyboard.

This a philospohical type question but I personally feel opposed to the idea that editing techniques improve recordings. I don't feel that an edited recording via midi or otherwise is "more" (better) than "raw fingers" unedited recordings of a trained pianist or "what the pianist would like the music to sound like". The best recordings I have listened to are live recordings and many of them have subtle inaccuracies in them but for me this does not take away from what the pianist would like the music to sound like. In fact these sometimes small inaccuracies actually make the recording more memorable and the "human" aspect comes out reminding me that we are not perfect beings.

Of course there are many live recordings where one would struggle to find any note errors if there are any at all! But is this the only standard of "greatness" that every single note is hit in an ideal manner? It can be one part of greatness but the expression really is the main factor in this all, you can have all the accurate notes but how are you going to improve the expression? Improving expression through editing the MIDI vai inputting values through a computer is extremely inefficient and almost all the time will be noticeable to the discerning ear.

The expression of music needs to come through, just the correct notes will not cut it and editing the music so that all the notes are correct will not necessarily improve expression. Trying to improve musical expression through midi seem to me to be an inefficient losing battle. Cutting and pasting retakes of phrases through non midi means of course is more effective though I still don't feel that a piece which is a segmentation of many "good" takes produces something that flows continuously in an natural way but rather something which is held into measured position with some artificial means. I personally like the drama of subtle inaccuracy when they pop up in live recordings, it reveals more of the pianist and makes the performance feel more organic.

It's curious that the more a performance is edited, the less of a "performance" we get. That's why, I think, extensive MIDI editing can very quickly put a performance on a slippery slope to a "conceptualization," that is, a "performance" that was not, for the most part, achieved using raw fingers.

Yes the more it is edited the more it is noticable that it has been edited especially through MIDI techniques through a computer. Setting all the volume velocity and duration values for each individual note really is like painting a picture by starting with the atoms!

The fact that you often can't differentiate between pure MIDI and raw-fingers-performance is a different matter.  That issue speaks to the effectiveness of the conceptualization, a product of good equipment and a feel for how to achieve a "real" sounding performance without ever touching the keyboard.

I don't believe that one cannot differentiate between "pure MIDI" (music data which is all entered in through a computer) and "raw fingers performance" (humans playing a piano). There is an obvious difference to me. This reminds me of the argument we had here: https://www.pianostreet.com/smf/index.php?topic=35977.0

[klavieronin]

To be fair I don't think the OP was talking about a "pure MIDI" performance, which naturally would sound very computer like. I think they were talking about capturing a real studio performance then making small adjustments in a DAW (like Cubase) and playing it back on a real piano. There is no way any body could distinguish a performance captured in MIDI then played back on a real piano from the actual performance itself. MIDI is more than capable or reproducing the subtle timings and rubatos of a real performance beyond what is distinguishable humans. And if it's true that these new pianos can record 1000+ velocities I would think that is more than enough too. In fact I doubt most people could distinguish between the 127 velocities currently possible in MIDI.

[themeandvariation]

In fact I doubt most people could distinguish between the 127 velocities currently possible in MIDI.

Are you included in that 'most people' category, Klavieronin? I can't tell how many times
I had to choose between velocities 84 and 85. Both not 'hitting it'..  Etc, etc....
No, the 'most people' position -  has little traction  in my musical world..

[klavieronin]

I can't tell how many times
I had to choose between velocities 84 and 85. Both not 'hitting it'..

What piano where you using? Don't forget the OP is talking about real pianos here, not sampled pianos with only a handful of velocity layers. I know exactly what you are talking about if you are using a sampled piano VST. That used to drive me crazy until I switched over to Pianoteq. It is a physically modeled virtual piano with 127 velocities. Here is a recording with 6 notes; 3 at 84, 3 at 85. If you listen closely perhaps you might hear a difference but within the context of a real piece of music I don't think you would notice. Tell me the order if you can hear it.

[themeandvariation]

Yes, Klavierinon, I am using the Sibelius samples (and note performer which is a bit more realistic). I haven't worked w Pianoteq. Yet, there is a noticeable gulf between the 127 in practically each step.. If were are talking about only 127 possibilities of volume, (which is what Sibelius has) I wonder how pianoteq can create the illusion of having a finer gradation.  Reverb/resonance of course can shift the equation, but without such niceties...?
No, I couldn't hear much of a difference.. But in the context we are discussing, I am always working with velocities within the arc of a phrase - (in the context of a piece) where a note in question can't find its 'true' dynamic. Are they really working on a 1000 possibilities of velocity?

(Pardon me for moving the goal posts with this ancillary jest: I guess this method would not take into account: squeaky benches, whisps of breathing from the performer blending w the high overtones, subliminal 'guttural' response, all the way to vocal obligato (a la Gould).  btw, it was Gould that suggested that cutting tape was the future as opposed to a single take.  Of course, he didn't apply a whole lot of rubato or employ a more dynamic fluctuation of volume, making it easier to do so, I would say).
I guess Ill have to check out Pianoteq to make a more informed opinion..

I've heard live the Yamaha Disklavier - and although impressed, it just felt to me like there was some 'lid' on the sound.. I somehow couldn't Feel the physicality as if someone were playing. Do you know what I mean?

[klavieronin]

To be honest I've never compared the Yamaha Disklavier to a real player side by side, so there could very well be something missing in the MIDI reproduction of a real performance. However, I'm always conscious of how easily our biases and prejudices can influence our experience. For example, I heard about a study that showed that the name of a corn chip can influence how crunchy people perceive it to be. We (humans) are very good at fooling ourselves.

There is definitely something "magical" about watching a live performer play but for a studio recording I genuinely can't think of any rational reason why the OP's scenario would detract from the music (provided, of course, that the captured MIDI is accurate to within a margin of error beyond what we can perceive - which I'm assuming is the case.) When I listen to a studio recording I want to hear the music, not the performer. I only prefer certain performers because of their interpretation. But maybe that's just me.

[fftransform]

I've encountered a couple of Disklaviers in airports/hotels playing back what I assume were live performances input into the device.  It's been a long time, but they sounded fine as far as I recall.  I think the problem is software; I don't think there's an interface for editing the MIDI directly that would give the engineer all the tools they need.  I am thinking of stuff like Finale or Sibelius, where there is no practical (like, not even close) way to adjust the individual dynamic or duration of each pitch (or within a given chord) over an entire passage, or apply various sorts of rubato/arpeggiation/etc in a systematic way.  Basically, atm it'd take more work to build a performance from scratch than just do it the old-fashioned way with a real pianist in a sound studio.

I know that IRCAM has developed some programs with graphical interfaces that can adjust a lot of those dynamical/phrasing aspects at the 'sheet music playback level' for writing spectral music, but those programs are really difficult to use and often need mathematical inputs from the user.  Something that was even more graphical (like adjustable mixer waves corresponding to different phrasing aspects or tempo shifts), and let you interact with the sheet music instead of code more along the lines of Finale, would make it doable.