Did you Know???

[lostinidlewonder]

- Did you know AI is getting more clever? Here it is representing what piano playing looks like. It actually took me a moment to realize the AI was an animation lol.

[j_tour]

- Did you know AI is getting more clever? Here it is representing what piano playing looks like. It actually took me a moment to realize the AI was an animation lol.

Well., I wouldn't say you're exactly wrong, but there's such a wide gulf between generalized AI (an impossibility without a robust ontology) and little automaton tricks like playing and composing little things.

For people who work in AI, there's no question that having little tricks is a long, long way off.

Maybe thirty years, if that.

And, no, that's not my opinion:  enthusiasm without ability tells no lies.

Remember Koko the gorilla or any number f simians or various elephants?  Mr. Ed?  Clever Hans?

These concotations may fool some illiterate rubes, but it is not there.

For starters and finishers we don't have anywhere close to an ontology for even restricted domains, such as biomedicine.

[j_tour]

dp

[lostinidlewonder]

Well., I wouldn't say you're exactly wrong, but there's such a wide gulf between generalized AI (an impossibility without a robust ontology) and little automaton tricks like playing and composing little things.

Yes for sure. This is just animating finger numbers to notes and processing it. I really wonder if this AI work in the way that it is given an audio file and it knows which fingers to use, that would be quite impressive. I feel however you'd have to input all the finger numbers and attach them to some kind of midi file. I guess the AI part is the processing the video to make the movements? I'm not computer expert lol. Some parts are quite convincing, others are alien like lol.

[j_tour]

I feel however you'd have to input all the finger numbers and attach them to some kind of midi file. I guess the AI part is the processing the video to make the movements? I'm not computer expert lol. Some parts are quite convincing, others are alien like lol.

I think that's possible — in fact, I'm almost 100% sure that a computer using what are a broad class of machine learning techniques called "supervised training" can handle the optical or aural challenges, if they're fed enough data that's been segregated into appropriate categories.

I'm not an expert, but I have had quite a bit of formal training in computer science (sort of an abortive second career).  However, the field of CS is vast, with a lot of specializations.

I'm not a fan of using "AI" or even "machine learning" as umbrella terms, since in those areas as well, there are quite a number of different techniques used (although they tend to be related, and I'd assume computer engineers or computer scientists are aware of the broad strokes of the different tools used).

[ranjit]

Finally something that's up my alley

I'm pretty sure that they have used an unsupervised deep learning model to come up with these results. Conventional (such as supervised) approaches which I've seen in robotics usually aren't that effective. Recently, there have been a flurry of papers which use deep learning models for similar problems.

Here, the video is being rendered from the raw audio. See the other video where they simulate Rachmaninoff playing a Rachmaninoff piano roll. They have another version of the software which recreates the hands from a MIDI file. You can see the difference -- there are basically no note errors when it comes to the MIDI ones, but there are some very prominent mistakes in the ones in the above video (see the hand crossings in the Chopin ballade). What this suggests, to me, is that the problem of rendering the piano notes from a recording hasn't been solved perfectly yet, and is one of the bottlenecks for this piece of software. However, it's still very accurate, just less than a human doing the transcribing.

I think we will get to the point of having accurate transcription software someday, perhaps 10 years into the future.

[lostinidlewonder]

I'm pretty sure that they have used an unsupervised deep learning model to come up with these results.....Here, the video is being rendered from the raw audio.

I think that is quite brilliant if AI can suggest whch fingerings work and I am actually quite confused how it could do that since multiple fingering options are always on the table, it seemed in the video above that it actually got it correct the majority of the times. I didn't slow down the video to check exactly but overall it looks quite ok.

There would be several technologies going on all at once if this AI really is determining the fingering through an audio file. First you need to be able to transfer the audio file into some kind of midi or sheet music signal, there are programs for this (google Audio to sheet music, programs like AnthemScore are quite good). This process however has error which alway crops up. Another process they would need is some kind of AI which can analyze the notes and come up with the efficient fingerings to produce the sound. This to me is even more astounding because it requires quite a lot of flexible knowledge and selecting between a set of possible solutions, to draw those conclusions from an audio file is bewildering. I totally get using MIDI files and inputting the finger numbers yourself, this process is not so surprising, but getting the AI to actually come up with human fingering combinations that will present the audio file, that is quite alien technology. This would produce a program which could suggest fingerings for any sheet music, something that I have not seen at all anywhere at the moment.

[j_tour]

I think that is quite brilliant if AI can suggest whch fingerings work and I am actually quite confused how it could do that since multiple fingering options are always on the table, it seemed in the video above that it actually got it correct the majority of the times. I didn't slow down the video to check exactly but overall it looks quite ok.

Well, it seems like Ranjit knows more than me, but I don't see any other way than supervised learning.  A support vector machine:  that's pretty basic, for example.  The basic idea is to create a "line" between good and non-good sets of, whatever, notes or fingerings.  It's an algebraic technique, pretty much.  The OCR is pretty much a solved problem.

This is not my expertise, so I hope Ranjit will elaborate a bit more.

There are a few other options, but let's keep it simple.

[lostinidlewonder]

It is just that there are so many options in fingering how does the AI know which one to use that produces less tension in a human hand? It must be fed a lot of fingering data but that is not really AI, how it manipulates that data in an intelligent way to solve different problems however is the AI part though I feel it will be with numerous errors, I hope to be proved dead wrong.

If we take a simple situation, say Mozarts Ronda alla Turca the Rh opening figure of :
BAG#AC,  DCBCE,  FED#E, BAG#A BAG#A, C

The appropriate solution is:
43213, 32124, 4321, 4321 4321, 5

I would be surprised if the AI came up with this movement even though it's a pretty simple situation it is a solution which benefits the human hand. Some humans would tend towards repetitive fingerings such as: 43213 43213 etc but this all uses the thumb in such a way that it provides less tension to the human hand. A robot doesn't feel anything in their hand so they could solve it like: 43235, 43235 etc and if it is programed to not care about the thumb being on a black note it could even choose: 32124, 32124 etc. So it might have a large database of fingerings inputted into it which it can draw from but if it then can intelligently use those to solve different situations in a manner which provides comfort for a human hand that would be quite astounding.

[j_tour]

If we take a simple situation, say Mozarts Ronda alla Turca the Rh opening figure of :
BAG#AC,  DCBCE,  FED#E, BAG#A BAG#A, C

The appropriate solution is:
43213, 32124, 4321, 4321 4321, 5

I''ll still wait for Ranjit, but I doubt it.

That sounds like what's called an "expert system," which is a subset of supervised training....actually not. 

That's more like how one creates what's called an "ontology" in the computer science sense, rather than the philosophical, weedy sense.

It's more like a database.

I'm not sure what the confusion is on your part, but I'm fairly sure I'm not wrong.

[ranjit]

I don't really know too much about it. Conventional machine learning algorithms usually don't give you exceptional accuracy. You need to manually create classifiers, and from what I've seen, that approach doesn't usually give really good results like the one in the video. In addition, I know that deep learning has kind of taken over computer vision and robotics over the last 2-3 years, and there have been a number of exceptional results. Given that these kinds of results were almost unimaginable 5-10 years ago makes me feel like they are not using conventional ML algorithms.

https://arxiv.org/ftp/arxiv/papers/1910/1910.13796.pdf

I imagine a classifier would work if you simply want to determine the fingering (side note: making an algorithm to figure out fingering seems like an interesting project and it looks like it could be done). But here, the actual video is being rendered, which is using some kind of virtual hand. So, either the hand has been reconstructed from a physiological perspective, or real-world image data has been mapped onto the hand. I'm leaning towards the latter.

I also found this paper, where they have used a random forest, which is basically what I think j_tour said. I've seen something similar before where there was a 2015 paper to solve a problem, and then another paper in 2019 which used a deep learning-based approach which blew it out of the water in terms of accuracy.

I don't know too much about these approaches, though, other than the fact that they work -- I mean, you can look up the model, but it's the nature of these algorithms that you don't know exactly what's going on, after you feed them the initial parameters and so on.

https://dl.acm.org/doi/10.1145/2856400.2856411


And to your point, j_tour, I think the random forest idea could work for simply finding out the fingering. It might even be possible to make a deterministic algorithm to give suggestions, based on rough distances and hand shapes, where you shift hand positions, etc.

[lostinidlewonder]

I'm not sure what the confusion is on your part, but I'm fairly sure I'm not wrong.

The confusion is how can it choose the correct solution when there are many possible solutions.

[j_tour]

The confusion is how can it choose the correct solution when there are many possible solutions.

I see.  Apologies if I came off a bit snarky:  not my intent.

I am a bit in over my head compared to ranjit on this topic.

But I could offer an analogy as to how and why raw computation might fail, in the sense of following a strict algorithm with control structures like "do....while" or "for every x less than 100, repeat, then stop."  Like, why is it that computers often fail to compute definite integrals, while differentiating a polynomial expression or anything else is simple.

That's not a great analogy, and it doesn't answer your question.

So, I guess I'm as much confused as you are on this point.

I can think of a few methods that might work in machine learning, but I don't want to throw a bunch of jargon around, especially since I don't fully understand how to implement a solution, from soup to nuts.

I could look it up, but that would be cheating!

[ranjit]

But I could offer an analogy as to how and why raw computation might fail, in the sense of following a strict algorithm with control structures like "do....while" or "for every x less than 100, repeat, then stop."  Like, why is it that computers often fail to compute definite integrals, while differentiating a polynomial expression or anything else is simple.

I mean yes, you're saying that it's not clear that there is a deterministic algorithm for fingering (differentiation vs definite integrals), but I think that if a large number of expert pianists sat down and collated their insights, I'm pretty sure you would get something which would work in over 99% of cases. Part of fingering is like calculating chess moves, just look a few steps ahead to where your fingers would be. That would probably be easy to code. And then you give everything weights in terms of how effective they would be. So if you have a weird configuration (e.g. in Ravel) and it overcomes some kind of "thumb-black key" barrier, that makes it more effective to use the thumb on a descending scale on a black key. Kind of like Tessitura for chords, perhaps?

Given all of this, I think you could get a pretty decent algorithm for figuring out fingering.

Anyway, probably everything I'm saying here is kinda obvious!

[lostinidlewonder]

I see.  Apologies if I came off a bit snarky:  not my intent.

I don't mind snarky, I get a lot worse from people lol. I didn't read what you asked as anything negative.

Given all of this, I think you could get a pretty decent algorithm for figuring out fingering.

This part I am really interested in because it will produce programs then which can help anyone with any pieces they are practicing and offer optimal fingering solutions. Of course error must come into it but I wonder if that error is like in the example I gave, where it choose a suboptimal fingering or fingering which does not understand what comfort is in a human hand. The idea of coming up with the correct fingering that suits the hand is intriguing to me, it would be even more interesting if you could input all your hand dimensions and the suggestions change based on what you can physically reach. Of course this is probably beyond the scope of what this program wants to achieve but it wouldn't be too far off.

The idea that an AI can solve piano fingering would be 100 times more important than this video representation I reckon.

[j_tour]

Of course error must come into it but I wonder if that error is like in the example I gave, where it choose a suboptimal fingering or fingering which does not understand what comfort is in a human hand. The idea of coming up with the correct fingering that suits the hand is intriguing to me, it would be even more interesting if you could input all your hand dimensions and the suggestions change based on what you can physically reach. Of course this is probably beyond the scope of what this program wants to achieve but it wouldn't be too far off.

Yeah, here I follow your thinking:  that is not obvious.

And, pace ranjit, look-ahead computations for games like chess are not at all obvious.

Consider that checkers (the board game) wasn't solved until....I think 1996 or so, with an implementation called Chinook, if I'm not mistaken.  Or, pretty close to about then.  And checkers is a fairly easy representation to translate into numerical terms:  it's a difficult game, but it's almost like solving a system of linear equations.

I mean yes, you're saying that it's not clear that there is a deterministic algorithm for fingering (differentiation vs definite integrals), but I think that if a large number of expert pianists sat down and collated their insights, I'm pretty sure you would get something which would work in over 99% of cases. Part of fingering is like calculating chess moves, just look a few steps ahead to where your fingers would be. That would probably be easy to code. And then you give everything weights in terms of how effective they would be.

Well, that would be my starting point:  probably create a CSV file of many thousands of inputs, and run it through an SVM engine.

It would surely fail a number of times, but probably work out fine most of the time.

[ranjit]

Consider that checkers (the board game) wasn't solved until....I think 1996 or so, with an implementation called Chinook, if I'm not mistaken.  Or, pretty close to about then.  And checkers is a fairly easy representation to translate into numerical terms:  it's a difficult game, but it's almost like solving a system of linear equations.

Oh, I wasn't thinking about "solving" fingering in that sense. I was just thinking about making an algorithm with sufficient predictive capability that it could suggest ideas based on fingering principles which already exist, given an unfamiliar piece of sheet music. I wasn't talking about surpassing humans or solving the problem completely, but something closer to a pianist friend giving suggestions.

[j_tour]

All right, here is the most unfortunate "Did you know ? ? ? ? ? ?"  ever.

So, I use Bing as a regular search engine because of reasons.

If you try to search for "private joker" and "come over to my house with my sister" (a famous scene from the movie Full Metal Jacket), it gives you impossibly crass results.

I was just looking for a brief clip of that scene with dialog.

And, that is with the adult content filter on.

Turn the filter off?

Unbelievable.

I'm not saying whatever, but it's VERY far from what I was searching for.  It's a strange world out there. 

No, I'm not naïf but I found it difficult to believe that a simple search for a well known scene in a famous movie pulled up such dross.

I mean yes, you're saying that it's not clear that there is a deterministic algorithm for fingering (differentiation vs definite integrals), but I think that if a large number of expert pianists sat down and collated their insights, I'm pretty sure you would get something which would work in over 99% of cases. Part of fingering is like calculating chess moves, just look a few steps ahead to where your fingers would be. That would probably be easy to code. And then you give everything weights in terms of how effective they would be. So if you have a weird configuration (e.g. in Ravel) and it overcomes some kind of "thumb-black key" barrier, that makes it more effective to use the thumb on a descending scale on a black key. Kind of like Tessitura for chords, perhaps?

Given all of this, I think you could get a pretty decent algorithm for figuring out fingering.

Anyway, probably everything I'm saying here is kinda obvious!

ETA, I just reread this, and, yes, that's exactly it.  Why else would one use a supervised learning machine? 

OTOH, Melanie Mitchell and others work in genetic machine learning, somewhat.

It's really textbook level we're talking about, here.  I favor temporal difference learning, and some results verify a certain amount of certainty.  But, it really doesn't matter much if one uses SVMs or TD and any of the k-n means methods of selection. 

Yes, I know about automaton theory and NFAs, but there's no chance for an undisclosed AI to run rampant, and it's only in abstract algebra that one can even give credence to the notion of a construction like a Turing machine, or Church's lamda calculus. 

But, I'm not a professional in CSci, I was just amused that looking for a youtube clip from a famous movie which includes a nice scene about a senior drill instructor punching some recruit in the nads brought up interminable pages of incestual videos.

I disapprove!

Well, not entirely, blah blah Monte Carlo predictive blah blah, but you know what I mean.

[lostinidlewonder]

Did you know the James Webb telescope cost $10 billion USD? On the 24 December 2021
the launch was so accurate that it saved enough fuel to give it a few extra years of life (it was meant to last 5.5years but could last longer than 10 years!!). It will sit in the Lagrange point (~1.5million kms away), where it can remain stationary with minimal gravitational forces acting upon it. Exciting discoveries are on their way.

https://webb.nasa.gov/content/webbLaunch/whereIsWebb.html





Did you know a Googlewhack is a contest to find a Google Search query that returns a single result.

For example if you google search this in quotations:

"Exciting discoveries are on their way"

You will only get one page result

[lostinidlewonder]

Did you know: "Yes, we can see." Is "Yes" in four language?

Yes: English
oui: French
ken: Hebrew
ci: Spanish

[Bob]

Is this true?

[klavieronin]

Did you know: "Yes, we can see." Is "Yes" in four language?

Yes: English
oui: French
ken: Hebrew
ci: Spanish

"Yes, we can see high" = "Yes" in 5 languages.

Yes: English
oui: French
ken: Hebrew
ci: Spanish
hai: Japanese

[lostinidlewonder]

Nice klavieronin.

WHat about: Yes, we can see higher.

Yes: English
oui: French
ken: Hebrew
ci: Spanish
hai: Japanese
ja: German

I guess you gotta say "higher" in an Aussie accent though lol. High-ya

[klavieronin]

 

Nice klavieronin.

WHat about: Yes, we can see higher.

Yes: English
oui: French
ken: Hebrew
ci: Spanish
hai: Japanese
ja: German

I guess you gotta say "higher" in an Aussie accent though lol. High-ya

 

[lostinidlewonder]

If you spell 'socks' out loud, you are saying 'that's what it is' in Spanish. (Eso si que es)

lol

[leigh anne]

Did you know that the piano was first called gravicembalo col piano e forte, which roughly translates to “soft and loud keyboard instrument"?

[lostinidlewonder]

Did you know that in Italian "piano" has many other meanings such as: softly, slowly, quietly, flat, plane, plan, floor? Sono confuso! I am going to play the piano piano and piano and piano and on the piano piano does that sound like a good piano or a bad piano?

[lostinidlewonder]

- Did you know that 'Ignition' of fusion reaction has been confirmed?

[lostinidlewonder]

- Did you know that the Adagio in G minor normally attributed to Albinoni is actually a 20th century composition created in the 1950s, some 300 years after the composers death?

https://www.britannica.com/topic/Adagio-in-G-Minor

[lostinidlewonder]

- Did you know that they used to measure tempo in terms of the inches the pendulum string length was at? I was reading some old music and was wondering why it had inches where the tempo marking was.

https://core.ac.uk/download/pdf/70971434.pdf  (see page 12)

[transitional]

Did you know that ice is colder than water? 

[lostinidlewonder]

Did you know that ice is colder than water? 

Did you know salted water can be colder than ice?

[lostinidlewonder]

- Did you know music therapy can help those who have had strokes?