Fun fact: Franz Liszt was the first pianist to perform concerts entirely from memory, and it's because of him that performing pieces from memory is standard practice today.
I think music has to be absorbed, then it literally becomes a part of you. It is stored somewhere inside the brain of ours. When it becomes organic then it develops freely through your creativity. When you have sheet music as ur security blanket, or guide even, you are neglecting full focus on sound production since there is that concerned about what the notes are actually doing on the page.