I think your friend, if not correct, is certainly on to something.
Let me give it a perhaps slightly different slant to it.
All around us there are what I call “open secrets”. Knowledge that would of great interest to us and that would definitely be in our best interest to have. From how to heal ourselves from any disease to how to perform the most complicated piece of music. When we hear that such “secrets” exist, we want them revealed to us. And we want them right now. But they are a special kind of secret. They are “secret” not because someone is purposefully hiding them from us, and we depend on this someone willingness to reveal them. They are “secret” because they demand some kind of effort on our part . They are open. They are there for all to see, and yet no oe can be bothered to look.
Take juggling, for instance. Most people who decide to juggle try it for a couple of minutes and give up in desperation: “This is too difficult, I will never be able to do it”.And the difficulty/impossibility is indeed real. Yet – and paradoxically – if you persist (I never met anyone that could not juggle after 15 minutes of guided effort), the difficulty vanishes surprisingly quickly, and you cannot even understand what was it that you found so difficult in the first place.
Or take reading. You cannot even begin to explain to an illiterate person who lived isolated in some rural area all his/her life why they should bother to learn to read. “Why should I learn to read?” Any advantage you may come up with is understandable only to literate people. The illiterate person will have to trust you that great benefits will accrue from his learning to read. (Which is why our very literate society makes reading compulsory for little children). Reading is indeed an “open secret”. No one is hiding it from you, in fact everyone is putting a lot of effort in trying to reveal it to you. Yet it will remain a secret unless you put some effort into it. And since an illiterate person cannot even fathom why they should put any effort toward it, the chances are that it will remain a secret.
It is as if there was a protection built around certain kinds of knowledge to avoid the unworthy to get it.
Music is such an open secret. Unless you put some effort into it, it will not yield its secrets.
Most people do not realise that the universe is multi-layered. As far as we know it could even be infinitely layered. Some people are happy to live in the first layer, the world of appearances: what we see, what we hear, what we taste what we smell, what we touch without realising that sensory perception is but a model of a reality that continually eludes us. Most people have even lost touch with this first sensual model and live in a world modelled by language, stuck forever in second hand verbal descriptions.
I always find amusing all this talk about how no one goes to concerts anymore, how one does not buy classical music CDs anymore and how this is going to be the end of classical music.
In fact, this will be the end of a few people making a lot of money out of it.
What keeps music going (classical or otherwise) is not the public. What keeps music going is the performers/creators.
There is a huge amount of people right now quietly playing the piano. They don’t make a big fuss about it, they do not perform in public, sometimes they will not even perform for anyone. They just play for themselves and do not tell anyone about it (I have several students in this category).
These are the people who actually guarantee that certain pieces remain in the repertory. If you want scores for some obscure pieces from some obscure composers, you better start playing it, since no publisher is going to print obscure music for Hamelin to play.
If you are a football watcher, you will experience a certain level of the game. The level that is open for all to experience. But if you actually play football, new layers of the game will reveal themselves to you. But playing takes far more effort than just watching. In fact most football lovers do not really “like” to play football, and would rather just watch it. Which is great for business, but not for your diabetes/heart condition.
Likewise in music, one needs to play a musical instrument to start to have access to its open secrets. But that requires effort. And no one wants that.
Music is not really for the masses. Music is for a few. This should not depress anyone. If you are not one of the few (evidenced by the fact that you are not willing to put the effort and enjoy that effort), just do something else. And if you are one of the few, why should you be depressed?
This is like a lot of (religious and otherwise) preachers, constantly demanding that you conform to their beliefs, when in most cases, they themselves cannot live up to it. In fact, it is a sure sign that one cannot live up to one’s beliefs when they start insisting that everyone else should conform to them.
So it is not really a case of liking or disliking music. It is more a case of not wanting to put in the effort. Like the illiterate they cannot understand why they should learn to read. And they never will since this understanding requires your ability to read in the first place.
In short, why should you care if people like music or not, if they want to put an effort towards it or not? It is not really your problem, is it? Concentrate instead in your liking of music, on the effort you are prepared to make towards owning that secret.
It should keep you busy for a while (and busy people do not get depressed).

Best wishes,
Bernhard.