Maybe if one's used to hearing the more "formal" sounds of Beethoven or Brahms they won't be accustomed to hearing the eccentric improvisatory-sounding nature of a lot of Liszt. From what I've read, he was one of the pioneers of an emerging "virtuoso" style of music in Paris in the early 19th century, which Beethoven's circle didn't dig, or something.
But Liszt has many fans, and he's rightfully at the top of the pantheon. I don't really mind those comments for the most part because for every hater there will likely be a fan, so he doesn't really need that much defending imo.
Personally, even if his middle period works are mostly some of my most-loved piano pieces, I'm not that big a fan of many of his early music, particularly the transcriptions. I feel that while he was an excellent transcriber, his real genius lay in his original works. I believe I've heard that these transcriptions were mostly showpieces which he sometimes performed to rake in cash.
Due to hearing his Apparitions (and the first teenage version of the Transcendental etudes; they deserve more attention in my opinion), I'm somewhat convinced that it was not age which turned him from showman to compositional great; he was a good composer all along, it just depended on what he wrote for (bravura arrangements as opposed to his original avant-garde works). He was inspired by Catholic philosophy from what I've read, and in his teenage years after his father's death he met with a monk (or a spiritual of some sort) who composed in a remarkably novel, avant-garde style free of compositional conventions. It's from this man that it is speculated that Liszt gained this aspect of him as a true Romantic and philosophical composer.
Since Liszt is a favourite of yours, might I suggest this documentary of his life, from childhood to his passing. Unfortunately it's in Hungarian and there are no subtitles, but it might be of interest to you (if you've not come across it before).
Some of the characters feel spot-on, at least with how I imagine their real-life personalities to have been; his father Adam as the demanding, obsessive stage-father; and the elderly Liszt's actor is spot-on in my opinion. How he dressed up as a Hungarian nobleman in a concert; his masterclasses; and so on.