Certainly it's true that some Romantic music can be self-indulgent, like some Romantic poetry. But just as in poetry I wouldn't let distaste for Wordsworth ruin Keats or Tennyson, likewise in music I wouldn't let Schumann ruin Beethoven or Brahms (and even Schumann does something right once in a while). If a composer represents his emotional experience well, and if it's reasonably universal, then what's wrong with that? Music about one's self isn't self-indulgent unless it makes no attempt to communicate with anyone else, or uses an idiosyncratic language with no meaning except to the composer. We're all human, so a musical expression of a composer's individual human sorrows can resonate with the listener. To go back to poetry, do you think that all lyric poetry is self-indulgent? Everything from Sappho to Li Bai to Sylvia Plath?
As others have pointed out, beauty is not necessarily the objective, even among composers in the periods you like. I think Vivaldi and Chopin really aimed at beauty as a primary goal. But Bach sure didn't. For him, beauty was a welcome side effect of exploring relations between lines and the contrapunctual potential of different themes. There's lot's of beauty in A Musical Offering, the Art of the Fugue, and the Well-Tempered Clavier, but it's a by-product, not the goal. If anything, Bach was, at least in those works, not all that interested in how they'd appeal to listeners, but in how they revealed his skill. Beethoven also seems often not to aim at beauty and sometimes not even particularly at the expression of his own feelings. Instead he seems to be interested in what you can do by breaking a musical idea into its smallest components and rearranging them, or in setting himself problems of how to get back to the tonic from a remote key at the end of the development section in a sonata form movement.
Plenty of Romantic music doesn't much appeal to me either, but to write it all off as self-indulgent goes a bit far, I think.