Quantum is right. At twenty, reject nothing. Plenty of time to start picking and choosing over the next half-century.
It is remarkable that you should be attracted to Ives, and an indication of a broad and receptive spectrum of musical response. Never question or modify the validity of your own response to suit what other people might say or think. On the contrary, treasure it; when you are older your independence will repay you a hundredfold. No particular tradition has grown up around Ives so you must play him according to your own feelings every inch of the way, which is a dynamic adventure in itself.
Also, if you feel driven to create your own sounds, improvisation, composition, then begin to do it at once. Don't leave it "till later". Everything feeds everything else.