If you only play for yourself, why not do what your heart pleases?
If you play for an audience however, know you will be judged (I'm stating the obvious here, but humans are creatures of strong opinions), and quite frankly, a blatant disregard for the composer's intentions will probably earn you harsh criticism.
I think there's plenty room for self-expression, for lack of a better term, in music, while still ahereing to the composer's desires. Take for example, Maurice Ravel, one of the most meticulous composers of all time. Despite being critical of performances down to the slightest rhythmic deviations (I'd like to start a count of the total number of "sans ralentir" in all his music), Ravel often did not specify how he wanted a melody shaped, or what tone color should be used. Often, he would mark measures with a simple "expressif." Now, what does this "expressif" mean? A dramatic gesture? Profound dynamic change? A tiny hint of rubato? In these tiny spaces a performer is free to interpret and express, without disregarding the score.