Well, Most of these markings were written by editors, Not by the composers themselves. It is simply one interpretation of the marking that the composer wrote. If something is marked allegro to you? What seems adagio to you? Even in some cases, a composers' vision for tempo is not really doable for the average performer (see Schumann Sonata No. 2). So you pick! Most people play too fast in general.
And here's me thinking that many pianists play too slow today, compared to the older recordings I tend to like Compare for example Chopin's 4th ballade which is played in 12-13 minutes by some modern pianists and in under 10 by Cortot.I think metronome marks depend a lot and cannot always be trusted. Many of Chopin's metronome marks are reasonable if difficult, while others just seem way too fast. Sometimes they were put there by the editor. Sometimes they just seem to have been set at a slightly too fast pace to guard the player against taking a too slow tempo. Sometimes it's just weird.
Timing has nothing to do with it. You can play the 'fast sections in a piece at lightning tempo, and take time between phrases that make it take the same time as someone else who plays slower but takes less time between phrases. Besides, speed is relative. Which feels faster? Quarter note = 160, or half note = 80? They are the same speed, but different internal tempos. You can cram lot of 'little' notes into a small space, as with the cadenzas in Ricordanza, or you can play those same cadenzas with inflections on every 3 or 4 notes. One results in a spacious, airy quality, while the other results in tightness and intensity. Sometimes, as with Chopin Etude Op. 10 No. 4, you want the strictness of tempo. But even in the Chopin, you should be considering the hypermeter--the longer beats at the start of each measure, every other measure, etc. This is what is missing from many pianist's playing, and as a result it yields a stressed, metronomic quality that is rarely enjoyable to listen to.