I wouldn't describe myself as a synesthete, but I do associate music with color. I don't associate entire tonalities with a given color, because there is too much possibility for ambiguity (C major can sound like a minor, F major/d minor, G major/e minor at a given time/progression). I associate colors with specific harmonies with specific notes--transposing it wouldn't be the same color, for example. But most of these "color harmonies" for me are highly complex, not major/minor/dim/aug/seventh, to which my color associations are minimal and subdued. Different voicing of the chord will give me a slightly different experience of color. For example, in the treble clef, Bb C# E F A (6th A maj chord with a superimposed Bb F fifth--it also works with a low G softly played concurrently three Gs down from the Bb, for some reason), gives me a dim greyish purple, a little brighter and more reddish purple if the A is voiced more prominently or if the entire chord itself (but not the bass tone) is louder.
Years ago, inspired by Sorabji's 'tropical nocturnes' (Sorabji is the composer I most associate with color, followed by Scriabin, some Szymanowski and Roslavets), I wanted to write a suite of pieces based on my favorite rare flowers, particularly Aroids. The organizing factor was primarily a sense of color to harmonies. On the most successful there was a sort of interchange of a dark magenta (root on b) and a frosty sort of green. It was something of a maddening experience, however, trying to get the colors right and to keep a sense of the colors without keeping the piece static (it amounted to contrapuntal manipulation of a sonority, with choice augmentations), something I don't want to go through again, especially since most of my compositions are microtonal now, which would just compound the difficulty. When I improvise I am often awash in colors, although it is often their changing that is more interesting than their native, respective colors. Every now and then I do come across a colorful chord that I jot down for future use, because it is so striking.