Warm water to warm the hands -- but then make sure that they are really truly dry, or they'll be colder than they were before. I have a set of fingerless gloves -- really just cover the palm and base of the thumb -- which go well up my lower arm. They help a lot. And I make sure that my body, as a whole, is warm -- whatever it takes. If your body isn't warm, or is trying to stay warm, your hands will certainly not be warm. If your practice space is genuinely cold -- say 60 or so (which mine is) you will need at least a heavy sweater or heavy sweatshirt, and a warm hat.
One does wonder what organists (like Bach!) did in the bad old days -- the buildings they worked in were quite unheated...