It sucks being anchor bass huh ....

I was a snare for 3 years and I didn't really notice any change in my fingers' ability. Though I did significantly notice back problems afterwards, so sitting at that hard piano bench is a constant ergonomic reminder.
As a matter of fact, I almost felt that after practicing drumming for a while, I was better at the bench, mostly because the vibration loosened up my fingers.
The basses at our school had harnesses that went symmetrically around the back- I don't know what the deal is with your harnesses... you should talk to the teacher. That isn't good to be putting all the weight on one shoulder.