All twelve positions of any subset should be equally accessible, aurally, visually and haptically; anything less will hamper musical flow. If black keys are a worry then the barrier is probably just physical. Fortunately, the solution is easy. Whenever you learn a new subset (scale or chord), practise it around the key circle. Do this in creative ways, not necessarily making exact transpositions. While that is good practice mentally it can result in pretty dull music. Better to let your mind run free within each key, using such motifs as occur to you, playing within C for a few seconds, then F, Bb, Eb and so on around all the scales. Do the same with the minors. Make the material simple to start with, little interlocking single note phrases in each hand, imitation baroque except with freer rhythm perhaps, whatever you enjoy. After a few weeks of doing this, all the keys will have sunk into your memory in an enjoyable way.