I think that the problem with your question is that you can only control the first part of it: You can choose to do something you are fairly good at, or you can choose something you love (assuming they are mutually exclusive – which is by no means a necessary assumption).
However you will have no control or even knowledge about fame or lack of. And I am taking fame here in its most broad definition: a significant contribution to any field.
Take Van Gogh. As he died, if he had any concerns towards fame (as a contribution to painting in particular and civilization and culture in general), he probably died feeling he was an utter failure – and so did most of his contemporaries. Did he love painting? I don’t know if “love” is the word here. He certainly could not leave it alone. He had a compulsion to paint. He never sold a single painting, and lived and died a poor man, mostly supported by his brother who never lost an opportunity to point out what a parasite he was. He had no idea, and could have no idea, that a couple of months ago one of his paintings would become the most expensive painting ever sold.
Take J. S. Bach. As he died, the style so dear to him was dying too. No one was that interested in his music. Even during his lifetime only a few musicians really appreciated what he was up to. Yet he couldn’t leave it alone. Most of his keyboard work was done for his own pleasure – something quite unheard of in his time – musicians them composed by request. NO one requested him to do the WTC. He did out of his onw interest. At the time of his death, did Bach had any inkling of what the future would hold in store for him? I doubt very much. He pursued his art without much – if any – thought for fame or posterity.
These two examples (almost every musician or artist of stature can be named here as further examples), should suffice.
Your question cannot be posed because there is no way to know. The choice is not between doing something you do well and can become famous through it, or doing something you love and dying in obscurity.
The choice is simply between doing something you love and doing something you don’t love. (Assuming of course that you have such a choice. Many times one has to do something one hates – paying taxes comes to mind)
All the rest is simply self-justification for timidity.
Best wishes,
Bernhard.