I'm walking proof of the fact that there's a bit of a gap - a gulf in some cases - between being able to sight-read a piece and being able to play it. I can sight-read Alkan's 'Quasi-faust', for example, but the fugal section is a garbled mess every time I play it. (As I don't study piano formally, this is unlikely to change in the near future.)
The usual adage applies to the preludes, though - which ones are difficult depends on what you can and cannot do well. (Taking the etudes as an example, I have large hands, so Op.10 No.1, No.9, No.11, and Op.25 No.4 aren't as difficult for me as they are for some, but I can't play extended passages of double thirds or double sixths to save my life, ruining Op.25 No.6 and No.8 for me.) Though the list of most difficult is usually topped by the F-sharp minor, B-flat minor, E-flat major, and D minor preludes (the aforementioned 8, 16, 19, 24), if you have the right technique they can actually be easier for you than some of the others. If you struggle with repeated notes, for example, the G-sharp minor will be hell on 88 keys for you, while the sheer unpredictability of the D major packs a lot of trouble into such a short piece. For whatever the Associated Board rankings are worth, I've seen the F-sharp major and A-flat major preludes turn up on the Grade 8 syllabus in various years, though this may have something to do with their length (in most performances they are exceeded only by the D-flat major prelude, which is not a particularly challenging piece next to some of the others).
There's no question that the E minor, A major, and C minor preludes are the easiest technically (making them not sound insipid is not so easy), and the B minor and E major preludes are, indeed, not much more difficult, so you could certainly handle them easily given what you already know. The E-flat minor and B-flat major preludes might be a good next step, along with either the F-sharp major or D-flat major preludes (in fact, you may want to try one of those first so you have something really substantial under your belt). Definitely leave the F-sharp minor, B-flat minor, E-flat major, and D minor preludes until near the end.