In the third measure of this excerpt, the stem directions as well as the placement of rests lead me to believe that the middle voice is still present (considering that some rests are placed above notes and others are placed below). If the bass and middle voices, in the third measure, were written on separate staves, here's how they would look:
Bass voice: D, (sixteenth rest), (eighth rest), (sixteenth rest), D, C, B natural
Middle voice: (sixteenth rest), E-flat, G-flat, B-flat, E-flat, (sixteenth rest), (eighth rest)
The fourth measure is more ambiguous in terms of identifying which notes belong to which voice, but from a "motivic" standpoint, I think the answer becomes clear.
Looking at the first beat of the fourth measure (in the excerpt), there is no confusion, because no rests are missing from the page:
Bass voice: B-flat, (sixteenth rest), (eighth rest)
Middle voice: (sixteenth rest), D, F-sharp, A
Soprano voice: A
The D5 and the B-flat3 that appear in the bass staff on beat 2 of the fourth measure don't have rests written above or below them, so it appears that either the bass voice or middle voice has disappeared. However, this is where a motivic analysis becomes helpful.
In the second measure and the beginning of the third measure, I'll consider the motive the bass voice plays: D, F, A, D. In the third measure, this motive is passed to the middle voice and transposed up by an interval of a minor ninth: E-flat, G-flat, B-flat, E-flat. In the fourth measure, a similar motive appears with two slight modifications in intervallic structure and rhythmic structure: the last note of the motive is an eighth note played staccato instead of a sixteenth note played staccato, and the interval between the first and second notes of the motive is a major third instead of the minor third in the original motive. So the third statement of the motive has the same
contour as the original motive; it consists of four
ascending pitches: D, F-sharp, A, D.
So, from a "motivic analysis" standpoint, the D5 on beat 2 of the fourth measure belongs to the middle voice.
In the context of the same measure and the next several measures, the B-flat3 (the B-flat immediately below middle C) could be the bass voice or the middle voice. If you called it the "middle" voice at this point, in measure 37, it would become the bass voice. I've noticed that this happens again a few more times throughout the fugue. So at this point it would become difficult naming the voices soprano, middle, and bass!

In my opinion, the B-flat3 in the fourth measure belongs to the lowest voice, the "bass" voice, and continues until measure 25, when the "middle" voice plays the E-flat4. Then the bass voice comes in again in measure 27 on D2.
I hope this helps, m1469. Good luck learning this fugue!
pnorcks