First thing you are doing wrong is thinking that you have a bad memory. It's perfectly fine that you can not memorize the melody in one hearing. Not many people can do.
You have to understand that this is a skill that can be improved.
What you are doing wrong is focusing on it too much. Do not write down what you hear, thus you are telling your mind that it can not memorize and you have to have a note. So, in small paces, with shorter melodies, say phrases of 2-3 bars, try to practice them with a recording and listen to it 2-3 times and than write it down. After that, you will reduce the amount of times you will hear to the phrase, then you move on to larger phrases and so on.
Btw, how is your memory with memorizing whole pieces? How do you memorize them? Easily? Do you find it hard? Do you always dependent on the scores?