You are definitely doing something right! :) Don't be so hard on yourself; be patient, it will happen. And, Never compare your progress to a man's. They are biological machines compared to a woman's chemistry! LOL!
If I may, I would like to ask you a few questions...
What is your exercise routine like? Walk me through the average week (weeks, if patterns change).
You said you need to start keeping a food log. Have you done that? What are your eating habits like?
Oh, and... have you had your body fat percentage calculated, and measurements taken?