Yes, it is frustrating. Results only come through persistence over time. Results that are slow and steady are the ones that will stick. But you know, you really do NOT want to eat whatever and balloon back up because you have already been down that road. You know that won't make you happy in the long run.
We have a choice to think of our bodies as chemistry labs or temples. If it is a chemistry lab then the goal is to do whatever you can to it to make yourself feel good, no matter the cost. If it is a temple, then you show it respect.