And the short, simple answer is that regardless of activity level, energy needs will increase along with body mass, so that an energy surplus on day 1 might not be an energy surplus on day 1000 with the same diet, even if activity level decreases.
I just read the question as 'What is it that compels me to eat exactly x number of calories to always go back up to the exact same weight?'
It's a good question and one I find fascinating. I don't think there is one single answer to it.
"Eat as if your life depends on it!"