I agree with you a bit, but its like getting over a hump. Once you get into it and it becomes routine it is not much harder for us than a youngster.
I see plenty of out of shape 20 somethings, who are huffing and puffing up the stairs, even thin ones. I dont really assume it is easy for them to jump up and get fit much quicker than us. Of course, they dont have all the aches and pains.