Overall calorie deficit does matter, but my experience has been that after a couple of days of the body settling back down, the damage doesn’t seem to be so extreme.
Again, in my experience, is to just park the bad days - the issue is the temptation to turn one day, into two/ weekend, into a week and then “start again Monday”. Write off a day and move on is the key.
If a bad (good?) day messes up a week in terms of goals, it’s only a week. Get back on track - the trend is your friend not day to day or week to week.