However, I have switched with my ex before. Then my week come up and she say "umm, no, I don't remember that". OK, so she burned another part of the bridge.
So yes, if simply switching is an option 100% agreement. If not, then I would enforce the schedule already in place. And if mom has a stink about dad being gone, then mom does not need to know he is gone since it leads to interference. JMHO