it is your responsibility to force the kids to go even if they do not want to go. Your failure to do so leaves you open to accusations of PAS. In other words, he will claim that you are "brain washing" the kids against him.
All that you can do is document his behavior. By all means, talk to your lawyer. The lawyer will have a better idea of how the judge will respond than anyone else.
