Tera, did you have custody of the children? By that I mean did it say in your court order that you had custody of the kids? (The court order that was in effect when you handed the kids over to their father) Or did you simply assume that you had custody because the kids lived with you?
What were his allegations for the restraining order? Were you served any papers indicating a court date?