Yes, he may be able to do that. If a divorce case is started here in the U.S., that doesn’t mean that the case cannot be moved to a foreign court later. In general, the state in which the kids have lived for the previous six months is their “home state”, and is the state where decisions will be made about custody.
If your ex-spouse lives in another country, absent any unusual circumstances, the court may order that the children be allowed to travel to that other country to visit him. The court will seek to determine what’s in the best interests of the children, and that generally means access to both parents.
In some situations, the other parent who is residing in the other country may then ask the courts in that country to rule on custody. Courts will generally seek to have the case heard in the geographical location that can best provide the court with accurate information about the children. If the kids have been living with your ex in Switzerland for six months, have begun attending school there, and have other ties with the country, courts may consider that other country to be the best place to hear any subsequent custodial challenges.
In some cases, a state will step in to assume jurisdiction to hear a custody matter, even if it’s not the state where the children have lived recently. If a child is present in a state and is in an emergency situation, a court may step in to hear the case even if it’s not the home state of the child.