I would contact a family law lawyer in your area to find out the answers to this. As a mom you continue to be there for the children and become used to being their sole parent legally.
Why do you ask? Is he saying he wants to give up his rights?
Quoting xtwistedxlovex:I've always heard unless there is a man willing to adopt them they won't cut paternity even if that's what both parents want.
If he wants to give up his rights, he needs to go through the court in which custody was established. You should be notified of the hearing and be able to speak at it.