What are the odds of child being taken?

I have a 6 year old child with cerebral palsy and a rare brain disorder, when she was born I had custody of her for the first year, both the "father" and I signed guardianship over to my mother who has graciously raised her for the last 5 years, I have always been a huge part of my daughters life and my husband and my son and myself just moved in to my moms over a year ago to help out and be closer to my daughter, the "father" has chose to stay out of her life, he would see her once in awhile. Last year he portioned to get visitations and joint custody of her, he got his every other week visits, but my concern is the custody, he wants joint custody with physical placement! We were never married, and like i said he signed off and chose to stay away for almost her whole life, what are the chances that he would get custody over my child?? We are from Iowa if that makes any difference in the law?