Paedophilia is not the same as child abuse.
Paedophilia is not illegal, as in its not illegal to find a child attractive, but it is illegal to touch them/rape them etc (obvs).
I think if I knew the person, and they had never abused a child (and could somehow prove it) then I would leave my fictional child with them.
I would happily be in the room with a gay person, but would feel uncomfortable being in a room with a rapist.