Pointing to anomalies does not negate the general principle. Biological women and men are different. Yes, there are anomalies and exceptions but these should probably be dealt with on a case by case basis, as they are RARE.
Let's talk about dogs. Mature male dogs have certain physical characteristics and behavioral traits. They are bigger, leaner, more muscular, and generally are very active and seek out female dogs in heat. Female dogs are slightly smaller, have more body fat, and go through heat cycles. They are interested in male dogs only during the estrus phase of their heat cycle. No one who works with animals would deny this. These differences are biologically ingrained. Occasionally you encounter anomalies among animals too, but absolutely NO ONE would deny this general rule because there are rare exceptions.
Humans are also animals. We are at least in part the products of biology. To deny this is to deny reality. A human adult male is not a woman, no matter how much he thinks he is.
It is not transphobic to say that transgender women are not the same as biological women. It is reality.