I agree with me wrote:
Maybe he's an only child and therefore knows his father is not a real man. He doesn't want to associate with someone who's not a real man.
Well, that would be a failing on his part (not his father's part). You can still respect a person who is not a real man, especially if he's your father. You can also disrespect your father if his actions lead to that (abuse, not being there, etc.).
This real man thing is a big thing for you. No need to feel insecure brother. Either go meet the criteria or decide the criteria I've set doesn't need to be met. I'm just a guy with an opinion. When you bring it up again and again, it appears you're being all defensive about it. No need brother! Either go meet the criteria or wash your hands of me and my comments!