I married my boss more than 10 years ago. (I am female.) We both figured we were risking our jobs by having a relationship. Perhaps now the risk would be more on him, but I definitely figured I was at the very least risking looking like a fool and a tramp in the office. But I also knew he was special and that our attraction was real. It was a small company, so the only real option was to leave, which one of us eventually did. But we dated at work for a long time. Neither of us told anyone at work, and I do mean no one. We didn't do PDA. Not just at work - never. We didn't communicate anything unrelated to work on work computers or really even talk inappropriately at work even in private. It also really meant we couldn't have any close friendships with coworkers, since we basically had to lie to everyone about our relationship.
Worth it. We have a few kids now and risking my job to date and marry him was 1000 times worth it. He says the same. I don't think it's "immoral" to fall in love with your boss / subordinate and follow your heart. Unprofessional - definitely. Risky - 100%. But quite possibly worth it.
The original poster should probably wait six months and make sure he and this woman really have a connection. But people are so uptight. Real life is messy but it can be wonderful.