We know that across the board, regardless of sport, women have a higher fat percentage than men. The normal fat percentage for women is about 18-20% of body weight, compared to 10-15% for men.So there's that.What's more, your observations seem to be borne out by an Australian study on swimmers:
Swimmers, especially female swimmers, face an energy balance conundrum. Elite swimmers typically undertake 4000-20,000 m per day in training, burning thousands of calories. However, the typical body fat levels of these athletes are significantly higher than runners or cyclists who expend similar or even smaller amounts of energy in their training. Many female swimmers have fought well-publicized battles with their body fat levels and with their coaches! They are generally prescribed "land training" (running or cycling) in addition to their many laps of the pool in the belief that it is a necessary treatment to produce lower skinfold levels."
http://www.sportsci.org/news/compeat/fat.htmlPossibly, this might have something to do with the fact that men burn fat more easily in exercise than women, and women depend more on fat stores when they exert themselves physically - and in fact (counterintuitively) sometimes they gain fat through exercise:
http://www.ncbi.nlm.nih.gov/pubmed/11706283