I do this every once in awhile and I have no issues. I bring something to read or have some work with me so if I don't feel like chilling - I'm good.
I'd say every one in 10 I feel lonely about it.
I just went on a cruise and we had a party of 13. But there were a bunch of sad singles on the boat. I really felt bad for them.