It’s likely not just the watch (although wearing a dive watch with a suit is never appropriate).
In any industry there is an image you must project. Every professional wears a uniform, but some are quite subtle. At least to outsiders.
Sounds like you are not really someone with the kind of background they are looking for, and you are quite lucky that you didn’t get hired or you would have been miserable.
Given all the details involved this is not really something with which internet forums can help. Perhaps your tailor can offer some insight into what exactly was off from the perspective of suitable dress for a banker.