Yes, there are smokers in Japan. And, contrary to popular belief, it’s not just foreigners doing it. As the penalty (and social stigma) is quite high for being caught with it, people are much more careful about possession as well as disclosure, keeping it a private matter. However, like most countries, go to the right nightclub (Shibuya or Roppongi) and you’ll find a source… just be cool about it. Being a jackass will definitely get you ignored.
You can also find young Japanese people smoking synthetic cannabinoids (called haabu, the Japanese pronunciation of herb). It used to be easy to obtain for many online, which doesn’t require meeting a sketchy dealer. But as of 2012 it’s also now illegal here. Possessing enough to considered a dealer comes with a five year prison sentence or a 5 million yen fine.