Smoking while walking in Japan is prohibited
In Japan, designated smoking and non-smoking areas are located indoors as well as outdoors. Local governments throughout Japan set their own laws for designated smoking areas and you will find that in urban areas, smoking on sidewalks is commonly prohibited.
No need to worry because, there are a lot established smoking areas available and they are clearly visible with signage that includes English. For example, train stations offer smoking booths that are usually located in areas near the entrance.