Smoking in public has been proven to be more harmful to nonsmokers, making it a huge public health concern with urgent need for proactive measures and attention by authorities. With the world moving towards the 4th Industrial Revolution, there is a need for reliable eco-friendly detective measures towards this harmful intoxicating behavior to public health in and out of smart cities. We developed an Interpretable feature learning framework for smoking behavior detection which utilizes a Deep Learning VGG-16 pretrained network to predict and classify the input Image class and a Layer-wise Relevance Propagation (LRP) to explain the network detection or prediction of smoking behavior based on the most relevant learned features or pixels or neurons. The network's classification decision is based mainly on features located at the mouth especially the smoke seems to be of high importance to the network's decision. The outline of the smoke is highlighted as evidence for the corresponding class. Some elements are seen as having a negative effect on the smoke neuron and are consequently highlighted differently. It is interesting to see that the network distinguishes important from unimportant features based on the image regions. The technology can also detect other smokeable drugs like weed, shisha, marijuana etc. The framework allows for reliable identification of action-based smokers in unsafe zones like schools, shopping malls, bus stops, railway compartments or other violated places for smoking as per the government's regulatory health policies. With installation clearly defined in smoking zones, this technology can detect smokers out of range.