Recent mainstream weakly-supervised semantic segmentation (WSSS) approaches based on image-level annotations mainly relies on binary image-level classification with limited representation capacity. In this paper, we propose a novel semantic learning based framework for WSSS, named SLAM (Semantic Learning based Activation Map). We firstly design a semantic encoder to learn semantics of each object category and extract category-specific semantic embeddings from an input image. The semantic embeddings of foreground and background are then integrated to a segmentation network to learn the activation map. Four loss functions, i.e, category-foreground, category-background, activation regularization, and consistency loss are proposed to ensure the correctness, completeness, compactness and consistency of the activation map. Experimental results show that our semantic learning based SLAM achieves much better performance than binary image-level classification based approaches, i.e., around 3% mIoU higher than OC-CSE occse, CPN cpn on PASCAL VOC dataset. Our SLAM also surpasses AMN amn trained with strong per-pixel constraint and CLIMS clims utilizing extra multi-modal knowledge. Code will be made available.