Accurate and unbiased examinations of skin lesions are critical for early diagnosis and treatment of skin conditions and disorders. Visual features of skin lesions vary significantly because the skin images are collected from patients with different skin colours by using dissimilar type of imaging equipment. Recent studies have reported ensembled convolutional neural networks (CNNs) to classify the images for early diagnosis of skin disorders. However, the practical use of CNNs is limited because the majority of networks are heavyweight and inadequate to use the contextual information. Although lightweight networks (e.g., MobileNetV3 and EfficientNet) were developed to save the computational cost for implementing deep neural networks on mobile devices, not sufficient representation depth restricts their performance. To address the limitations, we introduce a new light and effective neural network, namely HierAttn network. The HierAttn applies a novel strategy to balance the learning local and global features by using a multi-stage attention mechanism in a hierarchical architecture. The efficacy of HierAttn was evaluated by using the dermoscopy images dataset ISIC2019 and smartphone photos dataset PAD-UFES-20. The experimental results show that HierAttn achieves the best top-1 accuracy and AUC among the state-of-the-art light-weight networks. The new light HierAttn network has the potential in promoting the use of deep learning in clinics and allowing patients for early diagnosis of skin disorders with personal devices. The code is available at https://github.com/anthonyweidai/HierAttn.