Automated skin lesion analysis for detection and recognition is still challenging for inter-class diversity and intra-class similarity, and the low generic capability of a single Convolutional Neural Network (CNN) with limited datasets. This article proposes an end-to-end deep CNN-based multi-task web application for concurrent detection and recognition of skin lesion, named Dermo-DOCTOR, consisting of two encoders, where the features from each encoder are fused in channel-wise, called Fused Feature Map (FFM). For the detection sub-network, the FFM is used for decoding to obtain the input resolution of the output lesion masks, where the outputs of each stage of two encoders are concatenated with the same scale decoder output to regain the lost spatial information due to pooling in encoders. For the recognition sub-network, feature maps of two encoders and FFM are used for the aggregation to obtain a final lesion class. We train and evaluate the Dermo-Doctor utilizing two publicly available benchmark datasets, such as ISIC-2016 and ISIC-2017. The obtained mean intersection over unions, for detection sub-network, are 85.0 % and 80.0 %, whereas the areas under the receiver operating characteristic curve, for recognition sub-network, are 0.98 and 0.91, respectively, for ISIC-2016 and ISIC-2017 test datasets. The experimental results demonstrate that the proposed Dermo-DOCTOR outperforms the alternative methods mentioned in the literature, designed for skin lesion detection and recognition. As the Dermo-DOCTOR provides better-results on two different test datasets, even with limited training data, it can be an auspicious computer-aided screening tool to assist the dermatologists.