Coronavirus Disease 2019(COVID-19) spread rapidly around the world, causing a series of severe health crises. Automated segmentation of lung infections based on Deep Convolutional Neural Network(DCNN) from medical images such as CT, X-ray, etc, displayed a huge potential for accurate diagnosis and quantitative analysis. Most COVID-19 medical images show blurred boundaries, dense noise points, low contrast, and significant variation in the shape and size of lesions. Although various models based on UNet have been proposed, more optimisation is required to obtain accurate segmentation and meet complex computational needs. Furthermore, the existing COVID-19 infections segmentation DCNN based methods are only suitable for single modality medical images. To solve these problems, this paper proposes a symmetric Encoder-Decoder segmentation framework named MS-DCANet. We introduce Tokenized MLP block, a novel attention scheme that uses a shift-window mechanism similar to the Transformer to acquire self-attention and achieve local-to-global semantic dependency. MS-DCANet also uses several Dual Channel blocks and a Res-ASPP block to expand the receptive field and extract multi-scale features. In a large number of experiments on COVID-19 datasets using both X-ray and CT images, MS-DCANet achieved state-of-the-art performance compared with other UNet models. MS-DCANet can also improve trade-off accuracy and complexity. To prove the proposed model's strong generalisability, we also apply MS-DCANet to the segmentation of skin tumours from dermoscopy images and hand bone from X-ray images with satisfactory results.