Computed Tomography (CT) scans provide a detailed image of the lungs, allowing clinicians to observe the extent of damage caused by COVID-19. The CT severity score (CTSS) of COVID-19 can be categorized based on the extent of lung involvement observed on a CT scan. This paper proposes a domain knowledge-based pipeline to extract the infection regions using diverse image-processing algorithms and a pre-trained UNET model. An ensemble of three machine-learning models, Random Forest (RF), Extremely Randomized Trees (ERT), and Support Vector Machine (SVM), is employed to classify the CT scans into different severity classes. The proposed system achieved a macro F1 score of 57.47% on the validation dataset in the AI-Enabled Medical Image Analysis Workshop and COVID-19 Diagnosis Competition (AI-MIA-COV19D).