Malaria is a disease of global concern according to the World Health Organization. Billions of people in the world are at risk of Malaria today. Microscopy is considered the gold standard for Malaria diagnosis. Microscopic assessment of blood samples requires the need of trained professionals who at times are not available in rural areas where Malaria is a problem. Full automation of Malaria diagnosis is a challenging task. In this work, we put forward a framework for diagnosis of malaria. We adopt a two layer approach, where we detect infected cells using a Faster-RCNN in the first layer, crop them out, and feed the cropped cells to a seperate neural network for classification. The proposed methodology was tested on an openly available dataset, this will serve as a baseline for the future methods as currently there is no common dataset on which results are reported for Malaria Diagnosis.