In this research paper, we present an innovative system designed for the purpose of segmenting the layout of Bangla documents. Our methodology involves utilizing a sophisticated collection of YOLOv8 models, meticulously adapted for the DL Sprint 2.0 - BUET CSE Fest 2023 Competition that centers around Bangla document layout segmentation. Our primary focus lies in elevating various elements of the task, including techniques like image augmentation, model architecture, and the use of model ensembles. We intentionally lower the quality of a subset of document images to enhance the resilience of model training, consequently leading to an improvement in our cross-validation score. Employing Bayesian optimization, we determine the optimal confidence and IoU thresholds for our model ensemble. Through our approach, we successfully showcase the effectiveness of amalgamating anchor-free models to achieve robust layout segmentation in Bangla documents.