Uncertainty quantification is vital for safety-critical Deep Learning applications like medical image segmentation. We introduce BA U-Net, an uncertainty-aware model for MRI segmentation that integrates Bayesian Neural Networks with Attention Mechanisms. BA U-Net delivers accurate, interpretable results, crucial for reliable pathology screening. Evaluated on BraTS 2020, this model addresses the critical need for confidence estimation in deep learning-based medical imaging.