Infertility is becoming an issue for an increasing number of couples. The most common solution, in vitro fertilization, requires embryologists to carefully examine light microscopy images of human oocytes to determine their developmental potential. We propose an automatic system to improve the speed, repeatability, and accuracy of this process. We first localize individual oocytes and identify their principal components using CNN (U-Net) segmentation. We calculate several descriptors based on geometry and texture. The final step is an SVM classifier. Both the segmentation and classification training are based on expert annotations. The presented approach leads to the classification accuracy of 70%.