Various fonts give different impressions, such as legible, rough, and comic-text.This paper aims to analyze the correlation between the local shapes, or parts, and the impression of fonts. By focusing on local shapes instead of the whole letter shape, we can realize letter-shape independent and more general analysis. The analysis is performed by newly combining SIFT and DeepSets, to extract an arbitrary number of essential parts from a particular font and aggregate them to infer the font impressions by nonlinear regression. Our qualitative and quantitative analyses prove that (1)fonts with similar parts have similar impressions, (2)many impressions, such as legible and rough, largely depend on specific parts, (3)several impressions are very irrelevant to parts.