The effects of different pollen sources on fruit and seed characteristics of Paeonia ostii ‘Feng Dan’ were investigated using analysis of three different pollination treatments and pollen sources from fifteen cultivars in two successive years. The results showed that self-pollination decreased fruit setting, the number of seeds per fruit, seed volume, seed and kernel weights, and the linoleic acid (LA) concentration in the seed oil, but increased the concentration of oleic acid (OA) compared with cross-pollination. Among those pollen donors, P. suffruticosa ‘Yinhong Qiaodui’ produced the highest fruit set (87.5%); the lowest fruit set was obtained with P. suffruticosa ‘Mo Run Jue Lun’ (44.33%). The most seeds per fruit were achieved by P. suffruticosa ‘Mochi Jin Hui’. P. suffruticosa ‘Dahong Baozhu’ produced the largest fruit, which contained larger and heaviest seeds. The oil extraction ratio (26% to 31.6%) and the concentration of three major unsaturated fatty acids (UFAs) in seed oil also significantly differed among pollen sources. The content of OA, LA, and α-linolenic acid (ALA) ranged from 13.82 to 24.79, 12.09 to 21.84, and 23.50 to 38.64 g/100 g crude oil, respectively. Overall, pollen source has clear effects on seed yield and even on fatty acid (FA) composition of seed oil in tree peony.