Herbaceous peony (Paeonia lactiflora Pall.) is a well-known ornamental plant with abundant flower colors. However, our understanding of the underlying mechanisms of flower color formation is limited. In this study, a wild sample of herbaceous peony (collected from Heze, China) and eight cultivars with different colors were selected for experimental investigation. The Royal Horticultural Society Color Chart was used to determine flower color, and the anatomic structure; cell sap pH value; moisture content (MC); condensed tannin content (Ct); soluble sugar and soluble protein content of the petals; and content and composition of anthocyanin, flavonoids, and carotenoids in the petals were examined. 1) In the white, pinkish white, pale purple, purplish pink, and reddish purple cultivars, deeper color was associated with greater total amounts of anthocyanin (TA). Hypochromic effects were observed for kaempferol-7-O-glucoside (Km7G), myricetin-3-rhamnoside (My3R), and luteolin-7-O-glucoside (Lu7G). The accumulation of quercetin-3-O-glucoside (Qu3G) and lutein affected yellow color formation in the petals. 2) There are papillate epidermal cells in the petals of the wild P. lactiflora sample, ‘Lanyucangjin’, and ‘Dongjingnvlang’. 3) Cell sap pH and MC of the petals of white, pinkish white, pale purple, and purplish pink cultivars were greater than those of the purplish red and most of the reddish purple cultivars. 4) The Ct was greatest in the purplish red cultivars, whereas no condensed tannins were detected in the white, pinkish white, and pale purple cultivars. 5) There were no significant correlations among soluble sugar content, soluble protein content, and the other physiological indications.