A cytoplasmic male sterility (CMS) system is one of the most efficient ways to produce F1 hybrid seeds in pepper (Capsicum annuum). Restorer-of-fertility (Rf) genes are a critical component within the CMS/Rf system. The inheritance of Rf genes in pepper by joint segregation analysis was examined. The inheritance of Rf genes in the two progenies was controlled by two major additive-dominant epistatic genes and additive-dominant epistasis polygene. The two major genes had high additive effects and dominant effects. In addition, there existed significant epistatic effects between the two major genes. The major genes had high heritability in F2, BC1, and BC2 generations. Also, the fertility restorer characteristic can be selected during early generations of the breeding cycle.