Broccoli (Brassica oleracea L. var. italica) is the most economically important cole crop in the United States and currently has a farm-gate value of about $900 million (USDA-NASS, 2015). Horticultural characteristics of mature broccoli heads have traditionally been difficult to quantify in breeding programs because of many potential quality defects (Farnham and Björkman, 2011a). Starting in 2010, a public–private collaboration [East Coast Broccoli Project; National Institute of Food and Agriculture (NIFA) Project No. 2010-51181-21062, Specialty Crops Research Initiative] has worked to select and develop high-quality F1 hybrid broccoli suitable for eastern environments and production, providing an alternative to current broccoli hybrids specifically developed for western production.
Breeding and selection for superior horticultural quality of broccoli heads under conditions like those occurring along the eastern seaboard that may include abiotic stress involves the manipulation of many genes conferring complex traits for which phenotypic differentiation can be difficult to assess (Farnham and Björkman, 2011a; Heather et al., 1992). The need to develop a consistent, stringent, and robust means of identifying suitable or nonsuitable hybrids for the East became apparent during the course of conducting East Coast Broccoli trials at five regional test sites in South Carolina, North Carolina, Virginia, New York, and Maine. Although up to 10 attributes of heads produced by tested hybrids were evaluated in all regional trials, decisions to advance entries to subsequent trials have been based on assessments of mean overall quality scores. However, overall quality tends to be a subjective and difficult-to-define metric of horticultural acceptability that is hard to standardize when there are many raters evaluating multiple trials. With a goal of overcoming this dilemma, we postulated that a quality trait index could be devised that would take into account a variety of individual quality attributes and provide a more robust measure of superior quality.
At the outset of conducting experiments described herein, we deemed it important to determine if evaluations by independently operating raters correlate with one another and if ratings will have high validity. Because horticultural quality assessments of broccoli heads are partially subjective, we also wanted to determine if a useful evaluation index based on greater human consensus could be constructed. In addition, it was recognized early that the elimination of redundant traits could be helpful in streamlining quality evaluations to save time and expense in conducting relatively large-scale and numerous quality trials.
The above considerations motivated us to devise an approach for analyzing evaluation trials by developing head quality phenotyping indices that could be compared using the ICC. The ICC, introduced by Fischer (Bartko, 1966), is a statistic used when measurements or observations are made on the same test subject by multiple raters, and it is a useful means of testing interobserver reliability and consistency (Fleiss and Cohen, 1973).
Selection indices have been frequently tested in a variety of agronomic crops, including soybean (Glycine max L. Merr.; Bouchez and Goffinet, 1990; Byth et al., 1969), oats (Avena sativa L.; Eagles and Frey, 1977), peanuts (Arachis hypogaea L.; Chandra et al., 2003), and wheat (Triticum aestivum L.; Sharma and Duveiller, 2006). These studies were primarily focused on selection of low genotype-by-environment traits, such as yield (Byth et al., 1969; Chandra et al., 2003) or on multiple yield-related trait selection (Sharma and Duveiller, 2006). We are unaware of the use of a selection index in horticultural crops to improve evaluation of a relatively subjective quality trait.
With the above considerations in mind, the specific objectives of this work were to 1) compare trait scores made by different individual raters on the same plots and trials, 2) determine which individual traits were most associated with overall quality, 3) define unique linear trait indices (e.g., linear combinations of trait values) that might be used as alternative measures for identifying the best hybrids, and 4) determine if the indices increase rater consistency for ranking hybrids. Our ultimate goal was to develop a reliable and optimal linear combination of rating scores that can effectively identify the best adapted broccoli hybrids for East Coast production.
Björkman, T. & Pearson, K.J. 1998 High temperature arrest of inflorescence development in broccoli (Brassica oleracea var. italica L.) J. Expt. Bot. 49 101 106
Bouchez, A. & Goffinet, B. 1990 Evaluation of selection index: Application to the choice of an indirect multitrait selection index for soybean breeding Theor. Appl. Genet. 79 261 267
Byth, D.E., Caldwell, B.E. & Weber, C.R. 1969 Specific and non-specific index selection in soybeans, glycine max l. (Merrill) Crop Sci. 9 702 705
Chandra, S., Nigam, S.N., Cruickshank, A.W., Bandyopadhyaya, A. & Hardcrishna, S. 2003 Selection index for identifying high-yielding groundnut genotypes in irrigated and rainfed environments Ann. Appl. Biol. 143 303 310
Cobb, J.N., DeClerck, G., Greenberg, A., Clark, R. & McCouch, S. 2013 Next-generation phenotyping: Requirements and strategies for enhancing our understanding of genotype–phenotype relationships and its relevance to crop improvement Theor. Appl. Genet. 126 867 887
Farnham, M.W. & Björkman, T. 2011a Breeding vegetables adapted to high temperatures: A case study with broccoli HortScience 46 1093 1097
Farnham, M.W. & Björkman, T. 2011b Evaluation of experimental broccoli hybrids developed for summer production in the eastern United States HortScience 46 858 863
Fleiss, J.L. & Cohen, J. 1973 The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability Educ. Psychol. Meas. 33 613 619
Gamer, M., Lemon, J. & Singh, I.F.P. 2012 irr: Various coefficients of interrater reliability and agreement. R package version 0.84
Groemping, U. & Matthias, L. 2013 Relaimpo: Relative importance for linear regression in R: The package relaimpo J. Stat. Softw. 17 1 1 27
Heather, D.W., Sieczka, J.B., Dickson, M.H. & Wolfe, D.W. 1992 Heat tolerance and holding ability in broccoli J. Amer. Soc. Hort. Sci. 117 887 892
Johnson, J.W. & LeBreton, J.M. 2004 History and use of relative importance indices in organizational research Organizational Research Methods 70 238 257
Lunley, T. 2017 Leaps: Regression subset selection. R package version 3.0
R Development Core Team 2015 R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria
Sharma, R.C. & Duveiller, E. 2006 Farmer participatory evaluation confirms higher grain yields in spring wheat using a selection index for spot blotch resistance, maturity and kernel weight Euphytica 150 307 317
U.S. Department of Agriculture, National Agricultural Statistics Service. 2015. Vegetables 2015 Summary.