MN1264 (left), the noir maternal parent of GE1025, and MN1246 (right), the non-noir paternal parent of GE1025. Both are selections in the University of Minnesota grape breeding program and vary in their color and degree of cluster compactness.
A diagram of the camera, lighting, and backdrop setup for grape cluster image capture (not to scale). Grape clusters were suspended by an alligator clip in front of a flat white background and illuminated with a fluorescent light (Underhill et al., 2020).
An example of a single GE1025 grape genotype with raw (A), color-corrected (B), and segmented (C) cluster images. Raw (A) images were captured by a digital camera, and color-corrected (B) images were obtained using RawTherapee (Horváth and RawTherapee Development Team, 2017). Cluster images were then segmented (C) using Food Color Inspector (Albert et al., 2013).
A graphical representation of the individual morphological measurements processed from each grape cluster image. Traits collected included the cluster bounding box width and length, cluster central axis length, cluster perimeter length, and perpendicular (perp.) cluster widths at 25%, 50%, and 75% of the cluster central axis.
A flowchart of the grape cluster image capture, segmentation, and analysis process. Grape clusters were harvested when ripe (soluble solids >20%); then, images were captured in Nikon Electronic Format (.NEF) using a digital camera and gPhoto2 (Waugh et al., 2017). Images were adjusted using RawTherapee (Horváth and RawTherapee Development Team, 2017) and compressed as Tagged Image File Format (.TIFF) files; then, they were segmented by color using Food Color Inspector (Albert et al., 2013) and exported as Portable Network Graphics (.PNG) files. Morphological measurements were performed for the segmented images using a custom MATLAB (MathWorks, Natick, MA) script.
Actual calculated grape cluster compactness values (calculated using Eq. [4]) and cluster compactness values predicted by the partial least squares (PLS) model plotted against each other, along with their regression line (r = 0.90), in the test segment of the data comprised of 373 images with all genotypes in GE1025 represented.
Relative importance of each image-derived grape cluster morphological trait in the partial least squares (PLS) model. The area-to-perimeter ratio was the most important trait in the model, followed by aspect ratio (Eq. [1]) and compactness shape factor (Eq. [3]). In contrast, cluster roundness (Eq. [2]) was the least important trait in the model; the central axis length and percentage of cluster space pixels were also relatively unimportant.
A grape cluster image with similarly colored rachis and berries results in a poorly segmented image when using a color-based segmentation method. Red represents pixels marked as “berry” and green represents pixels marked as “rachis.” Here, many berry pixels are incorrectly classified as rachis (A) and many rachis pixels are incorrectly classified as berry (B). These errors can lead to inaccurate grape cluster measurements.
Histograms of grape cluster trait distributions of manually measured physical traits in the GE1025 (n = 121) population in 2017 with parental (MN1264, MN1246) averages denoted on each histogram. Calculated compactness is the cluster compactness found using the cluster weight divided by the square of the cluster length (Eq. [4]), whereas compactness is the cluster compactness is rated using Organisation Internationale de la Vigne et du Vin descriptor 204 (Organisation Internationale de la Vigne et du Vin, 2009).
Histograms of grape cluster trait distributions of manually measured physical traits in the GE1025 (n = 127) population in 2018. Calculated compactness is the cluster compactness found using measurements of the cluster weight and length (Eq. [4]), whereas compactness is the cluster compactness as rated using Organisation Internationale de la Vigne et du Vin descriptor 204 (Organisation Internationale de la Vigne et du Vin, 2009).
Histograms of grape cluster trait distributions of automatically measured image-derived traits in the GE1025 population (n = 121) in 2017. The aspect ratio is calculated by dividing the maximum cluster width by the central axis length (Eq. [1]). The compactness shape factor is calculated using the cluster area multiplied by 4π and divided by the square of cluster perimeter (Eq. [2]). The roundness is calculated by dividing the square of the cluster perimeter by the cluster area (Eq. [3]).
Histograms of grape cluster trait distributions of automatically measured image-derived traits in the GE1025 (n = 127) population in 2018. The aspect ratio is calculated by dividing the maximum cluster width by the central axis length (Eq. [1]). The compactness shape factor is calculated using the cluster area multiplied by 4π and divided by the square of the cluster perimeter (Eq. [2]). The roundness is calculated by dividing the square of the cluster perimeter by the cluster area (Eq. [3]).
Click on author name to view affiliation information
Grape (Vitis vinifera) cluster compactness is an important trait due to its effect on disease susceptibility, but visual evaluation of compactness relies on human judgement and an ordinal scale that is not appropriate for all populations. We developed an image analysis pipeline and used it to quantify cluster compactness traits in a segregating hybrid wine grape (Vitis sp.) population for 2 years. Images were collected from grape clusters immediately after harvest, segmented by color, and analyzed using a custom script. Both automated and conventional phenotyping methods were used, and comparisons were made between each method. A partial least squares (PLS) model was constructed to evaluate the prediction of physical cluster compactness using image-derived measurements. Quantitative trait loci (QTL) on chromosomes 4, 9, 12, 16, and 17 were associated with both image-derived and conventionally phenotyped traits within years, which demonstrated the ability of image-derived traits to identify loci related to cluster morphology and cluster compactness. QTL for 20-berry weight were observed between years on chromosomes 11 and 17. Additionally, the automated method of cluster length measurement was highly accurate, with a deviation of less than 10 mm (r = 0.95) compared with measurements obtained with a hand caliper. A remaining challenge is the utilization of color-based image segmentation in a population that segregates for fruit color, which leads to difficulty in differentiating the stem from the fruit when the two are similarly colored in non-noir fruit. Overall, this research demonstrates the validity of image-based phenotyping for quantifying cluster compactness and for identifying QTL for the advancement of grape breeding efforts.
Grape (Vitis vinifera) cluster compactness, defined as the difference between overall volume (shape) and volume of the cluster components (berries and rachis), is an important quality due to its influence on fruit quality (Tello and Ibáñez, 2017). The compactness of grape clusters has been shown to be a factor in susceptibility to fungal diseases like grey mold (Botrytis cinerea), ripe rot (Colletotrichum sp.), bitter rot [Greeneria uvicola (synonym Melanconium fuligineum)], and sour rot (a complex of wild yeasts and bacteria) because restricted airflow and higher humidity in compact clusters provide a favorable microenvironment for microbe growth (Hed et al., 2009; Marois et al., 1986). Compact clusters are also preferred by all generations of European berry moth (Lobesia botrana), which is an invasive pest in North America (Fermaud, 1998). Although insect and disease problems can be treated by chemical means, these methods are more effective when applied to loose clusters (Hed et al., 2011).
Despite the importance of cluster compactness, phenotyping compactness in a quantitative way remains difficult. The most commonly used assessment method is a visual rating described by the Organisation Internationale de la Vigne et du Vin (OIV) descriptor 204, an ordinal scale ranging from 1 (berries clearly separated, many visible pedicels) to 9 (berries deformed by compression) (Organisation Internationale de la Vigne et du Vin, 2009). Although this method is quick and inexpensive, requiring only a visual inspection, its accuracy relies on the expertise of the evaluator along with consistency in ratings across cultivars. If several evaluators are responsible for rating clusters, then differences between each can be an additional source of error. Difficulty can also arise if clusters are similarly compact, resulting in similar or identical OIV 204 scores that do not reflect population diversity. Because this rating scale was designed to rate established grape cultivars, this lack of specificity is limiting when evaluating breeding populations, for example, a population with scores exclusively between 3 (berries in loose contact with each other with some visible pedicels) and 5 (densely distributed berries, pedicels not visible, berries are movable) could be under genetic control and segregating for compactness, but the limited degree of variation quantified by this scoring system may not be sufficient for genetic mapping. An increased capacity to quantify variation in breeding or segregating populations would be advantageous for identifying loci for use in marker-assisted selection. This improvement would be particularly useful in cold-hardy or hybrid grape breeding programs, which often use non-vinifera grape species that can differ phenotypically from the European vinifera cultivars used to form the OIV descriptor 204 rating scale. Finally, the use of quantitative traits is better for statistical and genetic analysis because ordinal values cannot be as readily interpreted. Assumptions in QTL mapping include the normality and homoscedasticity of data, which can be difficult to meet when using qualitative traits.
Solutions using image-based methods have been implemented to evaluate grape cluster compactness in a more quantitative way. Several systems have been developed to measure various cluster components in a laboratory environment, including berry size (Cubero et al., 2014), berry count (Aquino et al., 2017), berry weight, and cluster weight (Diago et al., 2015). A method of measuring cluster compactness was developed by Cubero et al. (2015) which correctly predicted the OIV descriptor 204 score of a cluster 83.5% of the time. This process used semi-automated, color-based image segmentation to partition images of fruit clusters into berry, rachis, and background components, followed by automated morphological measurements of segmented images. Measured traits were used to construct a PLS model to predict visual compactness and evaluate which image-derived traits were significant to this prediction. However, this work evaluated only red-skinned V. vinifera cultivars grown using common production practices. In segregating breeding populations, field space is often limited and variations in berry and cluster shapes may be even more diverse than those seen in commercial cultivars. This application is where new phenotyping methods may be the most valuable: the ability to quickly and accurately evaluate progeny is key to breeding program efficiency, a need when working with long-lived, woody species like grape. The prospect of identifying genetic markers associated with cluster compactness (or traits that influence compactness, like rachis length, berry count, and berry size) is attractive because undesirable seedlings resulting from a cross could be discarded before planting, thus reducing time, effort, and resources.
The QTL related to fruit cluster architecture and compactness have also been identified, including those for berry weight (Ban et al., 2016; Costantini et al., 2008; Doligez et al., 2013; Lijavetzky et al., 2006; Mejía et al., 2007; Richter et al., 2019), berry size or volume (Mejía et al., 2007; Richter et al., 2019), rachis length, pedicel length, and cluster weight (Richter et al., 2019). A variety of cluster architecture traits were characterized by 19 QTL in a V. vinifera table grape cross, ‘Ruby Seedless’ × ‘Sultanina’, including those for rachis length, lateral shoulder length, and node number on the central rachis axis (Correa et al., 2014). Most QTL were found on chromosomes 5 and 18, whereas the main QTL corresponding to rachis length was on chromosome 9, and others were found on chromosomes 8, 14, and 17. Potential pleiotropy was observed among traits like rachis weight, rachis length, peduncle diameter, node number, and shoulder length. Architecture traits had relatively high broad-sense variability of 65% to 75%, meaning that these traits could be feasible breeding targets. Additionally, QTL for compactness were identified on chromosomes 2, 15, and 17 in a cross of V. vinifera GF.GA-47-42 × ‘Villard Blanc’, along with numerous other cluster architecture traits (Richter et al., 2019). A candidate gene called VvTFL1A, homologous to the Terminal Flower1 gene in Arabidopsis thaliana, was found to contain many polymorphisms and was responsible for 16% of flowering time variation, 13% of cluster width variation, and 25% of berry weight variation between different V. vinifera haplotypes (Fernandez et al., 2014). Specific haplotypes associated with combinations of these three traits were linked to specific cultivar groups, for example, the low-weight, small-cluster haplotype was found most frequently in wine grapes (Fernandez et al., 2014). Similarly, a study of V. vinifera flower transcriptome differences between loose and compact clusters found that genes related to cell growth, development, metabolism, and regulation were expressed in greater frequency in compact clusters, whereas genes related to hormone signaling and stimuli (including biotic stress and pathogen responses) were expressed more in loose clusters (Grimplet et al., 2017).
Although many studies have associated regions of the genome with compactness-related traits, each has identified these QTL in different locations on the genome, underscoring the importance of environmental influence and different genetic mechanisms on compactness-related traits. Tello et al. (2015) found that the number of berries per cluster and main rachis length were most influential in determining cluster compactness, both of which can be influenced by common vineyard management practices like gibberellic acid application and leaf pulling. Fruit cluster compactness itself is influenced by many traits, each of which is impacted by numerous environmental influences; a summary of these traits and influences is given in the review by Tello and Ibáñez (2017). Additionally, few of these compactness mapping studies have been performed in diverse, interspecific wine grape populations, and not one has studied cold-hardy cultivars specifically. Continuing to examine compactness and its related traits will help identify the scope of genetic and environmental influences along with suitability as a breeding target.
This study tested a method of image capture, segmentation, and analysis in a segregating population of hybrid wine grapes. Evaluating this method in a diverse, predominantly non-vinifera population helped determine whether it would be feasible to evaluate progeny in a hybrid grape breeding program or other segregating population. Image-derived traits were tested against conventionally collected traits in QTL mapping to determine whether they identified the same loci associated with compactness and cluster morphology. Additionally, correlations between traits were examined to determine the relationship between each, and a PLS model was constructed to determine the relative importance of traits when predicting cluster compactness and the potential of such a model to predict compactness accurately. Elucidating the nature of compactness and compactness-influencing traits will be helpful because moving toward fruit with optimal compactness will naturally aid in reducing disease and insect infestation.
Samples for this experiment were taken from a biparental F1 mapping population, GE1025 (n = 140), that was derived from a cross (MN1264 × MN1246) (Fig. 1) made in 2010. Each parent is a breeding selection in the University of Minnesota wine grape program, and the pedigrees of each are described by Teh et al. (2017). GE1025 is highly diverse, including ancestry from at least six Vitis species (V. vinifera, V. riparia, V. rupestris, V. labrusca, V. aestivalis, and V. berlandieri). Vines are located at the University of Minnesota Horticultural Research Center [Victoria, MN (lat. 44°52′08.1″N, long. 93°38′17.3″W)]; they were trained in a top wire, high-cordon system planted 0.9 m apart. MN1264 was assessed to have an OIV descriptor 204 score of 7, whereas MN1246 had a descriptor score of 3. Assessments were performed on fruit clusters produced in 2017 and 2018. The harvest process is the same as that described by Underhill et al. (2020); briefly, three clusters from each GE1025 genotype were collected at maturity (soluble solids concentration >20%) for laboratory image collection. The fruit colors present in GE1025 can also be found in detail in the work by Underhill et al. (2020).
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
The image collection and segmentation process described here and the equipment specifications have been detailed by Underhill et al. (2020). Each cluster was suspended by a clip from its rachis, and four digital images were obtained of each cluster with 90° rotations between each to produce 12 images per genotype. The design was based on the imaging setup used by Cubero et al. (2015), and a schematic diagram of the system used is provided in Fig. 2. In 2017, clusters with shoulders often extended outside the camera field of view and could not be processed through the analysis pipeline; therefore, these images had to be removed. Therefore, in 2018, shoulders (small, secondary clusters arising from the rachis) that extended outside the camera field of view were removed from clusters and not considered during the analysis. The software package gPhoto2 (Waugh et al., 2017) was used to automatically capture, download, and name image files (Fig. 3A). A color calibration card and the software package RawTherapee (Horváth and RawTherapee Development Team, 2017) were used to correct white balance and color in the images (Fig. 3B). Images were color-corrected, compressed, and exported for segmentation.
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
A detailed description of the segmentation process can be found in the work by Underhill et al. (2020). Food Color Inspector software (Albert et al., 2013) was used for segmentation, which uses a semi-automated process requiring the user to select 3- × 3-pixel squares to define a training set for each color class (berry, rachis, or background). One image from each genotype was used to define the color classes; the remaining images from noir (dark-colored) genotypes were then automatically processed using this training set. Images from non-noir (light-colored) clusters were processed individually to increase segmentation accuracy because color often varied between images and clusters. Segmented images were then exported for analysis (Fig. 3C).
After segmentation, images were processed using a custom MATLAB (MathWorks, Natick, MA) script. First, a 3- × 3-pixel median filter was applied to mitigate the effects of pixels that were not attached to any other region of interest; this prevented incorrect measurements later during the process. Berry and rachis cluster elements were classified based on their color (in segmented images, red areas represented berry pixels and green represent rachis pixels) (Fig. 3C), whereas background elements were classified according to their location within or outside of the fruit to differentiate backdrop space from empty space within the cluster (in segmented images, black areas represent background). This was accomplished by masking the entire cluster, subtracting the backdrop space, and then determining the proportion of each pixel class within the cluster. Using these separate elements and overall measurements of the cluster, traits were derived for the following: cluster area; cluster length measured by the central axis; percentages of empty space, berry, and rachis pixels within the cluster; cluster perimeter; ratio of the cluster area to the cluster perimeter; maximum cluster width measured by a bounding box; maximum cluster length measured by a bounding box; and cluster widths at 25%, 50%, and 75% of the axis length (Fig. 4). Additionally, the aspect ratio (Eq. [1]), cluster roundness (Eq. [2]), and compactness shape factor (Eq. [3]) were calculated from the cluster morphological measurements based on equations outlined by Cubero et al. (2015).[1][2][3]
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
A flowchart of the image capture, segmentation, and analysis process is provided in Fig. 5.
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
Physical features of the fruit were measured by hand after photographs were obtained. Fresh cluster weight, cluster length (maximum end-to-end length from top to bottom) as measured by a digital caliper, berry count, and 20-berry weight were recorded. When clusters contained less than 20 berries, the 20-berry weight was estimated using the average individual berry weight. Clusters were visually scored for compactness using OIV descriptor 204; possible scores ranged from 1 (the loosest; “berries clearly separated, many visible pedicels”) to 9 (the most compact; “berries deformed by compression”) (Organisation Internationale de la Vigne et du Vin, 2009). Compactness was also calculated using a ratio of cluster weight and length proposed by Tello and Ibáñez (2014) (Eq. [4]; referred to here as calculated compactness).[4]
Fruit was collected from 121 GE1025 genotypes in 2017, and from 127 genotypes in 2018, with 112 genotypes measured in both years. Manual physical measurements from each replication were averaged to provide a single representative value per genotype. Average values from each of the four cluster photos were used to give one value per trait. This resulted in a phenotype for each cluster, from which one set of average values per genotype was produced. To compare cluster length measurements between manual and digital measurements, digital lengths in pixels were converted to millimeters based on a 1-mm reference ruler acquired in the image. Statistical analysis was performed using R (R Core Team, 2017). Normality was tested using Shapiro-Wilk tests and visual assessments of Q-Q and residual plots. To assess the relationships between traits and phenotyping methods, the Pearson correlation coefficient (r) was calculated for relationships between all traits in each year. The R package “caret” (Kuhn, 2008) was used to construct a PLS model of the 2018 data using calculated compactness as the trait to be predicted and all image traits simultaneously as predictors. Three images from each cluster were randomly selected to train the model (1097 images total), whereas the remaining images were used to test the model (373 images total). Data from predictors were scaled and centered before analysis, and 20 random segments were used for cross-validation. Models were evaluated based on minimization of the root mean square error (RMSE) and predictive ability [coefficient of determination (R2)]. The relative importance of each variable was examined using the regression coefficients normalized to an absolute sum of 100. The R package “qtl” (Broman et al., 2003) was used to perform QTL mapping of cluster compactness traits. Non-normal traits were log-transformed before mapping. Traits from 110 individual genotypes in 2017 and 113 individual genotypes in 2018 were mapped using an integrated MN1264 × MN1246 linkage map composed of 3024 single-nucleotide polymorphism (SNP) markers that was previously developed (Teh et al., 2017). Discrepancies between the number of individuals phenotyped and the number of individuals used in QTL mapping stemmed from the limitation of not every GE1025 individual being genotyped during linkage map development. Interval mapping was used with the EM algorithm, and genotype probabilities were estimated using the Haldane function with an error rate of 0.0001. One thousand permutations were performed at both genome-wide and chromosome-wide levels to determine statistically significant logarithms of odds (LOD) at α = 0.05. QTL were considered significant at the genome-wide level when they exceeded the genome-wide threshold, and each QTL reported met or exceeded this threshold. The percent phenotypic variance explained by each QTL was also estimated.
Histograms for all trait distributions are provided in Supplemental Figs. 1–4. In 2017, cluster weight, cluster length, calculated cluster compactness (Eq. [4]), and berry count were all distributed normally in the population. The 20-berry weight and visual compactness as described by the OIV descriptor 204 were found to be non-normally distributed. In many cases, clusters classified as 1 on the OIV 204 scale were poorly formed. In 2017, the mean OIV 204 visual compactness rating in GE1025 was just over 3 (“berries in loose contact with each other with some visible pedicels”). Eleven out of the 121 individuals evaluated had average scores of 5 (densely distributed berries, pedicels not visible, berries are movable) or more, with none scoring more than 6, whereas 62 individuals had scores of 3 or less. In 2018, clusters were more compact on average, with a mean OIV 204 rating of 3.8, and 22 individuals had a score more than 5; 68 individuals had average ratings of 4 or less. Between years, the distribution of physical traits was similar (Supplemental Figs. 1 and 2), whereas clusters were moderately heavier and longer in 2018, with slightly greater berry weights and berry counts on average. Fruit color segregated in a 1:1 noir to non-noir ratio (χ2 test: P = 0.32 in 2017, P = 0.18 in 2018), which is expected in an F1 population based on the colors of the parents (MN1264: noir; MN1246: non-noir) and the two-gene (VvMYBA1 and VvMYBA2) model of color inheritance (Azuma et al., 2007). Six of the 12 image-derived traits were distributed normally in 2017; non-normal traits were percentage of berry pixels, rachis pixels, and space pixels, perpendicular width at 50% of the central axis, compactness shape factor (Eq. [3]), and roundness (Eq. [2]). Histograms of the non-normal traits did not show bimodal distributions; rather, deviations from normality were due to skewing (Supplemental Figs. 3 and 4). This non-normality was most likely due to the effect of the extremes because the population was skewed toward small, loose clusters. In 2018, cluster weight and cluster length were the only traits distributed normally, whereas calculated compactness, 20-berry weight, berry count, and visual compactness were all non-normally distributed. This was due to the data skewing toward lower values for all traits, again, perhaps, due to environmental stress on the vines. In contrast to 2017, most image traits were non-normally distributed in 2018; cluster perimeter length, perpendicular width at 50% of central axis, cluster area, and cluster area to perimeter ratio were the only normally distributed traits in the population. As in 2017, non-normally distributed traits were unimodal and tended to skew toward higher or lower values.
Full tables of correlations between traits for each year are available in Supplemental Table 1, whereas the correlation between years for each trait is available in Supplemental Table 2. Hand-measured length was compared with both the central axis and bounding box length to determine the ability of each method to estimate the physical cluster length. In 2017, the bounding box length and hand-measured length had a high Pearson correlation coefficient (r = 0.95), whereas the coefficient of axis length and hand-measured length were much lower (r = −0.27, P < 0.001 in both cases). On average, the bounding box method had a difference of −9.07 mm from the hand-measured length, whereas the axis method had an average difference of −21.94 mm. In 2018, the correlations were lower, with r = 0.88 between the bounding box and hand-measured length, and r = −0.004 between the axis and hand-measured length (again, both P < 0.001). Differences from hand-measured length were slightly higher in 2018, with the bounding box length having an average of −14.46 mm and axis length having an average of −29.09 mm different from hand-measured length.
The final PLS model was constructed using five latent variables and had an RMSE of 0.067 and an R2 of 0.81. These five latent variables captured 77.35% of variation contained in the image traits and 81.29% of variation in calculated compactness. The correlation between actual and predicted calculated compactness was r = 0.90 (P < 0.001) (Fig. 6), with differences between actual and predicted values that ranged from −0.33 to 0.30 g·cm−2, with a mean of −0.00043 g·cm−2 and sd of 0.068 g·cm−2. The area-to-perimeter ratio, aspect ratio, compactness shape factor, percentage of berry pixels, and percentage of rachis pixels were the five most important traits in the model (Fig. 7).
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
Full results of the QTL observed at a genome-wide significance level in both years are presented in Table 1 for manually measured traits and in Table 2 for image-derived traits. Between years, recurrent QTL for manually measured traits included those on paternal (MN1246) chromosome 11 and maternal (MN1264) chromosome 17 for 20 berry weight. QTL for image-derived traits did not recur between years.
Automated phenotyping techniques, such as those based on image analysis, are potentially advantageous because of their time-saving, labor-saving, or cost-saving efficiencies. By eliminating human subjectivity and error, data can become more accurate and better at capturing variation important in QTL mapping. The case of grape cluster compactness is one that suits this adaption well; manually measuring this trait requires time-consuming work that often includes multiple cluster and rachis evaluations along with berry counts and weights. Evaluating cluster compactness visually is subjective, and the most commonly used scale is not suited for all populations. Because 5 is the median of the OIV descriptor 204 scale, most of the GE1025 population was described in both years using fewer than half of the categories, and a lack of differentiation between these lower-scoring individuals made it difficult to fully capture variations in compactness. The ordinal nature of this scale means that each score may not be directly related to any quantifiable trait, and that the scale is not necessarily linear; for example, the difference between visual compactness ratings of 2 and 3 may not be the same as the difference between scores 3 and 4. GE1025, in particular, would benefit from a phenotyping technique that allows more compactness variation to be captured. Tello and Ibáñez (2014) developed a method of calculating compactness based on cluster weight and length (Eq. [4]) to provide a numeric value with which to compare clusters to one another. Their method was tested by assessing the correlation of this calculation with visual compactness of clusters; the relationship between these traits was significant, with r = 0.47 between calculated and OIV descriptor 204 visual compactness and r = 0.60 between calculated compactness and cluster compactness rank when ordered from least to most compact (Tello and Ibáñez 2014). This relationship was validated for GE1025 when calculated compactness was, again, found to be highly correlated with visual compactness, with r = 0.71 in 2017 and r = 0.77 in 2018 (P < 0.001 in both cases). This metric of compactness may be most helpful when traits like cluster weight and length are already being measured by hand because these measurements can be used to calculate a value that captures variations in compactness without additional work. Additionally, calculated compactness is more quantitative than visual scoring and helps differentiate clusters that would not otherwise be distinguishable using the OIV descriptor 204 scale.
Many image-derived traits were related to those measured by hand. For example, cluster area was strongly correlated to cluster weight in 2017 (r = 0.89), and roundness was correlated to both calculated and visual compactness in both years (r = 0.80 and r = 0.76 in 2017; r = 0.83 and r = 0.79 in 2018). These relationships, along with the fact that calculated compactness values were determined to be well-correlated to visual compactness, suggested that calculated compactness values could be predicted using image data. A PLS model was used to predict cluster compactness using image traits and to determine which traits were most useful for predicting cluster compactness. The final PLS model with five latent variables was effective at predicting calculated compactness, with an average difference of only −0.00043 g·cm−2 between actual and predicted values. These highly accurate estimations suggest that estimating cluster compactness from image-derived values alone is effective, and visual or physical measurements could be replaced by image analysis. The model performed best when clusters had low to moderate compactness values (<0.50 g·cm−2), with higher deviations above that range and a high likelihood—nearly 91% in this case—of underestimating cluster compactness. Clusters in this segment of the population were heavier on average, with shorter rachis lengths and slightly heavier berries. The combination of smaller physical size with tightly packed berries likely contributed to the error, because traits accounting for cluster size and length (area-to-perimeter ratio, aspect ratio) were relatively important in the model. Knowledge of cluster morphology in a population would be helpful for determining which image traits would be the most effective for predicting compactness because differences can occur between populations and years. Testing image-derived traits across a number of diverse populations would help to determine which traits are most reliably significant to predicting compactness and their suitability for use in evaluating segregating germplasm, and, ultimately, to evaluate marker–trait associations for use in breeding programs.
When comparing the bounding box length and axis length to the caliper-measured length, the bounding box length performed much better; in 2017, the bounding box length had a high Pearson correlation coefficient (r = 0.95), whereas the coefficient of axis length was much lower. The reason for length underestimation is likely due to the length of the peduncle excluded from the image by the clip suspending the cluster (≤10 mm) in contrast to hand measurements during which the entire peduncle is included in the length. The bounding box method performed better due to its measurement of maximum cluster length; the axis length measures the central axis of the cluster, which may or may not be its longest length. Additionally, image-based measurements are subject to the differences between two-dimensional and three-dimensional measurements; although humans can judge the longest length of a cluster and measure it, automated methods rely on that length being visible in the image. Taking the average of the lengths from several images can exacerbate this issue because the longest length observed could be skewed by shorter measurements observed due to berry occlusion or a change in the way the peduncle was held by the clip. Regardless, the highly positive, highly significant linear relationship observed between the bounding box and hand-measured cluster length makes it a good trait for image-based phenotyping. Implementing image-based measurements of cluster length could potentially save time and effort during data collection.
One issue that affected measurement consistency was variability in peduncle length between genotypes. Some genotypes had relatively long peduncles, whereas others were significantly shorter. The variation occasionally posed an issue while photographing the clusters because it was difficult to attach short peduncles with the clip, and the tops of clusters could extend outside of the camera’s field of view. Suspending the clusters (as opposed to laying them on a flat surface) was chosen based on the same rationale provided by Cubero et al. (2015) because the suspension eliminates cluster deformation and reduces shadow in the image, thereby improving image segmentation accuracy. Again, the variation in peduncle length is most likely the result of the population type; in a segregating F1 population, many traits will vary, sometimes dramatically, between genotypes. The population used in the study by Cubero et al. (2015) comprised a collection of cultivars and was likely free of this issue due to larger clusters with longer peduncles. In a study of a (V. rupestris × V. arizonica) × V. vinifera F1 population, peduncle length was associated with QTL on chromosomes 9, 10, and 12 (Viana et al., 2013). In another V. vinifera population, QTL for peduncle length were found on chromosomes 1 and 14 (Richter et al., 2019). Although peduncle length was not measured in this study, its segregation in the GE1025 population would make it a good candidate for mapping in future investigations, and measuring it via image analysis would be useful. Similarly, there was both intergenotype and intragenotype variations in the shoulder presence, along with variations in shoulder angle and size in GE1025. Shoulders, secondary branches that extend from the rachis, varied in their position from near the top of the peduncle to near the main berry cluster at varying angles. Angled shoulders originating near the top of the rachis led to an overestimation of cluster width because shoulders influenced measurements of the cluster widths at 25% or 50% of the central axis. Another issue with high shoulders arose when they extended above the field of view. This led to image processing issues because seven images in 2017 could not be analyzed due to the presence of multiple disconnected shapes in the image created by shoulders. When multiple shapes were in the image, a single bounding box could not be drawn. In contrast, shoulders that originated lower on the cluster did not have as large of an impact on measurements or image processing because they were effectively processed as part of the main cluster itself. Subsequently, in 2018, shoulders extending above the frame of the image (near the apex of the peduncle or with a high angle, extending the shoulder beyond the frame of view) were removed from clusters to reduce error and allow for correct image processing, whereas all other shoulders were left attached. Uniformly removing or retaining the shoulders would allow for more consistent measurements and is recommended in future studies.
Just like traditional phenotyping methods, the data obtained through image analysis is only as good as the method itself; like an untrained data collector, a poorly designed or poorly trained system (despite the removal of human judgement) will still perform poorly. With the image analysis system used here, one major source of introduced error was incorrect color segmentation of the initial images. This proved to be a challenge with non-noir fruit due to color similarities between berries and rachises; if the training sets for rachis and berry were of similar color, then the likelihood of correct pixel classification was low. Therefore, misclassified pixels (berry mistaken as rachis and vice versa) were difficult to correct. To decrease error, non-noir (green or light-colored) clusters were usually segmented individually (53 genotypes in 2017, 56 genotypes in 2018) with a unique training set for each image. Training sets often contained multiple berry and rachis color selections to increase accuracy because heterogeneous berry colors had to be sampled multiple times for pixels to be correctly classified. Berries with differential coloration were often difficult to correctly segment because the presence of green and red/blue colors in a single berry meant that a training set for berry color had to classify contrasting colors, which led to incorrectly classified pixels (Fig. 8).
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 145, 6; 10.21273/JASHS04932-20
Although the amount of classification error in images due to automated processing is difficult to quantify, non-noir clusters had a greater amount of error than noir clusters, and the error increased along with similarity between rachis and berry color. Often, non-noir fruit or fruit with a high proportion of “shot” (green and underdeveloped) berries contained a higher proportion of incorrectly classified rachis pixels, which led to right-tailed trait distributions. Additionally, collinearity between traits exacerbated this effect; for example, a higher percentage of rachis pixels necessarily meant a lower percentage of berry pixels. Misclassification issues did not pose the same challenges for noir fruit because sufficient contrast existed between the green-to-brown rachis and homogenous dark blue fruit color. The sole use of cultivars with noir clusters was likely what made this segmentation feasible and highly accurate according to Cubero et al. (2015). In GE1025, the F1 population used in this experiment, a large amount of color variation was observed due to genetic and environmental influences. To limit the error caused by color similarities and/or discrepancies, training sets for non-noir clusters were not automated; instead of using a single training set for each image for the genotype, different training sets were created for each image. Although this improved segmentation accuracy, the process was time-consuming and often still resulting in a high degree of error. This raises the possibility that a color-based image segmentation method may not be appropriate for populations in which fruit color is segregating. Better-suited methods may include morphological-based segmentation, where the shapes of round berries or linear rachises can be recognized and isolated from each other. This has been used in field imaging applications for measuring berry size and estimating yield (Mirbod et al., 2016; Nuske et al., 2014). Different imaging methods, such as hyperspectral, may help distinguish the berry and rachis tissues from each other in an image (van Roy et al., 2018). Additionally, images could be classified in a way that bypasses segmentation entirely, for example, training a convolutional neural network to recognize and categorize clusters by their compactness (Santos et al., 2020). Further research of an optimal image analysis method will be needed to find a solution that is broadly applicable across diverse germplasm.
QTL analyses of cluster compactness as a single trait have not been widely published, although one study found QTL associated with OIV compactness ratings on chromosomes 1, 2, 15, and 17 (Richter et al., 2019), and another found the VvTFL1A gene region on chromosome 6 to be related to compactness when comparing the transcriptomes of compact and loose clusters (Fernandez et al., 2014). In our investigation, QTL were observed on chromosomes 7, 12, and 18 for visual compactness and on chromosomes 12, 17, and 18 for calculated compactness. No QTL overlapped with the VvTFL1A region. Other QTL for compactness-related traits were also observed, which allowed comparisons to previously reported traits, including berry count, berry weight, rachis length, and cluster length. The most extensively reported of these is berry weight, with QTL on chromosomes 1, 8, 10, 11, 12, 15, and 18 appearing consistently between years in several studies (Cabezas et al., 2006; Costantini et al., 2008; Doligez et al., 2013; Mejía et al., 2007; Richter et al., 2019; Viana et al., 2013). In general, estimates of phenotypic variance explained by berry weight QTL in the reported literature are low, ranging from ≈2% to 17% of variance explained. In both 2017 and 2018, concurrent QTL for berry weight were observed in the GE1025 population on paternal chromosome 11. The proportion of phenotypic variance explained by berry weight QTL in GE1025 was consistent with previously reported results, with 12.68% to 16.87% of variance explained by the QTL detected across both years. However, other studies found a much larger percentage of phenotypic variation to be explained by berry weight QTL: up to 40.3% of variance explained by a QTL on chromosome 11 in a V. labruscana × V. vinifera cross (Ban et al., 2016) and up to 44% explained by a chromosome 7 QTL in a Picovine 00C001V0008 × Ugni blanc flb microvine population (Houel et al., 2015). Additionally, QTL found on chromosome 18 are often highly significant and explain a large proportion of phenotypic variance [up to 61% in Doligez et al. (2013)] due to colocalization with the known locus for seedlessness, SDI (Cabezas et al., 2006; Lahogue et al., 1998). No QTL were detected on chromosome 18 in this study, possibly due to the lack of segregation for the seedlessness trait.
Although QTL mapping for compactness-related traits using solely image-derived data has not yet been reported in the literature for grape, many traits of interest have been mapped previously using conventional phenotyping methods. Here, QTL identified using image analysis can be compared with those identified using manual measurements. In GE1025, a chromosome 9 QTL associated with cluster length was observed in 2017 across both phenotyping methods (hand-measured and image-derived). Rachis length, a closely related trait, was found to be significantly associated with a QTL on chromosome 9 in a ‘Ruby Seedless’ × ‘Sultanina’ cross, where it exhibited a broad-sense heritability of 76% and explained 24.1% of the phenotypic variance observed (Correa et al., 2014). This is consistent with results found in this study and helps confirm the presence of a length-influencing locus on this chromosome. Several genomic regions of interest were identified by both physical and image-based phenotyping methods. On maternal chromosomes 4, 9, 16, and 17 and paternal chromosome 12, overlapping QTL regions demonstrate that these different methods of phenotyping are identifying the same loci. Some of the relationships between traits are broad; for example, the overlap of roundness, cluster perimeter, bounding box length, perpendicular width at 25% of the central axis, and cluster weight on chromosome 4 in 2018 most likely reflected the relationship of these traits with cluster size. Although a clear line cannot be drawn from these image-derived traits to physical measurements, the fact that both methods identify the same region supports the use of image-based phenotyping for use in genetic mapping. Similarly, the two measurements of physical compactness (calculated and visually rated) identified the same regions on paternal chromosomes 12, 17, and 18. This suggests the calculated measurement of compactness is comparable to the OIV descriptor 204 scale, this supporting its use in situations in which cluster length and weight are already being measured. The compactness calculation is simple and quantitative and provides a numeric value instead of an ordinal rating, which allows for a greater number of data analyses.
A challenge in QTL mapping with grapes is the large amount of environmental variation that affects trait phenotype; coupled with differences in genotype × environment interaction responses between individuals, consistently detecting QTL across years can be difficult. Issues such as small population sizes, long juvenility periods, selection due to winter injury or plant death, and stress from closely planted vines can make it difficult to observe representative phenotypes in a mapping population. Often, only a single replication of genotypes exists due to the constraints of space and time. These factors along with environmental variations could be part of the reason why known QTL for cluster compactness-related traits were not detected in this population; for example, no QTL were found on chromosome 18 at the SDI locus, likely due to the lack of linkage because all individuals in the population are seeded. Many differences were seen between years in this population; only two QTL detected with manual phenotyping were observed in both years, and none of the QTL detected with image-based phenotyping was detected in both years. Several loci were associated with multiple image-based traits within, but not between, years, for example, on maternal chromosomes 9 and 17 in 2017, maternal chromosome 4 in 2018, and paternal chromosome 12 in 2018. This was likely due to environmental differences or genotype × environment interactions causing changes in fruit between years, the effects of which are magnified in phenotypes that rely on precise measurements. This is especially apparent in the low correlations between trait measurements between years (Supplemental Table 2). Additionally, some of these differences could be due to random error; as discussed previously, image segmentation was subject to error introduced by heterogeneous berry color or lack of contrast between rachis and berry. More years of data collection and replication of genotypes in the field would likely improve confidence in QTL detection and aid in elucidating the relative importance of QTL influencing certain traits. In terms of breeding strategies, genomic selection could potentially be more useful than marker-assisted selection because a large number of individual markers can more effectively capture variations due to minor QTL. This could be especially useful when selecting for polygenic, low-heritability traits related to cluster compactness.
This study demonstrated the advantages and challenges of an image analysis-based phenotyping system for quantifying grape cluster compactness. Evaluating more diverse segregating populations will help identify additional situations in which issues can arise in image segmentation or analysis and will aid in validating the use of this method for predicting cluster compactness in many population types. Continuing to explore the use of image-derived data for genetic mapping will help identify new relationships between quantitative traits and gene regions. Overall, this system allows measurements of grape clusters to be obtained quickly and accurately, producing useful data and reducing human error.
MN1264 (left), the noir maternal parent of GE1025, and MN1246 (right), the non-noir paternal parent of GE1025. Both are selections in the University of Minnesota grape breeding program and vary in their color and degree of cluster compactness.
A diagram of the camera, lighting, and backdrop setup for grape cluster image capture (not to scale). Grape clusters were suspended by an alligator clip in front of a flat white background and illuminated with a fluorescent light (Underhill et al., 2020).
An example of a single GE1025 grape genotype with raw (A), color-corrected (B), and segmented (C) cluster images. Raw (A) images were captured by a digital camera, and color-corrected (B) images were obtained using RawTherapee (Horváth and RawTherapee Development Team, 2017). Cluster images were then segmented (C) using Food Color Inspector (Albert et al., 2013).
A graphical representation of the individual morphological measurements processed from each grape cluster image. Traits collected included the cluster bounding box width and length, cluster central axis length, cluster perimeter length, and perpendicular (perp.) cluster widths at 25%, 50%, and 75% of the cluster central axis.
A flowchart of the grape cluster image capture, segmentation, and analysis process. Grape clusters were harvested when ripe (soluble solids >20%); then, images were captured in Nikon Electronic Format (.NEF) using a digital camera and gPhoto2 (Waugh et al., 2017). Images were adjusted using RawTherapee (Horváth and RawTherapee Development Team, 2017) and compressed as Tagged Image File Format (.TIFF) files; then, they were segmented by color using Food Color Inspector (Albert et al., 2013) and exported as Portable Network Graphics (.PNG) files. Morphological measurements were performed for the segmented images using a custom MATLAB (MathWorks, Natick, MA) script.
Actual calculated grape cluster compactness values (calculated using Eq. [4]) and cluster compactness values predicted by the partial least squares (PLS) model plotted against each other, along with their regression line (r = 0.90), in the test segment of the data comprised of 373 images with all genotypes in GE1025 represented.
Relative importance of each image-derived grape cluster morphological trait in the partial least squares (PLS) model. The area-to-perimeter ratio was the most important trait in the model, followed by aspect ratio (Eq. [1]) and compactness shape factor (Eq. [3]). In contrast, cluster roundness (Eq. [2]) was the least important trait in the model; the central axis length and percentage of cluster space pixels were also relatively unimportant.
A grape cluster image with similarly colored rachis and berries results in a poorly segmented image when using a color-based segmentation method. Red represents pixels marked as “berry” and green represents pixels marked as “rachis.” Here, many berry pixels are incorrectly classified as rachis (A) and many rachis pixels are incorrectly classified as berry (B). These errors can lead to inaccurate grape cluster measurements.
Histograms of grape cluster trait distributions of manually measured physical traits in the GE1025 (n = 121) population in 2017 with parental (MN1264, MN1246) averages denoted on each histogram. Calculated compactness is the cluster compactness found using the cluster weight divided by the square of the cluster length (Eq. [4]), whereas compactness is the cluster compactness is rated using Organisation Internationale de la Vigne et du Vin descriptor 204 (Organisation Internationale de la Vigne et du Vin, 2009).
Histograms of grape cluster trait distributions of manually measured physical traits in the GE1025 (n = 127) population in 2018. Calculated compactness is the cluster compactness found using measurements of the cluster weight and length (Eq. [4]), whereas compactness is the cluster compactness as rated using Organisation Internationale de la Vigne et du Vin descriptor 204 (Organisation Internationale de la Vigne et du Vin, 2009).
Histograms of grape cluster trait distributions of automatically measured image-derived traits in the GE1025 population (n = 121) in 2017. The aspect ratio is calculated by dividing the maximum cluster width by the central axis length (Eq. [1]). The compactness shape factor is calculated using the cluster area multiplied by 4π and divided by the square of cluster perimeter (Eq. [2]). The roundness is calculated by dividing the square of the cluster perimeter by the cluster area (Eq. [3]).
Histograms of grape cluster trait distributions of automatically measured image-derived traits in the GE1025 (n = 127) population in 2018. The aspect ratio is calculated by dividing the maximum cluster width by the central axis length (Eq. [1]). The compactness shape factor is calculated using the cluster area multiplied by 4π and divided by the square of the cluster perimeter (Eq. [2]). The roundness is calculated by dividing the square of the cluster perimeter by the cluster area (Eq. [3]).
Contributor Notes
We thank the research vineyard managers John Thull, Jennifer Thull, and Colin Zumwalde for maintaining our populations, as well as undergraduate assistants Stien Iverson and David Tork, who helped with data collection. Soon Li Teh and James Luby built the GE1025 linkage map.
Cluster images are available at https://conservancy.umn.edu/handle/11299/202560. Image analysis script is available at https://github.com/underhillanna/GrapeImageAnalysis.
Current address for A.U.: Grape Genetics Research Unit, U.S. Department of Agriculture, Agricultural Research Service, 630 West North Street, Geneva, NY 14456
M.C. is the corresponding author. Email: clark776@umn.edu.
MN1264 (left), the noir maternal parent of GE1025, and MN1246 (right), the non-noir paternal parent of GE1025. Both are selections in the University of Minnesota grape breeding program and vary in their color and degree of cluster compactness.
A diagram of the camera, lighting, and backdrop setup for grape cluster image capture (not to scale). Grape clusters were suspended by an alligator clip in front of a flat white background and illuminated with a fluorescent light (Underhill et al., 2020).
An example of a single GE1025 grape genotype with raw (A), color-corrected (B), and segmented (C) cluster images. Raw (A) images were captured by a digital camera, and color-corrected (B) images were obtained using RawTherapee (Horváth and RawTherapee Development Team, 2017). Cluster images were then segmented (C) using Food Color Inspector (Albert et al., 2013).
A graphical representation of the individual morphological measurements processed from each grape cluster image. Traits collected included the cluster bounding box width and length, cluster central axis length, cluster perimeter length, and perpendicular (perp.) cluster widths at 25%, 50%, and 75% of the cluster central axis.
A flowchart of the grape cluster image capture, segmentation, and analysis process. Grape clusters were harvested when ripe (soluble solids >20%); then, images were captured in Nikon Electronic Format (.NEF) using a digital camera and gPhoto2 (Waugh et al., 2017). Images were adjusted using RawTherapee (Horváth and RawTherapee Development Team, 2017) and compressed as Tagged Image File Format (.TIFF) files; then, they were segmented by color using Food Color Inspector (Albert et al., 2013) and exported as Portable Network Graphics (.PNG) files. Morphological measurements were performed for the segmented images using a custom MATLAB (MathWorks, Natick, MA) script.
Actual calculated grape cluster compactness values (calculated using Eq. [4]) and cluster compactness values predicted by the partial least squares (PLS) model plotted against each other, along with their regression line (r = 0.90), in the test segment of the data comprised of 373 images with all genotypes in GE1025 represented.
Relative importance of each image-derived grape cluster morphological trait in the partial least squares (PLS) model. The area-to-perimeter ratio was the most important trait in the model, followed by aspect ratio (Eq. [1]) and compactness shape factor (Eq. [3]). In contrast, cluster roundness (Eq. [2]) was the least important trait in the model; the central axis length and percentage of cluster space pixels were also relatively unimportant.
A grape cluster image with similarly colored rachis and berries results in a poorly segmented image when using a color-based segmentation method. Red represents pixels marked as “berry” and green represents pixels marked as “rachis.” Here, many berry pixels are incorrectly classified as rachis (A) and many rachis pixels are incorrectly classified as berry (B). These errors can lead to inaccurate grape cluster measurements.
Histograms of grape cluster trait distributions of manually measured physical traits in the GE1025 (n = 121) population in 2017 with parental (MN1264, MN1246) averages denoted on each histogram. Calculated compactness is the cluster compactness found using the cluster weight divided by the square of the cluster length (Eq. [4]), whereas compactness is the cluster compactness is rated using Organisation Internationale de la Vigne et du Vin descriptor 204 (Organisation Internationale de la Vigne et du Vin, 2009).
Histograms of grape cluster trait distributions of manually measured physical traits in the GE1025 (n = 127) population in 2018. Calculated compactness is the cluster compactness found using measurements of the cluster weight and length (Eq. [4]), whereas compactness is the cluster compactness as rated using Organisation Internationale de la Vigne et du Vin descriptor 204 (Organisation Internationale de la Vigne et du Vin, 2009).
Histograms of grape cluster trait distributions of automatically measured image-derived traits in the GE1025 population (n = 121) in 2017. The aspect ratio is calculated by dividing the maximum cluster width by the central axis length (Eq. [1]). The compactness shape factor is calculated using the cluster area multiplied by 4π and divided by the square of cluster perimeter (Eq. [2]). The roundness is calculated by dividing the square of the cluster perimeter by the cluster area (Eq. [3]).
Histograms of grape cluster trait distributions of automatically measured image-derived traits in the GE1025 (n = 127) population in 2018. The aspect ratio is calculated by dividing the maximum cluster width by the central axis length (Eq. [1]). The compactness shape factor is calculated using the cluster area multiplied by 4π and divided by the square of the cluster perimeter (Eq. [2]). The roundness is calculated by dividing the square of the cluster perimeter by the cluster area (Eq. [3]).
