Click on author name to view affiliation information
Measuring plant characteristics via image analysis has the potential to increase the objectivity of phenotypic evaluations, provides data amenable to quantitative analysis, and is compatible with databases that aim to combine phenotypic and genotypic data. We describe a new tool, which is implemented in the Tomato Analyzer (TA) software application, called Color Test (TACT). This tool allows for accurate quantification of color and color uniformity, and allows scanning devices to be calibrated using color standards. To test the accuracy and precision of TACT, we measured internal fruit color of tomato (Solanum lycopersicum L.) with a colorimeter and from scanned images. We show high correlations (r > 0.96) and linearity of L*, a*, and b* values obtained with TACT and the colorimeter. We estimated genotypic variances associated with color parameters and show that the proportion of total phenotypic variance attributed to genotype for color and color uniformity measured with TACT was significantly higher than estimates obtained from the colorimeter. Genotypic variance nearly doubled for all color and color uniformity traits when collecting data with TACT. This digital phenotyping technique can also be applied to the characterization of color in other fruit and vegetable crops.
Digital phenotyping aims to accurately describe a trait based on analysis of electronic images. Computer-based analysis of objects from digital images has the potential to increase the objectivity of data collection while reducing subjective characterization that is typically prone to bias. There are a number of computer image acquisition and analysis techniques for color in foods such as apple [Malus ×domestica Borkh (Leemans et al., 2002; Li et al., 2002)], banana [Musa cavendishii L. (Mendoza and Aguilera, 2004)], chicory [Cichorium intybus L. (Zhang et al., 2003)], as well as seed analysis (Granitto et al., 2002; Sako et al., 2001; Shahin and Symons, 2001) and meat (O'Sullivan et al., 2003; Tan, 2004). Color image analysis is also prevalent in floricultural crops such as lisianthus [Eustoma grandiflorum Grise. (Yoshioka et al., 2006)], and begonia [Begonia ×tuberhybrida Voss. (Lootens et al., 2007)]. Digital color analysis is also performed in plant pathology to quantify lesions on diseased leaves (Kwack et al., 2005).
Objective and systematic descriptions, trait ontologies, are being developed in the plant sciences for database retrieval and archiving (reviewed in Brewer et al., 2006; Ilic et al., 2007). This trend is stimulated, in part, by a desire to link trait descriptions to the growing databases of sequence information. Tomato has become the prominent model horticultural crop for studies in genetics and genomic sciences. With extensive resources, including 357,477 expressed sequence tags (National Center for Biotechnology Information, 2008) and a genome sequencing project focused on euchromatin (Mueller et al., 2005), there is promise for research that seeks to integrate emerging sequence resources with phenotypic variance. Fulfilling this promise will require extensive data for the traits studied. Immortal populations (e.g., recombinant inbred populations and inbred backcross populations) consist of nearly homozygous lines that preserve the genetic integrity of mapping populations. These populations serve as a resource and allow for replication of experiments and extensive analyses from different laboratories. Phenotypic and molecular characterization of such populations can be stored in public databases for use by other researchers. The Tomato Analyzer (TA) software application was developed to facilitate the collecting and sharing of data related to fruit size and shape and the identification of genes that contribute to quantitative variation in morphology (Brewer et al., 2006). We describe a new module implemented in TA that can accurately collect objective data for color from digital images.
Measuring color from digital images requires standardization and interpretation because digital devices use a color space that is not standardized, is nonlinear, and may vary between hardware devices and software applications. In the Red Green Blue (RGB) color space, each pixel is represented in the computer or interface hardware as values of red, green, and blue. In contrast, color spaces such as CIELab were designed to approximate human perception of color [Commission Internationale de l'Éclairage (CIE), 1978]. CIELab color space is a reference standard and is most commonly used for measuring object color. Color data collected in the dimensions of the CIELab color space can be archived and used for quantitative analysis of color.
Color holds an important economic role in horticultural crops. For fresh or processed products, color is one of the primary determinants of quality, along with texture, size, and flavor (Picha, 2006). In the case of tomato, color and color uniformity contribute to quality. The presence of yellow shoulder disorder (YSD) is a major quality constraint. YSD is a blotchy ripening disorder that is characterized by discolored regions under the epidermis of mature fruit. Cells from YSD tissue are smaller and more randomly organized, and the conversion from chloroplasts to chromoplasts is altered (Francis et al., 2000). Variation for YSD within fruit and among fruit in plots explained more than 75% of the variation for color (Sacks and Francis, 2001). Color disorders are also an economic problem. U.S. Department of Agriculture (USDA) processor grades are largely determined by the amount of off-color tissue in products (USDA, 2005). To improve the output of high-quality product, some processors structure contracts such that growers receive premiums for fruit based on color and color uniformity. Discoloration due to YSD also reduces lycopene and beta-carotene concentrations in tissue affected by YSD (Darrigues et al., 2008). A reduction in the incidence of YSD could benefit producers, processors, and consumers.
Our objective was to implement a new digital image analysis tool, Color Test (CT), as part of the TA software application (Brewer et al., 2006). The use of flat-bed scanners to acquire data has been reported (Kleeberger and Moser, 2002; Kwack et al., 2005; Shahin and Symons, 2001). However, the software available to analyze color images is not fully automated in these applications and requires many manual adjustments (Shahin and Symons, 2003). TACT is capable of collecting and analyzing color parameters in an efficient, accurate, and high-throughput manner. We evaluated a tomato population for color and color uniformity using TACT and a colorimeter and estimated variance components associated with these parameters. To assess the applicability of TACT to crops other than tomato, we tested the software using images of other fruit and vegetables for which color is an important trait.
The TA software was previously described by Brewer et al. (2006). Briefly, it was implemented in the programming language C++ using Visual Studio 6.0 (Microsoft Corp., Redmond, WA). The image input/output was made possible via the image processing library Computer Vision and Image Processing 3.7c. TA was designed to run on the Windows operating system, including Vista. The program is free and can be used for academic or private purposes (Van der Knaap, 2008).
TACT is designed to collect objective color measurement from JPEG images. We collected images using a flatbed scanner covered with a cardboard box to minimize the effect of shadow and provide a black background (Fig. 1). Images of fruit taken with a digital camera on a black background are also appropriate. Instructions for collecting, importing, and analyzing color from images are available in the TACT manual (Darrigues, 2007).
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 133, 4; 10.21273/JASHS.133.4.579
The TA software automatically recognizes and outlines images of fruit, and the color test module records RGB values of each pixel of the selected object and translates them into average L*, a*, and b* values from the CIELab color space (CIE , 1978). The algorithm implemented in TACT to convert RGB values to L*, a*, and b* values can be adjusted to account for the illuminant (D65 or C) and observer angle (2° or 10°). Converting RGB to L*, a*, and b* was accomplished in three steps according to CIE colorimetry standards (CIE, 2007). First, RGB values were scaled to a perceptually uniform color space [Eq. 1]:
Scaled RGB values were then converted to XYZ tristimulus values using the following relationships [Eq. 2]:
The XYZ values were converted to L*, a*, and b* values using the following relationships [Eq. 3]:
where
Yn, Xn, and Zn are the tristimulus values of the illuminant and observer angle. For illuminant C, observer angle 2°, Xn = 98.04, Yn = 100.0, and Zn = 118.11. For illuminant D65, observer angle 10°, Xn = 94.83, Yn = 100.0, and Zn = 107.38.
The L*a*b × values were then used to calculate chroma as
. Hue was calculated as
for a* > 0 and as
for a* < 0. The L* coordinate indicates darkness (≈0) to lightness (≈100) of color. Chromaticity coordinates, a* and b*, indicate color directions: +a* is the red direction, −a* is the green direction, +b* is the yellow direction and –b* is the blue direction. Hue is an angular measure from 0 to 360, which represents basic color. Chroma is the saturation or vividness of color. In addition to converting RGB values to L*, a*, and b* and calculating chroma and hue from these components, an algorithm was written for TACT to compute luminosity from the following relationship [Eq. 4]:
where maxCol is the highest of the R, G, and B values of an analyzed pixel, and minCol is the lowest value. Luminosity accounts for the variable sensitivity of the human eye to radiation at various wavelengths; it defines brightness. The output of the color analysis provides averaged values of R, G, B, luminosity, L*, a*, b*, hue, and chroma.
Several parameters in TACT can be adjusted by the user before analysis. To help differentiate cut surfaces of the fruit from adjacent peel, TACT was designed to allow the user to modify the outline by adjusting the minimum blue value from the RGB color space. In generating the data set for this study, we used a minimum blue value of 30 to define the boundaries. Another option was developed to let the user define two parameters with specific hue ranges of interest. TACT returns the proportion (%) of pixels that fall into the designated hue range. We defined our parameters as percentage of YSD (%YSD), which represents yellow, green-yellow color, and percentage of red (%RED), which corresponds to the desired red color of tomato internal tissue. Our hue ranges were 60 to 120 for %YSD and 0 to 48 for %RED.
To determine the relationship between colorimeter and TA data, we collected color readings from 247 standard Munsell color plates (X-Rite, Grand Rapids, MI) ranging from 2.5R to 10R (Red), 5Y to 10Y (Yellow), 2.5GY to 10GY (Green-Yellow), 2.5YR to 5YR (Yellow-Red), 5Y (Yellow), 5GY (Green-Yellow), 5G (Green), 5BG (Blue-Green), 5B (Blue), 5PB (Purple-Blue), 5P (Purple), and 5RP (Red-Purple). Absolute color measurements were collected for each standard plate with a colorimeter (CR300; Minolta, Ramsey, NJ). In addition, a JPEG image was collected from scanning each plate with a flatbed scanner (HP Scanjet 3970; Hewlett-Packard, Palo Alto, CA) and was analyzed with the color function of TA. Values for L*, a*, and b* were recorded from each method. Of the 247 standard color plates, 28 were chosen to span a range of colors observed in tomatoes. These plates were custom-made into a 28-patch color checker to be used for scanner calibration (Darrigues, 2007).
We tested the precision of our color measurements with three different scanners: HP Scanjet 3970, HP Scanjet 5300C, and Microtek ScanMaker 6000 (Microtek, Carson, CA). We collected a JPEG image of the 28-patch color checker at 200 dpi with each scanner. Each of the 28 patches was considered an individual object and was analyzed for color. We collected RGB data and converted it to estimates of L*, a*, and b* measurements for each patch using TACT. We also obtained colorimeter L*, a*, and b* data from each patch, and applied linear regression to determine slopes, y-intercepts, and regression coefficients. The TACT dialog box (Fig. 1) allows users to enter correction values for slope and y-intercept as a way to calibrate the device used in collecting images. These correction values, obtained by regression based on color standards, are entered as the inverse of the slope and the negative of the y-intercept for the L*, a*, and b* regressions (Table 1).
An inbred backcross (IBC) population derived from crosses with S. lycopersicum processing cultivars (OH832, OH8245, OH9241, and OH9242) was evaluated for color and color uniformity. The original F1 crosses were OH9242 × OH8245, OH832 × OH8245, and OH9241 × OH8245. Each F1 was backcrossed to the recurrent parent, OH8245, and individual BC1 progeny were again backcrossed to OH8245 to obtain BC2 plants. These BC2 were then selfed to the BC2S4 generation. Because the recurrent parent was the same and because OH9241, OH9242, and OH832 share a significant portion of their pedigree by descent, the IBC was considered a single population. Field trials were grown near Fremont, OH, at the Ohio Agricultural Research and Development Center (OARDC) North Central Agricultural Experimental Station in 2004 and 2005. An augmented design was implemented in both years to evaluate each IBC genotype (n = 179, r = 1) and replicated checks (n = 4, r ≥ 5). Each plot consisted of 20 plants per genotype spaced 30 cm apart, with plots spaced 150 cm apart. All field plots were planted and maintained following conventional practices (Precheur et al., 2004). The plots were harvested when 80% of the fruit were ripe.
Measurements of color were collected using a colorimeter (CR300) as described previously (Sacks and Francis, 2001) and from digital images that were analyzed by TACT. We used a flatbed scanner (HP Scanjet 3970) to scan the cut surface of the proximal end for 12 fruit within each plot and saved a JPEG image. Images were analyzed and color data were collected for two data sets: TA_Unadj, for which no modification was made to the automatic fruit boundaries, and TA_Adj, for which the boundaries were adjusted based on visual inspection to better represent the proximal end of each fruit. Figure 2 provides an example of the boundaries with the two TACT methods used to generate the TA_Unadj and TA_Adj data sets. The batch feature of TACT was used to analyze both data sets, whereby 30 to 50 images were selected per batch and analyzed for color.
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 133, 4; 10.21273/JASHS.133.4.579
Each data set generated from TACT consisted of L*, a*, b*, hue, and chroma values to represent absolute color. For tomato, improved color is characterized by lower L*, b*, and hue values and higher a* values for a more intense, red color. The interpretation of chroma is ambiguous as high chroma due to high b* values represents poor color, whereas high chroma due to high a* values represents good color. In addition, we measured color uniformity defined by the parameters %YSD and %RED.
Two color measurements using the colorimeter were also collected from the same fruit that were scanned for TACT color analysis. The two-point measurements were taken on mature red tissue and any discoloration present on the fruit shoulder. The difference between the two measurements, ΔHue and ΔChroma, provided an estimate of internal fruit color uniformity that is consistent with visual symptoms of YSD.
All statistical analyses were performed using SAS (version 9.1; SAS Institute, Cary, NC). Color standard data from colorimeter and TA were tested for normal distribution using the UNIVARIATE procedure. To determine the relationship between color data generated from the colorimeter and TACT, we used the regression procedure (PROC REG) to test the significance of the regressions.
The estimates of variance and standard errors were obtained using the restricted maximum likelihood (REML) method with the mixed model analysis of variance procedure (PROC MIXED). The model to estimate variance components for the IBC population was:
where Yijk was the color trait measured, μ was the overall mean, Gi was the effect due to the ith genotype, Tj was the effect due to the jth year, GTij was the effect due to the Genotype × Year interaction, and εijk was the experimental error. The percent total variance was reported to allow direct comparison between data sets. Using the estimates of variance components, broad-sense heritability (H) was determined using the following relationship:
where σ2 G was the estimate of the genotypic variance, σ2 G was the estimate of the year to year variance, σ2 GY was the estimate of the genotype × year interaction variance, and σ2 error was the estimate of the error variance. We assumed selection based on IBC line means across 2 years. To test whether methods of collecting data influenced the amount of phenotypic variance that we could partition into genotypic variance, the standard error associated with these estimates was used to perform a mean separation at α = 0.05.
To determine the precision and accuracy of the color data generated with TACT, we compared it to data collected with a colorimeter. The regressions of L*, a*, and b* values from the colorimeter onto TACT values showed a significant (P < 0.0001) linear relationship for all three parameters, with correlation coefficients greater than 0.96 (Fig. 3). Despite the linear relationship, the values between the colorimeter and TACT differed because the slope was not equal to 1 and the y-intercept was not equal to 0. The strong linear relationship suggested that calibration of the scanner used to generate digital images could be accomplished with simple adjustments to the equations used to calculate L*, a*, and b* values.
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 133, 4; 10.21273/JASHS.133.4.579
Three flatbed scanners were used to assess reproducibility and systematic differences in scanning devices for measuring L*, a*, and b* values from JPEG images (Fig. 4). The correlations between L*, a*, and b* values measured from the colorimeter and TA were high (r > 0.94) for all three scanners for each trait. Correlation values among scanners were the highest for L*, lightness, followed by a*, which measures color range from green to red. The lowest correlation values were found for b*, which measures color range from blue to yellow. We observed differences among the three scanners in slope and y-intercept values. For this reason, we implemented an option in the TACT dialog box (Fig. 1) to enter correction values for the slope and y-intercept as a way to calibrate the device used in collecting images.
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 133, 4; 10.21273/JASHS.133.4.579
To test whether TACT offered advantages over the colorimeter, we evaluated a breeding population for color and color uniformity using both approaches. Variance components for genotype, year, and the interaction genotype × year were estimated to elucidate the proportion of genotypic variance associated with each color parameter (Table 2). Among the three methods, the total phenotypic variation partitioned into genotype and genotype × year interaction ranged from 12% to 30%. The variance partitioned into year ranged from 0% to 9.6%. However, significantly more variance was partitioned into genotype using TACT than the colorimeter for all traits except chroma (Table 2). The proportion of genotypic variance for L*, a*, and b* measured with TACT was 2- to 4-fold greater than with the colorimeter. The ability to partition a greater portion of the phenotypic variation into genetic effects increases the potential for improving a trait by means of genetic manipulation.
In addition to evaluating absolute color (e.g., L*, a*, b*, hue, and chroma), color uniformity was also measured using two user-defined parameters in TACT, defined as %YSD and %RED, and Δ Hue and Δ Chroma from the colorimeter. Variance estimates partitioned ≈10% of the total variation for %YSD and 17% for %RED into the genotypic variance. With the colorimeter, 0% and 4% of the total variance was partitioned into the genotypic variance for Δ Chroma and Δ Hue, respectively. In addition, the error variance was lower for TACT than for the colorimeter for all parameters measured. Overall, 66% to 83% of the total phenotypic variance was in the error term for TACT, compared with 77% to 96% for the colorimeter. In our experimental design, error variance is equal to within-plot variance and complex interactions not controlled for in our sampling. The estimates of genotypic variance correspond to estimates of heritability that ranged from 0.11 to 0.17 for chroma, a*, and %YSD, and 0.211 to 0.275 for L, b*, hue, and %RED when TACT was used. With the colorimeter, H estimates were lower and ranged from 0.081 to 0.097 to L*, a*, and chroma, and 0.113 to 0.152 for b* and hue. With greater heritability estimates, we expect greater gains under selection for genetic improvement by measuring color with TACT.
We generated data sets to test two approaches to define fruit boundaries: TA_Adj and TA_Unadj (Table 2). The TA_Adj data set was compiled from images where boundaries were manually adjusted as needed. The second method, TA_Unadj, was used without making adjustments to the images before color analysis. TA_Unadj is less time-consuming, but is prone to include parts of the fruit, such as peel, deep cracks, or reflected light from the scanner that may bias the color values. However, between the two TA methods, there was no significant difference in the variance partitioned into genetic effects for all color parameters, suggesting that high-throughput analysis with TACT may be possible (Table 2).
We evaluated TACT with fruit and vegetable crops other than tomato. We tested potato (Solanum tuberosum L.), cucumber (Cucumis sativus L.), red plum (Prunus americana Marsh.), muskmelon (Cucumis melo L.), carrot (Daucus carota L.), and strawberry (Fragaria ×ananassa Duch). These selected crops and cultivars encompassed a range of colors and color uniformity (Fig. 5). TACT recognized the boundaries with precision for each crop, as expected with the contrasting colors between the background and the object (Brewer et al., 2006). The red-skinned potato had the highest L* value, which represents the brightest, closest-to-white tone (Table 3). TACT was able to detect a* values from −8 to 38, with the highest value given to the carrot with its orange tone. The carrot also had the highest b* value, lowest hue, and highest chroma, consistent with the deep orange color.
Citation: Journal of the American Society for Horticultural Science J. Amer. Soc. Hort. Sci. 133, 4; 10.21273/JASHS.133.4.579
To test the capability of measuring color uniformity, we defined the hue range of the first parameter as 70 to 120 for the proportion of pixels that fell into yellow-light yellow range. The red plum and muskmelon had 72% to 77% of the pixels falling into that range; the strawberry had 30% (data not shown). For the second parameter, we defined the hue range as 0 to 48 for the proportion of pixels that fell into red-orange range. The carrot had 25% of the pixels falling into that range, whereas the strawberry had only 15% of red-orange tissue. These results demonstrate that TACT can perform color analysis on a broad range of hue values and is not confined to the hue ranges common to the tomato fruit.
We developed a new module in the TA software application to collect objective color measurement based on digital images. This automated tool analyzes each pixel of a selected object and then translates it from RGB to L*, a*, and b* values. Our first objective was to test the accuracy of TACT against a colorimeter and to provide a calibration to account for differences in scanners. Empirical results demonstrated differences between devices due to hardware, software, or nonstandardized RGB values. Digital images collected from different sources can vary in color depending on the resolution, light source, and light quality. Three different scanners were used to scan color standards and test the precision among scanners with TACT. Although the L*, a*, and b* values computed from TACT correlated highly with those of the colorimeter for all three scanners, the slope and y-intercept values varied among scanners. Therefore, an option in TACT was developed to incorporate these values as a correction for L*, a*, and b*. Previously, an attempt to eliminate variability in brightness and color distribution due to scanner differences was reported (Shahin and Symons, 2003). However, the various calibration techniques exploited did not result in satisfying performance. In contrast, our technique in calibrating TACT was successful, user-friendly, reliable, and malleable.
Our second objective was to assess which technique can better partition observed phenotypic variation for color into genotypic variances. Compared with colorimetric data, the proportion of total phenotypic variance attributed to genotypic variance was significantly improved for all color and color uniformity traits when collecting data with TACT. Estimates of genotypic and phenotypic variances are the basis for determining heritability, which in turn provides an insight into the expected genetic gain and genetic improvement in a breeding program. We show greater line-mean heritability estimates for all traits measured with TACT, with the highest value for %RED. Moreover, the parameters for color uniformity measured with TACT, %YSD, and %RED are more informative in terms of variance estimates than the estimates for ΔHue and ΔChroma measured with the colorimeter. We hypothesize that is the case because color uniformity is better characterized when the entire surface is evaluated rather than the difference between two point measurements. Also, among the methods available with TACT, correcting the boundaries of the tomato fruit cut surface before color analysis (TA_Adj) does not partition significantly more of the genotypic variance from the total phenotypic variance compared with the analysis with unadjusted boundaries (TA_Unadj). The TA_Unadj is an appropriate method for high-throughput phenotyping as it does not require time-consuming manual adjustments before color analysis.
TACT was designed to be user-friendly with minimum requirements for running it, yet accurate and precise for collecting objective measurements. It facilitates data collection and management, and it requires equipment that is relatively more affordable. Tools for measuring color described in the literature require extensive environmental control, especially for the quality and quantity of light, shadow, and reflection. In contrast, the flatbed scanners we used to generate digital images for TACT color analysis only required a cardboard box as a cover to minimize the effect of shadow. For precision in the analysis of images generated from multiple scanners, we designed a user-friendly option in the dialog box of TACT for calibration relative to color standards. Other methods require a calibration of the system before each image is taken (Lootens et al., 2007) or with minimal calibration feasible (Wang-Pruski, 2006). In our experience, the application of TACT for color analysis from digital images is more accurate, precise, and less expensive than other methods.
Tomato Analyzer, as the name implies, was originally designed to analyze the morphology of tomato fruit. We developed a module for color measurement to expand the array of objective phenotypic analyses implemented. TACT was applied to other fruit and vegetables of various color and color uniformity. Overall, it was able to accurately capture and describe the characteristic color for each crop. Color uniformity was also well characterized for fruit that tend to have nonuniform pigmentation, as in the strawberry (Fig. 5). Its application could go beyond the color analysis of fresh crops. In food science, discoloration after processing or cooking can occur and is an important issue. Wang-Pruski (2006) reports the acquisition of potato tuber images to evaluate after-cooking darkening. Such discoloration could be measured with TACT by defining the specific range of hue values that best represent the undesired discoloration. We show that TACT is a tool that is reliable, precise, amenable, and affordable for digital image analysis of color.
Contributor Notes
We thank David Sullivan (Department of Mathematics and Computer Science, College of Wooster, OH) for his help in designing and optimizing the software application for TACT. We acknowledge Bert Bishop (Statistics and Computing Services, OARDC/OSU, Wooster, OH) for his suggestions on the analysis of the IBC population. We appreciate the help from Troy Aldrich for establishing field plots.
The development of Tomato Analyzer and its Color Test was supported by the National Science Foundation (grant no. DBI 0227541 to E.V.D.K.). A.D. was supported through the Ohio Agricultural Research and Development Center Research Enhancement Competitive Grants Program and through funds from the Mid-America Food Processors Association and from USDA/NRI grant 2004-01575.
Corresponding author. E-mail: francis.77@osu.edu.
