Abstract
Hands-on activities enhance learning and increase student satisfaction in horticulture courses. Nevertheless, hands-on activities can have widely different impacts on student learning. To achieve and maintain educational quality, instructors need to evaluate and improve activities. This research used text mining and sentiment analysis to gauge student sentiments about hands-on activities in a protected agriculture course. Students participated in five hands-on activities and submitted short reflective essays about them. Essay texts were separated into single-word (unigram) or three-word (trigram) objects. Unigrams were compared with general-use emotion lexica to extract student sentiments from the texts. Trigrams were used to assess essay content. All activities elicited positive sentiments among students. Trust, anticipation, and joy were the most prominent emotions identified. The activity focused on freeze protection was preferred over the other activities. Although other activities were also well received, they should be refined for future offerings. The presented method could be used to assess hands-on activities, leading to continuous improvement and successful implementation of experiential learning in horticulture courses.
Experiential learning is an integral part of agriculture education (Cheek et al., 1993). This teaching approach, which emphasizes the role of experience in the knowledge acquisition process (Kolb, 1984), is regularly used in horticulture courses and curricula. Experiential learning leads to better content retention and student performance in introductory horticulture courses (Bauerle and Park, 2012; St. Hilaire et al., 2009; Uchanski et al., 2015). Additionally, experiential learning enhances lower- and higher-order learning (Craver and Williams, 2014) and promotes skill development (Mahoney et al., 2015) in advanced horticulture courses. Experiential learning also aids in recruiting students into subsequent horticulture courses (Pritts, 2017). These benefits are consistent with findings in other science, technology, engineering, and mathematics (STEM) fields, where experiential learning positively influenced student learning and engagement (Freeman et al., 2014).
Previous research has shown that hands-on activities can have widely different impacts on student learning (Holstermann et al., 2010) and satisfaction (Henderson et al., 2018). Thus, to ensure educational quality, it is important to evaluate hands-on activities to identify which ones promote student learning and which need further refinement. Text mining is a tool that could be used for this purpose. Text mining consists of analyzing text data to gather insight into individuals’ preferences, feelings, and opinions (Hu and Liu, 2004). It uses set theory operations and labeled word lists (lexica) to categorize written language in an objective and scalable manner. Although this data analytics practice has become commonplace among businesses and marketers (Haywood and Mishra, 2019; Kim and Chun, 2019), text mining has limited use by horticulturists. Considering horticulture courses routinely generate text data from student assignments and evaluations, this is a missed opportunity. The present paper illustrates how text mining can be used to gauge student perception of experiential learning activities and inform teaching practices in a horticulture course.
Materials and methods
Greenhouse and Protected Agriculture (HOS 3430C) is a three-credit undergraduate course that combines lecture and laboratory at the University of Florida. During Spring 2018, the course had 37 students, 49% of whom were enrolled in horticulture majors (Horticultural Sciences or Plant Science) at the time they took the course. Five students were sophomores, 17 were juniors, and 15 were seniors. Twenty students were male and 17 were female.
Students participated in three faculty-led and two self-directed hands-on activities as a part of the course. Faculty-led activities were completed by all students as a group. Self-directed activities were completed individually by students after attending a demonstration of the activities by the instructor. Activity 1 consisted of building low tunnels by bending half-inch electrical conduits and installing 4-mil plastic over these hoops. Activity 2 consisted of preparing crops for overnight freezing temperatures by closing low tunnels, weighing down plastic with sand bags, and covering crops outside the low tunnels with hay. Activity 3 consisted of installing a 60% shadecloth in a 92-ft-long × 16-ft-wide hoop house. Activity 4 consisted of a morning or end of the day checkup of the teaching farm where students visually assessed crop, irrigation, and hydroponic system status. Activity 5 consisted of monitoring and adjusting nutrient solution pH and monitoring electrical conductivity (EC) in hydroponic systems. Activities 1 through 3 were faculty led, and activities 4 and 5 were self-directed. After each activity, students submitted a 200-word reflective essay about their experience with it. Student essays were submitted through the institutional learning management system. Reflective essay prompts included common and unique questions for each activity (Table 1). This follow-up assignment was intended to allow students a chance to reflect on the activities in the context of their individual learning process and personal and professional goals. This approach is consistent with recommended practices in fieldwork design (Remmen and Froyland, 2014) and provided the text data for the present research.
Questions included in reflective essay prompts provided to students after hands-on activities in a protected agriculture course at the University of Florida during Spring 2018.
Text mining and sentiment analysis.
Students’ essays, gender, major, and class standing were aggregated in a deidentified database according to institutional review board protocol 2019-00119. Essays were imported as text strings into statistical software (R version 3.5.1; R Foundation for Statistical Computing, Vienna, Austria) and were divided into single-word (unigrams) or three-word objects (trigrams) using the unnest_tokens command in R package tidytext (Silge and Robinson, 2016). Frequent trigrams were identified and used to assess the main content themes in each essay. The most important words in each essay were identified using the term “frequency-inverse document frequency index” [tf-idf (Silge and Robinson, 2016)]. This unitless index quantifies how important a unigram is to essays from each activity, compared with all essays submitted for the course. tf-idf was computed in a corpus formed by unigrams from all essays using the tf_idf command in R package tidytext (Silge and Robinson, 2016).
Total word counts per student per essay were produced. Highly common words that do not carry meaning (e.g., “the,” “of,” “to”) were removed using the anti_join command in R package dplyr [version 0.7.6 (Wickham et al., 2018)]. Unigrams that passed this filter were considered lexical words. Lexical word counts per student per essay were determined. Lexical density was calculated as the ratio of lexical to total words in each essay. Lexical density was compared among activities, genders, majors, and class standings using Kruskal-Wallis nonparametric analysis of variance. Where significant effects were identified, post hoc pairwise Mann-Whitney U tests were used to compare variable levels. Probability values were adjusted using the Bonferroni method.
Sentiment analysis was accomplished by comparing lexical words in each essay with words in term-sentiment lexica using the inner_join command in R package dplyr [version 0.7.6 (Wickham et al., 2018)]. Three general-use lexica were used: one developed to analyze microblogs (Nielsen, 2011) (subsequently referred to as AFINN), one developed to summarize opinions from product reviews (Hu and Liu, 2004) (subsequently referred to as BING), and one focused on word-polarity associations (Mohammad and Turney, 2013) (subsequently referred to as NRC). When using BING and NRC, overall sentiment was computed by subtracting negative from positive sentiment scores for each student. Analysis with AFINN produced overall sentiment scores directly. Overall sentiment scores were compared using the same statistical methods detailed above. The NRC lexicon was also used to calculate emotion scores for anger, anticipation, disgust, fear, joy, sadness, surprise, and trust—the eight basic emotions proposed by Plutchik (1994). Emotion scores represent the average number of words related to each emotion appearing in each student essay. To gauge the impact protected agriculture terminology had on emotion scores, sentiment analysis using the NRC lexicon was repeated after excluding lexical words present in the course description (University of Florida, 2019). The effect of protected agriculture terminology was assessed through Kruskal-Wallis nonparametric analysis of variance. Step-by-step annotated code used for this analysis is publicly available (Nunez, 2019). Graphs were generated using data visualization software [R package ggplot2 (Wickham, 2016)].
Results
Essay content and sentiment analysis.
On average, student essays contained 210 words. Between 3% and 11% of students missed their reflective essay submissions. Thus, the number of essays analyzed per activity was < 37. All students who submitted reflective essays participated in the hands-on activities. Student reflective essays were topical with respect to each activity. Frequent trigrams consistently referred to the activities carried out (e.g., “shade cloth installation”), the tools used (e.g., “the low tunnel”), or concepts applied (e.g., “the freeze protection”) (Fig. 1). First-person descriptions of the activities were also among the most common trigrams (e.g., “I completed my,” “I learned how,” and “I participated in”). Technical terminology was key in distinguishing between essays from each activity. Terms such as “low tunnel,” “freeze protection,” “shadecloth,” and “EC meter” were among the most important words corresponding to each activity according to tf-idf analysis (Fig. 2). Average lexical density was 0.38 for activity 1, 0.37 for activity 2, 0.38 for activity 3, 0.33 for activity 4, and 0.35 for activity 5. Essays from activity 4 had lower lexical density than essays from other activities (P ≤ 0.007). Essays from activity 5 had lower lexical density than essays from activity 3 (P = 0.009). All other contrasts were not significant. Essay lexical density was unaffected by major (P ≥ 0.36 in all cases) or class standing (P ≥ 0.06 in all cases). Gender affected essay lexical density only in essays from activity 1 (P = 0.03), where essays written by male students were more lexically dense than essays written by female students. All other contrasts were not significant.
Frequent three-words (trigrams) used in student reflective essays after hands-on activities where they (A) built low tunnels, (B) prepared the student farm for an overnight freeze, (C) installed shadecloth in a high tunnel, (D) performed a routine checkup in the student farm, and (E) checked and adjusted nutrient solution quality. Hands-on activities took place in a protected agriculture course at the University of Florida during Spring 2018.
Citation: HortTechnology hortte 30, 1; 10.21273/HORTTECH04527-19
Most important words in student reflective essays according to term frequency-inverse document frequency (tf-idf) index. Students participated in hands-on activities where they (A) built low tunnels, (B) prepared the student farm for an overnight freeze, (C) installed shadecloth in a high tunnel, (D) performed a routine checkup in the student farm, and (E) checked and adjusted nutrient solution quality. Hands-on activities took place in a protected agriculture course at the University of Florida during Spring 2018.
Citation: HortTechnology hortte 30, 1; 10.21273/HORTTECH04527-19
On average, all activities elicited positive sentiments among students regardless of lexicon used (Fig. 3). Greater than 74% of the students expressed positive sentiments about the activities. Activity 2 was best received by students (P ≤ 0.03 in all pairwise comparisons). Other activities did not differ in student perception. Major and class standing did not affect overall sentiments (P ≥ 0.11 in all cases). Gender only affected overall sentiment in activity 5, where male students expressed more positive sentiments than female students [6.94 vs. 4.88 (P = 0.02)]. Student essays contained terms associated with anger, anticipation, disgust, fear, joy, sadness, surprise, and trust (Fig. 4). Trust was the most prominent sentiment across all activities. Anticipation and joy were also common. Top 10 terms associated with these sentiments can be found in Supplemental File 1. Protected agriculture terminology did not have an effect in overall student sentiments (P = 0.38).
Overall student sentiments about five hands-on activities in a protected agriculture course at the University of Florida during Spring 2018. Students built low tunnels (activity 1), prepared the student farm for an overnight freeze (activity 2), installed shadecloth in a high tunnel (activity 3), performed a routine checkup in the student farm (activity 4), and checked and adjusted nutrient solution quality (activity 5). Sentiments were extracted using text mining and three general-use lexica: AFINN (Nielsen, 2011), BING (Hu and Liu, 2004), and NRC (Mohammad and Turney, 2013). Most essays reflected positive overall sentiments about hands-on activities in the course, as indicated by overall sentiment scores >0. Black dots indicate outliers.
Citation: HortTechnology hortte 30, 1; 10.21273/HORTTECH04527-19
Average number of terms associated with anger, anticipation, disgust, fear, joy, sadness, surprise, and trust in student essays about hands-on activities where they (A) built low tunnels, (B) prepared the student farm for an overnight freeze, (C) installed shadecloth in a high tunnel, (D) performed a routine checkup in the student farm, and (E) checked and adjusted nutrient solution quality. Hands-on activities took place in a protected agriculture course at the University of Florida during Spring 2018.
Citation: HortTechnology hortte 30, 1; 10.21273/HORTTECH04527-19
Discussion
According to experiential learning theory, hands-on experiences constitute the basis for student knowledge acquisition (Kolb, 1984). It follows that individual experience, as opposed to the experiences of others, is critical to learning. The frequency with which essays contained first-person descriptions of the activities suggests that most students had individual experiences that enhanced their learning process. Additionally, these results indicate student essays were a reflection about their experience in the hands-on activities.
Student essays were topical but casual. Common trigrams and tf-idf indices suggest essays focused on each hands-on activity, as expected from the assignment instructions. Lexical densities reported here are lower than typical lexical densities of newspaper articles or Wikipedia (Wikimedia Foundation, San Francisco, CA) entries (Analyze my Writing, n.d.), suggesting student essays were wordy, conversational, repetitive, or a combination of these. Considering text mining has been applied to texts as short as 180 characters (Nielsen, 2011), essay length was not considered a limitation in the present study.
All hands-on activities were well received by students, as evidenced by positive overall sentiment scores produced using three lexica. Positive emotions (trust, anticipation, and joy) were prominent in the detailed sentiment analysis. A considerable body of literature has documented the impact positive emotions can have on content retention and recollection (Willis, 2007). Additionally, positive emotions might explain the augmented interest in plants and plant science courses reported by students who participated in experiential learning activities (Pritts, 2017). Ultimately, the fact that hands-on activities elicited positive sentiments bodes well for students’ learning process and justifies the time and resources invested in creating these course features.
Experiential learning generally leads to positive academic outcomes (Bauerle and Park, 2012; Craver and Williams, 2014; Pritts, 2017; St. Hilaire et al., 2009; Uchanski et al., 2015), but not all activities have the same impact on learning (Holstermann et al., 2010). Activity 2 elicited the most positive sentiments among students across genders, majors, and class standings. It is notable that the activity focused on freeze protection emerged as the students’ favorite. Activity 2 took place the day before an overnight freeze. Thus, timing might have increased student interest in the activity. Previous research found topic interest influenced learning outcomes (Schiefele and Krapp, 1996) and student experience (Schiefele, 1996).
Student sentiments can be informative when evaluating hands-on activities. Activities 1, 3, 4, and 5 were not as well received as activity 2, suggesting they could be revised for future offerings of the course. Results from this analysis should be considered, along with student learning outcomes, activity relevance, and resource availability when planning and executing these activities in subsequent semesters. Thus, the method presented here can contribute to the evaluation and continuous improvement of hands-on activities.
Text mining of horticulture texts is not without its challenges. Lack of discipline-specific lexica is a current limitation. Terms such as “protected” and “structure,” which are routinely used in protected agriculture courses, are associated with trust by general-use lexica. Although excluding obvious protected agriculture terminology did not fundamentally change the results of the analyses presented, future studies should address this limitation by developing discipline-specific lexica that help interpret agricultural texts.
This study showed that experiential learning fostered individual experiences and elicited positive sentiments among students. The method described could be used to evaluate hands-on activities in horticulture courses in an objective and scalable way.
Literature cited
Analyze my writing. Lexical density 31 July 2019 <http://www.analyzemywriting.com/lexical_density.html>
Bauerle, T.L. & Park, T.D. 2012 Experiential learning enhances student knowledge retention in the plant sciences HortTechnology 22 715 718
Cheek, J.G., Arrington, L.R., Carter, S. & Randell, R.S. 1993 Relationship of supervised agricultural experience program participation and student achievement in agricultural education J. Agric. Educ. 35 1 5
Craver, J.K. & Williams, K.A. 2014 Assessing student learning from an experiential module in a greenhouse management course using hydroponics and recirculating solution culture HortTechnology 24 610 617
Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H. & Wenderoth, M.P. 2014 Active learning increases student performance in science, engineering, and mathematics Proc. Natl. Acad. Sci. USA 111 8410 8415
Haywood, M.E. & Mishra, A. 2019 Building a culture of business analytics: A marketing analytics exercise Intl. J. Educ. Mgt. 33 86 89
Henderson, C., Khan, R. & Dancy, M. 2018 Will my student evaluations decrease if I adopt an active learning instructional strategy? Amer. J. Phys. 86 934 942
Holstermann, N., Grube, D. & Bögeholz, S. 2010 Hands-on activities and their influence on students’ interest Res. Sci. Educ. 40 743 757
Hu, M. & Liu, B. 2004 Mining and summarizing customer reviews, p. 168–177. In: Proc. Tenth Special Interest Group on Knowledge Discovery and Data Science Intl. Conf. Knowledge Discovery and Data Mining, KDD ’04. Assn. Computer Machinery, New York, NY
Kim, E. & Chun, S. 2019 Analyzing online car reviews using text mining Sustainability 11 1611 1633
Kolb, D. 1984 Experiential learning: Experience as the source of learning and development. Prentice Hall, Englewood Cliffs, NJ
Mahoney, B.A., Tree, L. & Retallick, M.S. 2015 The impact of two experiential learning programs: The graduates’ perspective North Amer. College Teachers Agr. J. 59 319 325
Mohammad, S. & Turney, P. 2013 Crowdsourcing a word-emotion association lexicon Comput. Intell. 29 436 465
Nielsen, F.A. 2011 A new ANEW: Evaluation of a word list for sentiment analysis in microblogs, p. 93–98. In: Proc. European Semantic Web Conf. 2011 Workshop Making Sense of Microposts: Big things come in small packages. <http://arxiv.org/abs/1103.2903>
Nunez, G.H. 2019 Sentiment analysis. 11 Nov. 2019. <https://github.com/ghnunez/nunezlabuf>
Plutchik, R. 1994 The psychology and biology of emotion. HarperCollins, New York, NY
Pritts, M.P. 2017 Hands-on horticulture: A course for building enrollments in plant science courses HortTechnology 27 704 709
Remmen, K.B. & Froyland, M. 2014 Implementation of guidelines for effective fieldwork designs: Exploring learning activities, learning processes, and student engagement in the classroom and the field Intl. Res. Geogr. Environ. Educ. 23 103 125
Schiefele, U. 1996 Topic interest, text representation, and quality of experience Contemp. Educ. Psychol. 21 3 18
Schiefele, U. & Krapp, A. 1996 Topic interest and free recall of expository text Learn. Individ. Differ. 8 141 160
Silge, J. & Robinson, D. 2016 Tidytext: Text mining and analysis using tidy data principles in R J. Open Source Software 1 3
St. Hilaire, R., Sammis, T.W. & Mexal, J.G. 2009 Integrating hoop house construction and operation into an undergraduate general education horticulture class HortTechnology 19 445 451
Uchanski, M.E., Grover, K., VanLeeuwen, D. & Goss, R. 2015 Integrating hoop house construction and environmental data interpretation into an undergraduate general education plant science course HortTechnology 25 247 252
University of Florida 2019 Undergraduate courses in horticultural sciences. 31 July 2019. <https://catalog.ufl.edu/UGRD/courses/agl_ag_horticultural_sciences/>
Wickham, H. 2016 ggplot2: Elegant graphics for data analysis. Springer-Verlag, New York, NY
Wickham, H., François, R., Henry, L. & Müller, K. 2018 dplyr: A grammar of data manipulation. R package version 0.7.6. 31 July 2019. <https://CRAN.R-project.org/package=dplyr>
Willis, J. 2007 The neuroscience of joyful education. 64. 31 July 2019. <https://www.psychologytoday.com/files/attachments/4141/the-neuroscience-joyful-education-judy-willis-md.pdf>
Top 10 terms associated with trust, joy, and anticipation in student essays about hands-on activities in a protected agriculture course at the University of Florida during Spring 2018.