Curriculum Evaluation: An Outcomes Assessment of the Department of Horticulture at Iowa State University

in HortTechnology
View More View Less
  • 1 1Graduate Research Assistant, Department of Horticulture, Iowa State University, Ames, IA 50011
  • | 2 2Associate Professor, Department of Horticulture, Iowa State University, Ames, IA 50011
  • | 3 3Assistant Professor, Department of Agricultural Education and Studies, Iowa State University, Ames, IA 50011

The Iowa State University undergraduate horticulture program has been nationally ranked as one of the best in the nation. Regular and systematic outcomes assessment is one means to evaluate the program for purposes of sustaining this level of excellence. The purpose of this study was to conduct a survey of horticulture graduates to determine preparedness when entering the workforce, departmental effectiveness, and to evaluate how well graduates met departmental learner outcomes. A 59-question survey instrument was distributed electronically to a proportional sample of 221 horticulture alumni who graduated between 2000 and 2006. The response rate was 47%. Results showed that graduates found jobs quickly, and a majority (76.7%) noted that their first and current jobs were highly related to their degree. Furthermore, 41.8% rated themselves as adequately prepared, 37.9% as more than adequately prepared, and 15.5% as exceptionally well prepared for their first job. Respondents also ranked their abilities related to the 33 learner outcomes questions as good to excellent.

Abstract

The Iowa State University undergraduate horticulture program has been nationally ranked as one of the best in the nation. Regular and systematic outcomes assessment is one means to evaluate the program for purposes of sustaining this level of excellence. The purpose of this study was to conduct a survey of horticulture graduates to determine preparedness when entering the workforce, departmental effectiveness, and to evaluate how well graduates met departmental learner outcomes. A 59-question survey instrument was distributed electronically to a proportional sample of 221 horticulture alumni who graduated between 2000 and 2006. The response rate was 47%. Results showed that graduates found jobs quickly, and a majority (76.7%) noted that their first and current jobs were highly related to their degree. Furthermore, 41.8% rated themselves as adequately prepared, 37.9% as more than adequately prepared, and 15.5% as exceptionally well prepared for their first job. Respondents also ranked their abilities related to the 33 learner outcomes questions as good to excellent.

Outcomes assessment is a process used to continually improve student learning by systematically assessing the effectiveness of and adjusting the curriculum. Outcomes assessment provides a way for students to show their achievement of learner outcomes and their ability to use their gained knowledge effectively in the workplace (Mort and Messerschmidt, 2001). Assessment is most effective as a cyclical process that uses appropriate criteria and standards for learning quality. Data must be gathered and analyzed to determine how well student performance matches the set standards, and steps must then be taken to improve performance (Angelo, 1995). For curriculum assessment, the goal need not be to determine the success or failure of an individual—whether that is a student, course, or faculty member—but the goal is to assess the curriculum as a whole. Outcomes assessment can determine where the successes and failures lie, and can provide direction on how to build upon the curriculum to improve student learning (University of Arkansas, 2007).

Universities are being challenged to improve undergraduate education, to re-examine fundamental values, and to make education the primary goal (Suvedi and Heyboer, 2004). Outcomes assessment has become a required part of higher education, with the overall goal being to improve the quality of student learning (Newcomer, 2000; Pintar et al., 1999).

In 1996, the Iowa State University (ISU) horticulture program was ranked third in the nation (Gourman, 1996). To continue this level of success, regular and systematic assessments must be done to evaluate the curriculum. To this end, the ISU Department of Horticulture has developed a set of learner outcomes for all undergraduate courses offered in the department. These outcomes center on the core skills and knowledge that students need to demonstrate for the undergraduate to have met the objectives of the curriculum.

Learner outcomes for the ISU Department of Horticulture are summarized by the following four broad statements (Iowa State University Department of Horticulture, 2007):

  1. Graduates will have theoretical and practical scientific knowledge, which they will be able to apply to the efficient and sustainable production of horticulture crops;

  2. Graduates will develop professional skills in the areas of communication and leadership;

  3. Graduates will be aware of the many facets of horticulture throughout the nation and the world;

  4. Graduates will possess a sound ethical and value system, which will allow them to recognize moral and ethical conflicts and practice tolerance and celebration of diverse cultures and philosophies.

The ISU Department of Horticulture has implemented multiple methods, including direct and indirect measures, to determine whether learner outcomes are being achieved. These methods include student evaluations of the course and instructor, student portfolios, senior exit surveys, senior exit interviews, instructor curriculum review, and alumni surveys.

Other universities, including Pennsylvania State University, Oklahoma State University, Clemson University, and Virginia Tech, are using a similar approach to assess learning outcomes for their undergraduate programs (Craddock et al., 2003; Kahn, 2006; Scales et al., 1998; Scoggins et al., 2004). According to Kahn (2006), Oklahoma State University Department of Horticulture and Landscape Architecture's website describes in detail their outcomes assessment methods. Included in these methods is a requirement of a minimum grade point average within the major (2.50), student internship requirements, including a student presentation and employer evaluation of intern, and a comparison of how students compete nationally at educational events (i.e., ASHS Association of Collegiate Branches Horticulture Judging Contest).

The purpose of this study was to conduct an outcomes assessment, via an alumni questionnaire, of graduates from the Department of Horticulture at ISU. The questionnaire gathered information on graduate preparedness when entering the workforce, the effectiveness of the ISU Department of Horticulture, and the relevance of departmental learner outcomes. The survey questions were developed based on university, college, and departmental curriculum-based learner outcomes (Iowa State University, 2007; Iowa State University Department of Horticulture, 2007).

Materials and methods

A list of 470 former students who graduated between the years 2000 to 2006 was obtained from the ISU Department of Horticulture. Graduates were placed into seven strata based on the year they graduated (stratum 1 = graduates from 2000, stratum 2 = graduates from 2001, etc.). The stratum sizes were N1 = 68; N2 = 71; N3 = 74; N4 = 65; N5 = 81; N6 = 65; and N7 = 46. Of those graduates, a random sample of 221 was selected (n1 = 17; n2 = 35; n3 = 32; n4 = 36; n5 = 41; n6 = 31; and n7 = 29). Proportional allocation was used so that the sampled population reflected the true population.

A questionnaire was developed in consultation with the ISU Institutional Review Board. It consisted of 14 multiple choice, 35 Likert-type scale, six short-answer, and four open-ended questions, and was developed using SurveyMonkey software (SurveyMonkey, Portland, OR). Participants were asked about their demographics, education, employment, and perceptions of how well the undergraduate horticulture curriculum at ISU helped them achieve departmental outcomes and prepare them for employment in the green industry.

The survey was piloted to a sample of horticulture graduates and survey experts to control validity. Data collection was conducted via the web and through e-mail contact. To increase responses, the tailored design method was used, which consisted of four emails sent to each participant (Dillman, 2000). The first correspondence (30 July 2007) was an advance-notice e-mail informing alumni that they would be receiving the questionnaire. The following day (31 July 2007), participants received a letter describing the research project and a link to the questionnaire. Two weeks later (14 Aug. 2007), a reminder notice was sent to nonrespondents. On 21 Aug. 2007, nonrespondents were sent a follow-up letter, which included a link to the questionnaire. The SurveyMonkey software coded the completed surveys and compiled the data. The data were analyzed using SAS (release 9.1; SAS Institute, Cary, NC) to compute means, percentages, standard errors, 95% confidence intervals, weights, and chi-square tests. Survey results were compared with ISU Department of Horticulture senior exit survey data (2000–06) to evaluate reliability of the instrument.

Results and discussion

Of the 221 participants who received the questionnaire, 104 questionnaires were completed and usable, for a response rate of 47%.

Demographics.

The demographics of the 104 respondents showed that the average age was 27 years, the majority were male (70.1%), and 98% of respondents identified their ethnicity as white. Respondents represented all nine degree options within the ISU Department of Horticulture, with turfgrass management (45.2%), nursery crops production and landscape management (16.4%), and greenhouse production and management (8.7%) having the largest representation (data not shown). The distribution of undergraduates in the nine degree options during 2000–06 was similar to the respondents' distribution. The majority of the respondents (86.5%) currently have a full-time job. Most of these jobs were golf course related (28.9%), in the landscape design/build field (7.7%), or in research (7.7%).

First full-time job after graduating.

Participants were asked a series of questions about their first full-time job after graduating. On average, it took graduates <3 months to find a job (2.8 months). The most common job fields were golf courses (38.5%), nurseries (6.7%), and in landscape maintenance (6.7%). The remaining 51% of respondent jobs were distributed almost evenly between garden centers, greenhouses, sports turf, landscape design, landscape build, research, sales, and marketing (data not shown). When asked how closely related their first job was to their degree, 76.7% of graduates felt that their job was highly related. The majority of participants earned annual incomes for their first jobs of $20,000 to $29,999 (48.5%) or $30,000 to $39,999 (34.7%). These salaries were similar to the national average in 2006. The U.S. Census Bureau reported that 18- to 24-year-olds with a bachelor's degree and working full time, year round earned an average of $34,188 (U.S. Census Bureau, 2007). Almost half of the respondents noted that their first job had a minimum education requirement of a bachelor's degree (46.6%), followed by an associate's degree or 2 years of college (28.2%), and 17.5% required a high school diploma or less. When asked about how well their education prepared them for their first job, 43.7% of participants felt they were adequately prepared, 40.8% felt they were more than adequately prepared, and 13.6% rated their preparedness as exceptional.

Current full-time job.

A majority of participants believe their current job is highly related to their degree (70.2%), whereas 19.2% believe it is moderately related. Annual incomes increased with their current jobs such that 19.4% earn $20,000 to $29,999, 29.1% earn $30,000 to $39,999, and 29.4% earn $40,000 to $49,999. Again, the reported salaries are very similar to the 2006 national average. According to the U.S. Census Bureau (2007), 25- to 29-year-olds with a bachelor's degree and working full time, year round earned an average of $49,791.The minimum education requirement for current jobs was similar to that of first jobs with 52.4% requiring a bachelor's degree, 18.5% requiring an associate's degree or 2 years of college, and 13.6% requiring a high school diploma or less. Respondents have worked for their current employer an average of 2.5 years. When rating their preparedness for their current job, 41.8% of participants rated themselves as adequately prepared, 37.9% were more than adequately prepared, and 15.5% felt they were exceptionally well prepared.

Education.

Participants answered eight questions related to their overall undergraduate experience. The questions covered subjects such as overall quality of education and variety of courses offered. Seven of the eight questions received scores >4 on a five-point scale (1 = very dissatisfied, 2 = dissatisfied, 3 = neutral, 4 = satisfied, and 5 = very satisfied). The quality of instruction outside of their major received the lowest ranking of 3.96, whereas quality of education within the major received the highest score of 4.53 (Table 1).

Table 1.

Iowa State University horticulture alumni responses in relation to the quality of education and departmental satisfaction.

Table 1.

Participants also answered 33 questions related to the ISU Department of Horticulture curriculum outcomes (Table 2). The outcomes-related questions were divided into four categories: abilities in horticulture (six questions); professional skills (10 questions); professional abilities (nine questions); and abilities related to world perspective (eight questions). Respondents were asked to rate their abilities in these areas using a five-point scale (1 = very poor, 2 = poor, 3 = fair, 4 = good, and 5 = excellent).

Table 2.

Iowa State University horticulture alumni responses related to abilities in horticulture, professional skills and abilities, and world perspectives.

Table 2.

In the abilities in horticulture category, respondents ranked their ability to implement horticulture production strategies as the lowest, with a score of 3.96. The highest score (4.63) was the ability to apply basic horticulture knowledge. Respondents also felt strongly in their abilities to understand and use technical principles of horticulture (4.34) and to recognize plant stressors (4.31). Their abilities in plant identification and management of soil-based and artificial substrates were also high, at 4.1 and 4.2, respectively.

In the professional skills category, respondents rated their computer skills, such as word processing, (4.56), the ability to organize and interpret information on a computer (4.61), and electronic communication (4.74) highly. Participants also felt that their abilities were strong when defining problems and their solutions (4.54), working as part of a team (4.61), and that they had high standards of achievement (4.67). However, respondents felt less confident in their ability to debate issues (3.99), make oral presentations (4.22), write a concise report (4.28), and to motivate and organize others when problem solving (4.31).

When asked about their professional abilities, respondents felt confident in their ability to analyze and interpret data (4.31), in their resume and interview skills (4.35), and in understanding field terminology (4.47). Participants felt comfortable with their abilities to complete mathematical calculations (4.14), seek out opportunities for higher education (4.22), and use resources such as libraries and technical journals (4.23). The lowest-rated skill was the ability to interpret laws and regulations (3.87), followed by basic business concepts (4.02) and having a holistic perspective of the ecosystem (4.03).

In the category of world perspectives, respondents rated their ability to tolerate different beliefs as the highest-rated skill, with a 4.39. Graduates also rated their abilities to appreciate cultural differences, and recognize moral, ethical, and legal conflicts highly, at 4.28 and 4.26, respectively. Other abilities rated highly were the abilities to understand rural and urban influences on horticulture (4.17) and to appreciate the individual's role in sustainable management (4.30). The lowest-rated abilities were the knowledge of basic elements in the metric system (3.86) and knowledge of horticultural practices in other parts of the world (3.56).

Open-ended questions.

Participants were asked to list courses they found most useful in their career. Of the courses within the ISU Department of Horticulture, the courses listed most frequently included turf-related courses (25%), herbaceous plant identification courses (23%), woody plant identification courses (21%), and landscape maintenance and installation courses (12%). Participants also reported courses that they thought were the least useful, and the two courses listed most frequently were chemistry (20%) and statistics (9%). When asked which courses should be added to the curriculum, the most common responses were more courses on irrigation design, installation, and repair (14%). Additionally, they felt more business courses covering budgeting, accounting, and personnel management (13%) and landscape design courses (7%) should be added to the undergraduate curriculum.

Overall, graduates were more than satisfied with the quality of their undergraduate education in the ISU Department of Horticulture. They felt confident that they had attained the learning outcomes of the undergraduate horticulture curriculum, and that this education prepared them well for their careers. ISU Department of Horticulture senior exit surveys from 2000–06 show that students had a similar perception of their preparedness (data not published). However, alumni felt that they were weak in their ability to debate issues and lacked knowledge of horticultural processes around the world. Furthermore, alumni also stated that more courses on irrigation and business management would have been beneficial as part of their undergraduate curriculum. These comments were also reported in the senior exit surveys (data not published). Even with these weakness noted by respondents, they were able to quickly find their first jobs after graduating and were being paid well based on their reported salaries and in comparison with 2006 data from the U.S. Census Bureau (2007).

Findings from this survey will be used as part of the ongoing and cyclical outcomes assessment conducted by the ISU Department of Horticulture. These results and additional discussions will be used in making necessary adjustments to the curriculum and learner outcomes to enhance student learning.

Literature cited

  • Angelo, T. 1995 Reassessing assessment: Embracing contraries, bridging gaps, and resetting the agenda Amer. Assn. Higher Educ., AAHE Bul. April 1995 7 11

    • Search Google Scholar
    • Export Citation
  • Craddock, J.M., Stearns, D.T., McGann, M.R. & Kuhns, L.J. 2003 Incorporating industry input into the development of educational objectives for landscape management students North Amer. Colleges Teachers Agr. J. 47 41 46

    • Search Google Scholar
    • Export Citation
  • Dillman, D. 2000 Mail and internet surveys: A tailored design method Wiley New York

  • Gourman, J. 1996 The Gourman report 9th ed National Education Standards Los Angeles

  • Iowa State University 2007 Student outcomes assessment 22 Feb. 2008 <http://www.academicprograms.iastate.edu/assessment/guidelines.html>.

  • Iowa State University Department of Horticulture 2007 Outcomes assessment 22 Feb. 2008 <http://www.hort.iastate.edu/courses/outcomes.html>.

  • Khan, B. 2006 Program outcomes assessment report Oklahoma State University 22 Feb. 2008 <http://uat.okstate.edu/assessment/assessment_at_osu/outcomes/CASNR/CASNR_OA2006/Horticulture%20Undergrad%20Report%20%202005-06.pdf>.

    • Search Google Scholar
    • Export Citation
  • Mort, J.R. & Messerschmidt, K.A. 2001 Creating an efficient outcome assessment plan for an entry-level PharmD program Amer. J. Pharm. Educ. 65 358 362

    • Search Google Scholar
    • Export Citation
  • Newcomer, J.L. 2000 We teach that, don't we? Planning assessment and resounding to the results. Session T3A 16 21 Proc. 30th Amer. Soc. Eng. Educ. (ASEE)/Inst. Electrical Electronics Eng. (IEEE), Frontiers in Education Conf 18–21 Oct. 2000 Kansas City, MO

    • Search Google Scholar
    • Export Citation
  • Pintar, A., Aller, B., Rogers, T., Shultz, K. & Shonnard, D. 1999 Developing an assessment plan to meet Accreditation Board for Engineering and Teaching Engineering Criteria 2000 Proc. Amer. Soc. Eng. Educ. Proc. (ASEE) Charlotte, NC

    • Search Google Scholar
    • Export Citation
  • Scales, K., Owen, C., Shiohare, S. & Leonard, M. 1998 Preparing program accreditation review under Accreditation Board for Engineering and Teaching Engineering Criteria 2000: Choosing outcome indicators J. Eng. Educ. 87 207 210

    • Search Google Scholar
    • Export Citation
  • Scoggins, H.L., Latimer, J.G. & Barden, V.T. 2004 The Virginia commercial greenhouse industry: Current practices and future needs assessment HortTechnology 14 109 114

    • Search Google Scholar
    • Export Citation
  • Suvedi, M. & Heyboer, G. 2004 Perceptions of recent graduates and employers about undergraduate programs in the college of agriculture and natural resources at Michigan State University: A follow-up study North Amer. Colleges Teachers Agr. J. 3 22 27

    • Search Google Scholar
    • Export Citation
  • University Arkansas 2007 Assessment handbook: Core courses and general education 22 Feb. 2008 <https://www.ualr.edu/provost/?page_id=78>.

  • U.S. Census Bureau 2007 Current population survey: Annual social and economic supplement 22 Feb. 2008 <http://pubdb3.census.gov/macro/032007/perinc/new04_001.htm>.

    • Search Google Scholar
    • Export Citation

Contributor Notes

Corresponding author. E-mail vanderza@iastate.edu.

  • Angelo, T. 1995 Reassessing assessment: Embracing contraries, bridging gaps, and resetting the agenda Amer. Assn. Higher Educ., AAHE Bul. April 1995 7 11

    • Search Google Scholar
    • Export Citation
  • Craddock, J.M., Stearns, D.T., McGann, M.R. & Kuhns, L.J. 2003 Incorporating industry input into the development of educational objectives for landscape management students North Amer. Colleges Teachers Agr. J. 47 41 46

    • Search Google Scholar
    • Export Citation
  • Dillman, D. 2000 Mail and internet surveys: A tailored design method Wiley New York

  • Gourman, J. 1996 The Gourman report 9th ed National Education Standards Los Angeles

  • Iowa State University 2007 Student outcomes assessment 22 Feb. 2008 <http://www.academicprograms.iastate.edu/assessment/guidelines.html>.

  • Iowa State University Department of Horticulture 2007 Outcomes assessment 22 Feb. 2008 <http://www.hort.iastate.edu/courses/outcomes.html>.

  • Khan, B. 2006 Program outcomes assessment report Oklahoma State University 22 Feb. 2008 <http://uat.okstate.edu/assessment/assessment_at_osu/outcomes/CASNR/CASNR_OA2006/Horticulture%20Undergrad%20Report%20%202005-06.pdf>.

    • Search Google Scholar
    • Export Citation
  • Mort, J.R. & Messerschmidt, K.A. 2001 Creating an efficient outcome assessment plan for an entry-level PharmD program Amer. J. Pharm. Educ. 65 358 362

    • Search Google Scholar
    • Export Citation
  • Newcomer, J.L. 2000 We teach that, don't we? Planning assessment and resounding to the results. Session T3A 16 21 Proc. 30th Amer. Soc. Eng. Educ. (ASEE)/Inst. Electrical Electronics Eng. (IEEE), Frontiers in Education Conf 18–21 Oct. 2000 Kansas City, MO

    • Search Google Scholar
    • Export Citation
  • Pintar, A., Aller, B., Rogers, T., Shultz, K. & Shonnard, D. 1999 Developing an assessment plan to meet Accreditation Board for Engineering and Teaching Engineering Criteria 2000 Proc. Amer. Soc. Eng. Educ. Proc. (ASEE) Charlotte, NC

    • Search Google Scholar
    • Export Citation
  • Scales, K., Owen, C., Shiohare, S. & Leonard, M. 1998 Preparing program accreditation review under Accreditation Board for Engineering and Teaching Engineering Criteria 2000: Choosing outcome indicators J. Eng. Educ. 87 207 210

    • Search Google Scholar
    • Export Citation
  • Scoggins, H.L., Latimer, J.G. & Barden, V.T. 2004 The Virginia commercial greenhouse industry: Current practices and future needs assessment HortTechnology 14 109 114

    • Search Google Scholar
    • Export Citation
  • Suvedi, M. & Heyboer, G. 2004 Perceptions of recent graduates and employers about undergraduate programs in the college of agriculture and natural resources at Michigan State University: A follow-up study North Amer. Colleges Teachers Agr. J. 3 22 27

    • Search Google Scholar
    • Export Citation
  • University Arkansas 2007 Assessment handbook: Core courses and general education 22 Feb. 2008 <https://www.ualr.edu/provost/?page_id=78>.

  • U.S. Census Bureau 2007 Current population survey: Annual social and economic supplement 22 Feb. 2008 <http://pubdb3.census.gov/macro/032007/perinc/new04_001.htm>.

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 191 24 2
PDF Downloads 84 28 2