Abstract
With growing interest in food system solutions to address poor health outcomes related to preventable chronic diseases, organizations and researchers are examining the value of community gardens as interventions to promote individual and community health. Research suggests that participation in community gardens improves access to fresh, healthy foods and increases fruit and vegetable consumption. In addition to these physical benefits, research also documents a variety of social and communal benefits, by expanding social capital, stabilizing neighborhoods, and cultivating relationships. Unfortunately, most of these studies focus on a specific case, cross case, or intervention studies within a geographically specific locale. Learning lessons from successful community garden programs can be difficult because community gardens often rely on the synergy of a complex network of support agencies that assist in various technical and educational capacities. The purpose of the study was to demonstrate the use of a framework for program development and evaluation that stakeholders, including extension, can adopt to show program outcomes. The framework used a Delphi approach with a diverse panel of community garden stakeholders to reach consensus about program outcomes. The study demonstrated that the panel could reach consensus on a variety of short-, medium-, and long-term outcomes.
Community gardens are shared spaces that confer both physical and social benefits to participants (Draper and Freedman, 2010; Litt et al., 2011). These spaces are found in both rural and urban settings, across socioeconomically diverse neighborhoods, at workplaces, places of worship, social service sites, and in schools. With growing interest in food system solutions to address poor health outcomes related to preventable chronic diseases, organizations and researchers are examining the value of community gardens as interventions to promote individual health and community well-being. Research indicates that community garden participation improves access to fresh, healthy foods (Allen et al., 2008; Evans et al., 2015; Poulsen et al., 2014) and increases fruit and vegetable consumption(Alaimo et al., 2008; Hanson et al., 2017; Litt et al., 2011). In addition to these physical benefits, research also documents a variety of social and communal benefits, by expanding social capital, stabilizing neighborhoods, and cultivating relationships (Glover et al., 2005; Gorham et al., 2009).
Much of the research focuses on case, cross-case, and intervention studies within geographically specific locales. Draper and Freedman (2010) conducted a comprehensive literature review on the benefits, purposes, and motivations of community gardening in the United States. Of the 55 studies reviewed, case studies comprised 40% and interventions comprised 45%. This mirrors extension research, which emphasizes case and multisite case studies on individual programs (see examples in Blaine et al., 2010; Landry et al., 2015) or on specific topics within community gardening, such as garden design (Bradley et al., 2014) and management (Drake and Lawson, 2015a).
One outlier is a recent, comprehensive study by Drake and Lawson (2015b) that surveyed 445 community garden organizations from across the United States and Canada representing 8550 gardens that focused on community garden benefits and challenges. Respondents agreed on more than 75% of the primary or secondary benefits: food production and access (99.7%), nutrition/improved diet (99.5%), social engagement/well-being (99.5%), exercise/physical activity (98.6%), individual personal satisfaction (97.8%), education specifically about gardening (96.7%), environmental benefits (95.9%), intergenerational activities (94%), education (86.1%), intercultural communication (85.1%), and neighborhood revitalization (78.7%). These findings were further supported by the results of an open-ended question in which participants identified food production, social engagement, education, and nutrition as the most significant benefits. The authors noted, “While some academic literature focuses on a particular outcome—food production, nutrition, community engagement, etc.—it seems that organizations may accept multiple and varied outcomes.” Although this study provided a comprehensive lens toward program benefits, it did not demonstrate a process for achieving consensus among diverse stakeholder groups on the central outcomes that should be considered for an integrated development and evaluation model for community gardens.
The practice of community garden support is often a “complex web of interactions across scales as communities, local and extra local non-governmental organizations, and government agencies plan and implement community gardens” (Drake and Lawson, 2015b). In short, a patchwork of highly contextualized agencies and programs, which vary from town, municipality, county, and state, delivers garden support. Part of this complex web of garden support agencies are local county extension offices, which assist community gardeners in various technical and educational capacities through workshops, site visits, demonstration gardens, and plant clinics, to name a few services. Because of the diversity of stakeholders, the many potential services provided by extension, and the variety of physical and social outcomes, extension agents need a consistent framework for community garden program development and evaluation. We believe that a set of broadly applicable impact indicators for a program logic model would provide an outcome-driven framework to inform such efforts (Israel, 2001).
Impact indicators provide a clear and concise means for positioning community garden programs to address questions of program efficacy and impact. As such, it is critical that these programs clearly define, capture, and communicate their impacts (Starr and Hattendorf, 2013). Impact indicators also help to outline and evaluate pertinent strategies to address the challenges related to garden loss, including declining participation, lack of interest by gardeners, and loss of land (Drake and Lawson, 2015b).
Logic models provide an effective framework for educators and other key stakeholders to develop program plans for the development and evaluation of community garden programs based on agreed-upon outcomes (Israel, 2001). Table 1 outlines the short-, intermediate, and long-term outcomes included in the logic model framework. This framework facilitates program evaluation to extend beyond simply measuring learning outcomes by allowing stakeholders to specify intended behavioral and conditional impacts of community garden programs.
The outcome types used in a logic model for educational programs.
In the planning phase, these impact indicators (a vision of success) would be used to develop program objectives to guide the overall evaluation framework and associated evaluation tools (Israel, 2001). The logic model identifies a causal relationship between the program (inputs, activities, and participation) and its intended outcomes using a program theory of change. Israel (2001) explains that by organizing programs in this manner, the programs can measure, learn, and improve intended outcomes and make proactive changes if needed. In addition to supporting program planning and evaluation efforts, logic models can be used to communicate a community garden’s purpose and efforts and can be integrated into grants and funding. The present study addresses the need to develop these indicators as part of community garden planning and evaluation activities.
Developing a successful community garden program is a collaborative effort predicated on the ability of key stakeholders to effectively evaluate and communicate success. The purpose of the study was to demonstrate the use of a framework for program development and evaluation that stakeholders, including extension, can adopt to show program outcomes. It did so by using an innovative tool, the Delphi technique, which a breadth of stakeholders can adopt. The objective of the study was to identify the most meaningful outcomes (short-term, medium-term, and long-term) that could be included in a program logic model and be used to measure the success of such programs.
Methods
This statewide study used the Delphi technique to identify key outcomes that should be incorporated in evaluation frameworks across a variety of community garden programs (Delp et al., 1977; Linstone and Turoff, 2002; Warner, 2015). The Delphi technique provides a structured process for reaching a consensus among a panel of experts through multiple rounds of anonymous feedback, or iterations (Warner, 2015). It is frequently used in the educational context to develop consensus for program priorities and objectives that can help guide planning and evaluation efforts of programs.
The study began in Feb. 2017 and finished in May of the same year. The population for this study consisted of key community garden experts who held various roles in community garden programs across the state of Florida. First, an advisory committee representing state agencies, nonprofits, institutions of higher education, and various school systems was asked to create a list of potential expert panel members. The advisory committee selected individuals who had a breadth of experience and expertise through different roles in community garden programs and types of organizations, geographic diversity, and varied perspectives (Stufflebeam et al., 2012). The advisory committee provided a total of 101 unduplicated nominees. From this list, all potential participants were contacted by the principal investigator to provide them with additional information on the study and solicit their help. The 53 individuals agreeing to participate received a copy of the initial survey to provide time for thoughtful response.
The study used a series of three online surveys, which were validated using an expert panel of program evaluators, agricultural and horticultural educators, and state school garden coordinators not included in the study. The first round of the study used a survey with the open-ended item: “Please list all of the outcomes that result from a successful community gardens program. Make sure to consider short-term outcomes (changes in knowledge, attitudes, skills and aspirations), medium-term outcomes (behavioral change/adoption of practices), and long-term impacts (societal, economic and environmental) when developing this list.” This open-ended question item was used to create a comprehensive list of possible program outcomes.
Using the round 1 data, the researchers used a three-step process of content analysis to categorize the responses for developing the round 2 survey (Merriam, 2009). Three researchers coded together to develop the initial themes, whereas an external member reviewed them and provided feedback (Blair, 2015). First, the data were assessed line by line and provided with codes. The responses were then read again multiple times until categories became well-defined. Then the individual categories were examined to create themes based on meaningful relationships with other categories and subcategories. The researchers then used the logic model framework to organize the themes within the short-term, medium-term, or long-term outcome levels. The researchers used the logic model as the final step of data organization so that the subsequent surveys would solicit respondent evaluation based on similar outcome levels.
In the second survey, respondents were asked to rate items identified in round 1 on a seven-point Likert-type scale (1 = strongly agree, 2 = agree, 3 = somewhat agree, 4 = neither agree nor disagree, 5 = somewhat disagree, 6 = disagree, 7 = strongly disagree). The respondents were asked to frame their level of agreement based on how meaningful they felt it was to include such outcomes in the evaluation of a community garden program to ensure its success. The respondents were asked to consider feasibility to ensure that community garden educators and other key stakeholders would have the capacity to evaluate such outcomes (Table 2). The researchers used the criteria of two-thirds (66.67%) of group members choosing 1 = strongly agree or 2 = agree as the common definition of consensus, and items that met this criteria were included in the third survey (Boyd, 2003; Conner et al., 2013; Harder et al., 2010; Shinn et al., 2009).
Survey items from round 2 of the Delphi study asking participants to identify their level of agreement with the outcomes listed.
The third and final questionnaire asked the respondents to identify their level of agreement on the same Likert-type scale with the shortened list of outcomes resulting from round 2 (Table 3). The same two-thirds rule was applied to this round as the predetermined definition of consensus. The design of Delphi studies, including the number of rounds employed, is open to modification by the researchers. As most of the items achieved consensus following round 3, we decided to conclude the study at this point.
Survey items from round 3 of the Delphi study asking participants to identify their level of agreement with the outcomes listed.
Results
Breakdown of participant demographics
In each round of the survey, respondents provided information regarding the organization type that they represent, the role(s) they hold with the garden or garden program, the geographic region where their work takes place, and the number of years of experience they have working with community gardens. To delineate geographic area, we used the state of Florida’s Cooperative Extension Service districts. Overall, across all three rounds, the participants in this study had an average of ≈10 years (10.58) of experience working with community garden programs. Table 4 provides a breakdown of the rest of the respondent demographic information per round.
A breakdown of the participant demographics per round of the Delphi study including organization type, role(s) in the garden, and geographic area(s) of Florida where the program takes place.
Delphi results by round
Round 1.
Fifty-five outcomes were identified in the first round. In Table 5 these are broken down into short-, medium-, and long-term outcomes. These categories resemble the outcome levels within the logic model and were part of the researchers’ process of data categorization following the first round. There were slightly more long-term outcomes than medium- or short-term outcomes. The response rate for this round was 81% (n = 43).
Breakdown of the number of outcomes per outcome level as a result of the analysis of the round 1 survey responses.
Round 2.
In round 2, participants reached consensus on 28 of the 52 items that were initially provided as meaningful outcomes for community garden programs (Table 6). The response rate for this round was 75% (n = 40). The outcomes with the highest level of agreement in the short-term outcomes range centered on understanding and appreciation of the benefits of growing food, knowledge and skills for best practices in gardening, increased appreciation and knowledge of local food systems, and increased connection to community. The outcomes with the highest level of agreement in the medium-term range were predominantly focused on the increase in healthy food consumption, increased time spent outdoors, mentoring of new gardeners, and growing food to supplement diets. The outcomes with the highest level of agreement in the long-term outcomes range included increasing healthy food access, the creation of inclusive environments, improved mental health, garden sustainability, and increased educational opportunities for lifelong learners. Twenty-seven outcomes fell below the two-thirds threshold and were eliminated from further study.
Summary of the Delphi study round 2 results showing the percentage of participants who chose strongly agree or agree for the importance of the inclusion of each item in the evaluation of community garden programs.
Round 3.
In round 3, participants were provided with a shortened list of 28 items, with consensus achieved on 26 of them. Table 7 contains the summary data for this round. Forty-five of the 53 panel members responded in this round with an 85% response rate.
Summary of the Delphi study round 3 results showing the percentage of participants who chose strongly agree or agree for the importance of the inclusion of each item in the evaluation of community garden programs.
Conclusions
Using the Delphi technique proved to be a useful approach in gaining the input of a diverse panel of key experts in the identification of a set of agreed outcomes that are essential to community gardens. The panel reached consensus on nine short-term outcomes, nine medium-term outcomes, and eight long-term outcomes. As the panel of experts represented the diversity of stakeholders involved in community gardens across the state, it provided a comprehensive and holistic lens to the Delphi technique that included varied interests, expertise, and perspectives.
This process shows how a network of key stakeholders that exist in the context of community gardens can work together to develop an outcome-driven program. In addition, this model provides a basis for identifying a program’s performance measures because components that are important enough to include in a logic model should also be the focus for performance measurement (Hatry, 1999; Israel, 2001). It is important to take the results of this study into consideration when planning or refining a community garden program, but should not be viewed as a panacea as local needs should be integrated into planning decisions. The results have the potential to inform a comprehensive framework for community garden program development and evaluation that extension faculty and staff, and other community garden key stakeholders, can adopt and implement to consistently demonstrate positive program outcomes across geographic regions.
The results align with the outcomes drawn from the literature that relate to improving access to healthy foods (Allen et al., 2008; Evans et al., 2015; Poulsen et al., 2014) and increasing fruit and vegetable consumption (Alaimo et al., 2008; Hanson et al., 2017; Litt et al., 2011), while expanding the set of outcome indicators for the social and communal benefits that community gardens produce (Glover et al., 2005). Furthermore, the results support the findings on benefits of community garden participation described by Drake and Lawson (2015b) in their research including food production and access, nutrition/improved diet, social engagement/well-being, exercise/physical activity, individual personal satisfaction, education specifically about gardening, intergenerational activities, education, and intercultural communication. This study also highlighted additional outcomes related to improving the mental health of community garden participants (Pitt, 2014; Shanahan et al., 2015). It shows that by giving individuals an opportunity to reconnect to nature through gardening, those with issues of mental health , anxiety, and stress experience positive results stemming from their engagement with community gardens. In addition, this study highlights the educational opportunities that community gardens provide for lifelong learners (Merriam and Kee, 2014). By creating a better appreciation for themselves and nature, this study provides a rich set of outcomes that will help those planning for community gardens to be successful.
Several of the outcomes, such as increasing connections to community, demonstrating increased leadership and problem-solving skills, and gardeners mentoring newer and future gardeners, can address key community garden challenges. Each of these outcomes can be tied to pertinent strategies to address the challenges identified by Drake and Lawson (2015b) as primary to garden loss: declining participation and lack of interest by gardeners, followed by loss of land. Furthermore, the researchers found that the most significant challenges include funding, participation, land, and materials. Using the outcomes identified in this Delphi study in tandem with research-identified challenges to community garden success can assist horticulture professionals and support organizations with developing and implementing programs that focus on garden sustainability, and to intentionally address components that contribute to garden failure early on in garden development.
As the results of this study were organized similarly to a logic model, horticultural professionals tasked with developing educational activities may better understand how to influence participants’ knowledge, attitudes, skills, and aspirations necessary to achieve behavior change or the adoption of best practices (Israel, 2001). Likewise, those responsible for evaluating program performance can better understand when to administer evaluative tools and how they should be structured (Israel, 2001). Those who are currently engaged in community garden programs might consult the findings of this study to explore their program’s impacts and examine possible changes to align with desired outcomes. Future community garden programmers should apply appropriate outcomes to their planning and long-term evaluation activities.
Literature cited
Alaimo, K., Packnett, E., Miles, R.A. & Kruger, D.J. 2008 Fruit and vegetable intake among urban community gardeners J. Nutr. Educ. Behav. 40 94 101
Allen, J., Alaimo, K., Elam, D. & Perry, E. 2008 Growing vegetables and values: Benefits of neighborhood-based community gardens for youth development and nutrition J. Hunger Environ. Nutr. 3 418 439
Blaine, T., Parwinder, G., Dawes, A. & Snider, D. 2010 Profiling community gardeners J. Ext. 48 6FEA6 6 June 2017. <https://www.joe.org/joe/2010december/a6.php>
Blair, E. 2015 A reflexive exploration of two qualitative data coding techniques J. Methods Meas. Soc. Sci. 6 14 29
Boyd, B.L. 2003 Identifying competencies for volunteer administrators for the coming decade: A national Delphi study J. Agr. Ed. 44 47 56
Bradley, L., Lelekacs, J., Asher, C. & Sherk, J. 2014 Design matters in community gardens J. Ext. 48 6FEA6 6 June 2017. <https://www.joe.org/joe/2014february/tt9.php>
Conner, N.W., Roberts, T.G. & Harder, A. 2013 Competencies and experiences needed by entry level international agricultural development practitioners. J. Inter. Agr. Ext. Ed. 20:19–32. <https://www.aiaee.org/attachments/article/1503/Conner_Roberts_20102.pdf>
Delp, P., Thesen, A., Motiwalla, J. & Seshadri, N. 1977 Delphi: System tools for project planning. Natl. Ctr. Res. Vocational Educ., Ohio State Univ., Columbus
Drake, L. & Lawson, L. 2015a Best practices in community garden management to address participation, water access, and outreach J. Ext. 53 6FEA3 6 June 2017. <https://www.joe.org/joe/2015december/a3.php>
Drake, L. & Lawson, L. 2015b Results of a US and Canada community garden survey: Shared challenges in garden management amid diverse geographical and organizational contexts J. Agr. Human Values 32 241 254
Draper, C. & Freedman, D. 2010 Review and analysis of the benefits, purposes, and motivations associated with community gardening in the United States J. Community Pract. 18 458 492
Evans, A., Banks, K., Jennings, R., Nehme, E., Nemec, C., Sharma, S., Hussaini, A. & Yaroch, A. 2015 Increasing access to healthful foods: A qualitative study with residents of low-income communities Intl. J. Behav. Nutr. Phys. Act. 12 S5
Glover, T.D., Parry, D.C. & Shinew, K.J. 2005 Building relationships, accessing resources: Mobilizing social capital in community garden contexts J. Leis. Res. 37 450 474
Gorham, M.R., Waliczek, T.M., Snelgrove, A. & Zajicek, J.M. 2009 The impact on community gardens on number of property crimes in urban Houston HortTechnology 19 291 296
Hanson, K.L., Kolodinsky, J., Wang, W., Morgan, E.H., Jilcott Pitts, S.B., Ammerman, A.S., Sitaker, M. & Seguin, R.A. 2017 Adults and children in low-income households that participate in cost-offset community supported agriculture have high fruit and vegetable consumption. Nutrients 9:726
Harder, A., Place, N.T. & Scheer, S.D. 2010 Towards a competency-based extension education curriculum: A Delphi study J. Agr. Educ. 51 44 52
Hatry, H.P. 1999 Performance measurement: Getting results. Urban Inst. Press, Washington, DC
Israel, G.D. 2001 Using logic models for program development. 6 June 2017. <http://edis.ifas.ufl.edu/wc041>
Landry, S., Chittendon, N., Coker, C. & Weiss, C. 2015 Perceived effects of community gardening in lower Mississippi delta gardening participants J. Ext. 53 4RIB3 6 June 2017. <https://www.joe.org/joe/2015august/rb3.php>
Linstone H.A. & Turoff M. 2002 The Delphi method: Techniques and applications. New Jersey Inst. Technol., Newark, NJ
Litt, J.S., Soobader, M.J., Turbin, M.S., Hale, J.W., Buchenau, M. & Marshall, J.A. 2011 The influence of social involvement, neighborhood aesthetics, and community garden participation on fruit and vegetable consumption Amer. J. Public Health 101 1466 1473
Merriam, S.B. 2009 Qualitative research: A guide to design and implementation. 3rd ed. Jossey-Bass, San Francisco, CA
Merriam, S.B. & Kee, Y. 2014 Promoting community wellbeing: The case for lifelong learning for older adults Adult Educ. Qrtly. 64 128 144
Pitt, H. 2014 Therapeutic experiences of community gardens: Putting flow in its place Health Place 27 84 91
Poulsen, M.N., Hulland, K.R., Gulas, C.A., Pham, H., Dalglish, S.L., Wilkinson, R.K. & Winch, P.J. 2014 Growing an urban oasis: A qualitative study of the perceived benefits of community gardening in Baltimore, Maryland Cult. Agr. Food Environ. 36 69 82
Shanahan, D.F., Lin, B.B., Bush, R., Gaston, K.J., Dean, J.H., Barber, E. & Fuller, R.A. 2015 Toward improved public health outcomes from urban nature Amer. J. Pub. Health. 105 470 477
Shinn, G.C., Wingenbach, G.J., Briers, G.E., Lindner, J.R. & Baker, M. 2009 Forecasting doctoral–level content in international agricultural and extension education–2010: Viewpoint of fifteen engaged scholars J. Intern. Ag. Ext. Ed. 16 57 71
Starr, K. & Hattendorf, L. 2013 Rigorous yet realistic measurement. Mulago Foundation, San Francisco, CA
Stufflebeam, D.L., McCormick, C.H., Brinkerhoff, R.O. & Nelson, C.O. 2012 Conducting educational needs assessments. Springer, Berlin, Germany
Warner, L.A. 2015 Using the Delphi technique to achieve consensus: A tool for guiding extension programs. 12 Dec. 2016. <http://edis.ifas.ufl.edu/wc183>