Click on author name to view affiliation information
The purpose of this study was to determine if participation in the National Wildlife Federation’s (NWF) Schoolyard Habitat Program (SYHP) had an effect on the science standardized test scores or science grades of fourth-grade primary school students in Houston, TX. To conduct the study, five pairs of Houston elementary schools were selected as either treatment or control schools. The treatment group included a total of 148 fourth-grade students whose teachers reported using the NWF’s SYHP. The control group consisted of a total of 248 fourth-grade students whose teachers used a traditional science curriculum. To measure academic achievement, scores on a standardized science test and science grades were compared between the treatment and control students. Results from this study indicated Caucasian students scored higher than minority students on the Stanford standardized science exam. Significant differences existed in the Stanford standardized science exam scores between male and female students for the treatment group only. Overall, the results from this study also showed that the SYHP was equally as effective at science instruction as the traditional curriculum within the Houston Independent School District (HISD) after teachers gained familiarity with using the habitat for instruction.
According to the U.S. Bureau of Labor Statistics (2005), by the year 2014 there will be 197,000 job openings in the life and physical sciences, 507,000 engineering positions, and over 1.3 million jobs in computer and mathematics-related fields because of both new positions and retirements. However, there has been no growth in the education sector in these fields, and it is predicted the labor sector will be unable to keep up with the demand in these areas (Tytler et al., 2008). Less than a decade ago, the United States ranked 73rd out of 91 countries surveyed in the percentage of students graduating from colleges with degrees in science and math (National Science Board, 2006). These deficiencies have resulted in an urgent ongoing need for a more scientifically literate citizenry and increasing recruitment into science, technology, engineering, and math-related careers at a young age (Center for Science, Mathematics, and Engineering Education, Committee on Undergraduate Science Education 1999; National Science Foundation 1996).
Science literacy, as defined by the National Research Council (1996), is “the knowledge and understanding of scientific concepts and processes required for personal decision making, participation in civic and cultural affairs, and economic productivity” or understanding basic science skills. It is necessary to engage people in science when they are young to produce science literate adults (Butler, 2009). Even if educators engaged students when they were young and students exhibited spontaneous interest in science and math, they often became fearful or disdainful of math and science (Rutherford and Ahlgren, 1991). Students, in particular, did not see science as connected with society or offering ways to improve peoples’ lives, and regarded it as disconnected and removed from their own daily lives (Butler, 2009).
Most students who have become interested in science did so during middle school or earlier and attributed their interest to some specific education-related science-based experience (Maltese and Tai, 2009). According to Turner and Patrick (2008), students must be motivated to learn by tapping into their interests and by making science learning relevant to their experienced lives (Butler, 2009). Young people tend to be interested in those things around them with which they can directly connect, such as nature, the environment, and how things work (Turner and Patrick, 2008). Once engaged, research has suggested, it is necessary to address specific scientific principles to ensure students drew the connection between their experience and science (Mantzicopoulos et al., 2008). Furthermore, behaviors associated with intellectual challenge, interest, and engagement in science concepts included involvement with experimental work, problem solving, and interpretation and inference from observation as opposed to the tasks typically found in classrooms, such as memorization and recitation of facts (Tytler et al., 2008). Some research suggests that such activities were seen as real by students and that science learning was enhanced by participation in such enrichment activities as field trips, family science events, and clubs (Tytler et al., 2008). According to Tytler et al. (2008), greater attention needs to be paid to informing young people of the actualities of careers in science.
The popularity of using gardening and nature to teach science by incorporation of hands-on techniques and connection to lived experience has increased in the last several years. School gardening in particular has been a popular area of research in the fields of horticulture and education in recent years to investigate children’s learning outcomes from interactions with plants and nature (Klemmer et al., 2005; Pigg et al., 2006; Robinson and Zajicek, 2005; Skelly and Bradley, 2000). Unfortunately, Skelly and Bradley (2000) found that gardening was used as an instructional tool less than 10% of the time even though teachers reported gardening activities enhanced student learning.
Reports of the specific academic benefits students gained from participating in school gardening varied (Danforth et al., 2008; Klemmer et al., 2005; Pigg et al., 2006). Klemmer et al. (2005) found the science achievement of students who participated in a hands-on gardening program and science curriculum was higher when compared with students who only participated in a traditional science curriculum. Danforth et al. (2008) found students participating in the NWF’s SYHP had statistically significantly higher math scores when compared with peers in schools using traditional curriculum. Alternatively, Pigg et al. (2006) found no statistically significant difference in science scores for students who participated in a hands-on gardening program. Their research concluded that more studies and curriculum development were needed to successfully implement garden and nature science education programs (Pigg et al., 2006).
Research has also looked at the effects of participating in outdoor science activities on children. These programs often design active hands-on outdoor activities and exercises to teach science-related topics and can provide a fun environment for children to learn and apply these concepts to real settings (Alexander et al., 1995; Waliczek et al., 2003). Waliczek et al. (2003) found students who were in these programs engaged in application skills. This finding supported past research that showed school gardening programs provided children with the opportunity to apply school lessons (Braun et al., 1989).
The NWF’s SYHP is a certification process that schools are awarded after meeting certain criteria in the development of a wildlife habitat (NWF, 2011). Similar to the certification of a habitat through the NWF’s traditional Backyard Wildlife Habitat (BWH), both programs require that a yard has the four necessary elements to attract and sustain wildlife: food, water, cover, and a place to raise young. In addition to the NWF’s BWH program requirements, SYHP certification includes that the site be used as an educational teaching resource (NWF, 2011).
The SYHP connects students with their local environment using hands-on activities to enhance student learning and understanding of the application of learning outcomes to real-world experiences in science, environmental education, and stewardship (NWF, 2011). The purpose of this study was to determine if the SYHP was an effective method of teaching science as evidenced by student scores on the Stanford standardized science test and science grades of fourth-grade students in the HISD.
The sample population used for this study was drawn from fourth-grade students in HISD. Schools volunteered for participation in the study. Five control group schools were included in the study and were selected to be comparable to a treatment school with regards to student gender, ethnicity, at-risk status, and participation in the school lunch program. These schools did not have schoolyard habitats or use the SYHP in their science curriculum. The five treatment schools had teachers who used the SYHP curriculum and a habitat. The degree of participation by individual teachers in treatment schools varied and was dependent on the teacher and the teaching style. The treatment site teachers assured researchers they were using the SYHP sites for teaching on a regular basis. Teachers recorded the frequency their classrooms participated in the habitat to control for usage frequency as a variable in learning measurements.
A total of 546 fourth-grade students participated in the study. The number of student participants at the treatment schools was 298, and within that, the treatment group included a total of 148 fourth-grade students whose teachers reported using the NWF’s SYHP and for whom all measures of science achievement and demographic information were provided by the school district. The remaining 150 students from treatment schools did not have all measures of science achievement, demographic information, or teacher reports provided and were not used in statistical analysis; however, open-ended qualitative data were received from this group of students. The control group consisted of a total of 248 fourth-grade students whose teachers used a traditional science curriculum and for whom all measures of science achievement and demographic information were provided by the school district.
The study was conducted during a 15-week period between quarters 2 and 3 of the scheduled school year. Science grades given by individual teachers were recorded during this period to allow for comparison between the control schools and treatment schools and were on a 100-point scale.
Stanford science test percentile ranks were collected on sample fourth-grade students in both the control and treatment schools. The Stanford standardized test was required of all students in the HISD from grades 1 to 11, is given during quarter 4 of the school year, and compared student performance on a national level. Stanford science percentile rank scores assessed student achievement comparatively on a national level on a 100-point scale. A percentile rank score is interpreted such that a score of 75% indicates that student performed better than 75% of all other students who took the exam nationwide. The mean percentile rank score on a national scale is 50%, and the range of scores is from 0 to 100 even if every student performs well.
Teachers voluntarily documented the frequency in which their classrooms participated in the SYHP. This information allowed for teacher rankings in habitat usage values to be assigned based on the teachers’ responses. Teachers at control schools were considered nonusers of the habitat curriculum. Non-users also included any teachers at treatment schools who initially opted to participate, and then did not report habitat use during the study period for any reason. Teachers who used the habitat between 7 and 13 times during the study period were ranked as medium-users, and teachers who indicated using the habitat at least once per week during the 15-week study period were ranked as high-users. Within the sample of students, 248 were included in the non-use group, 125 in the medium-use group, and 23 in the high-use group.
Students at schools with habitats were asked to respond to open-ended questions regarding their experiences with the habitat. Students could choose not to respond to any statement. Students were asked to respond to the following questions: “Does learning in the schoolyard habitat help you understand science lessons better? How?,” “How does having a habitat make your school different or special?,” “Share with me a memorable experience you had in the habitat,” and “Do you share what you learn outdoors in the habitat with your family or friends?” Overall, 298 questionnaires were returned from treatment schools and used for analysis in this section of the study. Questionnaires for qualitative data were received from students for whom quantitative data were not available.
Overall, schools selected for both the control and treatment groups were largely minority, with 98% Hispanic and African American students (every school had over 64% Hispanic population specifically). Every school in both the control and treatment groups consisted of largely economically disadvantaged students (as determined by participation in the free and reduced lunch program), with only one school having less than 90% participation (the school had 87% participation and was in the control group). Schools in Texas are rated by the state based on standardized test scores with ratings including (from lowest performing to highest performing) unacceptable, acceptable (standard varies by subject, but generally 60% to 70% of students pass a specific subject area), recognized (at least 80% of students pass every subject area), and exemplary (at least 90% of students pass every subject area). Within our sample, every school ranked at least acceptable, with one school in each the control and treatment group ranking as exemplary, two to three ranking as recognized, and one to two ranking as acceptable (Table 1).
Demographic information regarding sample students’ ethnicity, participation in the free or reduced lunch program, and gender were obtained from the school district. In some cases, these data were not on file and were not associated with the student record.
The curriculum and treatment used in this study was the NWF’s SYHP. The SYHP curriculum was divided into two activity levels: kindergarten to grade 8, and grade 9 to grade 12. This study used the curriculum for younger students since fourth-graders were the target sample population. Applications were further subdivided into segments of grade levels to align with national academic standards (NWF, 2011). For this study, curriculum usage by treatment school teachers was at the discretion of each teacher. Teachers used the curriculum as they would any other materials available for classroom curriculum development and integration.
The NWF conducted a 4-d workshop for teachers that explained the program and provided lesson plans that showed how teachers could integrate state and national standards into the program (NWF, 2011). Workshop attendance was voluntary and teachers who attended were documented to use as a variable for analysis. Students were coded based on the number of days their teacher received training and included no training days (347 students), 1 d (20 students), 3 d (43 students), or 4 d (44 students). No teacher in this study attended 2 d of training.
To measure academic achievement, comparisons of the Stanford science test percentile rank scores between control and treatment groups’ fourth-grade data were evaluated. Quarterly science grades were also compared between the control and treatment groups for quarters 2 and 3 of the school year since the study was conducted only during those quarters. Teacher training days and teacher ranking of habitat usage were also used as variables for analysis. Data were entered and analyzed using SPSS Statistics (version 17.0; IBM Corp., Somers, NY). Data were analyzed using frequency statistics, analysis of variance (ANOVA), and multivariate analysis of variance (MANOVA). Levene’s test was used to confirm equal variances due to unequal sample sizes throughout the analysis. Qualitative data were recorded, analyzed for themes, and summarized.
Multivariate analysis of variance indicated that a significant difference on the Stanford science test scores existed based on the ethnicity of the student (P = 0.01) (Table 2). Post hoc analysis with a Bonferroni correction applied indicated Caucasian students scored 20 to 30 percentile ranks higher on the Stanford science test when compared with any other ethnic group in this study. This supported previous research that showed historical achievement gaps between minority groups and Caucasians have been fairly wide, with Caucasian scores on standardized tests being consistently higher (Haycock and Jerald, 2002; Ravitch, 1995). Since all of the Caucasian students (totaling 1.7% of the total population) were in the control group, Caucasian students were removed from further analysis to ensure ethnicity did not affect the results of this study. No significant differences existed between ethnic groups on science grades throughout the school year. No significant differences existed between students who participated in the free and reduced lunch program and those students who did not participate in the free and reduced lunch program on the Stanford science exam, quarter 2 science grades, or quarter 3 science grades (all P > 0.05).
However, a significant difference on Stanford science test percentile rank scores existed between males and females in the overall sample (P = 0.03). Post hoc analysis indicated male students scored, on average, 6 percentile ranks higher on the exam compared with female students. This finding was analyzed further to investigate if the difference was within the treatment group or control group, or both groups. A MANOVA was conducted with only the treatment group, and a significant difference between males and females persisted with male students scoring on average ≈6.5 percentile ranks higher compared with females. Within only the control group, no significant difference existed on the Stanford science percentile rank score based on gender (Table 3). This finding suggested males learned science better in the hands-on setting that the habitat curriculum provided and further investigation on targeting female science-learning is needed to support an alternative science curriculum. However, no significant differences existed between males and females with regards to science grades during quarter 2 (P = 0.89) or quarter 3 (P = 0.08).
A MANOVA test was used to determine differences between students whose teachers indicated different levels of usage of the habitat and the different levels of training for habitat use on mean Stanford science test percentile ranks and science grades. The MANOVA indicated that no significant differences on the mean Stanford science test percentile rank scores existed between students grouped based on teacher usage of the habitat (P = 0.36) (Table 4). This suggested teachers who used a traditional science curriculum had similar scores to eachers who had medium and high usage of habitats. There was also no significant difference between students based on the number of days of training their teachers received in preparing to use a habitat, indicating teachers who received no training taught science as effectively as those teachers who did receive training.
A significant difference existed between the mean science grades of those students who participated in the SYHP and those who did not in quarter 2 (P = 0.01), but this difference was no longer found in quarter 3 (P = 0.07) (Tables 5 and 6). Post hoc analysis with a Bonferroni correction applied indicated that, in quarter 2, students whose teachers did not use the habitat had higher grades compared with students whose teachers were ranked as either medium or high users. Therefore, students whose teachers used the habitat at any level had lower science grades in the first recorded period but similar grades in the second recorded period. This finding suggested students in the treatment groups may have improved their learning over the course of the study period. A similar finding was found with regards to SYHP teacher habitat training. In quarter 2, a MANOVA calculation resulted in a significant omnibus test with no significant differences in the post hoc comparison between science grades for students whose teachers received various levels of habitat training and students whose teachers did not receive habitat training (P = 0.01; no significant post hoc differences) (Table 6). However, this difference no longer existed in quarter 3 (P = 0.08) (Table 6), which suggested teachers may have become more effective at teaching science using the habitat after they had practice applying the principles learned during training and became equally effective teachers as when using traditional, familiar curricula. This finding supported research indicating teacher experience was related to student test scores, especially after initial experience is gained (Chingos and Peterson, 2010; Clotfelter et al., 2006).
Students at schools with habitats responded to open-ended questions about their experience with the habitat. Of the 298 students who responded, 260 (87.2%) felt as though the habitat helped them to understand science lessons, 18 students (6%) did not think the habitat helped them learn science, 18 students (6%) were indifferent, and 2 students (0.6%) did not respond. In the short answer response, students indicated the habitat improved learning through hands-on lessons, ability to observe an experiment or lesson, the real-life applicability of the lesson, and increased interest and fun in the lesson by being in the habitat. Direct quotes from students included: “It helps because science is in the outside world,” “It helps me because I can actually see what I am studying,” “It’s not fake models,” “It does help me because we use our hands to understand more,” “I can understand better because I can see the things I am learning,” “Motivated because I like to be outside,” “It helps me in a way that takes me closer to the real thing,” “The science lab is more fun.”
In response to the question asking “How does having a habitat at your school make it different or special?,” 85 students (29%) gave responses about nature, such as “It makes it different because a lot of creatures don’t have a home and they can find one.” In addition, 48 students (16%) gave some response indicating students at their school would be able to learn better, such as “Having a habitat at my school is special because I could go there and learn about fish, plants, and trees,” and “I think it is special because other schools can’t learn about it like my school,” and 45 students (15%) gave a response related to a schoolyard habitat being unique, such as “Having a habitat makes my school special because other schools don’t have it and we do,” and “People could say it is a remarkable school.” Other less frequent responses included those such as passers-by could see the habitat and understand the importance of nature or the environment, which was given by 11 students (3.7%) and food production was given as a response by 2 students (0.7%).
In response to the question, “Share with me a memorable experience you had in the habitat,” 81 students (27%) gave a response about planting seeds or transplants or growing plants, 80 students (27%) gave responses about viewing animals, 31 students (10%) gave a response related to water, either watering plants or viewing a pond, and 12 students (4%) responded about a specific lesson from the habitat.
In response to the question, “Do you share what you learn outdoors in the habitat with your family or friends?” 180 students (60%) responded positively, with 55 students (19%) indicating sharing with family specifically and 26 students (9%) specifying sharing with friends only. Negative responses indicating that the student did not share what they learned with others were given by 22 students (7.4%) and 15 students (7.5%) indicated sometimes or maybe sharing. No response to this question was provided by 81 students (27%).
In general, the qualitative data seemed to suggest students felt positively about learning in the habitat and thought the habitat was an interesting way to learn science which they share with their families and friends.
There has been controversy in the U.S. school system over students’ abilities to meet academic standards being presented to them and their ability to perform well on standardized exams (Meier, 2002; Whitaker, 2004). This concern is exacerbated when nontraditional curricula are suggested and critics argue that using curricula like that of the SYHP would distract students’ ability to learn and score well on standardized tests. Such curricula are often required to endure rigorous testing to show that student scores can match those of traditional curriculum and teaching methods (Pale et al., 2001).
The results from this study showed the SYHP was equally as effective at science instruction as the traditional curriculum within HISD after teachers gained familiarity with using the habitat for instruction. This finding supported other studies concerned with students’ academic achievement with an interdisciplinary or integrated curriculum, particularly those with an environmental agenda (Danforth et al., 2008; Klemmer et al., 2005; Lieberman and Hoody, 1998; Pigg et al., 2006; Stapp, 1969; Vaughn et al., 2003; Volk and McBeth, 1997). This may be due to a lack of experience of teachers in using this alternative method, especially since student learning seemed to improve the longer they used the habitat.
In addition to being equally effective as traditional curricula, student responses regarding their experiences in the habitat suggested increased interest and understanding in science lessons. Although the qualitative interpretations suggested that students were more interested in science using this alternative method of instruction, improved student learning did not seem to occur. This may be due to the nature of standardized testing revolving around memorization of fact rather than the process of inquiry. Increasing interest in science at this grade level is of increasing importance because of industry demand for individuals in science-related careers. Since most students who become interested in science do so during middle school or earlier, fourth-grade seems to be an opportune time to tap these interests with an education-related science-based experience (Maltese and Tai, 2009). According to Turner and Patrick (2008), students must be motivated to learn by tapping into their interests and by making science learning relevant to their experienced lives, which the curriculum such as the NWF’s SYHP attempts to accomplish, particularly with the largely minority group this study sampled (Butler, 2009).
This study has several limitations which should be addressed in future studies to clarify findings. Investigations from other school districts would assist in understanding the generalizability of the results found in this study before use of this curriculum can be extended. Additional investigation into why female students in the control group scored equally to males, whereas those in the treatment group scored lower and how to mediate this effect would help prevent this gap from developing as alternative curricula become more widely accepted. Furthermore, a pre- and post-test design on student scores on standardized tests would aid researchers in drawing conclusions regarding the effectiveness of the SYHP curriculum. In addition, it may be the case the objectives taught using the schoolyard habitat were not the same as those tested by the Stanford science exam. Finally, student grades are not standardized and may not be an accurate measure of learning because of individual teacher or school differences in evaluation and grading. Regardless of these limitations, this study is a step toward integrating a hands-on approach to science instruction that includes nature-based environmental education and suggests that alternative curricula may be a better way to engage students.
Contributor Notes
