Lesson plan Grade 11 Math
Wychcombe Commons Rain Gardens: A Randomization Test
A grade 11 lesson comparing runoff from twelve fictional parking bays with a randomization test, using a supplied simulation to estimate a p-value.
Objectives
- Students will compute and compare group means for two stormwater treatments.
- Students will estimate a p-value from a supplied randomization distribution.
- Students will explain why random assignment allows a causal claim that an observational study would not support.
Vocabulary
- Rain garden
- A rain garden is a shallow planted dip in the ground that catches rain running off a roof or driveway and lets it soak slowly into the soil. Its deep-rooted plants help clean the water.
Also: randomization test, treatment group, p-value, random assignment
Teacher overview
A rain garden is a shallow planted basin that takes runoff from a paved surface and lets it soak in, and the usual claim made for one is that it reduces the volume leaving the site during a storm. This lesson gives students the results of a fictional trial at Wychcombe Commons, a business park where twelve identical parking bays were randomly assigned to a rain garden retrofit or to conventional drainage, and asks whether the measured difference is larger than random assignment alone would produce. The misconception the lesson works against is treating any difference between two group means as a real effect: students first compute a difference of about forty-nine gallons per thousand square feet, then compare it against forty differences produced by reshuffling the same twelve measurements at random.
Procedure
Framing the question (7 min): Show the twelve measurements without labels and ask students how they would decide whether the rain gardens worked. Collect two or three proposals, then explain the design of the fictional trial: twelve bays of equal size, a random draw (for example, shuffled cards) assigning six bays to each treatment, and one storm measured for all twelve. Write the class question on the board as a comparison of two group means.
Computing the observed difference (8 min): Students find both group means and their difference on the handout. Circulate and check that students subtract in a consistent direction, since a sign reversal here reappears in every later step. Confirm as a class that the conventional bays averaged more runoff before moving on.
Modeling one reshuffle (12 min): Give each group twelve index cards and have them write one measurement on each card, ignoring which treatment produced it. Groups shuffle, deal six cards to each pile, compute the two means, and report the difference to the board. Ask what the collection of reported differences represents, and establish the key idea: these are the differences chance alone can produce when the treatment has no effect. Explain that the handout supplies forty such reshuffles so the class does not have to generate them all by hand.
Independent work on the handout (22 min): Students complete the ten handout items. Watch for two predictable errors, counting simulated differences that are merely positive rather than at least as large as the observed difference, and reporting the p-value as a count instead of as a proportion. Students who finish early should write one sentence stating the conclusion in terms a business park manager would understand.
Formative check (5 min): Ask for a show of hands on a single question. If a study produced an observed difference that thirteen of the forty reshuffles matched or exceeded, would that be evidence the treatment worked? The intended answer is no, because a p-value near 0.33 is the kind of result chance produces easily. If fewer than three-quarters of the class answers correctly, work that count aloud before the period ends.
Closing discussion (6 min): Ask what would change if the business park had simply compared bays that already had rain gardens against bays that did not, with no random assignment. Draw out that older, better-drained, or differently sloped bays might have been the ones chosen for retrofit, so the comparison would carry a cause the study could not separate from the treatment.
Student Handout
Wychcombe Commons, its twelve parking bays, and the storm record below are fictional. Each bay is the same size, and all twelve were measured during the same storm. Runoff is reported in gallons per 1,000 square feet of pavement.
| Group | Runoff readings |
|---|---|
| Rain garden bays | 118, 96, 134, 105, 88, 121 |
| Conventional bays | 152, 171, 139, 165, 148, 183 |
Forty reshuffles of these same twelve readings produced the following differences, computed each time as the conventional-group mean minus the rain-garden-group mean, and sorted from lowest to highest:
-38.0, -33.0, -29.3, -27.3, -24.3, -22.0, -19.7, -18.3, -16.3, -14.7, -13.0, -11.7, -9.7, -8.3, -6.3, -5.0, -3.7, -2.3, -1.0, 0.3, 1.7, 3.0, 4.3, 6.3, 7.7, 9.0, 11.0, 12.3, 14.3, 16.0, 18.3, 20.3, 22.7, 25.0, 27.7, 30.3, 33.7, 37.0, 42.3, 49.3
- Find the mean runoff for the rain garden bays. Round to one decimal place.
- Find the mean runoff for the conventional bays. Round to one decimal place.
- Find the observed difference, conventional mean minus rain garden mean. Round to one decimal place.
- What percent lower is the rain garden mean than the conventional mean? Round to one decimal place.
- Find the range of the rain garden readings.
- Find the range of the conventional readings.
- How many of the forty reshuffled differences are at least as large as your answer to item 3?
- Estimate the p-value by dividing your answer to item 7 by 40.
- Using a 0.05 cutoff, is the observed difference statistically significant? Answer yes or no.
- The bays were assigned to the two treatments by a random draw, six to each. Explain why that makes this an experiment rather than an observational study, and what it lets the business park conclude that an observational study would not.
From Water for Schools (waterforschools.org). This material is in the public domain (CC0) and may be copied, edited, and shared freely.
Answer Key
- Answer: 110.3 gal per 1,000 sq ft.
The six readings sum to 118 + 96 + 134 + 105 + 88 + 121 = 662, and 662 divided by 6 = 110.33. - Answer: 159.7 gal per 1,000 sq ft.
The six readings sum to 152 + 171 + 139 + 165 + 148 + 183 = 958, and 958 divided by 6 = 159.67. - Answer: 49.3 gal per 1,000 sq ft.
159.67 - 110.33 = 49.33 gallons per thousand square feet. Accept 49.4 if found by subtracting the rounded means; use the unrounded 49.33 when counting in item 7. - Answer: 30.9 percent.
49.33 divided by 159.67 = 0.3090, so the rain garden bays shed about 30.9 percent less runoff. - Answer: 46 gal per 1,000 sq ft.
The highest rain garden reading is 134 and the lowest is 88, so the range is 134 - 88 = 46. - Answer: 44 gal per 1,000 sq ft.
The highest conventional reading is 183 and the lowest is 139, so the range is 183 - 139 = 44. The two groups spread about equally, which is worth noting before comparing their centers. - Answer: 1.
Reading from the high end of the sorted list, only the final value, 49.3, is at least as large as the observed 49.3; that reshuffle reproduced the original split. The next value down, 42.3, falls short. - Answer: 0.025.
The estimated p-value is 1 divided by 40 = 0.025. - Answer: yes.
An estimated p-value of 0.025 is below the 0.05 cutoff, so the difference is larger than reshuffling alone comfortably produces, and the result counts as statistically significant at that level. - Answer: Random assignment makes it an experiment and supports a causal claim.
In an observational study the bays would have arrived in their groups for reasons of their own, such as age, slope, or how much traffic they carry, and any of those could explain a runoff difference. Assigning the treatment by random draw spreads those characteristics evenly between the groups on average, so the treatment is the only feature that differs systematically. That is what allows the business park to say the rain gardens caused the reduction rather than merely accompanying it. The claim still applies only to bays like these, at this site, during a storm like this one.
Differentiation
Support
Give students the two group sums already computed so items 1 and 2 begin at the division step. Provide a printed copy of the sorted simulation list with a blank column beside it for marking values that meet or exceed the observed difference, and let these students count with a highlighter rather than reading down the list mentally.
Extension
Ask students to repeat items 7 and 8 for a two-sided test, counting reshuffled differences at least 49.3 units away from zero in either direction, and to explain when a stormwater engineer would want the two-sided version. A second extension is to remove the largest conventional reading, recompute the observed difference, and describe how sensitive the conclusion is to a single measurement in a fictional trial this small.
ELL supports
Pre-teach "runoff," "treatment," "reshuffle," "significant," and "assignment" with a labeled drawing of a paved bay draining into a planted basin. Provide a sentence frame for item 10 reading "This is an experiment because ___, which means the difference is probably caused by ___," and allow item 9 to be answered by circling the correct choice on a printed line marked with 0.025 and 0.05.
Extensions
Have students design a follow-up trial for the same site that would test whether the reduction holds during a much larger storm, specifying how many bays they would need, how they would assign treatments, and what they would measure. A written design of this kind, half a page long, gives a natural place to practice the vocabulary of experiments and to argue about sample size before any data exist.