This unit is intended to develop the critical thinking required to evaluate statistical reports prepared by others. To carry out this evaluation, students will need to be familiar with making valid statistical statements of provided data and the process of selecting and presenting statistical information for a target audience. Such skills need to be practised and developed within a meaningful context.
This unit of work draws on students' science and geography knowledge. However, teachers might adapt this context whilst maintaining the structure of this unit.
- Comment on the validity of a statistical report with reference to:
- sample size
- proportions stated
- central tendency
- the presentation of data
- data collection
- Comment on the voice of a statistical report.
The focus of statistical literacy at Level 5 is to prepare the students to evaluate statistical reports prepared by others, with respect to the critical questions demanded at Level 6. This unit is intended to build such groundwork. The students will consider the validity of the reported statistics and, in turn, provide a range of valid statements and/or graphical representations that would satisfy the demands of a specified client or purpose. In considering the validity of statistical statements, the following guidelines may be of use.
Sample Size: In simple binomial or clear cut either/or situations, a sample size may be as small as 30. In more complex situations, with many possible contributing factors, such as determining a cause/effect relationship in medical research, the sample size would need to be much larger.
Comparing values across different populations/samples: If the population/sample sizes are very different, then proportions (or percentages) rather than raw values should be used.
Central Tendency: Although the term 'average' is commonly used, students should recognise the ambiguity associated with this term. If the data involved are numeric, then the mean or the median should be calculated. If there are only a small number of values from which to find the central tendency, then the median is generally the most useful. The mode should be used for finding the most common category. In general, for a large, representative sample of a normally distributed population, the mean, median and mode will be the same.
When evaluating a statistical report, the following should also be considered:
- In the case of time series data, has data collection been taken for a significant period of time to rule out seasonal effects and to show a long term trend?
- Do the statements outline the parameters/statistics clearly and without ambiguity?
- Are the graphical representations distorting the displayed data?
The learning opportunities in this unit can be differentiated by providing or removing support to students, and by varying the task requirements. Ways to support students include:
- restricting or increasing the length and complexity of text that students work with
- providing time and support for students to read and understand the report, before the mathematics session
- providing extended opportunities for students to investigate different statistical reports
- providing prompts for writing descriptive statements
- providing additional, targeted teaching around mathematical concepts (e.g measures of central tendency)
- strategically organising students into pairs and small groups in order to encourage peer learning, scaffolding, and extension
- working alongside individual students (or groups of students) who require further support with specific areas of knowledge or activities.
The context for this unit can be adapted to suit the interests and experiences of your students. For example, whilst this unit makes use of a text that is relevant in an environmental context, your students might be more motivated by a report that explores a sporting, human rights, or food context.
- Copies of/access to a questionable statistical report (you could use this report on water quality).
- Copies of/access to editorial piece from the New Zealand Herald.
- Access to the internet, research sites, and report-making resources
Session One
This session introduces students to a statistical report. This learning could be carried out as part of your literacy programme.
- Introduce the students to a questionable statistical report. You may have your own resources to draw upon, but could also use this report on water quality
- Have the students answer the following:
- Who is responsible for preparing this report?
- Who might use this report?
- What are the main findings of this report?
- Call the students back together and discuss their impressions of the report and responses to the three questions posed. Emphasis the responsibility of the the writers of the report in relaying accurate information and how damaging misleading information might be.
- Play the podcast of the interview from radio NZ where a scientist questions the validity of the report, from 0:00 to 2:48. While listening to these first few minutes, note the words:
- most
- time period
- stable
- Ask the students to explain what the scientist's concerns were. Draw attention to concerns over the terms most (when actually only 50 out of 400 waterways or 12% were involved), the time period (ten years was considered too short a time period to draw valid conclusions from), and stability (this was argued to be an incorrect conclusion).
- Set the students, in their groups, to the task of being editors. They must rewrite three key sentences within the first two sections of the report, to change the tone of the report to pay heed to the scientist’s concerns. They may need to listen to the whole podcast, read through more of the report and follow the links to do full justice to this task.
- Call the students back to report back on their editing.
- Have the students to summarise the key aspects covered in this discussion and critique of a statistical report. The key aspects should include data collection, purpose, target audience, fair/representative.
Session Two
In this session, show the students a report where the proportions given in the report are either misleading or ambiguous. You may have your own resources to draw upon, or, in keeping with the waterways context in the example for the previous session, you may choose to use the example below. It concerns the nitrogen levels in tested waterways.
- Give the students this editorial piece from the New Zealand Herald to read
- Ask the students to say what the main message(s) of the report were.
- Re-read the following extract from the report: “…..This rate of conversion will continue to have a telling impact on our waterways. Based on the projections above, nitrogen loads in New Zealand rivers would increase by 6 per cent by 2020. Canterbury alone has already seen a 27 per cent increase in nitrogen loads between 1996 to 2008, and given the conversion rates above will see a further 15 per cent rise by 2020. Many of our rivers are already unsafe for swimming in.”
- Ask the students why they think nitrogen levels might be monitored. Make a summary of the key points of this context. (…an indicator of pollution, leaching from farmland, increases may be due to increased stock numbers - particularly dairy stock, weather conditions such as flooding affects readings of levels). It is important for students to have an understanding of the context in order to evaluate and/or to construct a statistical report.
- Discuss who might be the target audience of this article (e.g., environmentalists, dairy farmers, river and lake users such as kayakers, swimmers, fishermen, etc). Considering the viewpoint of the readers, might the quoted percentage increase of nitrogen levels of this article be useful knowledge or possibly damaging to their interests?
- Arrange the students into small groups and ask them to find further information on the percentages nitrogen levels of NZ rivers. Some useful sites to begin their search are:
- Bring the groups back together and pool the combined efforts of their research to list percentage increases of nitrogen levels by region and date. Look also at the graph of nitrogen levels (figure 4b) on the Statistics NZ link which shows a peak in 2006 and then a drop. Use this to make calculations of the percentage increase (shown on the graph) from 1996-2006 and 1996-2007. Discuss the difference between these two values and which should be chosen. Discuss seasonal variation, the possibilities of differing weather patterns affecting readings and show how a trend-line could be drawn through the graph. (as an extension, this could be used to discuss how predictions can be made…).
- Get the students to prepare three ‘press statement’ sentences on the percentage increase of nitrogen levels in NZ rivers…One each for The Green Party, Fish and Game NZ and Federated Farmers.
- Compile a display, or have the students read out to share, their press statements.
Session Three
A group of secondary school students in Dunedin, as part of ‘Healthy Harbour Watchers’ have been measuring water quality data including nitrogen (listed as NNN) levels. This study will form the basis of the final three sessions of this unit. The Healthy Harbour Watchers data can be found on the link below.
http://en.wikipedia.org/wiki/Water_pollution_in_New_Zealand
- Arrange the students into groups and show them the Healthy Harbour Watchers results page on their website. Using the interactive map, change the data display to NNN and assign one harbour location per group of students to note the past readings by changing the date that the data is displayed. Have each group record the data for their location and calculate all the relevant statistics. This should include time series graphs and calculations of range, mean, median and mode.
- Call the groups back and discuss the relative merits of the different statistics.
- Give each of the groups, the task of choosing appropriate statistics and data displays to write a report for the Healthy Harbour Group with the aim of highlighting concerns with the nitrogen levels in the harbour. They may need to research what are considered acceptable levels. They might look at long term trends or they might highlight the times in the past when the levels were higher. Ask the students to summarise the key (negative) trends and/or statistics in their data. This activity should take some time and give the teacher plenty of opportunity to discuss the validity of choices, statements and conclusions the students are making while they plan and compile their report.
- Call the groups back to present their key findings.
- Discuss whether the levels and the long term trends of nitrogen levels in the harbour were of as much concern as they have just presented, or were they able to distort the picture they gave by careful choice of what they presented.
Session Four
- Bring the class reports from session three together and discuss the scenario of a journalist being given the task of reporting on the Healthy Harbour Watchers’ findings. Remind the students that the journalist has a responsibility to report in a fair and unbiased manner. The article is to be titled ‘Is it safe to play in our harbour?’
- The students should remain working in groups, but may need to undertake further research to complete their report. Encourage the groups to take a stand of yes or no, if their research suggests this is the case, but to be sure to back this stand up with citations, data displays and statistics. This activity should take some time and give the teacher plenty of opportunity to discuss the validity of choices, statements and conclusions the students are making while they plan and compile their report.
- Bring the class back together to share, to compare and contrast their reports.
- These reports will be used for session five, so try to find some way of making them available to the whole class. This could be a wall display, or a wiki, or a photocopied booklet.
Session Five
- Bring the journalists’ articles together from session four and discuss with the class:
- the validity of a statistical report with reference to:
- sample size
- proportions stated
- central tendency
- the presentation of data
- data collection
- the voice of a statistical report
- the validity of a statistical report with reference to:
- Give the students the task of each writing ‘a letter to the editor’ disputing the report, on the basis of at least one aspect of the statistics quoted by the journalist. The writer of this letter is to be from the water quality office of the regional council. This activity should take some time and give the teacher plenty of opportunity to discuss the validity of choices, statements and conclusions the students are making while they plan and compile their report.
- Bring the class together to share and discuss their letters to the editor.
Which voice was the most responsible…the students, the journalist, the water quality office?
How can you trust the statistics presented in a report?
Can you choose statistics to say just what you want, or is there only a little wiggle room?
How can you be sure of the real story? - Summarise their ideas over what is a responsible/fair/representative use of statistics, with regard to each of the following terms: data gathering, sample size, population, proportion, percentage, mean, median, mode, time period. Try to link in with clear guidelines such as those published in the explanatory notes for this unit.
Dear families and whānau,
Recently, we have been learning to evaluate statistical reports. This week at home, your child is to find an example of statistics used in media (e.g. in a newspaper article, a news segment, on social media) and evluate it. They should consider, if possible:
- the validity of a statistical report with reference to:
- sample size
- proportions stated
- central tendency
- the presentation of data
- data collection
- the voice of a statistical report.