A Tale of Two Questions Part 1: Primary

As part of our analysis of the EQAO data form our schools, I like to move way beyond a simple percentage figure that tells how many students are at or above provincial standard. In terms of looking for data that will have an impact on the way math is taught, I find it more useful to look at how students responded to individual questions. Good questions can often expose students' misconceptions which in might in turn give us pointers as to how adapt our teaching.
Two questions which I found particularly interesting in the Primary Math EQAO were these:

Both of these questions are on the section of the paper where calculators are not allowed.

I have shared these two questions with teachers, principals, and superintendents as well as parents at the first of our Math Cafés for Parents. I ask them to do the questions and then tell me which one they think was the most difficult to do. Some think the first one is more difficult because it has large numbers, whilst others think that the second one requires a bit more thinking even though the numbers involved are small.

I then ask them to estimate what percentage of students they think got the first answer correct. Bearing in mind that my audiences have heard and read a lot about how students are not taught the basics (whatever they are) and that they are not required to learn facts (which is definitely not true) I get a range of answers from 30% to 70%. 

I then tell them that provincially 88% of our students got this correct.

This is surprising for the people I show it to and I can get them to agree that this shows us that students, in general, can add two 3-digit numbers without a calculator. Now I don't know how many of these students did this mentally, or how many needed to write the numbers in columns to use a standard algorithm, or how many used an empty number line, or whether we would have got different answers if some regrouping was required. But I do know that for this question, the vast majority of students met the expectation, contrary to how adults expected them to perform. I then ask the question: do you think that this question shows that the vast majority of our students understand addition. Most people say yes.

The second question is more interesting. Again, I ask the question: what percentage of students do you think got this question correct. Typically I get a range of answers from 20% to 60%. 

The actual provincial results are that 43% of students got this correct.

I find this statistic incredibly informative. Firstly, this is not a 'gotcha' question. It is explicitly stated in the curriculum and a version has been used on many previous EQAO tests. In fact, previous EQAO tests would have had one answer as 9 but this was not an option on this year's test as too many students were choosing it! We actually gave this question as is to all the current grade 3s in our Board and 2% of them actually added '9' as a fifth option, drew a circle alongside and helpfully shaded this in! The actual provincial breakdown for this question is as follows:

Notice that almost as many students selected the incorrect answer '12' as the correct answer. These students have just added all the numbers that they see.

But why do so few get this correct? It isn't anything to do with not knowing their addition facts as we would have seen lower marks in the first question. Instead, I think that this is due to our students not understanding what the equals sign means (as I have written about before here).

For all the talk of 'going back to basics' and memorising facts, we are in danger of being blind to some extremely important mathematical concepts, in this case, the idea of equality. We can get students to develop a solid understanding of equality through the Concrete-Diagrammatic-Symbolic approach (as referred to in an earlier post here). For example, we can begin to develop a concrete understanding through models such as this: 

However, if our students experience nothing but a diet of 2 + 3 = , 9 – 6 = , 32 + 89 =  questions, then they will not develop a true understanding of equality thus putting their future math learning in jeopardy.