Part 2: Why Book Scrutinies Don’t Work in Maths (and what we can do instead)
This blog is the second of a two part blog challenging the common practice of carrying out ‘book scrutinies’ as a way of monitoring the effectiveness of teaching and learning in mathematics. Part 1 should be read first.
Ok, so if I’m advising against (and, in my role as a maths consultant, won’t carry out) book scrutinies, what is an alternative?
First we must return to our aims. Any process we engage in must be constantly focused upon what it sets out to achieve and how it can be continually improved it so it best meets these aims.
Why are we monitoring in the first place?
What can we change? The current reality and subsequent issues arising
1. Evidence from ‘Children’s Voice’: Meeting with our pupils
Reality: Sadly, in too many schools this simply doesn’t happen. In those that do meet with children, questions such as ‘Do you like maths? What have you enjoyed?’ And ‘What are you proud of?’ are commonly used. Adult talk and questioning can often dominate interaction. Capturing the interaction, and therefore feedback to wider staff, is difficult.
Issue: Where children are not met with, the evidence of learning is judged solely from recorded evidence (in books) where little, if anything, is known about the context. Where meetings are held with children, there may not be enough of a clear focus regarding what the discussion aims to elicit and how this will be used to inform whole school next steps in line with the school development plan. Some valuable insights may be gained, but children are often focused upon ‘trying to please’ and talking about what’s recorded in their books making it difficult to judge how effective the learning taking place was.
2. Evidence of Recording in Books:
Reality: What is in children’s books is most commonly recording that was generated as a result of being taught a procedure on that day (or very recently) and then completing similar activities/exercises. Problems relate to the operation or strategy being taught. Most work is recored by the child working alone.
Issue: This is evidence of teaching rather than evidence of learning. Most of us perform well when we’ve just been shown how to do something and/or have little variation within the activity (that which asks us to make decisions and think deeply). ‘Confirmation bias’ is more likely here, where the observer looks for evidence to confirm a belief or hypothesis rather than seeking actual proof. The quality of relationship and regard, between the person monitoring and the individual being monitored, is also likely to affect the judgements made (a further example of confirmation bias at work).
3. Evidence of Learning Journeys in Books
Reality: Activities are following a scheme to may ‘jump around’ from concept to concept with no obvious connection or meaningful application.
Issue: The mathematical concepts being used are not maximised by making connections explicit. Rather than using a scheme as a resource for ideas and to enhance subject knowledge and pedagogy, the teacher is using it as their planning. Instead of having ownership over the learning journey, and responding to questions, misconceptions and cross-curricular/real-life opportunities, the scheme is stuck to with little variation or evidence of ownership.
4. Evidence Using Photographs
Reality: Photos of children learning practically (using concrete resources) appear in every child’s book showing the same or similar task with an explanation of what they were doing underneath. This may result in 30+ colour photos being printed out and stuck in children’s books.
Issue: The cost to a teacher’s time (as well as the paper and ink) in copying, trimming, glueing in and annotating such photos is huge. Photos rarely have comments relating to pedagogical insight or children’s observations or questions and are present in the book as the teacher feels under pressure to ‘prove’ they’ve taught. (See below for more effective ways of using photos).
Changing Our Practice: Re-focusing upon what we’re aiming to achieve and how best to achieve the
Real (and very enjoyable) evidence of learning (based upon proven studies from Bruner, Skemp, Dienes, Piaget and others. All whom informed the development of the ‘Singapore Maths Curriculum’)
1.Evidence Using ‘Children’s Voice’:
Meet with a small group of children (2 or 3). Chosen at random, teachers choice, SLT choice (you choose!)
Ask the children to bring with them and then look back through the class photo diary and their books and choose something they learned recently.
Tell the children (because it’s true!) that you weren’t in the lesson but would love to learn to do what they can now do and you’d like them to be the teachers and you’ll be the student. (They LOVE this!)
Ask them to think about any resources they need to do this and collect them.
Get them to teach you the concept in the way they were taught.
(You may wish to have another SLT member either observing or videoing this interaction to use for reflection and subsequent feedback. Videoing is particularly useful and it allows the observer to concentrate on observing and can be shared with rest of staff at a later stage. This will help reduce the symptoms of ‘confirmation bias’ as described earlier as the children’s behaviour will be the focus for evaluation rather than being based solely upon someone else’s judgement).
Listen carefully to the children and give them time to discuss how best to teach you. Encourage them to reflect upon the class book and their maths books/journals to remind themselves of how the lesson was delivered and what they did.
Keep your questions simple and avoid approving or disapproving (verbally or with your facial expressions), instead be interested. Remember whyyou’re doing this; to elicit their experience of learning in this lesson, and don’t correct or guide them. Do probe and do push them to be as explicit as possible in their instructions.
What you will see will be the children’s version of the learning that took place. This places us in the camp of the learner and not the teacher. What was received rather than what was ‘taught’. The ‘customer experience’, as it were’ is all that matters.
When we subsequently share video of the experience with staff we can ask:
‘What do you see?’
‘What are they saying?’
‘What do they think is important here?’
‘How do they believe someone will learn this most effectively?’
By asking staff to cite evidence and discuss findings from the evidence we are avoiding relying upon our own judgement and giving ownership of effective practice to everyone involved in teaching. It’s far easier for a teacher to recognise and accept the reality of what’s working and what isn’t from their own children. This then gives us the valuable opportunity to ask what can be done differently or built upon to improve children’s learning. What expert knowledge can we turn to to develop use of manipulatives for example or children’s opportunities for reasoning and collaborative investigation? This approach creates cycle of determining what the focus for improvement is, gathering evidence reliably and without bias, planning next steps, investing in support (training or research), taking action and then collecting evidence once more.
2. Evidence Using Recording (a necessary part of the bigger picture):
As humans, we record to help us think, remember, communicate and reflect. When we record, we download our thoughts and allow our brain to focus upon the next task more fully. (Einstein referred to this frequently in his work). Recording enables us to revisit and recall these thoughts throughout a process or at a later stage. Recording helps us judge our progress and notice our mistakes, using them as valuable opportunities to improve. This makes recording is a vital part of mathematics (and one which I will discuss in greater depth in a later blog).
Recording is, however just that, ‘recording’. So WHAT are we recording?
In order to record we need to have experienced something. Imagine trying to write about an autumn day when you’ve never experienced it. You might have every writing skill you need to form the words, but without the first hand experience itself none of this would matter. How would you understand how to use the words you know to communicate what you saw and felt? It’s the same in mathematics. Our recordings need to be as a result of an experience, particularly at the conceptual stage (the point where you learn to understand what’s happening). In mathematics we refer to this stage as either ‘experiential’ (Bruner) or more recently ‘Concrete’ (Singapore). Without experience, we can not explore, conjecture, apply and eventually make sense of what we’re learning. As we experience something, we create pictures in our minds and often translate these images into drawings and diagrams. In mathematics this is the ‘Pictorial’ stage where we simplify and capture something real in a form we can imagine or reproduce. Finally, we use a word or phrase in our spoken language to express this idea and the abstract letters of our language to record this concept. This is the ‘symbolic’ (Bruner) or ‘Abstract’ (Singapore) stage. For example, if this experience had been around learning to understand what an apple is, I wouldn’t have to do much to convince you that the word ‘apple’ written down wasn’t the apple itself! Being able to say or write ‘apple’ wouldn’t be to proof of any understanding. Only when I spoke to you and asked you to share your knowledge based upon this word would I be able to determine your level of ‘learning’ so far.
So why then would we use written symbols in a maths book as evidence of learning?
There’s an even bigger issue in maths too. Let’s imagine our ‘apple’ is the maths being taught. Too many children are taught about ‘apples’ without ever having seen one; cut it up, eaten one, explored the varieties (you get the picture). They are simply told how to say and write the word alongside other fruit names. Maybe they even learn to recite a recipe or two involving ‘apples’. Their recording gives us the impression that they understand what ‘apples’ are (how to prepare them and cook with them etc.) but how could we actually find out if this is true?
So, without taking my apple analogy any further, I hope it’s obvious that, that when we look at book scrutinies in the light of how we might prove to ourselves, that someone truly understands something in any other part of life, they really don’t make sense.
Instead, our children need rich, conceptual experiences which they can secure in their minds by exploring them mathematical models and images and connect to abstract symbols. When these experiences are filled with talk, conjecture, exploration, application and connection, so that the brain uses the body’s senses and actively seeks to use what it already knows, it forms strong cognitive pathways it can later more easily retrieve.
As a result, maths books become learning journals filled with not only purposeful procedural practice (very necessary) but also problem solving journeys and rich sources of reflection. This could include notes, bar modelling and other pictorial approaches, notes and diagrams showing the process undertaken. (I will expand upon those point in a later blog as not to deviate from the focus of monitoring approaches in this piece).
3. Evidence Using Photographs: An alternative approach
Recording children’s experiences using photographs – but in ONE class book(not 30+ individual ones!)
Aim to capture learning when it’s happeningand use a few well chosen photographs to enable the whole class to recall their thoughts and actions and revisit that experience. Used both as a hard copy and electronically, a collection of photographs can stimulate discussions, recall thoughts and processes and be the basis of very valuable whole class, group and individual discussions. Additionally, they become a valuable prompt for teachers to reflect upon and use with the class when a concept is revisited in the future (so learning continues from were it left off). And finally, they become a very valuable and accurate insight into children’s learning share with parents, Ofsted, visitors and anyone else who would like an insight into how the children learn
Annotate the photos with comments made by the children at the time AND insights into the learning reflecting the teacher’s ongoing ‘research’ and improvement in their own understanding of how to teach this concept effectively (for example using Derek Haylock’s ‘Mathematics Explained For Primary Teachers’)
So instead of the same or similar photos x 30, we have maybe 6 photos with annotations that were collected to serve the children and teachers’ purposes and improve their learning. The fact that this recording also becomes a vehicle to share the learning journey with other interested parties is a bonus and NOT the reason for doing it.
As senior leaders, we’re focused upon improve standards for our pupils. This is a complex process which begins with teachers who have open minds and want to learn and change. Fear of being ‘checked up on’, judgement without context, practice that goes against the messages being delivered in training (i.e. talk and practical maths connected to pictorial and abstract recording creates the most effective outcomes) quite obviously won’t achieve this.
Education isn’t alone is continuing to use practices that fail to deliver the very outcomes they’ve been created to achieve. This might be reassuring on the one hand, but it is extremely dangerous on the other. Time, energy and resources in our schools are too precious to waste on approaches that do harm and don’t deliver. So stop. Do what works and then improve on this. Give people a voice and then listen very carefully. What do you hear?
Karen Wilding is an independent Primary Maths Consultant supporting schools and training teachers in the UK and internationally. She is a published author (Penguin/Random House), has appeared on BBC TV as a maths expert on “live Lessons’ and is a regular conference keynote speaker and presenter.
Next Time: ‘Why the traditional approach to marking in maths doesn’t work (and what we can do differently)!
Sign up here to receive updates and training information
Really useful, thought-provoking training with very clear explanations. Thank you!
A Matthews, Year 2 Teacher
Fabulous ways of understanding maths and how to make it easier to teach.
T Hartney, Year 5/6 Teacher
Karen's humour, presentation skills, manner and relationship with our staff was wonderful. Practical 'have-a-go' training which really made us think!'
S Flaherty, Head Teacher