Bravo Math – A Better Way to Learn

Apply to be a tutor, change a child’s life, and work on changing education forever!

Bravo Math Inc. is on a mission to help 5,000 students in Vancouver enjoy math at least five years above grade-level. If you want to join this revolution, you’re in luck, because Bravo Math Inc is hiring!

This won’t be like any tutoring or teaching job you’ve ever had.

If you join this company, you will not only work 1-on-1 with students, but join/start a small team that is employing rapid prototyping, UX, design, and a variety of other tools to create pedagogical breakthroughs. We will ensure that your teaching and your lessons get better and better every day until you are a master educator, and, you too, can train other tutors.

You need no experience in education or working with children, though you must pass a Vancouver Police Information Check because you will be working with minors. You’ll also need enthusiasm, patience, and an open-mind. Everything else is trainable.

Pay: $15/h for observation and training, $17.50/ for supervised tutoring, $20/ for unsupervised tutoring.

Apply Now!

Article: Is Education Reform a Mass Graveyard of Dreams?

Bill Gates tried to revamp American education. He brought his famous work ethic, endless political connections, the Gates Foundation, and billions of dollars.

How did he assess his efforts?

Photo credit: Thomas Hawk

"There's no dramatic change."
– Bill Gates

After countless reforms, elections, “Class Clowns”, presidents, premiers, panels, education ministers and secretaries, commissions, budgets, studies, academic articles, books, school boards, new buildings, charter schools, private schools, independent schools, vouchers, computers, networks, videos, online classes, correspondence classes, programs, philosophies, policies, curricula, standards, tests, textbooks, textbook versions, websites, Web 2.0, tinkering, fads, and trends of every type, the reality is this:

Elementary schools work about the same way they did 50 years ago.
High schools work about the same way they did 50 years ago.
Universities work about the same way they did 50 years ago.
Professional development works about the same way it did 50 years ago.
Teaching practices generally look a lot like they did five thousand years ago.

Improvements at small scales are common, but so are steps backward. With the exception of literacy, pedagogy is the one area of life where big breakthroughs just don’t happen.

Even incremental improvements are hard to sustain.

Have you ever heard a university say: “We eliminated selective admission because, over the past century, our teaching methods have improved by about 1%/year, so we can now do a great job of educating any high school graduate.”

Have you ever heard a similar statement coming from any high school, any elementary school, any corporate training program, or anywhere at all?

I’m guessing you haven’t.

Why is education so hard to improve?

Article: Education’s Insidious Forces

Here are some of the forces preventing major improvements in education. Many experts take one or two of them seriously, but Bravo Math will attempt to tackle them all.

The Intuitive Guide to Education

Ever played “Two Truths and a Lie”? Here are 9 statements. Which do you think are true and which are lies?

If you followed the links, you’ll realize every single one of those statements is wildly far off the mark.

Many educational beliefs are based in folk theory and unreliable subjective interpretations of personal experience. This is analogous to exorcisms, blood-letting, lead injections, and other medical beliefs from the 1400’s.

This is perhaps the mother of all problems in education: our intuitions lead us to the equivalent of exorcisms.

“If my students can do this, then it proves mastery.”

To plan lessons and assessments rigorously, teachers ask themselves “What evidence will I collect at the end of the unit to prove my students have achieved mastery?”

This is a giant mental trap and it contradicts basic science. Yet the trap ensnares people by the millions, from preschools right through world-leading science departments.

For example, despite acing physics courses, many Harvard students hadn’t actually learned concepts of high school physics. “They learn next to nothing,” says physics professor Eric Mazur. Even graduate students in physics good enough to find a Nobel Prize winner to supervise their master’s degree are often “clueless about physics”. How can this be? It’s actually quite simple. If a student masters the concepts of their physics class, then they will ace their physics assessments. But they can also ace those assessments without conceptual mastery. That is, they can perform calculations without thinking like a physicist and get an A. Now, if Harvard physics students flunk tests on high school physics concepts and and top-notch physics graduate students are clueless about physics, how are the rest of the world’s students really doing? What other mistakes are we making when measuring learning? (At a post-secondary level, many presidents of top universities appear to know nothing about research on learning.)

When I meet students for the first time, they often have gotten high grades in arithmetic of fractions and decimals. Then they tell me that “multiply” always means “increase”. (This is a common misconception even in university.)

Since the days of Karl Popper, we’ve known that falsification is generally logically stronger than confirmation. One can never prove that student has mastered multiplication, because multiplication is an abstraction and there are an infinite number of ways that it can manifest (or not) in a student’s life. There is no way to confirm that a student will always apply multiplication appropriately. But surely we can all agree that the notion of mastery has been disproved if the student believes that “multiplication” always means “increase”.

That’s the kind of evidence we should seek.

In other words, The truly rigorous question to ask is “What minimal evidence would prove that my students will not retain what they have learned? Or that they haven’t actually mastered the abstractions in the first place? That they will not be able to transfer their learning?”

Most teachers, including myself, can rarely answer that question with justified conviction. But we must try nonetheless and we need orders of magnitude more and better research to support us in this quest (see below).

When it comes to learning, that’s what teachers want, it’s what we’re paid for, it’s what’s expected, it’s what students want, it’s what parents want, it’s what administrators want, it’s what admissions officers and employers want. All this wanting creates bias. Relentless attempts to falsify the notion that learning has occurred is how we overcome this bias.

Teachers and Sabotage

If you wanted to slow the rate teachers improved their craft, how might you do it? You would implement the status quo! Specifically, you would:

Not require teachers-in-training to learn the most proven techniques for learning. You might suggest that teachers trust their intuition when cognitive scientists adamantly say not to.
Make teachers work alone, since innovation tends to come from diverse teams. You’d ensure that teachers learning by observing and collaborating with other teachers was rare, unpaid, logistically difficult, and a violation of norms.
Exhaust teachers by making their job more tiring and stressful than rocket science, a tougher and more sudden change than being a first-year lawyer, and so relentless that you can't find time to go to the bathroom.
Pay them so little that they can’t work on their craft because they work a second or third job and are sometimes low on blood plasma.
Deny them useful research and feedback. Are you in touch with your kindergarten teacher? Do you know exactly what effect their pedagogy had on you long-term? Teachers rarely get such feedback. How much do you know about the long-term effects your teaching has had? Can you point to evidence to support your guesses on the matter? Are such guesses based on a large collection of double-blind, randomized controlled field trials with big samples that measured how each lesson changed students’ opportunities, work habits, and attitudes over a 30-year time frame? [The answer is no because such a collection does not exist!] The evidence for many educational practices is scant and rarely causal because there is very little money spent on educational research. The pharmaceutical industry invests about 17% of revenues in R&D. The education sector invests about 0.1% of comparable “revenues” in R&D. [First minute.] Even the best evidence from cognitive science on how to improve teaching is, in an absolute sense, pretty weak because nobody knows how the brain works. Consequently, education lacks the kind of solid theories, data (direct biological measurements of learning), and experiments that cause counterintuitive breakthroughs that many other fields – such as chemistry, medicine, and computers – experience so frequently.

It is not the exclusive fault of socioeconomic status

It is true that socioeconomic status is a daunting obstacle in improving educational outcomes. But we know that it is not the only barrier to educational revolutions, because organizations that basically exclude the underprivileged do not claim radical advances in teaching either. Can you name a single large employer that has figured out how to hire average citizens and train them into becoming great employees? How about a university that eliminated entry standards because its pedagogy improved so much? Of course not. Not even educational publishing companies claim this.

There is always a better way.

By and large, most people, including the majority of educators, drastically underestimate how much lesson quality can vary. They frequently confound good mental effort with expected physical behaviors. They do not fully appreciate how sensitive attention and learning can be to seemingly trivial changes in a learner’s environment. [This sensitivity, part of “Tension Systems” is a foundation of social psychology.] Underestimating that sensitivity is where so many attempts to revolutionize education have gone wrong. Another way to think about lessons: “Could anyone, past, present, or future, find a better way to teach this? Or was that lesson the GOAT?"

Institutional reforms really are a graveyard of dreams

The book "Tinkering Toward Utopia" explains many reasons why it's so hard to move beyond the factory model of education. We all know what a "real elementary school" and a "real high school" are like and such notions have hardly changed in over a century. Why?

Individualization and teacher collaboration are far more exhausting and time-consuming than one might think, so teachers continue to work alone giving the same lesson to diverse students.
A "self-paced" course means the student is working by themselves, often in a solitary manner. The student is not working with other learners, not sharing or learning from positive social norms of classmates, not learning to communicate or collaborate. Self-paced schools also suffer from rampant cheating because if there are 200 students all writing a quiz on compound interest at different times, they are bound to tell each other what was on that quiz.
Whatever educational reform you have must survive persistently high teacher and administrator turnover. The people who believe in educational reforms often don't have long enough stays at any given school to implement them.
Educational reforms will inherently fly in the face of what many consider to be a "real school". Do you want to go up against angry parents, skeptical school board members, 100+ years of tradition, media griping, finger-pointing politicians, etc.?
Say you’re the world’s greatest multi-disciplinary teacher and you want to replace English 12 and a bunch of other courses with a multi-disciplinary course of history/literature/civics/drama. Will your students meet the government’s graduation standards? (Probably not.) Will students or parents be OK with that? (No.) Will school administrators and boards be OK with that? (No.) Will typical post-secondary institutions recognize your course? (No.) Will employers recognize that course? (No.) Major reforms to high schools are virtually impossible because the governmental graduation standards, post-secondary admission requirements, tradition, and employer interpretations would all have to shift simultaneously. Good luck with that.

The Trough of Sorrow

Some hunches around pedagogical difficulties and why we still see so much lecturing:

If a teacher lectures while students take notes and copy the teacher, this is easy for the teacher but ineffective. The teacher cannot tell if the students’ mental effort is rote or meaningful. If the students are not learning, the bad news is hidden from everyone.
Often, a teacher’s initial attempts to create active lessons are a huge amount of work for the teacher, way worse than lectures, and, if assessments are designed properly, you collect all the bad news about students immediately. This scares a lot of people away from non-lecture lessons.
Later and better iterations of active learning lessons are really hard for teachers but excellent for the students. But you have to get over the hump of initially crappy iterations.

The Way Forward

Lack of scientific evidence and research, lack of funding, entrenched traditions and bureaucracy, endless intuitive traps, thousands of years of stagnation...

How, then, does one improve education? What do we do next?

I’ve got a few hypotheses.

Article: Revolutionary Hypotheses

Vision

By June 30, 2023, we will have built the best damn math lessons ever made. They will be so fun and inspiring that grade 5 students will regularly beg us for more grade 10 math homework. We will repeat this feat with 5,000 grade 5 students will detonate an earthquake in education. They will be so easy to use for instructors, that high school students administer them to grade 5 students.

This vision rests on three main hypotheses and when it comes to the approach to those, Elon Musk said it best: “Constantly seek criticism. A well thought out critique of whatever you are doing is as valuable as gold… You should take the approach that you are wrong, that you the entrepreneur are wrong. Your goal is to be less wrong.”

My plan has Achilles Heels and I should find out what they are as cheaply and as quickly as possible.

So, please help me disprove the hypotheses below.

Hypothesis 1: The worst bottlenecks in K-12 academics are whole and rational number sense and introductory algebra. Relieving those bottlenecks will transform students’ entire lives. It’s best to ignore what’s going on in school if students are still struggling with these.

Students, teachers, and parents typically want to focus on work assigned by the math teacher. That could be fractions, geometry, data, probability, measurement, time/clock/calendar, algebra, money, shape and space, graphs and charts, etc.

I believe most of that should be ignored in a 1:1 setting. Regardless of what’s going on in school, students should lay a solid mathematical foundation of:

Whole number sense: +, -, ×, and ÷, what they mean, when to use them, and place value.
Rational number sense. This is mainly extending whole number sense to fractions, decimals, and ratios/percents.
Introductory algebra. This includes algebra’s generality, multiple representations of relations and inferences about them, and preserving equality.

In BC, this would roughly cover all number sense from grades 2 through 8.

With this solid foundation, there’s little to stop a grade 5 student from mastering grade 12 math.

This would transform almost any kid’s life.

If repeated at scale, it would be an earthquake in education.

Disproof of the hypothesis would be students who have mastered rational number sense and introductory algebra, but are still struggling badly in math otherwise. In 10 years in the math education field, I have never met or even heard of such a student, but you may be able to help me find them!

Hypothesis 2: By combining a multi-disciplinary team, rapid prototyping, and video-recorded user-testing, we will achieve pedagogical breakthroughs, including in the quality of educational materials, the compression of the average timespan for learning by at least 75%, and the ability to train older students to administer the lessons to younger students.

Here’s how I intend to differentiate typical pedagogical improvement from the status quo.

Status Quo	Bravo Math Initiative
Teacher develops lessons alone, inputting hard work and craft knowledge.	Diverse team develops lesson with hard work, craft knowledge, cognitive science, UX, and design. Drastically broaden the skill set used to develop lessons.
Teacher administers 1 veresion of each lesson per year.	Test and update 4 versions of a lesson per day. Accelerate feedback loop more than a thousand-fold.
Teacher improves next version of a lesson based on memory and student output.	Use all that (left) as evidence plus video of students learning and thinking aloud. Work with radically superior data.
Teacher must worry about covering the entire year of government-prescribed learning outcomes.	Focus on the bottlenecks of K-12 math regardless of student age.

Even a 5% improvement to each version of a lesson over 20 versions would be a 100% improvement. This would be a huge leap in education, an effect size virtually unheard of in education, at least at scale. The only question is how to make those 5% improvements happen.

Here’s what a typical day of lesson development would look like:

Time	Activity
9:00am	Teacher A administers Version 1 of a lesson to 1-5 students. Rest of team observes and records on video.
9:30am
10:00am	Everyone watches the video of Version 1 together. Everyone collaborates to develop Version 2 of the lesson.
10:30am
11:00am
11:30am	Teacher B administers Version 2 of the lesson to 1-5 new students. Rest of team observes and records on video.
12:00pm
12:30pm	Break
1:00pm	Everyone watches the video of Version 2 together. Everyone collaborates to develop Version 3 of the lesson.
1:30pm
2:00pm
2:30pm	Teacher C administers Version 3 of the lesson to 1-5 new students. Rest of team observes and records on video.
3:00pm
3:30pm	Everyone watches the video of Version 3 together. Everyone collaborates to develop Version 4 of the lesson.
4:00pm
4:30pm
etc.	Continue the cycle until the lesson and resources are just blazingly awesome.

Unfortunately, educational evidence for rapid prototyping learning activities, as far as I can tell, does not exist. The closest evidence I’ve found is called “Lesson Study” – and it works. That is, Lesson Study works despite the fact that it is, effectively, super slow prototyping, sometimes without video recordings, and without the kind of continuous engagement and talk aloud protocols.

If improvements in the lesson by just a few percent per iteration were possible, this would revolutionize education.

I’m actually well on my way to achieving it. Through years of iteration, I am now tutoring at this rate for introductory fractions, which is the most hated and feared topic in math.

This is an actual receipt from my tutoring lessons.

~4 years of fractions, < 6 hours

I believe such results are completely unheard of in education. And, yes, I will, at no charge, demonstrate these lessons for you or a student you have in mind. Just email me.

If you sit in on those lessons, I think you’ll see that practically anyone can learn to teach fractions this effectively. And practically any kid can learn fractions like this.

Not all of my lessons are of this quality. But for integers, they’re getting there. Same for introductory algebra. And once those are done, that will open the floodgates to massively higher achievement and enjoyment of math.

Mostly decisive disproof of this hypothesis would be significant documentation of evidence that rapid prototyping of lessons did not improve those lessons.

Hypothesis 3: The output of such rapid prototyping will be materials that learners love and older students can quickly learn to teach.

Lesson Study produces highly reusable lesson materials, despite all the differences among students, but those materials are made for teachers.

I believe it is possible to train 17- to 20-year-old students to administer those lessons as well. The rapid prototyping process should allow for sufficient time to collect and refine such activities, games, exercises, puzzles, projects, etc. Training an army of well-resourced math tutors is the path to scalability.

I have already hired one grade 12 student of mine to become a tutor and it is going well, despite the fact that I have very few resources available to her. Soon, I will hire more tutors. I can’t think of an obstacle preventing me from scaling this, nor am I sure what disproof of this hypothesis would look like as it would involve proving a negative.

Metrics of Learning

Say a teacher wanted to measure the notion that student has mastered division. Traditionally, the teacher would give a student a bunch of division exercises.

The problem is a student might divide because it says “division” at the top, not because they see the concept of division implied by the text. The student might then perform the procedures correctly while not knowing what they mean or why the work. The student might then get all the right answers and forget everything in a week.

Instead, one should test - i.e. attempt to falsify - the notion that the student has mastered division by giving some traditional exercises, but mostly:

Creating a variety of contexts and challenges of which only some are relevant to recently covered concepts. The student will have to be able to recognize when division is relevant and why, estimate answers, and interpret them.
Having the student create, from memory and imagination (no notes or textbooks or other assistance) a similar set of contexts and challenges for themselves.
Using interleaving, distributed practice, and elaboration to boost retention. This includes, say, a pop quiz on contents studied 6 months ago.

This style of exercise and assessment is unpopular because in the short-run, every subjective and obvious measure of learning sinks like a stone. It’s a lot easier to just do division procedures than to really think about a word problem and determine if division is relevant. But long-term measures soar, often with effect sizes of 200%. So, this is, then a design and UX problem: How do we make exercises that appear unproductive actually fun and attractive?