Generative AI is not a calculator
When talking about Generative AIs, such as ChatGPT, there has been a suggestion that they are to writing what a calculator is to maths. It is not a valid comparison to say that “maths adapted to the calculator, so other subjects need to adapt to generative AI”. This if for two core reasons, firstly (and centrally) because it is a fundamentally different challenge it addresses, but secondly (and more comically) because it sucks as a calculator.
The Calculator and Maths*
The adaptation to the calculator is an interesting comparison, and indeed there are entire rafts of rules on what can, and cannot be used with calculators in exams. The shift from “no calculators” to allowing graphing and programmable calculators did require maths exams to change. This meant testing more the understanding of the theory and model behind the challenge than being able to follow a specific formula to its conclusion. That means at very junior levels, where you are learning the model of addition you can’t use a calculator, but at 18 when looking at differentiation its perfectly ok as you’ve demonstrated knowledge of that basic model already. However at University some professors can still ban their use and require students to demonstrate they actually understand the model.
In maths there is normally a need to show both the method and the answer to gain the full marks. This ability to test the model is something that generative AIs struggle with. It is why they’re great at simple code generation, but rapidly struggle when it requires actually modelling to achieve the goal.
Generative Models are not calculators
The important element here is that while calculators have certainly changed how people study maths, and the sort of complexity that can be addressed, you have to understand the model. If a questions asks you to calculate the area under a curve, then you need to understand that you have to do integration. Now you could add that to a calculator and have it spit out the answer between two points, but what you’ll need is to show the intermediary steps before that answer.
This is not the case with Generative AI, as ChatGPT says:
Generative AI models are different from calculators in several ways:
1. Generative AI models can generate new content, while calculators can only perform mathematical operations on existing data.
2. Generative AI models are trained on large amounts of data, allowing them to generate outputs based on patterns and relationships within the data, while calculators follow a set of predefined rules.
3. Generative AI models can make creative decisions, producing outputs that deviate from training data, while calculators can only produce outputs based on their inputs.
4. Generative AI models are complex and can make mistakes or produce unexpected outputs, while calculators are straightforward and reliable.
These points from ChatGPT itself are spot on. A calculator requires you to know how it works first, and to follow the rules laid down, and Generative AIs can produce new answers, including wrong ones.
Calculators automate a boring and mundane tasks for people who understand the principle of how that task works. Now a person might not actually know how to do complex division, but they had better understand what division is before just banging on buttons. With a Generative AI I don’t need to understand how it works, or even the subject I’m pretending to have studied, to create an impression of knowledge.
The role of women in the early part of the French Revolution (1789–99) is often overlooked. Women such as Olympe de Gouges, the playwright and activist, wrote the philosophical foundations for gender equality in “The Rights of Woman and Citizen” and Theorigne de Mericourt, a passionate speaker who organized many rallies in support of the revolution, were extremely active in the earliest days of the revolution.
I asked ChatGPT about women in the French Revolution, and then edited it down to give the paragraph above. I put almost no thought into the content, beyond knowing I can’t just cut and paste it. This is where Generative AIs become a problem, particularly where their datasets or algorithms have been ‘tuned’ to exclude or minimize certain topics. Now the argument could be made that I’ve learned these things, and it is no different to reading a history book or attending a lecture, but that is where the bias of the model comes in.
Generative AIs are not unbiased
There are several studies that have shown that racists are idiots, and as AIs are not intelligent it comes as no surprise that they’ve regularly descended into racism. In order to prevent that companies invest significantly in people to moderate the learning sets and outcomes to eliminate racism and other negative elements. While this is clearly a good thing, the laws of unintended consequences can then kick-in:
Now there is a big missing item on that list, slavery, if you drill more on North v South, and ensure that it has to mentioned the specific topic of Slavery then it’s line is clear: it’s a bad thing, it led to the Civil War, however it is easy to not go down those elements and instead focus on just this core list for more detail. Here in lies a key way that Generative AIs are very different from calculators: The data and manner of their training impacts the outcomes.
Now this can help a teacher to identify the use of a Generative AI, and indeed questions can be created that the AI will refuse to answer or answer but not in the way intended by the question (Devil’s Advocate questions for instance). What it also means however is that you must accept that the source is biased, and not in a way that a standard history book is biased, it has been biased out of fear of what the AI might say. Thus examination of difficult subjects can instead veer away from root causes and focus on ancillary issues to the detriment of supposedly “constroversial” topics.
Generative AI will have a place in education
Generative AI will certainly have a place in the future of education. The ability to question and answer and follow a thread can certainly help people learn subjects and investigate topics. It’s adoption however is much more a student having access to a private tutor, and a question on whether they or the tutor are actually doing the work, than it is to a calculator.
And it sucks as a calculator
When Alan Turing created the Imitation Game he theorised that basic mathematics would be the easy part. With today’s LLMs however this isn’t the case. So to entertain ourselves a bit, here are some examples of ChatGPT trying to do mathematics.
It will be REALLY easy for Maths professors to check if their students are using ChatGPT.
