These questions requiring simulation reveal a stark divergence between the human brain and GPT-3. Humans don’t acquire math knowledge or language comprehension in the manner GPT-3 does. Children, for instance, do not simply absorb sequences of words until they can predict what comes next. They learn language by associating symbols with aspects of their pre-existing inner simulation. This foundation isn’t rooted in sequence learning but rather in the connection of symbols to components of the child’s inner world.
Humans have the remarkable ability to validate mathematical operations through mental simulation. When you add one to three in your mind, you intuitively recognize that the result should be four. You don’t need to check this on your actual fingers; you can imagine the operation. This capability stems from the accuracy of our inner simulations, which faithfully replicate the rules and attributes of the physical world.
However, GPT-3, despite its vast knowledge, cannot engage in the same level of simulation. GPT-3 can provide correct answers to math questions it has seen before, but its understanding doesn’t extend to the deeper, intuitive comprehension humans possess. For example, the cognitive reflection test, designed to evaluate the ability to inhibit reflexive responses and engage in active thinking, presents questions where human cognition excels, whereas GPT-3 mirrors the common errors made by humans.
