Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa, 2022
In 2022 a team led by Takeshi Kojima found that a large language model's accuracy on arithmetic word problems jumped if the answer was made to begin with a particular sentence.
With it, InstructGPT went from 17.7 percent to 78.7 percent on the MultiArith benchmark and from 10.4 to 40.7 percent on GSM8K, with no examples and no training. Asked to reason before answering, the model produced a chain of reasoning and then an answer that was far more often right. Whether this is a program is the question the exhibit asks. It is text that changes what a machine does, it was found by experiment rather than designed, and it works in no language but English. The collection opens with a table of operations for an engine that was never built and closes with a sentence addressed to a machine that was. This is the shortest work in the museum, and the only one every visitor can already read.