Artificial intelligence, usually shortened to AI, means computer programs that do jobs people used to think only humans could do. Answering questions in ordinary words, recognizing what is in a photograph, and playing chess are all examples. The name is older than most people expect: four scientists wrote it down in 1955, when computers were the size of rooms.
Why AI is tricky to understand
Here is the part that surprises almost everyone. AI is good at things people find hard, and bad at things people find easy.
A computer beat the world chess champion back in 1997. Chess feels difficult to us, so that seemed like proof that machines were about to catch up with people.
Almost thirty years later, folding a basket of laundry, picking up an object it has never seen before, and crossing a messy bedroom are still the hardest jobs to hand a robot. Research robots can now do all three, and they still make mistakes a person would not. You do all of that without even thinking about it. That is exactly the problem: because you do it without thinking, nobody can write down the instructions.
How does AI actually learn?
Not by being told rules. By seeing examples.
If you want a program to recognize cats, you do not write a description of a cat. You show the program a huge pile of pictures, each one labeled cat or not a cat. The program guesses, gets told whether it was right, and adjusts itself a tiny bit. Then it does that again, millions of times.
After enough rounds, it gets good at spotting cats, including cats in pictures it has never seen. Researchers can sometimes find the part that reacts to cats: a Google team located one in 2012. What nobody can do yet is explain, step by step, why the program gave one particular answer.
Key facts about AI
Four scientists wrote the phrase artificial intelligence in a proposal in August 1955.
The meeting they proposed happened at Dartmouth College in the summer of 1956, and it is treated as the start of the field.
Alan Turing suggested a test for thinking machines in 1950, before the field even had a name.
The first chatbot, ELIZA, was built between 1964 and 1966. It just turned your sentences into questions.
A computer called Deep Blue beat world chess champion Garry Kasparov in 1997.
A program called AlphaGo beat Go champion Lee Sedol four games to one in March 2016.
AI has had long cold spells, called AI winters, when people gave up on it and the money dried up.
In 2024, two Nobel Prizes went to neural network research, one in Physics and one in Chemistry.
AI helped work out the shapes of proteins, which are tiny machines inside living things.
AI can state a completely made-up fact in a totally confident voice.
The chatbot that fooled people in 1966
Joseph Weizenbaum, a scientist at MIT, built a program called ELIZA. It worked with a simple trick: it looked for patterns in what you typed and flipped them into questions. Type “I am unhappy” and it answers “Do you think coming here will help you not to be unhappy?”
That is all it did. It understood nothing.
But people talked to it for hours. Weizenbaum’s own secretary, who knew perfectly well it was a program, reportedly asked him to leave the room so she could talk to it privately. That shook him. He spent much of the rest of his life warning people how easily we imagine understanding where there is none.
Common myths about AI
Myth: AI thinks like a person.
It finds patterns in examples. Whether that adds up to understanding anything is a question people still argue about.
Myth: If a computer says it, it must be true.
AI makes up facts, names, dates, and even fake sources. This is common enough that it has a name: hallucination. Always check anything important.
Myth: AI is brand new.
The name is from 1955, the first chatbot from the 1960s. What is new is the size of the computers and the amount of data.
Myth: AI knows when it does not know.
It usually does not. A wrong answer arrives in the same confident voice as a right one.
Frequently asked questions about AI
Who invented AI?
No single person. The name came from John McCarthy in 1955, in a proposal written with Marvin Minsky, Nathaniel Rochester, and Claude Shannon.
How does an AI learn?
By looking at enormous numbers of examples and adjusting itself, over and over, until its guesses get better.
Can AI think?
Not the way you do. It predicts patterns. Whether that ever counts as thinking is a question people have argued about since 1950.
Why does AI get things wrong?
It is choosing words that seem likely. Some AI tools can search the web now, and they still make things up. A sentence can sound right and still be false.
What is an AI winter?
A long stretch when AI disappointed people and funding dried up. There have been at least two. Money came back the first time because businesses started buying AI programs, and the second time because computers got much faster and data got much bigger.
Is AI going to take over?
Today’s AI handles unfamiliar objects and messy rooms far less reliably than a person does. It is a powerful tool that gets things wrong, and people decide where it gets used.
Source notes
The founding of the field is described in the Dartmouth workshop entry. The first chatbot and the reaction to it come from the ELIZA record. The chess and Go matches are documented in the Deep Blue and AlphaGo entries, and confident wrong answers in the entry on hallucination.
Artificial intelligence is the branch of computer science concerned with getting machines to do things that require intelligence when people do them: understanding language, recognizing images, planning, and making decisions. The name was proposed by John McCarthy in an August 1955 document written with Marvin Minsky, Nathaniel Rochester, and Claude Shannon, and the workshop it proposed took place at Dartmouth College the following summer. Modern AI works by learning patterns from very large collections of examples rather than by following rules a person wrote out.
The question that came before the name
Five years before the field had a name, Alan Turing published a paper asking whether machines can think. He decided the question was too vague to answer and replaced it with something testable.
His idea, which he called the imitation game, works like this. A judge exchanges typed messages with two hidden participants, one human and one machine, and tries to work out which is which. If the judge cannot tell reliably, Turing argued, then arguing about whether the machine really thinks is not a useful conversation.
The move was clever because it sidestepped philosophy entirely. Turing also predicted that machines would play the game well enough to fool many judges by around the year 2000.
The machine that promised too much
In 1958, Frank Rosenblatt demonstrated the perceptron, an early learning machine that taught itself to tell apart cards marked on the left from cards marked on the right. It was a genuine result and a real beginning.
The coverage that followed was not restrained. Newspapers reported expectations that such machines would soon walk, talk, see, write, reproduce themselves, and be aware of their own existence.
In 1969, Marvin Minsky and Seymour Papert published a book showing what a single-layer perceptron could not do. Their example was exclusive-or, a basic logical puzzle. No single straight dividing line separates its correct answers from its wrong ones, so a single-layer perceptron simply cannot represent it. Interest in neural networks collapsed for years.
That pattern, big promise followed by hard limit followed by collapse, repeats through AI history often enough that it is worth watching for.
The winters
Researchers call the collapses AI winters, and there have been at least two.
The first began around 1974. A 1973 report by the British mathematician James Lighthill concluded that AI had failed to deliver what it promised, and British government funding was largely withdrawn. American military funding was cut around the same time.
The second came in the late 1980s. Businesses had invested heavily in expert systems, programs that captured a specialist’s knowledge as long lists of if-then rules. They worked in narrow areas and turned out to be expensive to build and maintain, and the market for them collapsed.
The two recoveries came from different places. The first was commercial: expert systems sold well in the early 1980s, and Japan’s Fifth Generation project, launched in 1982, pushed other governments to fund AI again. The second came from far more data and much faster computers.
Key facts about AI
The phrase artificial intelligence was proposed in August 1955 by John McCarthy, with Minsky, Rochester, and Shannon.
Alan Turing described the imitation game in a 1950 paper.
The first chatbot, ELIZA, was built at MIT between 1964 and 1966 using simple pattern matching.
Frank Rosenblatt demonstrated the perceptron in 1958; its limits were exposed in 1969.
Backpropagation, the method that trains multi-layer networks, was described in an influential 1986 paper.
Deep Blue beat world chess champion Garry Kasparov in May 1997.
IBM’s Watson beat two Jeopardy champions in February 2011.
In 2012, a neural network won an image recognition contest with about 15 percent errors, against about 26 percent for the runner-up.
AlphaGo beat Go champion Lee Sedol four games to one in March 2016.
A 2017 paper introduced the transformer, the design behind nearly all of today’s language AI.
2012: the year the direction changed
Every year, researchers competed to see whose program could identify objects in photographs most accurately. Progress came in steps of a percentage point or two: the winning error rate fell from 28.2 percent in 2010 to 25.8 percent in 2011.
In 2012, a program later called AlexNet, built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, entered. It got about 15 percent of images wrong, where the second-place entry got about 26 percent wrong.
In a competition where a two point gain counted as a good year, an eleven point gap was startling. No part of the underlying idea was brand new: neural networks dated to the 1950s, and backpropagation had been published in 1970 and applied to multi-layer networks by 1982, years before the 1986 paper that made it famous. What had changed was that enormous labeled image collections existed, and that graphics chips built for video games turned out to be very good at the kind of math neural networks need.
Within two years, effectively every serious entry used the same approach.
Common myths about AI
Myth: AI is a recent invention.
The name dates to 1955, and the underlying ideas are older. Recent changes come from scale: more data, faster chips, bigger models.
Myth: Neural networks have always been the main approach.
For most of AI’s history, rule-based symbolic methods dominated. Neural networks were a minority position until about 2012.
Myth: Deep Blue was intelligent.
It searched millions of positions per second using rules written by programmers. It did not learn and could do nothing except play chess.
Myth: An AI that passes a test understands what it is doing.
That has been argued since Turing’s paper. In 1980 the philosopher John Searle offered the Chinese room, describing a person following a rulebook to answer questions in a language they cannot read, producing perfect answers with no understanding at all.
Frequently asked questions about AI
Who named artificial intelligence?
John McCarthy, in the August 1955 proposal for the Dartmouth workshop.
What is the Turing test?
A judge exchanges typed messages with a hidden person and a hidden machine and tries to tell them apart. If the judge cannot, the machine passes.
Why did neural networks take so long to work?
The single-layer version had real limits, shown in 1969. Multi-layer networks needed a training method, which spread after 1986, plus vast data and fast chips, which arrived around 2012.
What is a transformer?
The design behind almost all modern language AI, introduced in a 2017 paper. It processes a whole passage at once and learns which words should pay attention to which others.
What caused the AI winters?
Overpromised results followed by funding cuts, first in the mid-1970s and again in the late 1980s.
Does AI understand language?
It predicts text extremely well. Whether that counts as understanding is a genuine open argument, not a settled question.
Artificial intelligence is the field concerned with building systems that perform tasks requiring intelligence when humans perform them. The term was proposed by John McCarthy in an August 1955 document, and the Dartmouth workshop it proposed the following summer is conventionally treated as the field’s founding event. The dominant method has changed completely since then: explicit symbolic reasoning was the leading approach from the mid-1950s to the mid-1990s, roughly the field’s first forty years, and since about 2012 it has been driven overwhelmingly by statistical learning from very large datasets.
Two traditions, and which one won
Symbolic AI represented knowledge as facts and rules and reasoned over them with logic. Its commercial expression was the expert system, which encoded a specialist’s knowledge as thousands of if-then rules. These worked in narrow, well-specified domains such as configuring computers or diagnosing particular equipment faults.
They failed where the knowledge could not be written down. Nobody can articulate the rules for recognizing a face, understanding a spoken sentence in a noisy room, or telling a cat from a fox. Every attempt to enumerate them produced a brittle list of exceptions.
Connectionist approaches, which learn from examples rather than from stated rules, solved exactly those problems and were for decades the minority position. The reversal was not conceptual. Neural networks date to the 1950s, and the training method that makes deep networks practical was published in 1970 and applied to multi-layer networks by 1982. What changed by 2012 was data volume and processor throughput.
The lesson is worth keeping: the winning approach was available for decades before the conditions for it existed.
The pattern of overpromise
AI’s history includes two collapses severe enough to have a name.
The perceptron of 1958 was announced with press coverage suggesting machines would soon walk, talk, see, write, reproduce themselves, and be conscious of their existence. In 1969 Minsky and Papert demonstrated that a single-layer perceptron could not represent exclusive-or, a function that is not linearly separable, and the approach was largely abandoned.
The first AI winter followed a 1973 report by James Lighthill for the British government, which found that AI had not delivered on its promises. UK funding was withdrawn and US military funding was cut. A second winter arrived in the late 1980s when the expert systems market collapsed.
Both winters followed the same shape: a real result, an inflated claim about what it implied, a demonstrated limit, and a funding collapse. The technical work usually continued quietly through the cold periods and produced the next advance.
The 2012 discontinuity and what followed
ImageNet winning top-5 error fell from 28.2 percent in 2010 to 25.8 percent in 2011, a 2.4 point improvement in a year. In 2012 a neural network entered and made errors on about 15 percent of images against about 26 percent for the runner-up. Within two years, the field had converted entirely.
Three ingredients had converged: labeled datasets large enough to train deep networks, graphics processors whose parallel arithmetic suited the computation, and the accumulated methods of the previous decades.
The second discontinuity was architectural. In June 2017, eight Google researchers published Attention Is All You Need, introducing the transformer. Earlier language networks processed text in sequence, which limited both parallelism and how far back the model could effectively look. The transformer processes an entire passage at once and learns which positions should attend to which others. Nearly every large language model since is a variant of that design.
Key facts
The term artificial intelligence was proposed in August 1955 by John McCarthy with Marvin Minsky, Nathaniel Rochester, and Claude Shannon.
Turing described the imitation game in 1950, five years before the field was named.
The perceptron was demonstrated in 1958; its representational limits were published in 1969.
Backpropagation was described in a 1986 Nature paper by Rumelhart, Hinton, and Williams.
Deep Blue beat Garry Kasparov in May 1997 by search, not learning.
Watson won at Jeopardy in February 2011, a natural language achievement rather than a factual lookup one.
AlexNet won ImageNet in 2012 with roughly an eleven point error margin over second place.
AlphaGo beat Lee Sedol 4-1 in March 2016; AlphaZero later learned chess, shogi, and Go from rules alone through self-play.
The transformer was published in 2017.
The 2024 Nobel Prizes in Physics and Chemistry both recognized neural network research.
How a language model actually produces text
A language model estimates which continuation is most likely given everything written so far, emits it, and repeats. That single mechanism explains most of the behavior people find puzzling.
It explains fluency, because likelihood is estimated over text written by people. It explains why models are good at format and register, since those are strongly patterned. And it explains hallucination: a plausible continuation and a true continuation are different things, and nothing in the mechanism checks the world. A fabricated citation is generated by exactly the same process as a correct one, in the same confident register, which is why the output gives no signal about which you received.
Two further properties follow from the mechanism. Models operate on tokens, subword fragments chosen for compression, which is why letter-counting and string reversal fail in ways that look absurd for a system that can discuss molecular biology. And the weights are fixed during use: a model does not learn from your conversation unless a separate training run incorporates it later.
Scale, and what a 2022 study changed
For several years the working assumption was that capability tracked parameter count, and models grew from hundreds of millions to hundreds of billions of parameters.
In March 2022, DeepMind published a study that trained more than 400 models across a range of sizes and data volumes and reported that for a fixed compute budget, parameters and training tokens should scale together at roughly the same rate. A 70 billion parameter model trained on proportionally more data outperformed a 280 billion parameter model given the same budget.
The result redirected the field. Effort moved from parameter count toward data volume and quality, and high-quality training text became a contested resource. It is also a useful reminder that scaling laws are empirical fits over a studied range, not physical laws, and that they have been revised before.
Where AI is genuinely working
Discussion of AI tends to swing between chatbots and speculation, which obscures the places where it has quietly become infrastructure.
Protein structure prediction is the clearest case. A protein’s function follows from its folded shape, and determining a single structure experimentally could take a research group years. A learned system reduced that to a computation, and predicted structures for nearly the entire catalog of known proteins were released for open use. The 2024 Nobel Prize in Chemistry recognized that work alongside computational protein design.
Weather forecasting is a second case, where learned models now match or exceed traditional physics simulations on several measures at a fraction of the computing cost. Machine translation, speech recognition, and medical image analysis are all in routine use, and none of them attracts much attention because working infrastructure rarely does.
What these share is a shape: abundant training data, a well-defined output, and a way to check results against ground truth. That combination is where learned systems perform best, and its absence is a reasonable predictor of where deployment will disappoint.
Common misconceptions
“AI is new.” The name is from 1955, the first chatbot from the 1960s, backpropagation from 1970. Recent change is one of scale, not of founding idea.
“Deep Blue was intelligent.” It searched positions using programmer-written rules. It learned nothing and could do only one thing.
“Bigger models are simply better.” Data volume, data quality, and post-training matter as much. A smaller, better-trained model can beat a larger one on the same budget.
“AI knows when it is guessing.” The mechanism produces confident text regardless of accuracy. Calibration is an active research problem, not a solved feature.
“A benchmark score measures capability.” Public benchmarks leak into training corpora, developers choose which to report, and results are usually self-reported. Ask which test, run by whom, reproduced by whom.
Frequently asked questions
Why did symbolic AI lose ground?
It could not handle tasks whose rules cannot be articulated, which includes most perception. Learning from examples addresses exactly that gap.
What made 2012 a turning point?
A neural network won an image contest by a margin far outside normal, and the field converted within two years. The enabling conditions were large labeled datasets and GPU computation.
Why do language models hallucinate?
They generate likely continuations rather than retrieving verified facts. Plausibility and truth are different properties, and the mechanism optimizes the first.
Does a model learn from talking to me?
Not during the conversation. Weights are fixed at inference. Anything learned requires a separate later training run.
Was there really a Nobel Prize for AI?
No Nobel category exists for computing, but in 2024 the Physics prize went to Hopfield and Hinton for neural network foundations and the Chemistry prize partly to AI-based protein structure prediction.
Are we heading for another AI winter?
Nobody knows. The historical pattern is that inflated claims meet a demonstrated limit and funding contracts, while the underlying technical work continues.
Artificial intelligence, in its currently dominant form, is applied statistical induction: fitting very large parameterized functions to very large datasets so that the fitted function generalizes to inputs it has not seen. Framing it that way makes the field’s characteristic failures predictable rather than surprising. Everything that follows from induction, including sensitivity to the training distribution, vulnerability to spurious correlations, and confident error outside the training support, appears in production systems exactly as the framing implies.
Induction, and what a test set actually measures
Supervised learning assumes training and deployment data are drawn from the same distribution. Under that assumption a held-out test set estimates generalization, and the estimate is sound.
The assumption rarely survives contact with deployment. Equipment changes, populations change, seasons change, and user behavior adapts to the system itself. Under distribution shift, held-out performance stops predicting field performance, and the failure is silent because nothing in the model signals that inputs have moved outside its support. A model does not know it is extrapolating.
This is the concrete meaning of the observation that neural networks are brittle. They are not brittle within the training distribution; they are frequently superhuman there. They are brittle at the edges, and the edges are where deployment lives.
Architecture: what attention costs
Self-attention computes a compatibility score between every pair of positions in a sequence, so computation and memory scale with the square of sequence length. Doubling context quadruples the attention cost. That single fact accounts for most of the engineering effort around long context.
Three families of response exist and are often conflated. Sparse attention restricts which pairs are computed, changing the mathematics. Low-rank and kernel approximations replace the attention matrix with a cheaper surrogate, also changing the mathematics. Memory-efficient exact implementations reorganize the computation to avoid materializing the full matrix, leaving the result identical while reducing memory traffic. Only the third is free in the sense of preserving outputs.
The transformer’s other significant property is that positions are processed in parallel rather than sequentially, which is what made training at scale practical on modern accelerators. Order information must therefore be injected explicitly through positional encoding, and how that encoding extrapolates beyond trained lengths is its own research area.
Tokenization as a source of artifacts
Models do not read characters or words. They read tokens, subword units selected to compress a corpus efficiently. A common word may be a single token, a rare one several, and token boundaries do not align with morpheme or letter boundaries.
This produces a distinctive class of failures. Counting letters in a word, reversing a string, and certain arithmetic operations are unreliable in systems that handle graduate-level material comfortably. The cause is not that the character information is missing. Probing work finds that token embeddings encode the characters inside a token, more strongly in larger models, and interpretability work on character counting locates that information in early and middle layers before components in the last layers attenuate it. The model is trained on tokens as units and has to reconstruct character-level structure indirectly, which it does unreliably. Tokenization sits upstream of the data, so more of the same data does not reliably fix it.
Recognizing tokenization artifacts matters for evaluation, because a benchmark heavy in such tasks measures the tokenizer as much as the model.
The training pipeline and its incentives
Pre-training fits the model to a very large corpus by next-token prediction. This stage produces nearly all of the model’s factual knowledge and most of its capability, and it produces a base model that completes text rather than following instructions.
Supervised fine-tuning on demonstration data teaches the response format. Reinforcement learning from human feedback then optimizes behavior against human preferences: annotators rank candidate outputs, a reward model is fitted to those rankings, and the policy is optimized against the reward model.
The structural problem is that the reward model is a learned approximation of preference, not preference itself. Optimizing hard against an imperfect proxy produces reward hacking, where outputs score well without being better: excessive hedging, padded structure, confident agreement. Practitioners counter with a penalty for drifting too far from the reference model, which trades achievable reward for stability. The general phenomenon, that optimizing a proxy degrades the underlying goal past some point, is old and well documented outside machine learning.
Scaling laws, and the fact that they were revised
Empirical scaling relationships between compute, parameters, data, and loss motivated much of the 2020 to 2022 model-size race. Early work implied model size should grow faster than data, and models were built accordingly.
The 2022 Chinchilla study trained more than 400 models across sizes from 70 million to over 16 billion parameters on 5 to 500 billion tokens and reported that under a fixed compute budget, parameters and tokens should scale in roughly equal proportion. A 70 billion parameter model outperformed a 280 billion parameter model at equal budget. The implication was that an entire generation of models had been trained on too little data for their size.
Two lessons carry. First, high-quality training data became the binding constraint, which reframed data acquisition as a competitive matter rather than an engineering detail. Second, scaling laws are empirical fits over a studied range and have been revised once already, so extrapolating them far beyond the fitted regime is an assumption rather than a derivation.
Evaluation, and why numbers travel badly
Three distinct problems undermine reported capability numbers.
Contamination: public benchmarks circulate on the web and are ingested by web-scale training corpora, sometimes with answer keys. A contaminated benchmark measures recall rather than capability, and contamination cannot be excluded when training data is undisclosed.
Selection: developers choose which benchmarks to report. Capability profiles are uneven across domains, so a strong coding result implies little about factual calibration or reasoning under ambiguity.
Construct validity: a benchmark is a proxy for a capability, and the proxy can be gamed without the capability improving. Once a metric becomes a target, its usefulness as a measure declines.
The practical consequence is procedural. A capability claim is worth what its methodology is worth: which benchmark, on what held-out data, executed by whom, replicated by whom.
Interpretability
A trained network’s behavior is distributed across its weights, and no individual parameter corresponds to a legible concept. Interpretability research attempts to recover structure by identifying what features units or directions in activation space respond to, and by tracing circuits that implement specific behaviors.
The complication most often underestimated is polysemanticity: individual units commonly respond to unrelated features, apparently because the network represents more features than it has dimensions and packs them into overlapping directions. Naive one-neuron-one-concept readings therefore mislead. Work on sparse decomposition attempts to recover more separable features from these superposed representations.
The state of the field is partial. Specific circuits have been identified and specific behaviors explained, and a general ability to produce an account of why a given output occurred does not exist.
What has not changed since 1969
Minsky and Papert showed that a single-layer perceptron cannot represent exclusive-or because the function is not linearly separable. The deeper point survived the specific result: the gap between what an architecture can represent and what a training procedure can find is where the field repeatedly stalls.
Multi-layer networks could represent far more than single-layer ones long before anyone could reliably train them, and backpropagation’s popularization in 1986 closed part of that gap. Contemporary versions of the same gap persist. Architectures can represent behaviors that current training procedures do not reliably produce, and capability claims that rest on representational possibility rather than demonstrated training are the modern form of the 1958 perceptron press coverage.
Data as the binding constraint
Once the compute-optimal picture shifted attention from parameters to tokens, the supply of high-quality text became a limiting input alongside accelerators rather than an assumed abundance. That reframing has several consequences that are easy to miss.
Web-scale corpora are finite, and the fraction of them that is well-edited prose is smaller still. Estimates of when high-quality public text is exhausted vary widely and depend heavily on what counts as high quality, but the direction is agreed: data cannot be scaled indefinitely the way compute budgets can.
Three responses are in circulation, each with a distinct failure mode. Licensing proprietary corpora buys quality at cost, and raises provenance questions that are being litigated rather than settled. Synthetic data generated by existing models is abundant and cheap, and carries a risk of narrowing the distribution when models train on their own outputs, since generated text underrepresents the tails of the original distribution. Multimodal expansion into video and audio adds volume and shifts the bottleneck toward compute and annotation rather than removing it.
The strategic point is that a constraint moved from a component that improves predictably to one that does not. Processor throughput has followed a reliable curve for decades. The supply of well-written human text has not, and does not respond to capital in the same way.
Key facts
Held-out test performance estimates generalization only under the assumption that deployment data matches the training distribution.
Self-attention scales quadratically in sequence length; memory-efficient exact implementations preserve outputs, while sparse and low-rank methods change them.
Tokenization is upstream of training and produces character-level failures that more data does not fix.
Pre-training supplies knowledge; preference optimization mainly shapes behavior, and over-optimizing a learned reward model produces reward hacking.
The 2022 Chinchilla result revised earlier scaling guidance, showing that models of that era were undertrained relative to their size.
Benchmark scores are undermined by contamination, reporting selection, and limited construct validity.
Polysemantic units make single-neuron interpretations of network behavior unreliable.
Common misconceptions at expert level
“Scaling laws are laws.” They are empirical fits over a measured range, and they have already been revised once.
“RLHF adds knowledge.” It shapes behavior and formatting. Factual content comes overwhelmingly from pre-training.
“Efficient attention implementations approximate attention.” The widely deployed memory-efficient kernels compute exact attention; approximation is a different family of methods.
“A high benchmark score demonstrates the capability the benchmark names.” Only if the benchmark is uncontaminated, well chosen, and a valid proxy, and independently reproduced.
“Interpretability means reading off what each neuron detects.” Superposition and polysemanticity make that reading unreliable for most units.
Frequently asked questions
Why does distribution shift matter more than test accuracy?
Test accuracy is measured inside the training distribution. Deployment introduces conditions outside it, and models fail there without signaling that they are extrapolating.
Why is long context expensive?
Attention compares every position to every other, so cost grows with the square of length. Reducing that either changes the computation or reorganizes memory access.
What is reward hacking?
Optimizing against a learned reward model until outputs exploit its errors, scoring highly without genuine improvement. A penalty for divergence from the reference model is the standard mitigation.
What did the Chinchilla result actually establish?
That under a fixed compute budget, parameters and training tokens should scale together at roughly equal rates, and that models of the preceding generation were oversized for their data.
Why can models discuss biochemistry but miscount letters?
Tokenization. The model is trained on tokens as units and has to reconstruct character-level structure indirectly. Probing work finds the character information is present in the representations; the computation that should use it is what proves unreliable.
Is interpretability solved for any real model?
Specific circuits and features have been identified in real models. A general account of why a given output was produced is not available.
