Prediction, Not Retrieval
5 min read · 4 principles · apply it by lunch
The model itself has never looked anything up. Not once. When your app searches the web, it pastes the results in as more text for the model to read.
When a model last handed you a figure you couldn’t check on the spot, what did you picture it doing to produce that number? Whatever you pictured, it wasn’t that. Spend three minutes this morning asking it something you already know the answer to, and the picture corrects itself.
In 2017 Ashish Vaswani and seven colleagues at Google published a paper on machine translation called ‘Attention Is All You Need’. It introduced the transformer, and that is the architecture underneath every model you use. What a transformer does is narrow: given a stretch of text, work out which fragment is most likely to come next, then repeat. There’s no index behind it, no lookup, no library at the back of the room. The trained model is a very large set of weights encoding statistical regularities in the text it learned from, which is why it can hand you a citation with a real journal, a plausible author and a volume number that has never existed. It didn’t fetch the wrong record. It produced the shape of a citation. Almost every disappointment you’ve had with these systems follows from that one fact, and yet most people’s working picture is still a search engine that talks.
Prediction, Not Retrieval is Day 103 of 365.
In the Daily, each of its four principles is explained in full, with a one-page template to apply it the same day. A new framework arrives every morning at 7am for a year, across all fifteen courses of the FutureMBA curriculum.
$29 a month · cancel any time · 7-day money-back guarantee