AI
What the machines can do, what they can't, and why the gap is the whole story. The long view on AI, hold the hype.
AI Won't Replace Software Engineering (Yet)
But it will raise the bar.
I have a boring opinion: AI is a tool.
Like a text editor, a compiler, grep. And like every tool, it's clueless, easily nudged off track, and sometimes too eager to please. That's not a flaw of character. It's a stochastic model, not a person.
It is a groundbreaking tool. My third most-used after the text editor I'm typing this in and the shell running it. No tool has rooted itself this deep, this fast. Long-abandoned projects look polished. Ideas I shelved as too tedious are a prompt away. It's almost magic.
The numbers feel magic too:
- OpenAI processes ~6 billion API tokens per minute; the Library of Congress, every three seconds.
- Anthropic went from $87M to $30B in revenue in 27 months; a 345Γ run.
- NVIDIA hit $4T in July 2025 and $5T in October 2025; the first company at either line.
The headlines are magic too:
- Frontier models now score ~2Γ expert virologists on the Virology Capabilities Test; biology benchmark scores rose >4Γ from Nov 2022 to April 2025.
- Two of the four 2024 Nobel Prizes went to AI work; Hassabis and Jumper for AlphaFold, Hopfield and Hinton for neural networks. First Nobels for AI, ever. Two in one year.
- On 2-hour tasks, AI agents beat human experts 4Γ. On 32-hour tasks, humans win 2-to-1. There's a duration cliff in agentic work, and almost all real engineering lives on the wrong side of it.
Still, I'd taper expectations. The underlying tech is human language, not a model of the world. Language is a symptom of intelligence, not the cause. AI is still a tool.
Replacing Software
I've been coding with AI assistants for over two years. The shortest summary I have:
The best intern I could ask for. It listens, chips away, makes some bad assumptions. On familiar ground, it shines. Off-trail, the output is off-trail too. With clear direction, it solves most of what a working engineer hits in a day.
I've watched it fall flat in narrow places (LaTeX Tikz, custom diagrams) and broad ones (overweighting the current file instead of reaching for world knowledge). It's not just me:
- METR ran a randomized controlled trial: 16 experienced open-source devs, 246 real GitHub issues. With AI tools enabled, they took 19% longer to ship. Afterwards, the same devs estimated AI had sped them up by 20%. A 39-point gap between feeling and measurement, on their own codebases.
- MIT's Project NANDA found 95% of organizations investing in generative AI saw "little to no measurable impact on P&L" despite $30β40B in enterprise spending. Internal builds succeed at one-third the rate of purchased ones.
- Goldman Sachs, March 2026: "no meaningful relationship between AI and productivity at the economy-wide level". Only 10% of S&P 500 management teams quantified AI's impact on a specific use case. Just 1% quantified it on earnings.
- Only 21% of generative AI users have redesigned a workflow around the tool. The rest are layering AI on top of processes that don't need it.
- Hallucinations are the #1 documented AI failure mode: 38% of catalogued incidents, more than nonsensical output (25%) and fabrication (15%) combined.
- AI incidents tracked by the AI Incident Database climbed from 149 (2023) to 233 (2024) to 362 (2025). The trendline is accelerating, not slowing.
My workflow is unrecognizable from two years ago. The objective hasn't moved: ship software that scales. Code is one piece. AI helps with most of them, but someone still has to orchestrate top-to-bottom. AI turns a rowboat into a crewed ship, and the rower into the captain. They route, monitor, and adjust. I find AI roughly useless without me at the wheel, nudging it back on course.
This may age badly, but I'll stake it: the software engineering industry keeps growing for a while yet. It contracts later, when models meet or beat human intelligence and the captain stops being load-bearing. For now we're building the ship and flying it at the same time.
So if we aren't being replaced, what are we doing?
Raising The Bar
When the compiler arrived, demand for engineers went up. More ambitious projects became feasible; the harder problems got reached for, not skipped. Productivity tools tend to do that β they raise the ceiling before they shrink the floor.
The lesson: lean into where AI is strong. Onboarding, legacy code, the routine. I haven't written an argparse CLI by hand in over a year, and I don't miss it. Side projects that would have taken months take days. The cobwebs in my codebases are fun to clear, not tedious. Professionally, however, it's a different story.
The difference is simple: I read every line of code I ship at work. I don't read every line I ship at home. Side-project stakes are low. if it breaks, someone files an issue or sends a patch. Production stakes aren't.
When models pass human intelligence, I'll stop reading the code to fix it and start reading the code to learn from it. It'll know more than I do. Until then, intelligence is bounded by context windows and language. I'm not staking a career on a pile of tokens and linear algebra β neither by skipping the code review, nor by quitting the field.
A few real obstacles on the way there:
- Only 3% of developers "highly trust" AI output. About 46% explicitly don't. 75% still ask a human when they don't trust the model.
- Hyperscaler AI hardware depreciates fast enough that ~$400B in annual depreciation outpaces their combined profits. The denominator can't keep up with the numerator.
- Art history grads (3% unemployment) are now more employed than computer scientists (6.1%) and computer engineers (7.5%). Twenty years of "learn to code" inverted in twenty-four months.
- US data centers used 183 TWh of electricity in 2024 β roughly all of Pakistan's annual consumption β projected to grow 133% to 426 TWh by 2030.
Those are problems for us, as humans, to fix.
As engineers, we do what we did when the compiler shipped, and the IDE, and the package manager: we build more. Better. Sometimes the bottleneck is human review; sometimes it's reaching past where the model can see. The bar moves up.
Every engineer now runs a small team of capable agents. We can build anything we can specify. We can serve very small audiences with very specific software, or very large ones with very hard software. The work isn't going away. It's getting harder, and more interesting, at the same time.
Still The Captain
I trust the model on the easy stretches. I read every line on the hard ones. That doesn't change until the ship doesn't need a captain. And from where I'm standing, mid-flight, it still does.
){kind=link}
There's No Place Like $HOME
A ten-year journey of mostly Vim, then VS Code, then back to Vim.
Let me take you through ten years of text editors.
I've been programming primarily via a shell since 2016, and my dotfiles go back to 2017. I wrote the thesis that you should care deeply about the tools you use. In 2026 I use the same tools I did then. But a lot else has changed.
I entered big tech, and noticed how few engineers around me shared my tool suite. Most lived in a GUI editor where the terminal was optional and a shell was something you bolted on. I found it increasingly hard to communicate intent with colleagues. So, out of FOMO, I decided to try something different. After a decade of Vim, I changed my daily driver to VS Code.
Then, around 2025, the work itself changed. My workload and my approach to it were suddenly different.
IDE For Me?
At first it was pretty nice. The tooling was integrated: version control was a panel instead of a man page, the file tree sat right there, a debugger and a run button one click away. Nothing needed a name before I could use it; every action was a menu, every menu a button. For someone starting out it lowers the floor, and you can be productive before you understand the machine underneath.
My "Oh, this just works" moment was subtle but delightful: my compiler would print a filepath, line, and column in its output, and one day I realized I could click it and land on that exact spot in the source. It was exactly what IDEs promise: it just worked.
I took many other lessons from the IDE into Vim, and almost all of them collapse to a single keystroke: go to definition (gd, yes I had to setup language servers), find all references (gr), rename a symbol everywhere it lives (F2), the lightbulb quick-fix (F4), language-server autocomplete, a Problems panel for the squiggles (<leader>xx). I even bound the fuzzy file opener to Ctrl-P, the same key VS Code uses for the same thing.
It had quirks. A beat of lag sat between the keypress and the glyph, just long enough to notice; a one-keystroke edit became a command palette, a fuzzy match, an Enter; opening a 5,000-line file spun the fan up like a jet on a runway. The terminal lived in a cramped strip at the bottom, a few lines tall, where my whole workflow used to live. And the editor I tuned on my laptop was never quite the editor waiting on my desktop. Each one was small. Together they were ceremony.
By the end, though, it felt like maintaining a delicate garden. I would weed away panes so I could see more code and then they would sprout back up after a reboot.
My typical workflow would be:
- My cursor sits in a function I just wrote. It calls an API I don't know, so I take a hand off the keyboard and mouse over it, waiting for the indexer to wake up.
- Right-click the symbol, scan the menu, click Go to Definition. A new tab opens in another file and scrolls me somewhere I didn't pick.
- That definition calls a bare helper. Hover it, wait for the tooltip, click Go to Definition again. A third tab piles on.
- Read the helper, then hunt the tab strip for the tab that was mine. Click back, mis-click a neighbor, correct.
- Back in my function, drag-select the wrong argument with the mouse, then fix the call.
- Click into the terminal pane, run the file, and wait for it to come back.
$HOME, sweet, $HOME
The thinking then is the same now: an IDE built for everyone optimizes you for average. This doesn't mean IDEs are bad or that people who use IDEs should feel bad (even John Carmack swears by Visual Studio, the full Microsoft IDE, not VS Code). It just means tooling is worth the investment when you live in code (even your IDE). Before Vim I used Xcode, Sublime Text, and Atom, and I still think Sublime was the GOAT back then.
With shell tools, I am able to craft the perfect environment for me. I like coding at the speed of thought. I feel more comfortable navigating, modifying, and manipulating the code all via the keyboard. I like completely automatable environments. I like being in control.
My typical workflow is:
- My cursor sits in a function I just wrote. It calls an API I don't know, so I read it in place. No mouse, nothing to wake up.
gdjumps to the definition, and the jump is recorded so I can come back.- That definition calls a bare helper.
gdagain, another hop on the jump list. - Read the helper.
Ctrl-o Ctrl-owalks the jump list back the way I came, onto my function. - Back in my function,
ci(swaps the argument in normal mode, no selection to drift. Ctrl-wto the tmux pane, recall the run command from history, Enter; the output prints in place.
Same six steps. Not one of them takes my hands off home row.
The AI of it All
Confession: FOMO was not the only reason I decided to try VS Code. My dotfiles had gone stale, mostly untouched for five years. Reviving them meant sinking days into a revamp before I could code again, or switching to VS Code and coding immediately. I chose the latter.
But then AI flipped that premise on its head. Particularly, Claude Code changed everything for me in 2025; can you spot when I started using it?
The cost of maintaining dotfiles was the cost of the tokens to do it.
But you may ask, with tools rapidly changing and shifting around AI today, will the terminal even matter tomorrow? In my opinion, it will matter even more.
The cost of writing code is minimal; the cost of reading code is still your reading speed. I read every line of code I ship to production, and you should too. The task has now shifted, but the thinking has not: the more time you spend reading code, the more time you should invest in tooling to assist you with it.
Reading code is not linear. You jump from your function to an API, to its helpers, and back. That's the motion both lists above describe, and the motion an editor lives or dies on.
How does Vim do this any better than an IDE? Because it's already designed for it.
Open a file in Vim, start typing, and nothing lands. You're in normal mode, where the keys are a vocabulary for moving, not typing: j/k/h/l step through the buffer, / jumps to anything you can name, gd lands on a definition and gr lists every reference, Ctrl-o walks the jump list back the way you came, % hops to the matching brace, ci( rewrites what is inside the parens with no selection to drift. Every one of those is a way to read. Writing is a mode you ask for, a deliberate i before a single character lands, and you leave it the moment you are done. The resting state of the editor is the thing you do most: looking at code that already exists, moving through it the way you read it, without waking an indexer, without ever reaching for the mouse.
An IDE is built the other way around. The cursor blinks at an insert point the second a file opens, primed to type, and every reading motion is a second layer bolted over that: a menu to find, a panel to define, a click to chase a reference. The default is write; reading is the thing you add. That used to be a preference. AI made it the whole job. When writing a line costs a sentence of English, the expensive part is no longer producing code, it is reading the code that came back, often code nobody at the keyboard wrote. Vim built its grammar for exactly that motion in the nineties, decades before the motion became the bottleneck.
You might object that agents change even this: you read their work as a diff in a review pane, not a buffer you move through. But a diff is a keyhole. To trust a change I still open the files it touches and follow what it calls, and that tracing lives in the editor, not the review pane. The pane shows me what moved; Vim is where I find out what it meant.
Whatever Works, Works
I tried something new. Don't drop everything you use and replace it with my stack; find one part of your workflow, see if it can be improved, and keep what works. If I had to pick somewhere to start, give Vim keybinds a go.
Learning a new editor is hard. Learning a key at a time is sustainable.
One afternoon it took me until lunch to notice my mouse had died. I'd been somewhere I didn't need it. There's still no place like $HOME.
Artificially Unintelligent
mini(wins), max(losses)
Most problems in AI come down to optimization. Neural networks train by minimizing a loss. Good Old-Fashioned AI plays games by maximizing a heuristic evaluation. The architectures look nothing alike, but the engine underneath is the same: pick the parameter (or move) that pushes some number in the right direction.
For a chess engine, that number comes from a "fitness function": a heuristic that takes the board state and produces a score for the position. The AI then prioritizes moves that maximize its fitness while trying to minimize the adversary's. This is the famous minimax algorithm, and it goes something like this:
def minimax(board, depth, maximizing):
if depth == 0 or board.is_terminal():
return evaluate(board)
if maximizing:
value = float('-inf')
for move in board.legal_moves():
child = board.apply(move)
value = max(value, minimax(child, depth - 1, False))
return value
else:
value = float('inf')
for move in board.legal_moves():
child = board.apply(move)
value = min(value, minimax(child, depth - 1, True))
return value
We can do something interesting here. We win by maximizing our score and minimizing the opponent's. But what happens if we negate the fitness \(f(x)\), say by returning \(-f(x)\)? We start minimizing our own score and maximizing the opponent's. We start, in effect, trying to lose.
def dumb_evaluate(board):
return -evaluate(board)
One line. That's the whole thing. The algorithm doesn't change; it still faithfully maximizes whatever score it's given. We just give it a worse score, and it faithfully drives the game off a cliff.
With this, we get some pretty entertaining games. I present to you Artificial Unintelligence.
Smart vs Dumb
Eight hand-picked games where Smart starts at a material deficit, often facing a Black army arranged into a deliberate visual pattern, and still wins. The set is ordered from the most ordinary to the most theatrical. Every game ends in checkmate (1-0).
Standard Match
The control. Standard opening, no FEN setup, no visual pattern, just Smart vs Dumb. Smart wins on g6 with a quiet bishop sacrifice and a queen mate. Sets the baseline before things get strange.
Pawn Cross
Black's king sits dead center inside a small + of pawns. Smart has only K + R against this miniature cross and threads the rook around the arms, stripping pawns one at a time before delivering Ra1#.
Zigzag Fence
Black's pawns form a perfect zigzag fence across ranks 4β7. Smart's queen alone walks the fence end to end, picking off pawns on the diagonal and converging on Qe2#.
Bishop Constellation
Black's eight bishops sit on the long diagonals like a star map; Smart's pawn phalanx sits below them. 36 moves of slow promotion warfare yield four white queens and a final Qc1# from the corner of the board.
Four Knights
Smart starts with K + four knights symmetrically posted on a1/c1/f1/h1; Black has the entire opening army. Knights jig their way into a cooperative net and mate before Black can mobilize a single major piece.
Skull Mask
Black's pieces draw a skull: rooks for eye sockets, queens for temples, knight and bishops for jaw and teeth. Smart's king starts in the corner and tip-toes out while a rook and queen deconstruct each feature, ending Qd5#.
Spiral Vault
Black's pieces spiral outward from the king in a vault formation, a chaos of bishops, knights, rooks and queens stacked on every square of the upper half. Smart's R+B+Q carves a path inward through 27 moves of attrition to Qg7#.
Mosaic Blitz
Black's bishops and knights tile the back two ranks in a perfect checkerboard mosaic. Smart's pawn wave cracks open the mosaic, promotes three queens in five moves, and ends in a queen-and-queen double-mate.
AI Foundations
Ball don't lie, and neither does linear algebra.
In the future, AI will be the last problem humans solve by themselves. This final frontier is called AGI: Artificial General Intelligence. It will do just about everything a human can, and if it can't do it it will learn it.
As humans we learn in the brain and evolve through evolution. Machines learn in the cloud and evolve in the cloud, on processors with 80 billion transistors apiece, fed by datacenters that draw the power of a small city, programmed to do nothing but learn and evolve.
After they are born, nothing will be the same. Every problem we've been trying to solve, from medicine for every disease to a unified theory of physics, will be solved. Just not by us.
But that's AI of the future. Today, AI is seeping its tentacles into just about everything. It's taken over the voice assistants of the past, and turbocharged our code completion engines. What used to be ML is now AI, in part due to the generative wave: chatbots that hold a real conversation, image models that paint on demand, code completion that finishes your sentences before you do.
Today, I'll talk about what AI is, in concrete terms.
What Is AI
Three definitions, that you will certainly come across in most textbooks:
"The field of study that gives computers the ability to learn without being explicitly programmed."
β Arthur Samuel, 1959, coined while building a checkers program at IBM. The most-quoted definition in every textbook.
"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E."
β Tom Mitchell, 1997, from Machine Learning (McGraw-Hill). The formal definition.
"Artificial intelligence is the science and engineering of making intelligent machines, especially intelligent computer programs."
β John McCarthy, 1956, who coined the term "artificial intelligence" at the Dartmouth Workshop.
In short, AI is giving intelligence without explicit programming. It started with symbolic AI: logic, rules, search, puzzles. These developed into expert systems, using hard-coded rules for medicine, engineering, finance.
The early confident promises ran out, twice. The 1974-1980 AI winter. The 1987-1993 winter followed the collapse of the LISP-machine industry and expert systems' failure to scale into the enterprise. Funding evaporated both times. The current boom is the longest sustained period of progress AI has ever had, which is either a sign that the technology finally works or a sign that the third winter will be uglier than the first two when it comes.
Game-playing AI was the comeback story between the winters and the modern era. IBM's Deep Blue beat Garry Kasparov at chess in 1997 with brute-force search and a hand-crafted evaluation function. IBM's Watson took Jeopardy! in 2011 with statistical NLP layered over a curated knowledge base. DeepMind's AlphaGo beat Lee Sedol at Go in 2016, combining deep neural networks with Monte Carlo tree search in a game whose branching factor had been considered intractable. AlphaZero learned chess, shogi, and Go from self-play in a matter of hours the next year.
The modern era starts around 2012, when AlexNet won ImageNet with a deep convolutional network and a pair of GPUs, dropping the image-classification error rate by ten points and making deep learning the default. Word2vec (2013) showed that words could live as vectors in a learned space. The Transformer paper (2017) replaced recurrence with attention. GPT-3 (2020) showed that scale alone, billions of parameters and trillions of tokens, was a research direction. By late 2022 a public-facing chatbot crossed 100 million users in two months, and the industry has not stopped pouring concrete since.
Mathematically, an AI is nothing more than a function fitting by optimization. Simpler AIs can learn linear relationships and need just a few linear algebra equations at their disposal. More complex ones, such as language models, employ advanced techniques like neural networks, loosely modeled on the brain's layered neurons. As they are more sophisticated, they too need more sophisticated mathematics: calculus, statistics, and information theory. You don't need advanced maths to train or deploy AI models, but to make advancements in frontier AI you certainly will. You won't understand backpropagation if you don't understand derivatives and the chain rule; and thankfully, that's not our goal today.
The goal here is to use AI to solve day-to-day problems, not derive new theorems for the future. At least, not yet.
In Concrete Terms
The goal is to train an AI model, which is nothing more than the learned function mapping training data to predictions. Your training data contains the features you wish to train on, the input variable (i.e., square footage when predicting house price). The thing you're predicting is called the label or target (i.e., the actual house price).
Parameters are the internal values the model learns, such as the famous weights that correspond to how large language models store their knowledge. Along the way, you'll optimize hyperparameters, which are essentially settings you choose before training (i.e., learning rate, batch size, regularization strength). The learned function improves via a loss function, which measures how wrong the model is. For example, if you have a set of features you wish to train on and the labels that you know to be ground truth, you can simply minimize for the mean-squared error:
$$ \mathrm{MSE} = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2 $$
The dataset is usually broken into a:
- Training Set Data the model learns from, usually 70β80% of your data.
- Validation Set Held-out data for tuning hyperparameters.
- Test Set Final unseen data for honest performance evaluation.
The split exists because of overfitting: the failure mode where a model memorizes training data instead of learning the pattern. A model that scores 99% on training and 60% on test isn't smart; it's a lookup table. Generalization is the opposite, where the model performs about as well on data it's never seen as on data it has. Holding out a test set is the only way to know which one you've got.
Underfitting is overfitting's mirror twin: the model is too simple, misses real patterns, and is wrong on both training and test. The bias-variance tradeoff is the search for the sweet spot: a model complex enough to capture the signal, simple enough not to memorize the noise. Most hyperparameter tuning is, in effect, walking that line.
After your model is trained, you make predictions via inference.
Metrics answer the question "did it work?" For classification, the headline is accuracy (fraction correct), but accuracy lies when classes are imbalanced. Precision is the fraction of positive predictions that were right; recall is the fraction of actual positives the model caught. F1 is their harmonic mean. For regression, MSE or RMSE is the default; MAE when outliers shouldn't dominate.
Basketball IQ
In most academic AI introductory texts, a problem would be built upon:
- Iris dataset 150 flowers, four features, three species
- MNIST 70,000 handwritten digit images, ten classes
- California / Boston housing geographic and structural features, sale price
But, I find these to be too academic. My closest relation to Iris is that it's my grandmother's name. So I would like to step through a problem relevant to today. Literally today. Like, the NBA finals.
I like the fact that this problem is simple and easily relatable. But it's actually not an ideal AI problem. While it seems like a lot of data, an entire NBA season and playoffs, it really boils down to 82 regular season games per team and roughly 80 playoff games league-wide. Not ideal. But it'll be fun to watch our prediction play out in real time.
The full system, code, and whitepaper live at github.com/illyastarikov/artificial. Briefly: the model is a four-head ensemble (Elo + logistic regression + two LightGBMs), the features are 55 per-game columns, the label is whether the home team wins, and the dataset is 21,742 games split chronologically. The detailed model description, the metrics on the test set, and the live 2026 predictions are in the NBA Finals section below.
You don't have to figure out what regression or what sort of problem this is yet, we're going to do that in the next section.
Learning Paradigms
AI doesn't learn one way. There are four broad paradigms, ordered below from most-deployed in industry to most-research-flavored. Real systems mix and match, but every approach falls into one of these buckets.
Supervised learning
You have inputs and the correct outputs. The model learns the mapping. The dominant paradigm in industry.
Classification Predict a discrete category.
- Algorithms: logistic regression, SVMs, random forests, gradient boosting, k-NN, naive Bayes, neural nets.
- Famous uses: Gmail spam filtering, breast cancer detection from mammograms, ImageNet (AlexNet, 2012), credit card fraud detection.
Regression Predict a continuous number.
- Algorithms: linear/ridge/lasso regression, regression trees, gradient boosting (XGBoost), neural nets.
- Famous uses: Zillow's Zestimate (home prices), Uber surge pricing, demand forecasting at Walmart.
Unsupervised learning
No labels, no ground truth. The model has to find structure on its own: groups of similar things, hidden axes of variation, outliers that don't fit any pattern. Less common than supervised learning in deployed products, but it's how you make sense of data you've never seen before.
Clustering - Algorithms: k-means, DBSCAN, hierarchical, Gaussian mixtures. - Famous uses: customer segmentation, news topic grouping (Google News), gene expression analysis.
Dimensionality reduction - Algorithms: PCA, t-SNE, UMAP, autoencoders. - Famous uses: visualizing high-dim data, image compression, eigenfaces (face recognition, 1991).
Anomaly detection - Algorithms: isolation forest, one-class SVM, autoencoders. - Famous uses: fraud detection, network intrusion, manufacturing defect detection.
Association rules - Algorithms: Apriori, FP-growth. - Famous uses: Amazon "frequently bought together", market-basket analysis.
Semi-supervised & self-supervised learning
Mostly unlabeled data with a sprinkle of labels (semi-), or labels invented from the data itself like "predict the next word" (self-).
- Famous uses: Google's early image search labeling, word2vec (2013), and the entire pre-training era of modern AI.
Reinforcement learning
An agent takes actions in an environment and learns from rewards. No labeled examples; feedback comes from outcomes.
- Algorithms: Q-learning, SARSA, policy gradients, actor-critic, DQN, PPO.
- Famous uses: AlphaGo beating Lee Sedol (2016), AlphaZero, OpenAI Five (Dota 2), DeepMind's Atari agents, robot locomotion, data-center cooling at Google.
Tip-Off
Enough theory. The NBA Finals problem is supervised at heart: every past game has a known winner, every feature has a known value, the model learns the mapping. The next section runs it.
NBA Finals
This bracket is the model's call from the start of the 2026 playoffs, before a single game tipped. The sections below walk through the model behind it, how accurate it is on held-out games, the bracket it produced, and where it has already gone wrong. How It Played Out grades the call against the real postseason.
The Model
Four predictors look at the same game from different angles: a chess-style team rating, a straight-line classifier, a tree ensemble that picks the winner, and another tree ensemble that picks the scoring margin. Their four predictions get blended together and run through a calibration step so the stated confidence matches the real hit rate. The full math, the hyperparameters, and the code are in the whitepaper.
Results on the Test Set
Across 60 independent training seeds on 3,261 held-out games:
| Metric | Mean | Std |
|---|---|---|
| Log-loss | 0.634 | 0.016 |
| Brier score | 0.215 | 0.004 |
| Accuracy | 65.9% | 0.8% |
| Expected calibration error | 0.023 | 0.006 |
65.9% accuracy beats the all-time NBA home-win rate (~60%) and lands within about a point of the ~67% that strong public game-prediction models have historically hit calling winners straight up. An ECE of 2.3% means when the model says "65% chance of winning," teams in that bucket win about 65% of the time. The calibration is honest.
The Start-of-Playoff Bracket
Before the first game, with all sixteen teams alive, a vectorized Monte Carlo simulator runs 100,000 best-of-seven bracket trials respecting the 2-2-1-1-1 home schedule, each game drawn from the model's per-game probabilities. A full bracket sim takes 30 seconds. The top four teams by title odds:
| Team | P(Conf. Finals) | P(Finals) | P(Champion) |
|---|---|---|---|
| SAS | 80.4% | 48.2% | 36.3% |
| OKC | 86.8% | 44.7% | 30.3% |
| DET | 69.7% | 41.9% | 14.6% |
| BOS | 54.3% | 29.9% | 10.4% |
By title odds, SAS leads at 36% β a 2-seed the model rates like a 1-seed, in a sim where 2-seeds collectively win it all more often than 1-seeds do (47% to 45%). OKC is a close second. The East trails badly: DET and BOS fill out the top four, while the two teams that would actually emerge from it β NYK and CLE β sit way down at 3.6% and 1.0%. Hold that thought.
Where the Model Missed
3 of the 12 series completed in the first two rounds:
- PHI over BOS, Round 1, 3-1 comeback. Boston led 3-1; Philadelphia won the next three to take the series 4-3, the 14th team in NBA history to erase a 3-1 deficit. The model gave BOS 91% to advance.
- MIN over DEN, Round 1. A 6-seed beating a 3-seed. The model gave DEN 85%.
- CLE over DET, Round 2, Game 7. Detroit had the higher seed and the pre-series Elo edge; the model gave DET 79%. Cleveland won 4-3.
All three were calls the books also missed. No model nails every series; the question is whether it's biased. ECE of 2.3% says no.
Scenarios
Each of the 100,000 Monte Carlo trials is itself a complete bracket realization. Aggregate them and you get the title odds above. Look at individual trials and you get a distribution of brackets, some of which are pure chalk, a few of which are total chaos.
The most-frequent realization across the 100k trials is SAS over DET β the same SAS that tops the title odds. The model's single best bracket and its overall favorite agree on the champion. (Pure chalk, every top seed advancing, would crown OKC; the model likes SAS enough to override the seed line.)
The deepest Cinderella the simulator produced is ORL β an 8-seed β over OKC. The model crowns an 8-seed about once every 500 runs (0.2%). It exists in the distribution, but it's not where you put your money.
The full whitepaper, code, and per-trial scenarios live at github.com/illyastarikov/artificial.
How It Played Out
The model's favorite was SAS β 36% to win it all, and the single bracket it would have printed. SAS is in the Finals, so the headline call is alive.
It nailed the West. SAS and OKC were its top two by a wide margin, and they are exactly the teams that met in the West Finals; the favorite, SAS, came through. Across the first two rounds it went 9 of 12 β a 75% hit rate, the three misses (Philadelphia's comeback over Boston, Minnesota over Denver, Cleveland over Detroit) all upsets the books missed too.
It missed the East. The model loved DET and BOS β 42% and 30% to reach the Finals β and pegged the teams that actually got there, NYK and CLE, at 14% and 7%. The upsets that did it, Boston and Detroit falling, are the same ones on its miss list. So of the four conference finalists, the model got the West pair and missed the East pair: two of four, not a sweep.
A 36% favorite is a favorite, not a promise, and a model can read the field well and still be wrong about which top seeds fall. But the one bracket it would have bet has SAS cutting down the nets β and SAS is one series away. Game 1 is June 3; the title is still open.
The Last Problem
AI is function fitting by optimization. The function got enormous, but the engine is the same one Arthur Samuel described in 1959. What's changed is scale, and what scale buys is generalization. A model that fits 21,742 NBA games well enough predicts the Finals to within a few percent of where the books are. A model that fits the internet well enough holds a conversation in any language about any topic.
Thankfully, the function that solves the last problem hasn't been fit yet.
Token Anxiety
Approaching daily Pro limit Β· resets in 6h
range anxiety /reΙͺndΚ Γ¦ΕΛzaΙͺ.Ι.ti/ noun The fear that an electric vehicle will run out of charge before reaching a destination or charging station.
There's a similar phenomenon in the AI community: token anxiety.
token anxiety /ΛtoΚ.kΙn Γ¦ΕΛzaΙͺ.Ι.ti/ noun The fear that an LLM will exhaust its context or its credits before arriving at a solution.
You know the moment. You're chipping away at a hard problem, on the best model available, on the highest effort setting. You're almost there, and then the banner drops: Approaching daily Pro limit Β· resets in 6h. Will you make it to the terminating token?
That's one flavor; here's another. You hear AI is the future, you don't want to fall behind, so you buy one of the big-lab plans. You polish a shelved project here, spin up a new project there. A month in, you've used 12% of your quota, and you can't decide whether to feel relieved or guilty.
Or here's a third. You work in tech, your employer encourages AI use, and your usage caps may as well be unlimited. You watch the cool projects your colleagues are shipping and want one of your own. You know AI fluency is a big part of performance now, so you start hunting for ways to spend all those tokens.
Three scenarios, two failure modes. An empty tank, and a full one; not having enough tokens, and having too many.
Empty Tank
When tokens are scarce, the symptoms are both predictable and compounding.
Rationing. You start asking the model less than you'd otherwise ask. The cap forces a triage that the work doesn't actually need. Each prompt becomes a small negotiation with yourself before it ever reaches the model.
Attachment avoidance. The first thing you cut is context. You stop pasting the file and describe it in prose. You hint at the error instead of showing it. Each attachment costs tokens, so you give the model less to ground on, and it generalizes from less. The output drifts toward generic, and you don't always notice why.
Model degradation. When the output disappoints, you assume the model was overkill and step down. Pro to Flash, Flash to Flash-Lite. Each step is a small concession: this question doesn't really need the smart model. Sometimes you're right. The trick is that you stop noticing when you're wrong.
Premature compression. The same anxiety that pushed the downgrade pushes you to trim the conversation before it's full. You /compress early to stretch the session, or auto-compression catches you mid-flow. The plan, the failed attempt, the thread you were following: gone. You restart on a thinner version of the problem and the model takes a wrong turn it had already corrected once.
Session splitting. Eventually you skip compression and start over. Work that should live in one continuous thread gets chopped into three. You spend tokens re-explaining where you were, paid in setup instead of progress.
Provider hopping. When Gemini finally caps out, you bounce to ChatGPT. ChatGPT is congested, you bounce to Mistral. Each switch loses the context you'd just rebuilt. You're chasing free tokens and paying in continuity.
Meter watching. By now /stats is a tic. You glance at the percentage between every prompt and weight each question by what it might cost. That's the wrong frame. The question is whether the answer is worth having, not whether you can afford to ask.
Settling. When every prompt feels taxed, you stop iterating. You accept the first draft because you can't afford another round. You stop asking for refactors. You stop asking for alternatives. The output is okay. Okay becomes the ceiling.
Burn out. Living inside these constraints is exhausting in a quiet, attritional way. You're working with a model that should make things easier, and instead you're rationing your access to it. Eventually you decide it isn't worth the friction, and you reach for the tools you trusted before.
Full Tank
You'd think the cure was more tokens. It isn't.
When tokens are plentiful, every incentive points toward burning them. You think you're falling behind. AI is showing up in your performance review. The dollars you spent on a plan demand to be amortized. So you use, and use, and use.
Trivial offloading. You start asking the model things you would have done in two seconds yourself. Renaming a variable. Looking up a flag. Reformatting a paragraph. The model's latency is higher than yours, but the tokens are "free," so you keep doing it. The muscle for the small things atrophies.
Drift. The bigger work catches the same habit. You let contexts grow long with stale code, dead chats, and abandoned plans. You retry from scratch instead of iterating. You let the model wander without guidance, because it has plenty of room to figure it out. The output gets worse, not better, and you respond by spending more tokens on it.
Project sprawl. With a great number of tokens comes a great number of side-projects. You start three demos in a week. Two are vibes, one has a real idea, and none of them ship. By Friday there's a fourth. The repos outpace the finished work. I'm still figuring out how to keep the portfolio from turning into a graveyard.
Burn out. This is the one I want to be honest about, because it's the reason I'm writing this.
When you're paying for the tooling, you want your money's worth. $20, $100, $200 a month is real money. So you use AI every chance you get. In the checkout line? Check out the long-running operation on your phone. Reading before bed? Perfect time to kick one off. Just woke up? Perfect time to check on it. The financial cost is fixed; the effort cost feels free.
It isn't. You're babysitting agents, holding the architecture in your head, deciding what they should work on next. You never stop thinking about the work. The model gets to forget. You don't.
We're not AI agents; we're human. We need rest, whether we choose it or not. The pause is where the work consolidates, where the bad ideas drop out and the good ones surface, where you remember what you were trying to build in the first place. None of that happens while you're checking on a build from the grocery store.
My current practice is to downgrade my plan once every few months for a month at a time. The lower cap forces me into intentional use, and the spare hours go into the things that aren't coding. It's the only thing that's worked; I'm curious if you've found others.
The Middle Lane
Range anxiety in EVs didn't get solved by 1,000-mile batteries. It got solved by chargers along the route, by trip planning, by drivers learning their cars. The fear faded as the fit got better.
Token anxiety won't be solved by infinite limits either. The cure isn't a bigger battery; it's knowing the route. Decide what the work is worth before you ask. Spend where the answer earns it. Hand some of the small tasks back to yourself, so the big ones get the version of you that actually shows up.
A low tank exhausts you quickly. A full tank exhausts you slowly. Only the middle is sustainable.
