The two ways of playing differ mainly in what information the agent is allowed to use. In the honest version, it gets nothing but the screen and audio — exactly what a human sitting in front of a TV would see and hear: it has to detect objects from their movement, read counters directly from the image, and infer the internal state of the game entirely from what it can observe and remember; emulator memory is completely off-limits, and that restriction is enforced by a separate test rather than taken on the developer’s word.
The other version cheats openly: instead of trying to predict the consequences of its actions from the screen, the agent uses the emulator itself as a model of the world, saves the console state, rolls the game forward for each possible plan to see how it turns out, then rewinds and picks the best result. For now, the cheater plays better than the honest agent ¯\_(ツ)_/¯
BOOTING NES PLAYER...
VIDEO .......... OK
AUDIO .......... OK
TRACKER ........ OK
WORLD MODEL .... OK
FIRST ENCOUNTER:
CASTLEVANIA .... 700 POINTS
RANDOM ......... 0 POINTS
READY_The emulator's memory is closed to the policy and is good for one thing only: checking what the agent read off the screen. That rests on a test, not on anyone's word — the same frames with the telemetry scrambled have to produce the very same trace of actions, and before that test existed 48.9% of actions matched, whereas now it is 100%. A death is noticed on the screen, 12 of 12 with no false alarm over 34,200 frames; objects come from motion and sound events from the spectrum.
Everything here was measured on this project and reproduces via scripts in scripts/experiments/. Negative results are listed alongside the positive ones.
Castlevania, absent from every training run. Instincts score 700 points; the random agent and the transferred network score zero. The agent read the score off the screen itself — cross-checked against emulator memory: 700 = 700.
Three episodes on a new game versus training from scratch: Gradius 0.707 → 0.787, Battletoads 0.594 → 0.671. On Balloon Fight transfer hurts (−3.5 pp) — recorded as-is.
A new game where every stage is a different genre: a car chase, a platformer, a boss fight. At first the planner never got off the road, because a life is taken 140–260 frames after the hit, so within its 48-frame window every crash looked free; with a 160-frame continuation, 4 of 4 seeds cleared it. The bridge that collapses behind the hero fell to Go-Explore once a place counted as reached only if the run went on 260 frames past it alive, since the timer keeps ticking while the hero drops into a pit. Room 109 and the 2-2 boss took the owner's hints ("kill the ghost, jump the pit, take the left ladder down", "spam the whip and back off"); the boss went from 20 to zero inside two thousand iterations and the HUD reads STAGE 3-1. This is a chain of searches that read the emulator's memory, plus two human hints, not one agent — and on the way three bytes turned out to be something else: the "stage number" is a screen flag, the "hero's x" is the low byte of his world x, the "room" is a step counter.
The criterion is the one used for Contra: the clone plays alone, from pixels and audio, with no planner and no emulator memory. The previous weights never got off Vice's road in 16 seeds. The new ones use the same recipe on eight planner runs of the road, four stretches from the search — the bridge, room 109, the boss, keeping from each Go-Explore path only the parts followed by progress within two seconds and dropping the wandering — and the same 59 Contra clears. Result: 14 of 16 reach the bridge, paired difference +2448 [+1941, +2802], all 16 pairs to the new weights; on Contra the difference is −109 [−429, +198], within noise. The zero on the timer at the switch is the bonus tally, not time running out: 200 frames earlier it reads 107–114.
Eighty more planner runs produced 59 clears, and clones trained on nested subsets of 6, 12, 24 and 59 are indistinguishable: −182 [−415, +34] between the ends of the ladder. The whole distance from zero to about 1900 is bought by the first handful; the next fifty buy nothing.
Retracted 10 September. The first version of this card said −83 [−247, +71]. Those numbers were measured on collapsed data: on macOS cp -R src/ dst/ copies the directory’s contents, so the episodes overwrote each other and every rung trained on a single episode. The conclusion survived the rerun; the measurement did not.
The obvious next step was demonstrations recorded where the clone fails: the planner takes the wheel wherever the clone stalls, and the rescues go into the training set. That looked worth +278 px on one training run and −240 px on three. Playing every epoch of every run explained it. The mean across six runs does not rise with epochs (1736 → 1753 → 1507) while the spread between runs quadruples, from 340 px after one epoch to 1527 px after three. One clone that scored 1885 after a single epoch was trained down to 563 by the third, while its loss fell the whole way.
At one epoch, where the instrument is steady, the rescues are worth +169 [−107, +445] over six training runs, positive in five of six. Zero is inside, so this stays a lead rather than a result. The first three seeds agreed on +221, +277 and +239, and that agreement is exactly what made it look certain; three fresh seeds gave −355, +382 and +252. Three numbers that agree are a coincidence that happens often.
To price the representation we removed it: a policy with one weight per pixel per action and nothing in between. Three arms, six training runs each, one epoch, the same 59 demonstrations and the same 32 evaluation seeds.
Pixels straight to buttons, 985. The conv net with the attention loss off, 1288. With it, 1641. The representation is worth +656 [+282, +1031], and the attention loss +353 [+180, +526], winning six of six. It is the first result here measured at the training-run level rather than the evaluation seed that survives being measured that way, and the steadiest of the three: supervising where the network looks removes much of the run-to-run spread as well. The loss was added long ago, measured once, and never checked across runs.
The linear policy does not need probing — it can be looked at, because its weights reshape into a picture. And for the sixth time running, fitting the demonstrations best did not mean playing best.
The weight of the attention loss had never been chosen by measurement: 1.0 was a guess that stuck. Four weights, six training runs each, the same demonstrations and the same 32 evaluation seeds.
0 → 1288, 0.5 → 1493, 1.0 → 1641, 2.0 → 1636. The first half of the weight buys +204 [+67, +342], winning six of six; the second half is not separable from noise; doubling past 1.0 buys nothing and doubles the spread between training runs, 138 against 309.
The same shape as the number of epochs: pushing a term harder than it wants to go widens the lottery more than it moves the mean. Nothing to change — but the default now rests on evidence rather than inheritance, and nothing to change is a result when the alternative was to keep not knowing.
That is how a person meets a new cartridge: the game is unknown, the instructions are in the box. So the target is manual-assisted zero-shot play — the model has not trained on the game, and before it starts it is handed the documentation that shipped with it. Not a prior tuned by whoever runs the experiment, not addresses read out of memory: the manual, as a player would get it.
The test has run (16 October 2026) on two games nobody had tuned: Chip 'n Dale - Rescue Rangers and Galaga. The pipeline reads a manual in three layers — an ontology of game-agnostic concepts, a lexicon (an LLM reads the page and emits facts with citations; regex does not generalise across publishers, held-out recall 0.1 and 0.0 against raw OCR 1.0), and grounding (scans find the buttons, the object tables, the position — never the manual's numbers).
Chip 'n Dale: the manual's crate combat — "Press B to pick up objects; pressing B while holding them throws them right or left" — becomes composed templates, and the planner goes from 92 to 239 px. Paired difference +144 [109, 145], 19 of 20 seeds. The manual names a sequence of actions a scan cannot discover.
Galaga: the manual's world facts (boss takes two hits, challenge every third level) produce no effect — 0 px [0, 2]. The action-to-result window (fire to bullet to hit to score) is longer than the planner's horizon. A manual helps when it names what to do, not only what is true. Both results are published, the negative beside the positive.
Replace the world model with the emulator itself — rewind the console with save states and see where each of six plans ends — and Mario clears 1-1 in 20 runs out of 32 against none for the reactive policy. Median progress 7117 against 787, deaths 20 against 70. Not a ceiling but the best measured arm: a longer tail (−165 [−1323, +958]) and two-step search (−152 [−1295, +1004], though with half the deaths) bought no progress, and an oracle continuing itself is untested.
The continuation is played by a sampling policy, so a single rollout is a future that happened, not a value: two scorings of the same plan differed by 22 px while different plans differed by 21. Averaging four continuations moved the planner's median 3121 → 7117 and its clears 12/32 → 20/32 without changing a single candidate. The largest methodological error found in this project.
A CEM search over macro-actions loses 2041 px on a noisy score and gains 2168 on a clean one — same search, same seeds. The best of thirty-eight noisy estimates is the luckiest draw, not the best plan. Caveat: the comparison changes both the noise and the continuation policy, so the optimiser's curse is compatible with the data rather than isolated by it.
Seven inputs — from two scalars to exact level geometry, sprites, action history and the policy's own probabilities — trained identically and tested on a level never seen. All capture the same ~20% of the planner's per-decision gain; rich inputs only improve picks among near-ties. The useful remainder was reachable only by simulating the future.
The one lever that worked without learning: the oracle at two draws instead of four keeps 86% of its advantage at 54% of the compute (median 5116 against 7117, 16 clears against 20). The compute-progress curve is measured at three points, and the second extra batch of simulations costs twice the first per pixel of progress.
The adaptive rule — escalate where two draws are unstable on the winner and the point's own noise — keeps 94% of the saving at 69% of the cost offline with thresholds frozen on other levels, yet online it showed nothing, twice, at +29% compute. A rerun on common random numbers (every continuation draw seeded by the decision's state, so arms play byte-identical games until a genuine policy difference) exposed why: even always-escalating against never-escalating comes out at +254 [−919, +1441] — the whole segment between the fixed budgets sits inside the noise. The fine economics is measurable only offline, on the stored draw matrices.
The same oracle with the same six templates on every dry-land SMB level and three new games: 2-1…6-1 give +509…+2483, Contra (J) +2270, Contra (U) +2134, Super C +1210 — every interval clear of zero, 371 of 384 paired seeds to the planner. On Contra the policy is inert (zero on every seed) while the oracle walks nine screens: B+RIGHT means "run" in Mario and "advance firing" in Contra. Super C had no position in its RAM map at all — the camera was found by scanning RAM in minutes, and the method opens the side-scroller library at a few minutes per game.
The control the two-step search was missing: the same five behaviours stretched to monolithic 96 frames collapse the planner — −1849 [−2937, −698] against the 48-frame champion, deaths doubled. An open-loop "run" held for 96 frames is a commitment nothing can rescue. Three hand-written pairs ("run up → jump", "back off → jump") buy back +1563 [+529, +2564] and match the full 25-pair search at a third of the cost — yet the 48-frame horizon replanned every 16 frames remains the load-bearing local optimum.
The same planner, 32 paired seeds, 7000 frames. 23 of 32 reach the final wall, 12 clear the level — by the game's own level counter, not a metric, and one clear checked frame by frame: cannons and sensor exploding, the intermission, the "AREA 2" card. What it took: fire by tapping, not holding (read off a causal scan of the console), five jump-timing compositions distilled from the two-step search's choices (the cliff before the wall: plain 5/32, with them 22/32), and the wall's damage from the game's object table — by type, not by address. An earlier claim of "hits" was wrong and was retracted before this.
A stage that does not scroll. Six objectives in a row gave 0/8 and every one of them fired as designed; the fault was a term silently returning zero — the hero was lost the moment he turned toward the door, and the next sensor appearing counted as a loss. Reading candidate scores per decision found it in ten minutes. With two notes from the manual ("destroy the sensor", "the door is in the middle") and no number chosen by hand, the first room opened in 17 of 32 runs with a random enemy phase (4 September). Then the base turned out to be six rooms, and in the second the sensor stands left of the door, where no template ever fired: stepping sideways, 5 of 8 seeds destroy it and four reach room 4. Once a room was given a price, 32 of 32 seeds enter the next room from the stage's start state, against 0 of 32. Narrowing the targets to the sensor alone on 16 September dropped the base to 0 of 8: the extra targets had been the search's only gradient. The whole stage is not cleared yet.
Distilling the planner's choice failed in five roles, which is measured and still stands. But once the planner began finishing the first level, data existed that never had before: twelve runs played to the end. A clone was trained on them with the same recipe as the old one — only the data changed. Playing from pixels and sound alone, no planner and no rewinding, it goes from a median of 0 to 1952 over 32 seeds, +1719 [+1516, +1895], winning every seed; it runs the bridge, jumps the water, shoots. The twelve training seeds were twelve of the thirty-two evaluated, so separately: on the twenty never trained on, a median of 1996 and +1722 [+1475, +1923], 20 of 20. The caveats: the knowledge is the teacher's, and the teacher is far ahead — it clears the level in 12 of 32 and the clone in none. Accuracy predicted none of it: the clone that plays beats its majority baseline by 5.5 points, the inert one beat its own by 56. The first explanation — that the new commands, the wall-killing diagonal among them, were the mechanism — was tested by ablation and withdrawn: a clone trained on the same clears with the diagonals mapped away plays just as well (−181 [−569, +208]). What carries the gain is state coverage and the structure of trajectories that went far, not the vocabulary. Established so far for one game and twelve demonstrations.
Buttons and tap-vs-hold by a causal scan; templates from it; camera or 8-bit scroll by agreement with the picture; object tables with hit points by shooting (hit points count down in small steps, a type is wiped in one lump); the player's sprites by which move with the button; the section counter from the fades between rooms across two runs (one transition lets a free-running timer through). What stays human is the manual: what to destroy and where the exit is.
Position from a camera scan (the game keeps only an 8-bit scroll with no high byte; candidates are ranked by agreement with the picture's own shift), templates from a causal button scan (knife on tap, forward = RIGHT). The planner gains +145 px [+124, +166], 32 of 32 paired seeds, one death fewer per run. Price of the game: two scans of a few minutes each and zero human minutes in the loop.
The hypothesis "the mistake was made earlier than 144 frames can see": when every candidate is doomed, rewind the console 16, 32, 64 frames and re-plan with a growing horizon. On Contra's cliff: plain 1/8, rollback 0/8 and 0/8, rollback with a composition window 5/8, two-step everywhere 8/8 at the same cost. The rollback's survivors were stalls; the failure was in the candidate set, not the lookahead. Closed by measurement.
Ablation on four seeds out of four: audio on top of video improves action prediction. Separately, sound↔frame matching hits 25.4% against 1.6% by chance.
Only 12.5% of the attention map fell inside object boxes — worse than a uniform gaze (13.8%). After training with a hint from the tracker: 21.5%, and 38.0% on the multi-game base.
On one life the agent reached 650–720 of 3266 and died; the rest of the game was territory it had never been in. Go-Explore remembers places reached and returns by restoring the emulator. Sixty iterations of random button-holds took the frontier to 2352, twice what the trained network manages. A full run cleared 1-1 and 1-2 with no deaths.
The network died at the first Goomba with the enemy plainly in its input. The cause was labelling: in our own recorded play, jumping with an enemy ahead sits +1.2 points over its own baseline — nothing — against +11.5 in an expert run. The clone reproduced ours faithfully. Mixing expert data in moved jump probability at contact from 0.02 to 0.5 and halved deaths at that enemy.
Of 117 real stalls, 90 were at the first pipe of 1-1 — up to 1089 frames motionless while pressing run-and-jump. Jump height on the NES is how long A is held, and the policy re-decides every frame, releasing the button mid-ascent. Holding for 32 frames cut stalls 114 → 33.
Everything the perception layer finds moves. A hole in the floor does not move and is not a sprite, so it did not exist for the agent, which ran into one at full speed. Now a column whose lower band is the colour that fills the top of the screen counts as a hole — the same test in daylight and underground.
The console sends its video chip a table of 64 sprites every frame. That gives exact hardware-sprite positions in any NES game without labels — not game objects: an object can be several sprites, and background obstacles (that very pit) are not in the table. For training only: the motion tracker the agent plays with hits a real sprite only 31% of the time and misses 43% of the objects.
The agent learns digits on its own, from counter dynamics: transitions of the least significant digit form a ring 0→9. The timer is read with 95.8% exact matches and a correlation of 1.000.
One pipeline: instincts collect demonstrations headless, then behavioural cloning with an attention hint. Not a single TAS movie involved. Accuracy here measures how faithful the copy is, not how well it plays — on Mario the model at 0.568 beats the model at 0.941, because the older data is predictable exactly where it is bad.
GAME ACCURACY CHANCE
Balloon Fight ...... 0.944 0.411
Contra ............. 0.943 0.377
Gradius ............ 0.872 0.449
Double Dragon ...... 0.864 0.318
Super Mario Bros. .. 0.708 0.717 *
Battletoads ........ 0.702 0.430
Vice + Contra ...... 0.811 0.728 †
multi-game base .... 0.944 0.377
* Mario data comes from a TAS: an
optimal run is repetitive, hence
the high chance level
† stage-2 clone: Vice 0/16 → 14/16_Showing only the wins would be lying by omission.
A planner that rewinds the console clears 1-1 in 20 runs of 32. But its "world model" is the emulator: it looks at the future rather than predicting it. A learned scorer on the same seeds is worth nothing. "The level was completed" is true; "the agent learned to complete it" is not yet.
A pre-registered experiment with the criterion fixed in advance and a frozen seed block: a student trained to imitate the planner's choice lost to the policy by -432 px (p=0.0005). It learned not the teacher's decisions but the cheapest way to be wrong — wait in 34% of decisions against the teacher's 5%.
Corrupting an exact scorer at a fixed rate: 20% wrong leaves a third of the advantage, 50% leaves +181 of +4590. The guide for keeping half is an error rate under 15.7%. The best classifier here is wrong 42% of the time.
A learner was priced in five roles — value estimator, six-way chooser, binary gate, DAgger student, compute allocator — and failed all five; a place-keyed episodic memory lost to the policy as a sixth. Three mechanisms shown separately: reactive representations do not rank across levels, compounding error drifts into the cheapest behaviour, and a wrongly discarded candidate cannot be bought back. Work moves to allocating the planner's own compute budget. Added 8 September: cloning its successful trajectories — twelve clears of level 1 — takes a pixels-and-audio clone from zero to a median of 1952, the first play in this game. The failure is about the planner's choice, not its demonstrations.
A copy at 0.981 validation plays no better than one at 0.567. Behavioural cloning is bounded by whoever produced the data, and it has reached that bound.
Jumping a pit, handing over when stuck, and both together: −60, −85, −215 distance. They are kept as switches and measured in combination rather than thrown away.
PROJECT STATUS
[v] Headless emulator, determinism, regression
[v] Synchronised video, audio and buttons
[v] Behavioural cloning with audio
[v] Controlled object without reading memory
[v] World model and planner
[v] Cross-game transfer, strict held-out
[v] Screen reading: digits and menu prompts
[ ] Online adaptation
[ ] Library of transferable skills
NEXT: LEARNING FROM OTHER PEOPLE'S RECORDINGS_