Problems and negative results

Observation: Contra's second level is a pseudo-3D base, the camera does not scroll, the planner's value (level × 4000 + camera) stands still. The first recording of level 2 also exposed a harness bug: the auto-continue fired on "camera = 0" and paused a live game; fixed by conditioning on lives.

Next: the same class of problem as scene-segmented games (Ninja Gaiden, Kung Fu): a room/door counter or a signal from the object table. A level-start state is saved for the scan.

What it was: wall damage was 66 minus the sum of four addresses; 66 was the sum in one savestate, and three of the four addresses turned out to be object type bytes, not HP. At a real arrival the sum was 38 before a shot was fired, and the campaign reported "28 hits" on every seed.

What it is: the object tables were found (type per slot, HP per slot); the wall is four objects with 72 HP. The damage term is rewritten by type, and the only kill criterion is the level counter. Then the wall fell: 2/8 from the arrival state in the lab, then 12/32 from power-on. The withdrawn claim stays in the journal next to its tables.

Measurement: three rollback variants (survivor committed whole; compositions at the rescue point; a danger window) pass Contra's cliff 0/8, 0/8 and 5/8 against 8/8 for the two-step search at the same cost. The survivors are stalls, the doom returns, the depth escalates to the cap.

Conclusion: the remaining failures on the road are candidate-set failures; the cheap cure is five jump-timing pairs distilled from the search's choices (cliff 7/8 at half the price). The rollback code stays behind a flag; the branch is closed.

The measurement: adaptive draw allocation — two rollouts for everyone, two more where the early ranking is unreliable. Offline the rule is impeccable: a trigger built on winner instability and the point's own noise, frozen on five other levels, keeps 94% of the 2→4 saving at 69% of the cost against an independent reference. Online — nothing, twice: −503 on its own noise, −202 [−1438, +1068] on common random numbers, at +29% compute.

The cause: the criterion asked for a point above the line between oracle-2 and oracle-4, and the endpoints themselves are indistinguishable: o4−o2 = +254 [−919, +1441] on 32 paired seeds under fully shared noise. The medians differ twofold (3124 against 7106 — the distribution is bimodal around clearing the level); the means do not. The line was strung between two statistically indistinguishable points.

The verdict: pre-registering a criterion without first drawing the noise floor is a methodological mistake, recorded as one. Online at this scale certifies only effects the size of the planner itself (+3800 over the policy); the fine economics of draws is measurable offline, on the stored matrices. What remains are the instruments: the CRN harness (arms play byte-identical games until a genuine policy difference) and the offline calibration stand.

Measured: a learner was priced in every role defined for it: value estimator (~0 over a constant online), six-way chooser (-432 px on frozen seeds, p=0.0005), binary gate (AUC 0.57 against 0.583 for a single airborne bit), DAgger student (below the majority baseline), compute allocator (worse than uniform thinning at matched compute). A place-keyed episodic memory lost to the policy by -209 as a sixth.

Cause — three mechanisms shown separately: reactive representations do not rank across levels (seven battery inputs capture the same ~20%); compounding error drives each model into its cheapest behaviour — waiting, deferring, cautious templates; and hard pruning is asymmetric — a wrongly discarded candidate cannot be bought back with more simulations.

Outcome: the programme closes with a working engineering result: uniformly halving the simulations keeps 86% of the gain at 54% of the compute, and the continuation is sequential allocation of the planner's own budget, not another student.

Measured: the continuation after a plan is played by a sampling policy, so a single rollout is one future that happened. Two scorings of the same plan differed by 22 px while different plans differed by 21, and the best of six candidates agreed with itself in half the decisions.

Result: averaging four continuations moved the planner's median 3121 → 7117, its 1-1 clears 12/32 → 20/32 and its deaths 42 → 20 — with no change to the candidate set. The largest methodological error found here: the instrument was defined wrongly, not the model.

A dead end on the way: a deterministic continuation makes the target perfectly reproducible and drops the result to +2458. A greedy policy stalls at a pipe, so the value starts describing a world that never happens. Variance had been traded for bias.

Symptom: every classifier trained to choose between "let the policy play" and "jump" collapsed into never intervening.

Cause: in 40% of decisions jumping and deferring produce byte-identical futures — 88% of them airborne states where the button does nothing. The label is the probability that jumping is better, which on an exact tie is zero. In the set the classifier trained on, such points are 45%.

Result: nearly half of training was a systematic push towards refusal. A bug in the target rather than a fact about the game; a tie should be labelled as a tie.

Symptom: a training-free nearest-neighbour readout beat the trained network, 0.707 against 0.562.

Cause: every run walks the same level. 74.6% of held-out points have a training point at exactly the same world x, and 99.3% within four pixels. Memorising the level's geography survives that split, and copying a neighbour's label works.

Consequence: offline comparisons of representations made on that split are optimistic, including those used to reject designs. The online results are unaffected. What is needed: blocks by world coordinate with a purged buffer around the test region, and held-out levels for real transfer.

Measured: a pre-registered experiment — architecture, target, loss and criterion fixed before the data finished collecting, one run on 32 seeds no arm had touched. The student lost to the policy by -432 px (p=0.0005) and beat only a constant.

What it learned instead: wait in 34% of decisions against the teacher's 5%. A spurious wait costs 2 px inside the horizon the label is measured over — the cheapest error available — and 432 px across a run. 83% of those waits land on the slowest states, which are exactly the ones that need an action to leave.

The cost of error, measured separately: an exact scorer corrupted uniformly keeps a third of its advantage at 20% wrong and +181 of +4590 at 50%. The guide for keeping half is under 15.7%, against 42% for the best classifier here.

Symptom: "always take the same template" scored +3178 against the policy and matched a full planner to within three pixels. Three conclusions were drawn from it and reported.

Cause: the score was short-circuited to a constant, but the oracle's scoring loop kept running and appended its real numbers to the same list. The maximum then took the oracle's answer wherever any plan gained more than a pixel. The "constant" chose its own template in 42% of decisions and played like the planner in the rest.

Result: the three conclusions are withdrawn and a real constant is worth -413 against the policy. The commits with the wrong headlines stay in history beside the retraction: a wrong step is also a result, and it explains why the control exists.

Measured: two Double Dragon models, same recipe, different data — 0.864 and 0.618 accuracy, a gap of 25 points. Made to play, ten paired runs: 149.4 against 146.2 points. No difference at all.

Cause: the metric answers how predictable the cloned policy was, not how well it plays. The old instincts stood in a corner repeating one manoeuvre, which clones beautifully. The fixed ones react to the situation, and reactive behaviour is harder to guess from four frames.

Consequence: the accuracy column in the model table ranks cloneability, not skill. Against its baseline it still says something useful — whether anything was learned at all — but a higher number does not mean a better player.

Symptom: the model trained, reached 0.42–0.48 accuracy and "just played badly".

What actually happened: all 41 episodes were a static title screen. The game never started: its title cannot be passed from a cold boot by any input, and the integration expects a save state.

Why it slipped through: accuracy cannot tell "weak play" from "a model that memorised a still image". We only found it by looking at the frames.

Outcome: a start-state flag was added, data re-collected, accuracy 0.864. The earlier transfer result for this game was retracted from the log.

Symptom: a third of the frames in a two-minute recording were frozen.

Cause: the title-screen detector memorised the picture at episode start. Starting from a save state, the "title" it memorised was the first level's street — so every similar frame looked like a return to the menu, and the agent pressed START, which is pause. Blind START pulses in the first seconds hit the running game too.

Outcome: frozen frames 33.9% → 7.3%, longest freeze from 1.1 s to 0.1 s. The score per run doubled.

Measurement: only 12.5% of the attention mass landed inside object boxes, against 13.8% for a uniform gaze. The network was systematically staring at scenery.

Cause: behavioural cloning rewards any feature correlated with the action, and the background works as well as an enemy.

Outcome: a penalty for looking outside tracker boxes. Now 21.5%, and 38.0% on the multi-game base against 10.4% by chance. The price is 2–3 pp of accuracy.

Symptom: in a beat-em-up the agent punched away from the enemy.

Cause: when sprites overlap the motion-difference tracker sees a single blob. It went to whichever track was nearer, the other fighter ghosted at its last position, and the winner's centre slid into the gap between them — so the sign of the direction became noise.

Fix: two tracks may claim the same blob, and it is then cut between their predicted positions. Frames in which the agent is striking a tracked enemy rose from 254 to 462.

But not everything: the score did not move — +10 on a standard deviation of 24 across 12 runs, five wins and five losses. Recorded as it stands.

Measurement: on an unseen game the multi-game base without fine-tuning covers 9.7 units of distance against 126 for a random agent. It confidently presses what was right in its own games, and that actively gets in the way.

Contrast: few-shot with three episodes does work (+7–8 pp). So features transfer, behaviour does not.

Slot attention learns reconstruction (error 0.027 → 0.012) but splits the frame into a positional speckle instead of objects.

Cause: the NES background is itself a repeating tile pattern, and a speckled decomposition explains it just as well as an object-based one. The method has no other gradient to follow.

Resolution: the classic motion tracker stays. Plans for a second attempt are in the experiment log.

Source scale: 933 NES recordings, 763 games, 240 hours of frame-exact input — and the button presses are already in there, nothing to recognise.

Obstacle: the recordings were made in other emulators, and some drift off mid-run. Comparing three cores showed no single winner: fceumm is better on eight recordings, quicknes twice as good on two.

Plan: pick the core per recording and keep the verified prefix up to the breaking point.

Symptom: after adding a second emulator next to the main one, every game started returning a different resolution and sample rate. Zero errors in the logs.

What caught it: regression tests with golden frame and audio hashes, added the same day.

Outcome: core switching now happens in process memory only, shared directories are left alone. Silent model degradation would have been the worst possible outcome.

At first it looked like the core failed to play the death sound. In fact the lives counter in memory decrements about 120 frames after the sound, so the event was being looked for in the wrong place.

Outcome: sounds are attributed to events through a backward window, and the clustering threshold was tightened — otherwise the jingle merged with the music.