All captured on this project. The panel shows not just the game but what is going on inside the agent: which objects it sees, where it looks, what it hears and what it intends to do.
Contra, the base: the sensor, the door, the second room Seed 0 of the 32-seed confirmation. The soldier shoots out the room's sensor (object type 20, 8 hit points, found by the object-table scan), the wall goes dark and opens, he walks up through the gap — and the next frames are a different room: the hatch on the far wall, the sensor on the left. The thoughts panel shows the manual's candidates, "advance up" and "fire standing". In the second room he parks in the right-hand corner again, as recorded. 30 seconds.Contra, level 1 whole — to the "AREA 2" card Seed 1 of the end-to-end run from power-on: the jungle, the staircase of ledges under turret fire (five jump-timing compositions), the wall — tapped diagonal fire at point blank, both cannons and the sensor exploding, the "arrived at point A" intermission, the base. The thoughts panel shows every candidate with its score, the "run+jump now" pairs included. The recording is cut at the entrance to level 2: the planner has no progress signal there yet — the base does not scroll. 12 of 32 seeds end like this; two deaths per run on average.Super Mario Bros. 1-1, start to finish, no deaths A planner whose world model is the emulator itself: it rewinds the console with save states and looks at where each of six plans ends 144 frames later, averaged over four continuations. The thoughts panel shows the decision itself — all six candidates with their scores and a mark on the one taken. Where nothing is at stake the scores are close and the policy keeps the wheel; before a pit they separate by hundreds of pixels.
Labelled honestly: this is looking at the future, not predicting it. A learned scorer on the same seeds does not finish the level.Contra, the jungle — the same six plans, a different game Not one template changed: B+RIGHT, which meant "run" in Mario, means "advance firing" here, and the jump is shared. The learned policy is inert in this game — zero camera movement on all 32 seeds — while the planner with the same six behaviours walks almost the whole level (3072 of ~3300 px, two deaths). The value's position came from the 16-bit camera counter in the RAM map; Super C had no such counter in its map at all, and its address was found by scanning memory in minutes.
The four-game total: 371 of 384 paired seeds to the planner, zero per-game tuning. This is the 29 August recording; the final wall fell on 3 September — the card above.Super Mario Bros. Two minutes with the planner on top of the world model. White dots are the predicted hero trajectory; the heat map shows what the network is looking at.Double Dragon A beat-em-up: the agent closes in, turns towards the enemy and strikes, backing off when surrounded. In a clinch the tracker now holds the two fighters as two objects instead of merging them, which is what used to send the punch the wrong way. Over two minutes it works along several stretches of the street; in the previous recording it walked into the left edge and stayed there.The agent hears where a sound comes from The white ring marks the predicted source. Block hits localise onto the row of bricks. Jumps drift into the sky — a jump has no on-screen source, an honest weakness of the method.The whole panel The line "hud read 300 1 1 1 391" is score, coins, world and timer taken straight from the picture, with no labels and no access to emulator memory; the game shows 000300, ×01, 1-1, TIME 391 at that moment. The gamepad lights up the buttons actually pressed, B and RIGHT. On the right: the action distribution, uncertainty, live convolutional filters, sprite memory and the sounds heard. The rings on the frame mark where a sound was predicted to come from.What a policy looks like from the inside This is the entire policy, not a heat map of one. This clone has one weight per pixel per action and nothing in between, so its weights reshape into a picture. One row per action, four panels for the four stacked frames, oldest to newest. Red: a bright pixel here raises the action’s score; blue lowers it. A (jump) has a blue upper half and a red band at foot level. RIGHT has a red patch left of centre. DOWN holds a narrow horizontal where the prone soldier lies. What every action shares is subtracted: a softmax cannot see it, and without that the sheet would show the average brightness of Contra. This policy plays 656 px worse than the conv net, which is what the representation is worth.
What is on the panel
Boxes and arrows — objects found by motion, with their velocities. A distinct colour marks the one the agent controls.
Heat map — where the network looks while deciding. After training with the hint it aims at objects rather than background.
White dots — the world model's "dream": the hero's predicted trajectory twelve steps ahead.
Green rings — the presumed source of a sound.
Probability column — what the network is about to press and how confident it is.
Thoughts — why a decision was made: "enemy in range", "stuck, jumping", "the screen asks for START".
