Getting started

How to run it

Everything runs locally: macOS on Apple Silicon and Linux x86_64 out of the box, Windows through WSL2. Tested on macOS 26, Python 3.14, PyTorch 2.13 with Metal.

We distribute no ROM images — there are none in this project. Your own dumps are imported into the emulator with one command.

Trained models live in a release of the repository and download with one command: uv run python scripts/fetch_models.py. They are optional: the instincts play without any model, and training one for a new game takes about half an hour.

Install

Requires uv, Astral's Python package and project manager; install it first (curl -LsSf https://astral.sh/uv/install.sh | sh on macOS and Linux).

git clone https://github.com/\
  Recluse/NES-Player.git
cd NES-Player
uv sync

Your ROMs

uv run python -m retro.import \
  /path/to/roms

Watch it play

./start.sh

A graphical menu: game, mode, checkpoint, video recording.

Full cycle on a new game

No ready-made playthroughs needed: instincts play by themselves, their runs become the training set, and the network learns from it. About half an hour on a laptop.

# 1. Instincts play and record episodes (headless, ~1000 fps)
uv run nes-player explore --game Gradius-Nes-v0 \
    --record datasets/explore_gradius --loop --max-frames 3600

# 2. Behavioural cloning: video + audio + attention hint
uv run nes-player train-bc --episode datasets/explore_gradius \
    --out runs/bc_gradius --audio --attn 1.0 --epochs 3

# 3. Play with the trained model, window and sound on
uv run nes-player play --game Gradius-Nes-v0 \
    --checkpoint runs/bc_gradius --window --realtime --hd --auto-start

Useful flags

--core nestopia      another emulation core (fceumm by default)
--state default      start from the integration's save state: needed for
                     games whose title screen cannot be passed cold
--planner            planning on top of the world model
--ghost runs/ego_x   predicted trajectory on the panel
--sound-loc runs/av  show where the sound comes from
--video-out out.mp4  record the run with audio
--loop               play indefinitely (for streaming)

Reproduce the experiments

Every number on this site comes from one of these scripts.

# first encounter: instincts vs random vs the transferred base
uv run python scripts/experiments/zero_shot.py

# transfer to games absent from training
uv run python scripts/experiments/heldout_transfer.py

# where the model looks: attention mass inside object boxes
uv run python scripts/experiments/cam_focus.py <episode> <checkpoint>

# comparing emulation cores on third-party recordings
uv run python scripts/experiments/core_compare.py

Integrity check

The project ships regression tests with golden frame and audio hashes of a fixed run. They are what caught a silent emulation-core swap that would have degraded quality without a single error in the logs.

uv run pytest -q     # 48 tests