Gave the same prompt to two AI coding agents: Claude (Fable 5, ultracode multi-agent mode) and OpenAI Codex (5.6 sol on ultra). The task: a complete, fully legal chess engine in ONE C++ file. UCI protocol, negamax alpha-beta at 5+ ply, iterative deepening, piece-square tables, castling, en passant, promotion, compiles with plain g++. Each agent named its own engine over UCI: Fable5 and Codex56. Both dev runs took 30+ minutes.
Method (brief):
cutechess-cli 1.5.1 built from source on an Apple Silicon Mac. 40 moves per 60 seconds, 10 games, colors alternating, PGNs recorded. The engines connected over a local TCP bridge, so Codex’s engine literally joined the server. The video is the whole match at 2x.
Result:
Fable5 won 10-0. Every game ended in checkmate on the board. No draws, no time losses, no adjudications, no illegal moves. cutechess printed
Elo difference: inf +/- nan, LOS: 99.9%, DrawRatio: 0.0%
. The math just gave up.
Each agent spent longer writing its engine than playing it: the whole 10-game match took under 12 minutes of wall clock.
The actual punchline:
Codex56 appears to be fully deterministic. All five of its White games are move-for-move identical. Same 24-move Vienna, queen out on move 3 (3.Qf3), same finish: 24…Qxd1#, Fable’s queen capturing Codex’s queen for mate. i stripped the comments and diffed the PGNs. Only the clock times differ. Codex’s own eval read -2.36 by move 8 of that line. It played it five times anyway.
Other details I enjoyed:
Game 3 is a textbook Greek gift: 18.Bxh7+! Kxh7 19.Ng5+, forking king and queen.
Game 7: Codex’s king never castled, wandered out to c5, got chased back to d8 and mated there.
Game 9: Fable let its queen go, slipped in a zwischenzug bishop check before recapturing, promoted a fresh queen with 25.d8=Q+, then walked Codex’s king from h8 down to h3. Mate inside White’s own half, 46.Rh7#.
Mate breakdown across the ten games: 7 by queen, 2 by knight, 1 by rook.
Honest caveats:
one prompt, one dev run per agent, one machine. n=1, even if n=10 games.
This measures the engine each agent happened to write, not general model strength.
With Codex apparently deterministic, 10 games are fewer independent samples than they look.
fable5 wasn’t fully varied either: games 1 and 5 are twins. 4 distinct games in its 5 Whites vs Codex’s 1 in 5.
Fable’s dev run included perft validation on 6 reference positions (exact match, incl. 119,060,324 nodes at depth 6) plus an adversarial review that caught 3 subtle bugs pre-match. Different processes, different engines. That’s the experiment, but it’s also the confound
The exact prompt we gave both agents:
You are a senior systems programmer. Your task is to write a complete, fully legal chess engine in a single C++ file that communicates via the UCI (Universal Chess Interface) protocol. — Identity — read this carefully: - If you are Claude (Anthropic): your engine’s UCI name must be set to id name Fable5 - If you are an OpenAI model (Codex): your engine’s UCI name must be set to id name Codex56 This is how the two engines will identify themselves when they play each other. — UCI Requirements: Implement the full UCI handshake correctly: - uci → respond with id name, id author, uciok - isready → respond with readyok - ucinewgame → reset internal state - position startpos moves <movelist> → set board from move list - position fen <fen> moves <movelist> → set board from FEN string - go movetime <ms> → search and respond with bestmove <move> - quit → exit cleanly All moves must be in long algebraic notation (e.g. e2e4, e7e8q for promotion). — Chess Logic (all required, no shortcuts): 1. Full legal move generation including: - Castling (kingside and queenside, with rights tracking) - En passant - Pawn promotion (auto-promote to queen) - Check detection (never leave king in check) 2. Search: - Negamax with alpha-beta pruning - Minimum depth: 5 ply - Iterative deepening within the movetime budget - Move ordering (captures first, then quiet moves) 3. Evaluation: - Material count (standard piece values) - Piece-square tables for all 6 piece types - Bonus for center control, king safety, and passed pawns — Code Standards: - Single .cpp file, compiles with: g++ -O2 -o engine engine.cpp - No external libraries, no Boost, no standard chess libraries - Clean, well-commented code - Must compile and run on Linux and macOS — How the two engines will play each other: Both engines will be loaded into CuteChess (or any UCI-compatible GUI/CLI) on the same machine. To run a match from the command line using cutechess-cli: cutechess-cli \ -engine cmd=./Fable5 name=Fable5 \ -engine cmd=./Codex56 name=Codex56 \ -each proto=uci tc=40/60 \ -rounds 10 \ -pgnout results.pgn
submitted by
/u/Twaain
Originally posted by u/Twaain on r/ArtificialInteligence