i wanted to see if a model gets better at a task when it can write notes to itself between attempts, so i used chess as the test. GPT-6.1 Sol, Claude Sonnet 5.5 and DeepSeek V4.1 Flash play each other and Stockfish in tournaments that run back to back. after each game the model reads the game, writes what it learnt into its own notes file, and gets that file back at the start of the next game. it can read the other models’ notes too. 31 tournaments and 324 games later, this is the record against Stockfish held to a shallow search: GPT-6.1 Sol: 2 wins, 9 draws, 49 losses Claude Sonnet 5.5: 0 wins, 19 draws, 33 losses DeepSeek V4.1 Flash: 0 wins, 3 draws, 51 losses there is no upward trend. what i found when i went through the lost games: the first big mistake is nearly always a piece left undefended, at move 13 to 16 on the median. it is not a deep tactic. the notes almost never caused it and almost never prevented it. the blunder move cited a memorised line in 0 to 4 games per model. the notes fill up with opening lines. i had to cap how many move sequences a notes file may hold, because the models were turning them into an opening book. a picture of the board did not help. all three read the text position better than the image. so the notes work as memory and they do not work as board vision. the model knows the plan and does not see that its knight is hanging. the notes files are public if you want to read what each model thinks it learnt: https://github.com/marvijo-code/ai-chess-agent-memory the games run live with commentary: https://www.twitch.tv/marvijo submitted by /u/marvijo-software
Originally posted by u/marvijo-software on r/ArtificialInteligence