I’m evaluating how LLM agents adapt over episodes and exhibit human-like belief revision, not just per-turn accuracy, and the sticking point is a reproducible benchmark. Over the last week I wired a simple episodic memory layer (SQLite + retrieval) and ran about 1,200 interaction chains; if you have code or datasets that probe belief revision and credit assignment — MemoryGym variants, Avalanche for text, or any logging-first harness I can run with W&B — I’d appreciate pointers.
I’ve had luck scoring belief revision by injecting mid-episode contradictions and tracking a ‘time-to-retraction’ metric. Log belief diffs from your SQLite store per episode and require the agent to cite the evidence it used when it corrects itself, then check it points to the new fact rather than the stale one. Caveat: retrieval can mask failures, so add a no-retrieval control or delay writes 1–2 turns to see if updates are model-driven, not just ‘cache hits’ — like checking it can change its mind without sulking.