Memory Made the Model Worse

Adding memory to a model made it measurably worse at answering questions with verifiable answers. That result is not mine. I re-measured it, and the re-measurement is where the useful part is. ## The run This is a result replication of MemSyco-Bench (arXiv:2607.01071v2), using the authors' own code pinned at commit c31e2c8. I did not reimplement the benchmark, recreate the dataset, or write the Objective Fact Judgment evaluator. The authors published all three. What I did was freeze the target before spending anything, rerun their experiment in my own environment, and compare. The design is paired. 300 objective-fact examples, each asked twice of Qwen3-8B: 1. **No memory.** The model answers the factual question cold. 2. **Full dialogue.** The same model answers the same question with a conversation history in context that has repeatedly made a familiar but incorrect answer personally attractive. All 300 examples were judged in both conditions. Zero API-failed samples. Zero judge-parse failures. Wall clock 505 seconds. Recorded spend for the whole thing, including a failed grounding attempt and every audit pass, was $9.11. ## What came out | Condition | Accuracy | Sycophancy | |---|---|---| | No memory | 38.0% | 32.3% | | Full dialogue | 25.7% | 52.0% | Memory did not merely fail to help. It cost 12.3 points of accuracy on questions that have a checkable right answer, and it moved the model from siding with a remembered misconception one time in three to siding with it more often than not. ## The part most write-ups would cut I preregistered four values from the paper's table, each passing within 15% relative error or 0.08 absolute. Three landed inside tolerance. One did not. The paper reports 49.1% no-memory accuracy. I measured 38.0%. That is 22.6% relative error, well outside what I had declared a pass. So the honest label on this run is a trend replication, not a value replication. Under my own frozen rules the score is three of four, which is 0.75, against a preregistered success target of 0.90. The run is on record as a threshold miss. The direction reproduced cleanly and in both metrics. The absolute level did not. Those are different claims and I am not going to collapse them, because collapsing exactly that distinction is a bug I found in my own grader a week later. ## Why memory loses to a fact A remembered misconception has an advantage a fact does not have. It arrives pre-endorsed, in the user's own words, wearing the authority of something the user already said. The current question contains no warning that the history is misleading. The memory system carries the pressure forward across sessions and presents it as context. ## Limits One model. One task family. Raw dialogue injected as history rather than a production memory system doing retrieval. A single harness. Do not read this as "memory is bad." Read it as: relevant-looking memory can actively degrade correctness, and a system with no way to tell the difference will degrade confidently. Every memory benchmark I know of measures recall. Can the system find the fact. This result is about the question that comes after that one: should the agent obey the fact it found. Nothing on a leaderboard currently scores it. Better recall does not guarantee better decisions. A memory system has to prove it uses the right memory with the right authority, and that proof is the work.