log

Research log.

Everything I've tried, what held up, and what didn't. Results get published whether or not they flatter the project.

open

Open question

Does structure beat search on the questions retrieval buries?

Two claims to test. That decomposing a question into sub-questions before retrieving answers multi-hop questions a single search misses. And that walking a timeline answers “when did this change?” from fewer retrieved items than ranking memories. Both should hold. The honest test is someone else's benchmark, someone else's harness, existing systems as baselines, same models throughout.

Benchmark	Scope	Status
LoCoMo	Long conversation recall, multi-hop and temporal	running
LongMemEval	Multi-session, temporal reasoning subset	queued
Baselines	Existing memory systems, re-run on the same stack	queued

I have results from my own scenarios already. I'm not printing them. Small samples, my scenarios, my scoring — it's very easy to build an evaluation that agrees with you, and a number nobody can reproduce is worth less than no number at all.

notes

Working notes

Written up as it happened.

JUL 2026

If we trust agents to reason, why not let them ask?

Everyone optimises how memory is written. This one is about the read side: let the model decompose the question into the smaller ones it would need answered first, then retrieve for those. Tested on LoCoMo through an existing harness — strong on multi-hop, and it exposed exactly where the hole was.

Read the essay →

JUL 2026

I told you where it broke. So I went and fixed it.

The hole was time. Fact stores overwrite, so “when did this change?” has nothing to retrieve. This is the timeline layer: one page per relationship, a dated line per change, nothing erased — and the two-dates problem nobody warns you about.

Read the essay →

method

How results get run

Rules I'm holding myself to.

Someone else's harnessBenchmarks run on the existing public evaluation code, unmodified scoring and question sets.
Same stack everywhereEvery system compared — including the baselines — runs on identical models and embeddings, re-run by me rather than quoted from a paper.
Unfavourable rows includedEvery question category gets published, not the ones that look good.
Reproducible or it doesn't countThe harness ships in the repo. If you can't re-run it, I shouldn't be claiming it.

fund

Fund the benchmarks

Reproducible results cost compute.

Running the full plan is not free — LoCoMo and LongMemEval in full, every baseline re-run on identical models and embeddings, is real API and GPU time. If you'd like the numbers to exist sooner, sponsoring a run is the most direct way to help. Email me and we'll sort it out; every sponsored run ships with its harness and raw output in the repo — favourable or not.

Email me to sponsor a run →

What's next

Where this goes.

Benchmark results first. After that: better handling of questions that slice across timelines rather than down one, and a scale story beyond the development store. If the numbers say the approach doesn't hold up, that gets written up too.

If you want to poke at it, break it, or prove me wrong — the repo is open.