How Eva remembers months of group chat

Eva sits in a Discord server and talks with a friend group for months. At some point the chat is way too long to fit in the model's context window, so something has to be compressed. This is how Eva does that, and I think it's one of the coolest parts of her.

The usual way

Most chat agents use what's called a rolling summary. When the chat gets too long, a model writes a summary of the old part, and from then on the agent sees that summary plus the newest messages. When that gets too long again, it summarizes the summary plus the new stuff. And so on.

It works, but every round squeezes the old stuff again. Something said in March might have been summarized ten times by October, and each time a few details fall off. Who said what is usually the first thing to go. And for a friend group, who said what is the most important part.

One line per message

Eva's memory starts small. Every message gets compressed into one short line, at most 512 bytes, by a cheaper model. Say these four messages come in:

#0  Pia: anyone up for padel saturday? 10am at Gothia
#1  Max: can't, my sister's wedding in Borås
#2  Eva: I'll remind you on Friday Pia
#3  Lo:  I'm in!! bringing my new racket

They become four lines:

0+1 2026-10-14|Pia asks who wants padel Sat 10:00 at Gothia
1+1 2026-10-14|Max can't come: sister's wedding in Borås
2+1 2026-10-14|Eva promised to remind Pia on Friday
3+1 2026-10-14|Lo joins padel, bringing a new racket

The 0+1 at the start is the line's address. It means it starts at message 0 and covers 1 message. The date is there so Eva knows when it happened. The model that writes the lines is told to keep people's own words where it can and to always credit the right person by name.

Two lines become one

Then neighbouring lines get merged in pairs. 0+1 and 1+1 become 0+2, one line that covers messages 0 and 1:

0+2 2026-10-14|Pia asks for padel Sat 10:00 at Gothia, Max out (sister's wedding, Borås)
2+2 2026-10-14|Eva to remind Pia Fri, Lo joins with new racket

And then those two become one:

0+4 2026-10-14|Padel Sat 10:00 Gothia: Pia organises, Lo in, Max out (wedding), Eva reminds Pia Fri

The important part is that a merge only reads the two lines right below it. It never goes back to all the original messages. So it's cheap, and every line is written once and then left alone. If two lines are short enough to fit in 512 bytes together, they just get glued together with no model call at all.

So what you end up with is a tree. Every message is a leaf, and every merged line covers twice as many messages as the two lines it came from:

messages 0 to 7, each row merges the two above it

 0     1     2     3     4     5     6     7
  \   /       \   /       \   /       \   /
   0+2         2+2         4+2         6+2
      \       /               \       /
         0+4                     4+4
             \                 /
                     0+8

What Eva actually reads

Eva doesn't read the whole tree. She reads one row of lines that covers everything from the first message until now, oldest first. The smart part is which lines get picked.

When that row passes 96 KB of text, the pairs that are both old and small get merged first, until it's under 48 KB again. The priority is how long ago a stretch ended divided by how many messages it covers. So a line from last winter that covers a single message is very due for a merge. A line from yesterday isn't.

What you get looks a lot like how you remember things yourself:

<memory>
0+1024 2026-01-03..2026-03-20|the whole winter in one line
1024+256 2026-03-20..2026-04-02|two weeks in one line
...
1800+1 2026-10-14|one message from yesterday
</memory>

And under that, the newest messages word for word, about the last 12 KB of chat.

She can zoom back in

Nothing is thrown away. Every line at every level stays in a database table, and so do the original messages. Eva has two tools for getting back to them.

zoom opens a line into its two halves, and she can keep going all the way down to the actual message. recall searches the original messages by their words and shows the messages around each hit.

So if someone asks "what did Max say about the wedding?", she can open 0+4, then 0+2, then 1+1, and read what Max actually wrote. The row she reads is compressed, but the memory under it isn't. The one exception is when someone presses Forget me. Then their messages and their lines really are erased, as the privacy page says.

When it runs

The compression doesn't happen on every message. It starts when the part that isn't compressed yet passes 48 KB of text. Eva writes the new lines right away, but she only swaps them into her prompt once the chat has been quiet for 30 minutes.

The reason is caching. The AI provider caches the start of the prompt, which makes replies faster and cheaper, and changing the memory block throws that cache away. As far as I know that cache is gone after about 30 quiet minutes anyway, so that's a free moment to swap. If the chat never goes quiet, she swaps anyway once the uncompressed part passes 96 KB.

What it costs, and what can go wrong

I tested it on a year of my own friend group's chat, about 52,000 messages. Building the whole tree took 4,801 calls to the small model and cost about $3 at list price.

It's not perfect though. The lines are written by a small model, and a small model can get a detail wrong. That's why Eva is told to zoom until she has the whole thing before she guesses. I'm not sure yet how often she skips that and guesses anyway.

Where the idea comes from

The tree isn't my idea. It comes from Victor Taelin's OptMem and his UniiChat spec, including the merge priority. What I did was fit it into Eva: the merging runs in the background, the swap waits for a quiet chat, and every line keeps track of who said what.

End of feature

She remembers your group chat.

Price$15

a month per server

Replies1,500

a month, model included

Add to DiscordSee Eva

Numbers are Eva's defaults on 19 October 2026 and may change. Spotted something wrong? hello@evachat.app