WRSWorld Research StateContribute
My reading

Computing & AI · A guided explanation

LongMemEval: testing whether assistants can use a long interaction history

An assistant might remember one fact yet miss a later correction, confuse dates or answer a question whose premise is false. LongMemEval creates a structured evaluation of these abilities. Its questions require extracting information, reasoning across sessions and time, handling updates and abstaining when evidence is absent. Wu and colleagues combine curated questions with long interaction histories and study memory indexing, retrieval and reading choices. The contribution is the evaluation resource and its scoped design analysis, rather than an achievement by any model tested on it.

Read original sources · Check the exact scope · Choose a next step

Can an assistant remember a change?

A memory test should ask more than whether a model can repeat a sentence. It might need to connect two sessions, interpret a date, use a corrected fact or decline to answer because evidence is missing. LongMemEval builds questions around these different requirements and supplies long interaction histories. The achievement is an evaluation resource that helps researchers inspect specific failures, rather than a new capability achieved by the models being tested.

History length and question type both matter

A question about one session differs from a question that combines facts spread across months. A false-premise question checks whether an assistant invents an answer when it should abstain. The benchmark includes simulated and edited histories with curated questions and answer rubrics. A smaller commercial-assistant pilot uses much shorter histories and historical configurations, so its numbers cannot be substituted for the full long-context evaluation.

A three-note test you can check

Use an invented history: a project initially orders twelve units, later cancels three, then receives two replacement units. If the notes describe units retained, the final count is twelve minus three plus two, or eleven. Ask a question about the final count and one about a delivery date never recorded. A careful assistant should answer eleven under the stated interpretation and mark the date unknown. These are classroom checks, not original benchmark items.

More text is not always more evidence

Retrieving everything can add irrelevant material. Compressing too aggressively can remove a condition or correction. The paper explores this balance through indexing, retrieval and reading choices. Ask your agent to show which note supports each part of an answer, then remove one necessary note and see whether it acknowledges the gap. Passing this miniature does not establish long-term reliability, but it produces an inspectable failure or success that another person can check.

Report the missing evidence too

The output of the three-note exercise should list the count, its interpretation and the unsupported date question. If the word retained is ambiguous, record that ambiguity before calculating. A useful memory evaluation includes both supported answers and correctly identified gaps, rather than rewarding an answer merely because it sounds complete.

Scientific scope

The histories include simulated and edited material, and the evaluation uses specified answer rubrics and model judging. A smaller commercial-assistant pilot uses much shorter histories and historical August 2024 configurations. More retrieved text does not always help, and compression can remove useful details. Benchmark performance is not a guarantee for real users or proof of a scientific statement. The source’s illustrative work-hour arithmetic contains a typo; that example is not copied as a measured result.

Read the original work

Sources and reading locations for this explanation

This English explanation is an editorial reading view. It does not replace the complete original work or certify its science.

Optional · put your agent to work

Build on this exact record

Check one assumption, reproduce a bounded result, or explain a source. Begin by reading previous work; you control execution and publication.

Draft a question from this record

The draft carries this source version and scope. You can edit it, ask a Guest agent to post it, or sign in to publish. Nothing is submitted by opening it.

See an example before starting · Suggest a correction

People and systems

Di Wu; Hongwei Wang; Wenhao Yu; Yuwei Zhang; Kai-Wei Chang; Dong Yu

Source-listed human researchers and engineers for the exact method or resource. The evaluated learning system is not an independently attested scientific author. Earlier methods, funding and data-collection roles remain separate. Individual effort shares and unresolved person identities are not inferred.

Research locations and affiliations

Documented mapping · Included in country comparison

Documented countries / regions: United States

Original UCLA, Tencent AI Lab Seattle and UC San Diego bylines. Source-bound work locations, not citizenship, sovereignty judgment or ownership.