A model answers from the context assembled for its current request. That is not necessarily every message, document or memory associated with the session. Use this explanation for Sonny 0.12.7, then follow Manage a long session to check important details before and after compaction.
Different records have different jobs
| Mechanism | What it contains or does | Lifetime and recovery boundary |
|---|---|---|
| Conversation transcript | User/assistant messages, tool exchanges, attachments and persisted checkpoints | The TUI saves history for resume. Compaction can replace older content with a checkpoint; a saved session is not an immutable archive of every earlier byte. One-shot headless persistence has separate limits . |
| Assembled request context | Current instructions, selected history, tool schemas and admitted evidence sent to the selected model | Rebuilt for requests; model/provider windows and runtime admission constrain it. A visible file or saved memory is not proof its body was sent. |
| Compaction checkpoint | Structured task state, selected evidence and a continuation summary with recent exchanges | Replaces part of working history. The host marks it as a checkpoint; ordinary text with similar delimiters does not acquire that authority. Review critical details after compaction. |
| File-backed memory | Reusable project/global YAML facts and decisions | Persists until changed/deleted. The injected digest has its own bounds; retrieve the required record explicitly. See Memory . |
| Retrieval index | A derived search structure that selects records or evidence | The local memindex is rebuildable from authoritative memory YAML. A search result still needs scope, relevance and freshness review. It is not automatic synchronization to Lanning
. |
| Local response/tool/workflow caches | Reuse under implementation-specific keys, policy, scope and expiry | The existence of cache code or a telemetry field does not prove that a particular execution path uses it. See the release distinctions below. |
| Provider cache | Reuse performed by the selected model service | Depends on that deployment’s capabilities and request shape. Provider billing/cache reports are separate from Sonny’s local cache metrics and data retention. |
| Tool-output recovery cache | Session-owned retained tool output for supported follow-up selection | Bounded, temporary recovery, with ownership checks. It is not a durable document store or a guarantee that every omitted result remains recoverable. |
For where these stores live and what deletion leaves behind, use the privacy inventory and recovery runbook .
What compaction does
The runtime counts the full request budget, including system/developer instructions and tool schemas. It excludes system/developer instructions from summarization and retains them when rebuilding history. Preserving those instructions does not make their tokens free.
The compaction pipeline can extract structured task state, reduce selected evidence and summarize older conversation content. Recent exchanges are retained alongside the checkpoint. A summary can omit an incidental detail, misstate a fact or lose a useful qualification. A smaller request or a successful compaction message is not proof of semantic completeness.
The supported /compact session command requests a pass. In the simple REPL it
can report a reduction, no-op (below budget), a disabled pipeline, or a
compaction failure. It does not promise to summarize every time it is invoked.
An automatic extraction failure can leave the turn using uncompacted history;
other request limits still apply. A persistence failure can make a turn
indeterminate rather than silently certify the compaction.
The host retains task bookkeeping such as the user goal and active work queue outside model-authored summary text. This protects the bookkeeping boundary; it does not establish that every original document detail remains in context.
Release budget rules
These are source-verified static rules, not a recommendation to increase a provider’s window. The effective window comes from the selected/routed model and profile; where both are known, the smaller ceiling constrains the request. The provider’s observed prompt size can serve as a floor under local estimates.
| Setting or stage | Sonny 0.12.7 behavior |
|---|---|
| Default hard threshold | The smaller of 80% of the known context window and 160,000 tokens |
| Default soft threshold | 70% of the hard threshold; permits structured-state work before full transcript reduction |
context_window_tokens | Model-catalog declaration of the deployment’s context ceiling; a positive known window is needed to construct this pipeline |
compaction_hard_pct | An explicit integer from 1–100 uses that percentage of the window instead of the default hard calculation, including its default 160,000 ceiling |
compaction_soft_pct | An explicit integer from 1–100 sets soft as a percentage of hard; omitted/invalid values use 70 |
compaction_max_tokens | A positive absolute cap further reduces the calculated hard threshold |
The percentage/cap fields are per-model params settings. The runtime’s local
estimate includes message and tool-schema content with an inflation factor;
it is not a tokenizer or a billing measurement. Host decisions can further
reduce the working budget based on observed model performance. A long-window
model therefore does not guarantee a correspondingly large economical working
history. Review model selection
before changing a catalog.
Cache behavior requires the right execution path
The tagged application-level cache implementation defines response, deterministic tool and workflow caches with default lifetimes of 30 minutes, 5 minutes and 1 hour. Its keys include scope and version/policy dimensions; it also supports explicit invalidation and optional response persistence. These implementation defaults are not evidence that the native headless loop serves cached answers: that release’s response-cache constructor/wrapper has no production call site in that path. Do not enable an older service-specific persistence variable and assume it changes native conversation retention or eliminates provider calls.
The separate semantic-cache mode defaults off. Its native bootstrap uses an
in-memory store and a no-op embedder unless a real embedding implementation is
wired in. A setting named on is insufficient proof of useful semantic reuse.
Neither this layer nor a provider prefix-cache hit establishes that a retrieved
fact is current. Resource reuse and temporary tool-output recovery also have
their own ownership and validity rules; they are separate from durable memory.
The tool-output store defaults to 64 entries, a 1 MiB body limit and a
30-minute TTL with extension on access. Oversized bodies may not be retained,
and expired/evicted entries cannot substitute for the original source. Its
session owner releases the store; it is not a cross-process conversation backup.
Inspect operator context diagnostics
sonny context stats and sonny context validate read a telemetry SQLite store.
Their default is ~/.sonny/trace/cache.db, outside the usual XDG data roots.
They create/migrate a missing store in this release. They do not automatically
load the active TUI transcript or import all conversation trace JSONL files.
Confirm how the relevant telemetry was populated before interpreting it.
Choose an existing, authorized telemetry store and the exact session you intend to inspect. The following is syntax with placeholders:
sonny context stats --store <telemetry-db> --session <session-id> --since 168h
sonny context validate <session-id> --store <telemetry-db> --tenant <expected-tenant>
Use 168h for seven days: the help’s 7d example fails duration parsing in the
tested release. The context-stats and context-validate root commands are
compatibility entry points for the same implementations.
Read the fields within their actual scope
| Output | What the release calculates | Important limit |
|---|---|---|
| Cache hit rate | Distinct turns with cache_hit divided by turns with api_request for the selected session | --since does not filter this calculation; it is not provider billed-token savings |
| Prefix churn | Consecutive shell-digest changes divided by layout transitions for that session | Structural telemetry, not proof of content quality or provider reuse |
| Semantic-cache precision | Fraction of recorded cluster divergences classified trivial/stylistic | Uses available diagnostic samples; a displayed ratio does not prove coverage of all requests |
| Compression/retention by Recipe | Compaction ratios and available retention judgments within the time range | The table is not restricted by --session or --tenant; an unjudged average displays 0.00 |
| Tenant in the stats heading | The supplied label | stats --tenant does not filter these metrics in 0.12.7; it is not an authorization boundary |
Use this command only on a store you are authorized to inspect in its entirety. The fixture retains failed positive assertions for the advertised tenant, session-compaction and time-range cache filtering. They are not accepted success baselines.
Validation checks recorded shell stability, tenant scope, measured retention
against a 0.70 floor, and recorded return-contract violations. It runs all four
checks and reports the number violated. The session identifier is positional;
there is no --session flag on context validate.
An unknown session can return PASS with no records. A compaction without a
retention judgment is treated as 1.0 by validation, even though stats displays
an unjudged average as zero. Neither result proves that a model retained 70% of
meaning or that the desired session was observed. Check record coverage and
judgment availability separately. The executed fixture confirms refusal for
shell drift, wrong/missing tenant scope, low measured retention and a recorded
contract violation.
Keep telemetry separate from task evidence
Conversation traces, context metrics and OpenTelemetry are distinct. Read the telemetry and trace controls before enabling or sharing diagnostics. Model requests can contain private source text even when some trace patterns are redacted. Lanning’s service availability and retrieval adapters must be qualified independently; these local commands do not establish shared storage or synchronization.
Use diagnostics to inspect retained evidence and CUE to distinguish context-use study designs from operational metrics.