Start with one question: did a tool fail, did the request contain the intended source, or did the result meet the task? A trace helps investigate those questions. It does not replace inspecting the file, calculation or external service affected by the run.
This guide targets Sonny 0.12.7, source/build
192e590345c6a4bf1ef8f72b043b318084ea84d6. Installed Linux checks generated a
real no-tool trace against a controlled provider, inspected it through the CLI
and opened its HTML views offline. The model text and reported usage in that
fixture are synthetic. No live telemetry collector or upload service is
qualified by those checks.
Choose the evidence for the question
| System | Answers | Boundary |
|---|---|---|
| Diagnostic/configuration commands | Which version, configuration, connection or execution prerequisite applies? | A parsed configuration or available tool name does not prove a task ran. Start with the relevant subsystem guide. |
| Conversation trace | What requests, replies, tool results, approvals and failures were recorded? | Can contain private conversation data. Its completion record is not independent verification of every claim. |
| Context telemetry | What context-assembly and cache measurements were recorded? | Separate opt-in data and filters; see context and compaction . |
| OpenTelemetry | What spans, metrics and process relationships were exported? | Separate exporter/content controls and collector trust boundary. |
| Verification artifacts | What output and independent checks support a task outcome? | An existing file or successful process exit alone may be insufficient. |
| Recipe evaluation | Does a reply meet the chosen case assertions? | See the fixed non-coding evaluation ; lexical tests and advisory judges have different limits. |
| Optimization | Does a measured candidate improve the chosen evaluation, and can it be adopted? | Proposal, local score, holdout, approval and publication are separate decisions. |
| Background analysis | What configured pipelines consume events and produce findings or candidates? | Configuration validation is local evidence; the installed build can report runtime assembly unavailable. See analysis availability . |
| CUE | What evidence supports context-use claims under a specific design? | Planning, imported evidence, hypothetical reports and authorized measurement must stay distinct. |
| Training/export | Can reviewed data become local cases or eligible training records? | Collection and verified trace export have different eligibility policies. |
Record the version, Recipe/source revision, model/provider, client, workspace, input and expected outcome. Keep the original artifacts under the intended access rules. For a tool problem, compare its actual returned failure and side effects with the requested operation before rerunning it.
Inspect a trace without changing it
Local conversation traces normally live under
<data-dir>/traces/<session-id>/trace_*.jsonl, where the data directory is
resolved for the account and environment running Sonny. Hosted services can
use a tenant-specific trace root. Find the intended process’s data paths with
configuration
; do not guess that another account or
client uses your local store.
The following command forms inspect your selected trace:
sonny trace list
sonny trace summary <trace-file-or-reference> --json
sonny trace tools <trace-file-or-reference> --json
sonny trace errors <trace-file-or-reference> --json
sonny trace show <trace-file-or-reference> --json
Replace the reference with an actual file, session directory, trace ID or
unique prefix. A session reference can combine resumed segments. Keep the
resolved paths from sonny trace path <reference> --json with the report so
that a later reader knows which files were inspected.
Try a public synthetic trace in separate input and output directories:
umask 077
mkdir -m 700 trace-input trace-output
curl --fail --location 'https://sonny.sh/examples/diagnostics/sample-trace.jsonl' -o trace-input/sample-trace.jsonl
sonny trace --trace-dir ./trace-input summary ./trace-input/sample-trace.jsonl --json
sonny trace --trace-dir ./trace-input html ./trace-input/sample-trace.jsonl --audience team --out ./trace-output/review.html
The download contains only documentation-fixture data and expired temporary
paths. It is an actual native trace, not a live session or a measured provider
benchmark. Review review.html locally before sharing it.
The installed checks also exercised events, status, tail --no-follow and
agents. They left the original trace bytes unchanged. Removing only the
writer’s summary footer in a separate test copy still produced the same event,
request, response and token counts. A malformed appended JSON line was reported
under the header’s parse problems; a zero command exit did not mean every input
line was valid. Inspect those problems and missing coverage before drawing a
conclusion from partial data.
Keep identifiers separate
| Identifier | Meaning |
|---|---|
| Session ID | The application/session-store identity; continuation and ownership use their own rules. |
| Trace ID and file | A trace segment; one session can have multiple segments. |
| LLM correlation ID | A provider-routing label such as sonny_…, sonny_eval_… or sonny_opt_…; it is not an authorization credential. |
| Evaluation case and run IDs | The assertion/campaign being measured, not necessarily an interactive session. |
| Optimization round and candidate digest | The proposed change and evidence it was compared against. |
| CUE plan, archive and receipt digests | The retained design/artifact identities; not interchangeable with provider sessions or a proof that a scheduler ran. |
The parser’s final header can contain facts absent from the first raw header: the fixture emitted several header updates during normal tracing. Do not assume one immutable header or infer a correlation ID from an unrelated filename.
Provider transport is scheme-specific. The tested openai fixture carried no
LiteLLM session field on the wire, even though the parsed trace contained an
application LLM correlation ID. Source contracts describe different transports
for LiteLLM, opt-in OpenAI-compatible gateways, Bedrock and Vertex; they are not
a universal session header and are not live-provider qualifications here. Keep
identifiers opaque and free of secrets. See provider selection
.
Interpret usage and outcomes carefully
The fixture supplied three input and five output tokens per response. Its summary showed six input and ten output tokens across two API responses. Those are fixture-reported counts, not paid usage or a cross-provider billing standard. Check missing usage, cache-read/write treatment, pricing revisions, retries and judging/optimizer calls before estimating a real run’s cost.
The same trace had a completed conversation and an unresolved authoritative outcome. An internal completion/quality score did not turn it into verified training evidence. Inspect the actual outcome, verifier evidence and resulting artifact rather than treating any one score as a task certificate.
Capture and sharing controls
Conversation tracing is on by default in the tested local run. Its trace file was mode 0600 inside a 0700 directory. Credential-shaped synthetic text was replaced by a redaction marker, while ordinary confidential prose remained. Pattern redaction is not consent to disclose the rest of a conversation.
--no-trace and SONNY_TRACE=0 suppressed the parent trace in both installed
checks. Each run still created an internal mcp-server trace. Do not describe
these controls as a process-tree-wide guarantee that no trace file exists.
Inspect the applicable trace roots and the actual client/service behavior.
Context telemetry is a separate opt-in system. OpenTelemetry is also separate:
its source-default instrumentation is off unless enabled by SONNY_OTEL or
an OTLP endpoint. OTEL_SDK_DISABLED=true overrides it; SONNY_OTEL_CONTENT
controls raw span content and defaults off. The fixture explicitly disabled the
OTel SDK. A live collector, exporter transport and child relay still need their
own deployment qualification; disabling OTel does not disable JSONL traces.
The local trace writer has no automatic retention cleanup in the pinned source.
sonny trace is read-only. Tracing off does not remove old traces or the
unredacted session replay store. Use reviewed privacy and backup procedures
for retained files, and inspect training/export
before any
collection, dataset construction or upload.
HTML views apply their audience at construction: operator retains the content
the CLI shows; team and executive reduce it. In the synthetic fixture, only
the operator HTML retained the private marker. All three views rendered with
network disabled and JavaScript disabled, with no remote requests. Audience
selection is still not a substitute for reviewing your actual report.
Continue with evaluation , optimization and CUE .