Choose and connect models

Choose a model by what the task needs and what your account, endpoint and Sonny client can actually use. This guide is for a user with Sonny 0.12.7, an approved provider or local runtime, and access to a model it serves. It explains setup, inspection and routing; Supported Providers gives the release-derived transport, credential, URI and capability matrices.

The executed examples use controlled Chat Completions and native Ollama protocol fixtures. They qualify Sonny’s schema and dispatch behavior without private source access. They do not certify a hosted account, a downloaded local model, model reasoning quality or a provider’s current product catalog.

Start with the task Share link

A model’s advertised context size is not the entire usable task budget. System instructions, history, retrieved material, tool definitions/results and reserved output also consume context. A text model does not acquire image understanding because a document filename appears in a prompt. Extract supported text or use a qualified image/document path first.

Keep the names separate Share link

An OpenAI-compatible interface describes a wire format. Model features, endpoint security, authentication and retention still need checking. The provider table separates CLI provider IDs from accepted Recipe URI schemes; these namespaces are not interchangeable.

Configure one approved connection Share link

Keep secret values out of catalogs and Recipes. Configure the catalog source’s credential through Provider credentials , using its source ID and declared credential need. A stored key for another source or a bare provider account may not answer this need. Follow the selected provider’s access terms and your organization’s approved endpoint.

These are complete schema templates. Replace their marked model identifiers with models your endpoint actually serves before making a request:

• Hosted account catalog template : OpenAI transport, source work-hosted, OPENAI_API_KEY credential need and YOUR_HOSTED_MODEL to replace. An approved compatible gateway can use an explicit base_url and its own credential variable instead.
• Local runtime catalog template : native Ollama at http://127.0.0.1:11434, source work-local and YOUR_INSTALLED_LOCAL_MODEL to replace. This does not install a runtime or pull a model. A remote endpoint needs its own transport/authentication review.
• Complete review Recipe : one default profile asking for category primary, with a prompt to compare approved evidence.

For example, download the review Recipe and the catalog template matching your setup into a new directory:

mkdir sonny-model-review
cd sonny-model-review
curl --fail --location https://sonny.sh/examples/models/review.yaml --output review.yaml
curl --fail --location https://sonny.sh/examples/models/hosted-account.template.yaml --output models.yaml

For native Ollama, download local-runtime.template.yaml from the link above as models.yaml instead. Edit the catalog and inspect the YAML before using it. The model identifier is a replacement field, not a current model recommendation.

The dated hosted-protocol fixture and local-protocol fixture contain the exact synthetic model/source definitions exercised during qualification, with only loopback ports substituted by the runner. Their hosted-fixture and local-fixture models exist only in the controlled fixtures. Their synthetic prices are test data, not vendor prices. Do not copy these fixture values into a real account setup.

Inspect, then make a small request Share link

First validate the Recipe and inspect selection without endpoint probes:

sonny recipe validate ./review.yaml
sonny models list --models ./models.yaml --no-check --json

Check the returned primary winner, source and model rows. This catalog command is an offline inspection, not a service-health or authentication test. Static Recipe validation also does not validate every raw URI/model combination; review warnings as well as exit status.

Use sonny models detail <source>/<model> --models ./models.yaml --json with your actual source and model identifiers for a deployment’s category settings and capability decisions. The angle-bracket fields denote replacements. A setting being declared is distinct from a live model accepting it.

After configuring the source’s credential and model, make a bounded, tool-free request with the selected catalog:

SONNY_MODELS_FILE="$PWD/models.yaml" sonny run --new --file ./review.yaml --no-tools --no-trace --prompt 'Note A: 30/40 answered within one day. Note B: 35/50. Give both rates and identify missing satisfaction evidence.' --result-json ./result.json

Expect A = 75%, B = 70%, and no claim about satisfaction. Independently check the answer, the intended endpoint and the result envelope’s completion, usage and error fields before assigning larger work. The controlled fixtures returned those values; they do not establish that your chosen model reasons correctly. --result-json writes Sonny’s result envelope; it does not impose a JSON schema on the model’s own answer.

This run sends the prompt to the selected service and persists local session, control and result state. --no-tools disables model tool use for this headless request. --no-trace suppresses conversation tracing, not provider transmission or all local state. Tool-enabled work needs a separate permission and execution review.

Understand precedence before changing a key Share link

For the primary catalog, an inspection command’s --models wins over SONNY_MODELS_FILE; otherwise the standard configuration file applies. User models.d files still participate after the primary, sorted by filename. Changing just the primary file does not remove those drop-ins.

Installed catalog/Recipe-source imports and the project chain also participate. Later definitions can replace a source’s model data; operator-owned endpoint, credential and routing authority remain protected separately. Inspect the merged result rather than infer a winner from one file. The model configuration reference records the layer order and trust boundary.

For profiles, a reachable explicit provider/model pair wins over category or URI settings. Without that pair, a category wins over a URI and emits a warning. A category that cannot resolve can fall back to the registry’s most-capable reachable model, also with a warning. A profile combining an unreachable pin and a category can take that fallback during load. A misspelled category can therefore produce a successful response from an unintended role.

At the endpoint boundary, an explicit operator catalog URL wins over a generic provider URL. An env_prefix source’s resolver can take its derived endpoint variable ahead of its literal URL, but resolution does not establish admission. In this release, a fresh headless run with a different prefixed URL refused with “primary routing provenance changed; reload the selected pack” before dispatch. The same-endpoint prefixed-credential counterpart succeeded. Catalog inspection still showed the literal URL, so inspect actual admission and dispatch separately; the distinct-endpoint positive remains nonzero in qualification.

A Recipe’s own endpoint/credential combination must be admitted: an unsigned Recipe could not send the fixture key to an undeclared endpoint; declaring that connection in the operator catalog allowed the reviewed counterpart. An endpoint override is not permission to forward an arbitrary key.

Control fallback deliberately Share link

Several mechanisms have different scopes:

The installed category fixture switched to a standby for model-not-found 404 and rate-limit 429 responses. A reachable pinned profile did not widen its model when that model failed. A malformed response failed without contacting the standby. A stream that has already delivered content is not safely interchangeable with a fresh request; inspect partial output before retrying.

Release limitation: an ordinary HTTP 400 unsupported-parameter error also switched models in the installed headless category path, although the independent SDK/classifier check rejected that same class of 400 as unavailable. The cause is not established. Do not assume a parameter error always stops Automatic routing or treat a response from the standby as proof that the original parameters worked. The strict parameter-refusal positive remains nonzero in qualification.

Approve the data boundary for every candidate you permit, including a fallback in another service. If work needs one exact route, use the supported explicit selection/lock and inspect its behavior rather than relying on a category name. A source outage and a failed task-quality check are separate problems.

Read usage and model attribution carefully Share link

sonny models prices --models ./models.yaml shows configured/cached prices. --refresh explicitly fetches configured pricing endpoints. Prices used to rank candidates are not a bill, and an absent price is not proof of a free service. A cheaper route still has to meet the task’s evidence requirements.

In the controlled run, result.json reported token usage across the task and its completion audit. A failover case had two outer provider_attempts but four actual HTTP POSTs. The result’s model remained the nominal selected model while the captured receiving endpoint showed the standby answered. Those fields alone do not identify every serving model or count every internal request/retry. Use the attribution available in your actual client and the provider/gateway’s request records for an audit; inspect their retention/access settings too.

Diagnose the failing layer Share link

Continue with provider/URI/capability reference , credentials , configuration and trust , tool/server security boundaries , Recipe evaluation or the non-coding cookbook .