Guide — Custom model providers

Scope

This document summarises the steps and criteria for implementing custom model providers with the Strands Agents SDK inside Shell Sentinel. It serves as an internal checklist and complements the official documentation.

Prerequisites

Know the strands.models.Model hierarchy (review examples such as BedrockModel).
Understand Messages, StreamEvent and ToolSpec types.
Python client for the proprietary LLM service (sync or async).
Declarative configuration in conf/ and credentials via environment variables.

Implementation flow

Define configuration: create a typed ModelConfig and expose get_config/update_config.
Initialise the client: resolve credentials securely, instantiate the remote client and register logging.
Implement stream(...): convert inputs, adapt to StreamEvent, handle errors; use asyncio.to_thread for sync SDKs.
Support tools: reuse stream in structured_output(...) with Pydantic ToolSpec conversion.
Register the provider in smart_ai_sys_admin.agent and conf/agent.conf.

Additional considerations

Use DEBUG logging for troubleshooting.
Document new parameters in user manuals when operators are affected.
Never hardcode tokens or endpoints.
Run smoke tests before TUI integration.

Practical case: OpenAI Responses API

providers.openai.api accepts chat_completions (default, /v1/chat/completions) and responses (/v1/responses). max_tokens is normalized to max_completion_tokens for Chat Completions or max_output_tokens for Responses; reasoning_effort and reasoning.effort are normalized for the selected endpoint too.

The migration addresses an HTTP 400 when gpt-5.6-sol receives function tools and active reasoning through Chat Completions. reasoning_effort: "none" avoids the error only by disabling reasoning (reasoning_tokens=0). Responses supports function tools with medium reasoning without that degradation.

{
  "model_id": "gpt-5.6-sol",
  "api": "responses",
  "params": {"reasoning_effort": "medium", "max_tokens": 32768}
}

The effective request uses reasoning.effort and max_output_tokens; the real configuration uses 65536. Optional temperature: 0.3 is supported but is not a default. Export OPENAI_API_KEY from the environment.

Optional stateful mode can use previous_response_id, but reconstructed multi-turn reasoningContent does not yet retain full reasoning continuity.

Practical case: LM Studio

OpenAI-compatible local server (/v1/*); configure base_url, api_key and model_id.
Start with lms server start; tune client_args for timeouts.
Native REST API (/api/v0/*) exposes metrics and max_context_length.

Practical case: Mistral AI (Path A+)

ShellMistralModel wraps Strands MistralModel (official mistralai v2 SDK).
Configure providers.mistral with reasoning_effort: high and max_tokens: 16184 by default.
Run make test-mistral when MISTRAL_API_KEY is available.

Practical case: Cerebras

Integrate cerebras_cloud_sdk with SSE streaming.
Configure providers.cerebras with model_id, params, client_args and api_key_env.
Convert ChatChunkResponse to native events with metadata for usage and timing.