Guide — Custom model providers

Scope

This document summarises the steps and criteria for implementing custom model providers with the Strands Agents SDK inside Shell Sentinel. It serves as an internal checklist and complements the official documentation.

Prerequisites

• Know the strands.models.Model hierarchy (review examples such as BedrockModel).
• Understand Messages, StreamEvent and ToolSpec types.
• Python client for the proprietary LLM service (sync or async).
• Declarative configuration in conf/ and credentials via environment variables.

Implementation flow

1. Define configuration: create a typed ModelConfig and expose get_config/update_config.
2. Initialise the client: resolve credentials securely, instantiate the remote client and register logging.
3. Implement stream(...): convert inputs, adapt to StreamEvent, handle errors; use asyncio.to_thread for sync SDKs.
4. Support tools: reuse stream in structured_output(...) with Pydantic ToolSpec conversion.
5. Register the provider in smart_ai_sys_admin.agent and conf/agent.conf.

Additional considerations

• Use DEBUG logging for troubleshooting.
• Document new parameters in user manuals when operators are affected.
• Never hardcode tokens or endpoints.
• Run smoke tests before TUI integration.

Practical case: LM Studio

• OpenAI-compatible local server (/v1/*); configure base_url, api_key and model_id.
• Start with lms server start; tune client_args for timeouts.
• Native REST API (/api/v0/*) exposes metrics and max_context_length.

Practical case: Cerebras

• Integrate cerebras_cloud_sdk with SSE streaming.
• Configure providers.cerebras with model_id, params, client_args and api_key_env.
• Convert ChatChunkResponse to native events with metadata for usage and timing.

External references

• Strands Agents documentation
• BedrockModel example