Runtime Contracts

AI Core exposes the same runtime concepts through FFI, the Rust SDK, and the process-host binary. SDK users do not need source-level Rust structs to understand the contract: the host applies config, installs packages, starts or resumes work, and stores durable state outside AI Core.

What AI Core Owns

AI Core owns live execution state:

• model endpoint registration and dispatch by model_id;
• routing/default model selection;
• plugin metadata, package install, and package reload checks;
• Wasm action execution;
• pipeline start/resume checkpoints;
• runtime events for transcription, routing, model calls, tools, plugins, and pipeline progress;
• in-memory license material while a normal marketplace runtime is running.

AI Core does not own users, accounts, billing, application databases, queues, secret stores, long-term pipeline output storage, or UI settings.

Host-Owned Config

The host configures runtime state with a JSON-equivalent snapshot:

{
  "models": [
    {
      "kind": "openai_compatible",
      "url": "http://127.0.0.1:1234/v1",
      "model_id": "qwen/qwen3-1.7b",
      "token_ref": "secret:models.routing_local.token",
      "enabled": true
    },
    {
      "kind": "openai",
      "url": "https://api.openai.com/v1",
      "model_id": "gpt-4.1-mini",
      "token_ref": "secret:models.default_openai.token",
      "enabled": true,
      "local": null
    },
    {
      "kind": "built_in_local",
      "model_id": "qwen3-1.7b",
      "enabled": true,
      "token_ref": null,
      "local": {
        "path": "/models/llm/qwen3-1.7b.gguf",
        "context_tokens": 4096,
        "backend": "cpu",
        "max_concurrent_requests": 1,
        "max_pending_requests": 0
      }
    }
  ],
  "local_llm": {
    "backend": "metal"
  },
  "model_selection": {
    "routing": "qwen/qwen3-1.7b",
    "default": "gpt-4.1-mini"
  },
  "plugin_model_bindings": [
    {
      "plugin_id": "example_pipeline",
      "slot_id": "draft",
      "model_id": "qwen3-1.7b"
    }
  ],
  "plugins": [
    {
      "plugin_id": "example_pipeline",
      "public": {
        "language": "en"
      },
      "secrets": {
        "api_token": "secret:plugins.example_pipeline.api_token"
      }
    }
  ],
  "speech": {
    "whisper": {
      "model_path": "/models/whisper/ggml-large-v3-turbo.bin",
      "coreml_encoder_path": null,
      "language": "auto",
      "max_concurrent_requests": 1,
      "max_pending_requests": 0
    }
  }
}

Supported model kinds are openai, openai_compatible, claude, and built_in_local. The built-in local kind requires a runtime artifact that includes native local LLM support and an explicit .gguf path.

Runtime artifacts may ship in different build variants. Some builds include both local speech-to-text and local model execution, while others omit one or both capabilities. Check the core download page on the site for the current artifact matrix.

Local LLM backend preference is host-owned config. local_llm.backend applies to all built_in_local models and defaults to auto. A model can set local.backend to override the global preference for that one .gguf file, which is useful when a small router fits on GPU but a larger default model should start on CPU. Supported values are auto, cpu, and metal. The value is a preference, not a hard guarantee: AI Core tries the preferred backend first, then falls back to automatic backend selection when the native provider reports a backend-level failure.

Local LLM concurrency is per configured built_in_local model. local.max_concurrent_requests defaults to 1 and must be greater than zero. local.max_pending_requests defaults to 0, which rejects a new native call when all active slots are busy. Inside one process, the native provider keeps one resident llama_model per model/backend key after first use; multiple FFI handles share it. Each active request still creates its own llama_context and KV cache, so increasing concurrency increases peak context memory. Rust hosts can use LOCAL_MODEL_UNBOUNDED_CONCURRENT_REQUESTS, and JSON hosts can pass 4294967295, for an effectively unbounded active limit. That should be reserved for hosts with their own external backpressure.

Whisper uses the same admission-control shape under speech.whisper. speech.whisper.max_concurrent_requests defaults to 1; max_pending_requests defaults to 0. With defaults, one resident Whisper context is kept per process/model path after first use, and concurrent transcription calls are rejected while that context is busy. Rust hosts can use WHISPER_UNBOUNDED_CONCURRENT_REQUESTS, and JSON hosts can pass 4294967295, for an effectively unbounded active limit.

These managers are process-local. Separate OS processes do not share resident GGUF weights or Whisper context pools. Keep durable queues, retries, and backpressure in the host.

Each non-local model can carry its own token_ref. AI Core stores only the reference; the host resolves it through a secret store and gives the provider the raw value at call time. OpenAI and OpenAI-compatible models use the OpenAI chat-completions JSON shape and Authorization: Bearer .... Claude uses the Anthropic messages JSON shape and x-api-key.

The core runtime has one app-level routing model and one default model. Plugin packages can declare generation model_slots, and the runtime seeds missing plugin_model_bindings from each slot's fallback when it points at core.default or another concrete model name. Routing is never a plugin slot.

Secret values are references such as secret:plugins.example_pipeline.api_token. The host resolves those references through Keychain, Keystore, KMS, Vault, Kubernetes secrets, environment injection, or another host-owned secret store.

Package Activation

AI Core installs .aip packages after validating:

• package format and manifest file hashes;
• plugin id and version compatibility;
• requirements.core;
• plugin API version;
• Wasm entrypoint and ABI version for Wasm packages;
• Marketplace approval and license material in normal marketplace runtimes.

Install failure keeps the previous package version active. Installed package files are kept in memory for the current runtime session so package execution and pipeline steps can read bundled skills, schemas, and pipeline.json.

No-marketplace runtime artifacts are a separate local, enterprise, or white-label mode. They accept packages produced for no-marketplace distribution, including plaintext unsigned packages or developer-signed packages built with aip-pack-no-marketplace.

Pipeline State

Pipeline calls exchange a serializable cursor:

{
  "schema": "ai_core.pipeline_state.v1",
  "run_id": "run-1",
  "task_id": "task-1",
  "plugin_id": "example_pipeline",
  "status": "waiting_for_approval",
  "next_step_index": 1
}

The host stores this state and any completed step outputs. On resume, the host passes state, previous_outputs, new user_text, and a control value:

• "continue" to advance;
• "retry" to rerun the completed step with host-supplied user_text. The same step still receives its skill, output schema, and earlier outputs.

If the package pipeline.json declares a top-level input contract and the start request is missing required values, AI Core can return a checkpoint before step 0. The host asks the user for missing data and resumes with the same pipeline state.

Events

Runtime events are for host logging, UI progress, and diagnostics. Common event areas include transcription, router decisions, model request/response, package install, plugin action execution, pipeline checkpoints, and Wasm progress.

Language bindings should expose events as host-language objects or JSON arrays instead of leaking internal Rust event types.

Version Rules

Packages declare runtime compatibility in requirements.core. AI Core parses that semver expression before install/reload activation. Ship runtime artifacts and packages from compatible release lines, and treat package/runtime mode as part of the artifact contract.