Speech To Text

AI Core can use a native local Whisper provider for speech-to-text when the runtime artifact includes that capability. SDK users do not need the speech-whisper source crate; they need a compatible AI Core artifact and a local Whisper model file.

Requirements

Use an AI Core runtime artifact that includes local speech-to-text support for your target platform. Ship or download a compatible ggml Whisper model:

ggml-large-v3-turbo.bin

This is a speech recognition model only. It is not an LLM and cannot be used for routing, reasoning, or chat.

AI Core currently expects WAV PCM16 or raw PCM16 input at the provider boundary. Hosts should convert M4A, MP3, FLAC, OGG, WebM, or platform recorder formats to WAV/PCM before calling the runtime.

Model Path Precedence

When the runtime uses the default Whisper model path helper, path resolution is:

1. SPEECH_WHISPER_MODEL=/absolute/path/to/ggml-large-v3-turbo.bin
2. SPEECH_WHISPER_MODEL_DIR=/absolute/path/to/models/whisper
3. platform app data fallback, for example on macOS: ~/Library/Application Support/AI Core/models/whisper/ggml-large-v3-turbo.bin

For app bundles, put the model in the app resources directory:

Your.app/Contents/Resources/models/whisper/ggml-large-v3-turbo.bin

For services or CLI deployments, put the model in a stable local directory and set:

export SPEECH_WHISPER_MODEL_DIR=/opt/ai-core/models/whisper

For user-selected models, store the selected absolute file path in host-owned settings and pass it in runtime config instead of relying on directory scans.

FFI Config

Configure speech through the normal AI Core config snapshot. In ai-core-ffi, this native provider is linked only when the shared library is built with builtin-speech-whisper or builtin-local-all.

{
  "speech": {
    "whisper": {
      "model_path": "/models/whisper/ggml-large-v3-turbo.bin",
      "coreml_encoder_path": null,
      "language": "auto",
      "max_concurrent_requests": 1,
      "max_pending_requests": 0
    }
  }
}

Language policy:

• config language is a provider-level override;
• request locale is a per-request hint from the host;
• effective precedence is config language, then request locale hint, then auto.

Concurrency and memory policy:

• max_concurrent_requests defaults to 1 and must be greater than zero.
• max_pending_requests defaults to 0, so a concurrent call is rejected while the active Whisper slot is busy instead of waiting inside Core.
• Rust hosts can use UNBOUNDED_CONCURRENT_REQUESTS, and JSON hosts can pass 4294967295, for an effectively unbounded active limit. That is a deliberate memory-risk choice and should require external backpressure.
• The native provider keeps a resident whisper_context pool per process/model path after first use. With defaults, that pool has one context.
• Raising max_concurrent_requests allows more simultaneous transcriptions, but each active slot needs its own native Whisper context memory.
• Separate OS processes do not share the pool. A pool of four worker processes can load four Whisper context pools.

Use the host service, UI, or job system for durable queues, retries, and backpressure. max_pending_requests is only an in-process admission queue.

FFI build variants:

# Native Whisper transcription only.
cargo build -p ai-core-ffi --release --features builtin-speech-whisper

# Native Whisper transcription plus native local GGUF chat.
cargo build -p ai-core-ffi --release --features builtin-local-all

FFI hosts call ai_core_transcribe_audio_json with host-owned audio bytes and a small request JSON:

{
  "format": "pcm16",
  "sample_rate_hz": 16000,
  "locale": "ru-RU"
}

The function is exported even in FFI artifacts without builtin-speech-whisper, but those artifacts return transcription_provider_unavailable instead of loading a Whisper model.

CLI Usage

ai-core run-audio /path/to/input.wav --language ru --debug

run-audio transcribes the audio first and then routes the transcript through the same model/plugin flow as run-text. The Whisper provider resolves the model path through SPEECH_WHISPER_MODEL, then SPEECH_WHISPER_MODEL_DIR, then the platform fallback path for ggml-large-v3-turbo.bin. Use --debug to print the transcript before routing.

Core ML Encoder

Some macOS artifacts may support Core ML acceleration. Those deployments need an additional encoder artifact beside the ggml model:

models/whisper/ggml-large-v3-turbo.bin
models/whisper/ggml-large-v3-turbo-encoder.mlmodelc/

If your selected AI Core artifact requires Core ML, check both paths before enabling local transcription in the host UI. If either file is missing, hide or disable local speech-to-text instead of constructing the provider.