aktagon/llmkit-rust

llmkit (Rust)

One Rust API for Anthropic, OpenAI, Google, and 20+ other providers — including local models through Ollama and vLLM. Switch providers without rewriting your request.

Async, built on tokio and reqwest.

Also available for Go, TypeScript, and Python.

Install

[dependencies]
llmkit = "1.0"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Quick Start

use llmkit::builders::anthropic;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let c = anthropic(std::env::var("ANTHROPIC_API_KEY")?);
    let resp = c.text()
        .system("Be concise.")
        .temperature(0.3)
        .prompt("Why is the sky blue?")
        .await?;

    println!("{}", resp.text);
    println!("{} input tokens", resp.usage.input);
    Ok(())
}

The typed builder is the only public surface as of v1.0.0. One mental model — client.<capability>().<chain>.<terminal> — across every capability.

Runnable counterparts to every code block below live in examples/ and are exercised by tests/examples.rs against a mock HTTP server, so the call shapes shown here are guaranteed to execute against the real builder surface.

Providers

Per-provider factory functions in llmkit::builders:

ai21       anthropic  azure      bedrock    cerebras   cohere
deepseek   doubao     ernie      fireworks  google     grok
groq       jan        llamacpp   lmstudio   minimax    mistral
moonshot   ollama     openai     openrouter perplexity qwen
sambanova  together   vertex     vllm       yi         zhipu

Or use the generic new_client(ProviderName::OpenAI, key). 30 providers, 4 API shapes (OpenAI-compatible, Anthropic Messages, Google Generative AI, AWS Bedrock Converse). Bedrock auth uses SigV4; other providers use API-key auth.

API

Text — one-shot prompt

let resp = c.text()
    .system("You are helpful")
    .temperature(0.7)
    .max_tokens(200)
    .prompt("What is 2+2?"