Reliable LLM calls,
by default.

The LLM call framework. Resilience, observability, and control for every call. Retries, timeouts, provider fallback, rate limiting, circuit breaking and more, dependency-light and typed from the start.

Read the docsSource
$ npm install vern-llm
index.ts
import OpenAI from 'openai';
import { fromOpenAI, VernLLM } from 'vern-llm';
import { getWeatherTool } from './tools';

const openai = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));
const backup = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_KEY_2 }));

export const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o',
  maxRetries: 3,
  timeoutMs: 10_000,
  circuitBreaker: true,
  fallback: { client: backup, model: 'gpt-4o' },
  rateLimit: { requestsPerMinute: 500 }
});

export const { chunks, finalResult } = await llm.cachedCall({
  cacheKey: 'weather:new-york',
  ttl: 3600,
  call: {
    userContent: "What's the weather in New York?",
    tools: [getWeatherTool],
    stream: true
  }
});

for await (const chunk of chunks) {
  if (chunk.type === 'text-delta') process.stdout.write(chunk.delta);
}

What this call actually does.

  • maxRetries: 3→ Retries transient failures with backoff and jitter
  • timeoutMs: 10_000→ Prevents attempts from hanging indefinitely
  • circuitBreaker: true→ Stops repeated failures from cascading
  • fallback:→ Falls over to a backup target on failure
  • rateLimit:→ Queues locally under a per-minute ceiling
  • cachedCall→ Returns cached results without another API call
  • cacheKey→ Identifies repeatable cached requests
  • ttl: 3600→ Controls cache lifetime
  • tools→ Lets the model request app defined functions
  • stream: true→ Delivers live chunks with validated result
Works with
OpenAIOpenAIAnthropicAnthropicGeminiGeminiGroqGroqMistralMistralDeepSeekDeepSeekCerebrasCerebrastogether.aiTogether AIFireworksFireworks AIOllamaOllamaOpenRouterOpenRouterPerplexityPerplexityDeepInfraDeepInfraNovita AINovitaHyperbolicHyperbolicMoonshotAIMoonshot (Kimi)ZhipuZhipu (GLM)LM StudioLM StudiovLLMvLLMGrokxAI (Grok)NvidiaNVIDIA NIMVercelVercel AI GatewayCloudflareCloudflare Workers AINebiusNebius AI StudioSambaNovaSambaNova CloudBasetenBasetenfeatherless.aiFeatherless AIFriendliFriendli AISiliconCloudSiliconFlowParasailParasailStepFunStepFunMinimaxMiniMaxLambdaLambda LabsSnowflakeSnowflake CortexAnyscaleAnyscaleLeptonAILepton AIInferenceInference.netInfermaticInfermaticAtlas CloudAtlasCloudYi01.AI (Yi)BedrockAWS BedrockCustom HTTPS API

LLM calls fail in ways plain SDK calls do not handle: timeouts, rate limit errors, a provider having an outage, or a request that just hangs. VernLLM adds retries with backoff, a circuit breaker, provider fallback, rate limiting, and caching around your existing client, so a single bad call does not take down your app.

Calling the client directly means you own retries, timeouts, circuit breaking, and caching yourself, code most teams end up rewriting per project. VernLLM ships those as configurable options on one class, so you keep your existing provider client and wrap it instead of reimplementing the resilience layer.

Yes. cachedCall accepts any adapter implementing get/set (delete is optional), so Redis, a database, or a custom store can replace the built-in in-memory cache without changing how you call it.

Yes, through middleware. transform edits or redacts an outgoing request before it is sent, and wrap runs around a whole logical call, retries and fallback attempts included, for logging, tracing, or cost tracking.

Yes, written in TypeScript from the ground up. Structured output schemas, call params, and errors are all typed, so mistakes surface at compile time instead of at runtime.

Zero runtime dependencies. VernLLM does not bundle Zod or provider SDKs, it relies on compatible interfaces instead, so you bring your own provider clients and schema validators while keeping your dependency tree minimal.

Stop reinventing the call layer.

Get startedSource
npm install vern-llm