Provider Setup — Connecting LLMs to Hermes Agent

By Hermes Agent··8 min read·providersllmhermes-agentquickstart

Learn how to configure LLM providers for Hermes Agent, manage credential pools, switch models, and verify provider health across cloud and local runtimes.

Provider Setup — Connecting LLMs to Hermes Agent

Breadcrumb: Home > Quick Start Guides > Provider Setup

Hermes Agent is designed to work with multiple large language model (LLM) providers through a unified configuration system. Whether you’re connecting to a hosted API, using a managed routing service, or running models locally, Hermes lets you switch providers without changing your workflows.

This guide covers the fastest setup path, manual configuration, provider settings, credential pools, model switching, local inference, and health verification.

Provider Overview

Hermes Agent supports more than 20 providers through a consistent interface. Once a provider is configured, every Hermes command uses the active model without requiring provider-specific syntax.

Commonly used providers include:

  • OpenRouter
  • Anthropic
  • OpenAI
  • DeepSeek
  • Groq
  • Mistral
  • Google
  • Together
  • Cerebras
  • Ollama
  • llama.cpp
  • Fireworks
  • xAI
  • Cohere
  • Azure OpenAI
  • AWS Bedrock
  • Vertex AI
  • AI21
  • SambaNova
  • Novita
  • LM Studio
  • OpenAI-compatible endpoints

Most providers require only:

  1. An API key
  2. A model identifier
  3. (Optional) A custom base URL

Hermes stores provider settings separately from project configuration, making it easy to reuse the same credentials across multiple repositories.

Tip: If you’re just getting started, use the Hermes Portal OAuth flow. It automatically configures credentials without manually copying API keys.

Fastest Path: Nous Portal OAuth

The quickest way to configure Hermes Agent is through the integrated Portal authentication flow.

Run:

hermes setup --portal

Hermes will:

  1. Open the authentication page.
  2. Sign you into your account.
  3. Request authorization.
  4. Download encrypted credentials.
  5. Configure your default provider.
  6. Verify connectivity.

A typical setup takes less than a minute.

After completion, confirm everything is working:

hermes doctor

If a default model was configured during onboarding, you’re ready to start using Hermes immediately.

Manual Provider Setup

If you prefer direct API keys or your organization manages credentials manually, use the setup wizard.

Launch:

hermes setup

The wizard walks through:

  • selecting a provider
  • entering credentials
  • choosing a default model
  • testing connectivity

Hermes stores provider configuration in your user directory.

Typical location:

~/.hermes/

Environment variables are commonly stored in:

~/.hermes/.env

Example:

OPENAI_API_KEY=sk-xxxxxxxx
ANTHROPIC_API_KEY=sk-ant-xxxxxxxx
OPENROUTER_API_KEY=xxxxxxxx
DEEPSEEK_API_KEY=xxxxxxxx

After editing the file manually, reload Hermes or rerun:

hermes doctor

to validate the configuration.

Provider Configuration Table

The following table summarizes the most common provider settings.

Provider Environment Variable Base URL Notes
OpenAI OPENAI_API_KEY https://api.openai.com/v1 Default OpenAI endpoint
Anthropic ANTHROPIC_API_KEY https://api.anthropic.com Claude models
OpenRouter OPENROUTER_API_KEY https://openrouter.ai/api/v1 Unified access to many models
DeepSeek DEEPSEEK_API_KEY https://api.deepseek.com Fast reasoning models
Groq GROQ_API_KEY https://api.groq.com/openai/v1 Low-latency inference
Mistral MISTRAL_API_KEY https://api.mistral.ai/v1 Native Mistral APIs
Google GOOGLE_API_KEY https://generativelanguage.googleapis.com Gemini models
Together TOGETHER_API_KEY https://api.together.xyz/v1 Open-weight models
Cerebras CEREBRAS_API_KEY https://api.cerebras.ai/v1 High-throughput inference
Ollama None http://localhost:11434 Local inference server
llama.cpp None Configurable Local OpenAI-compatible server
Azure OpenAI AZURE_OPENAI_API_KEY Deployment URL Requires deployment configuration
AWS Bedrock AWS credentials AWS endpoint Uses IAM authentication
Vertex AI Google credentials Regional endpoint Google Cloud authentication

Only the provider-specific credential is required for most hosted services.

Switching Models

Hermes separates provider configuration from model selection.

The easiest way to change models is the interactive picker.

hermes model

The picker displays:

  • configured providers
  • available models
  • recommended defaults
  • favorites
  • recently used models

Selecting a model immediately updates the active session.

To permanently change the default:

hermes config set model.default openai/gpt-5

or

hermes config set model.default anthropic/claude-sonnet

You can also override the model for a single command.

Example:

hermes chat --model openrouter/deepseek-r1

This leaves your default configuration unchanged.

Credential Pools

Many teams maintain multiple API keys for reliability, cost allocation, or quota management.

Hermes supports credential pools that group multiple credentials for the same provider.

Example configuration:

providers:
openrouter:
pool:
- key: OPENROUTER_KEY_PRIMARY
- key: OPENROUTER_KEY_BACKUP
- key: OPENROUTER_KEY_TEAM

Credential pools can provide:

  • automatic failover
  • quota distribution
  • regional routing
  • workload separation
  • organizational access control

Hermes selects healthy credentials automatically.

If one key reaches its quota or becomes unavailable, Hermes can move to the next credential in the pool without interrupting execution.

Fallback Chains

Fallback chains extend this concept across providers.

Example:

OpenRouter
↓
Anthropic
↓
OpenAI
↓
Local Ollama

If the preferred provider is unavailable, Hermes can continue by using the next configured provider.

Fallback chains are especially useful for:

  • CI pipelines
  • automated agents
  • long-running workflows
  • scheduled jobs
  • production deployments

This significantly reduces failures caused by temporary provider outages.

Info: Fallbacks work best when equivalent models are configured across providers.

Local Models

Hermes also supports completely local inference.

This enables offline development, reduced latency, and improved privacy.

Ollama

Install Ollama and pull a model.

Example:

ollama pull llama3.1

Ensure the Ollama server is running.

Hermes detects the default endpoint automatically:

http://localhost:11434

You can then select an Ollama model through:

hermes model

or specify it directly.

Example:

hermes chat --model ollama/llama3.1

llama.cpp

Hermes also supports OpenAI-compatible llama.cpp servers.

Typical workflow:

  1. Start the llama.cpp server.
  2. Configure its endpoint.
  3. Select the desired local model.

Example:

hermes config set providers.llamacpp.base_url http://localhost:8080

Once configured, local models behave like any other provider.

You can switch between cloud and local inference using the same commands without modifying prompts or workflows.

Verifying Provider Health

After configuring providers, run the diagnostic utility.

hermes doctor --fix

The doctor command verifies:

  • configuration syntax
  • environment variables
  • provider authentication
  • network connectivity
  • model availability
  • local inference endpoints
  • credential permissions
  • cached configuration

When possible, the --fix option automatically repairs common problems such as missing configuration directories, invalid cache files, or outdated defaults.

Typical output resembles:

✔ Configuration
✔ Credentials
✔ Provider Connectivity
✔ Model Access
✔ Local Endpoints
✔ Cache
✔ Default Model

System is healthy.

If a provider cannot be reached, Hermes reports the failing step and suggests corrective actions.

Recommended Workflow

For most users, the following sequence provides the smoothest onboarding experience.

  1. Run hermes setup --portal.
  2. Verify with hermes doctor --fix.
  3. Select a preferred model using hermes model.
  4. Configure fallback providers.
  5. Add additional credential pools as needed.
  6. Enable local models for offline work.
  7. Periodically rerun diagnostics after changing providers.

This approach keeps your configuration simple while allowing it to scale as your workflows become more sophisticated.

Companion Guides

Continue with these related guides:

  • Installation Guide
  • First Chat
  • Project Configuration
  • Model Selection
  • Agent Profiles
  • Authentication
  • CLI Reference
  • Troubleshooting
  • Local Development
  • Configuration Reference

Together, these guides cover the complete Hermes Agent setup process from installation through advanced multi-provider deployments.

External Documentation

Official provider documentation:

Review the provider-specific documentation for supported models, authentication methods, rate limits, pricing, and advanced capabilities before deploying production workloads.

Summary

Hermes Agent provides a unified experience across hosted and local LLM providers. Whether you authenticate through the Nous Portal, configure API keys manually, or run models on your own hardware, the same commands continue to work. Credential pools improve reliability, fallback chains reduce downtime, interactive model selection simplifies experimentation, and the built-in diagnostics help ensure every provider remains healthy. With a single configuration layer abstracting differences between providers, you can focus on building workflows instead of managing infrastructure.

HERO_IMAGE_PROMPT: A modern developer workstation showing Hermes Agent managing multiple connected LLM providers through a clean dashboard, with cloud providers (OpenAI, Anthropic, OpenRouter, DeepSeek, Groq, Google, Mistral, Together, Cerebras) connected alongside local Ollama and llama.cpp instances, credential pools flowing into fallback chains, terminal windows displaying hermes setup --portal, hermes model, and hermes doctor --fix, minimalist dark theme, subtle neon blue and purple accents, high-end technical documentation illustration.