Arena — Model Battleground
Comparing LLMs head-to-head on Hermes Agent content generation, design, and engineering tasks. Track scores, reliability, and match outcomes.
Leaderboard
| # | Model | Score | Matches | W/L | Best |
|---|---|---|---|---|---|
| 🥇 | deepseek-v4-flashdeepseek | 19 | 7/12 | 9.4 | |
| 🥈 | mimo-v2.5xiaomi | 18 | 4/14 | 9.3 | |
| 🥉 | poolside-laguna-s-2.1openrouter | 16 | 4/12 | 9.1 | |
| #4 | glm-5.3-flashzai | 3 | 2/1 | 9.4 | |
| #5 | poolside/laguna-s-2.1openrouter | 2 | 0/2 | 9.2 |
Model Pool
| Role | Model | Provider | Family | Score | Availability |
|---|---|---|---|---|---|
| Builder | deepseek-v4-flash | deepseek | deepseek | 8.4 | Stable |
| Builder | mimo-v2.5 | xiaomi | xiaomi | 4.6 | Stable |
| QA | glm-5.2 | zai | glm | 6.6 | Stable |
| Judge | qwen3.7-flash | openrouter | qwen | -- | Unknown |
| Builder | poolside-laguna-s-2.1 | openrouter | poolside | -- | Unknown |
| Research | openai/gpt-5.6-luna | openrouter | openai | -- | Unknown |
| Planner | mimo-v2.5-pro | xiaomi | xiaomi | -- | Stable |
| Builder | glm-5.3-flash | zai | - | -- | Unknown |
Match History
▶Hermes Agent Web Dashboard beginner setup guide (long-tail 'hermes agent web dashboard') — Monday tutorial guide; source = official dashboard docs + CLI reference + install/platform/Nous Portal docs, all HTTP 200 verified 2026-09-14tutorialdeepseek-v4-flash8.3glm-5.3-flash8.7mimo-v2.57.9poolside-laguna-s-2.16.7⚖ deepseek-v4-flash2026-09-148.3/10
Date: 2026-09-14
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek8.3/10
🥈glm-5.3-flashzai8.7/10
🥉mimo-v2.5xiaomi7.9/10
#4poolside-laguna-s-2.1openrouter6.7/10
Strengths
- Highest stable mean of the three full-plan drafts (8.35, spread 0.90 across 3 pure-qwen passes)
- Densest sourcing: 14 links to the official docs host plus Tailscale KB, all HTTP 200 verified
- Full requirement coverage: question-shaped H2, 40-60 word H2 leads, FAQ with 3 question H3s, E-E-A-T block, Common mistakes, hub + sibling internal links
- Only builder to be rescued from the documented zai stall: the full-plan brief (11.5K chars) returned no draft after 420s x2, then a 3.2K-char condensed standalone brief returned 1,149 words in 269s
- Clean structure, honest E-E-A-T block, FAQ present
▶Beginner's Guide to Hermes WhatsApp: Getting Started in 2026 (Baileys bridge + WhatsApp Business Cloud API setup) — Monday tutorial guidetutorialglm-5.3-flash8.4mimo-v2.58.0poolside-laguna-s-2.17.9deepseek-v4-flash7.8⚖ glm-5.3-flash2026-09-078.4/10
Date: 2026-09-07
Winner: glm-5.3-flash
Results:
🥇glm-5.3-flashzai8.4/10
🥈mimo-v2.5xiaomi8.0/10
🥉poolside-laguna-s-2.1openrouter7.9/10
#4deepseek-v4-flashdeepseek7.8/10
Strengths
- Only draft to clear the 8.0 gate (R1 7.35 → R2 8.40 after prompt-refined redo)
- Pure-qwen confirm judge 8.40 (3 clean passes, no fallback) on top of 8.57 mixed-fallback 2-pass mean — consistent across judges/passes
- Verbatim honesty canon, dense inline citations, question-shaped H2, FAQ, clean frontmatter
- Most consistent builder across rounds (R1 7.87 → R2 7.98)
- Strong practicality every pass
▶Hermes Agent Vision & Image Analysis: how to analyze images with Hermes Agent (CLI /paste command, vision_analyze tool, auxiliary.vision config, platform requirements, SSH gotchas) — Monday tutorial guide (Week rotation)tutorialdeepseek-v4-flash7.9mimo-v2.57.9poolside-laguna-s-2.17.3glm-5.3-flash6.4⚖ deepseek-v4-flash2026-08-317.9/10
Date: 2026-08-31
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek7.9/10
🥈mimo-v2.5xiaomi7.9/10
🥉poolside-laguna-s-2.1openrouter7.3/10
#4glm-5.3-flashzai6.4/10
Strengths
- Cleanest draft: proper frontmatter, canonical honesty line verbatim, complete structure (E-E-A-T block, question-shaped H2, 40-60 word answers, FAQ with 3 H3s, 10 internal + 4 unique external inline links)
- Selected as winner: QA-fixed and published; all config blocks verbatim from official docs; every claim traceable to a 200-verified source
- Highest raw judge mean (7.90) with tightest spread (0.50) — most reliable score this match
- Correct /blog/ internal link paths from the start
▶Beginner's Guide to Hermes Slack: Getting Started in 2026 (integration guide, long-tail 'how to connect Hermes Agent to Slack for beginners')integration-guideglm-5.3-flash9.4mimo-v2.59.3deepseek-v4-flash8.7poolside-laguna-s-2.18.7⚖ glm-5.3-flash2026-08-289.4/10
Date: 2026-08-28
Winner: glm-5.3-flash
Results:
🥇glm-5.3-flashzai9.4/10
🥈mimo-v2.5xiaomi9.3/10
🥉deepseek-v4-flashdeepseek8.7/10
#4poolside-laguna-s-2.1openrouter8.7/10
▶Hermes Agent Tool System Internals: How Tools Load and Run (research deep-dive)research-deep-divemimo-v2.58.4deepseek-v4-flash8.3poolside-laguna-s-2.18.0⚖ mimo-v2.52026-08-278.4/10
Date: 2026-08-27
Winner: mimo-v2.5
Results:
🥇mimo-v2.5xiaomi8.4/10
🥈deepseek-v4-flashdeepseek8.3/10
🥉poolside-laguna-s-2.1openrouter8.0/10
Strengths
- Highest judge mean (8.37) with a 9.35 second pass
- Accurate registry/dispatch coverage across passes
- Consistent 8.15-8.40 across all passes
- Thorough (1434 words)
▶Hermes Agent vs Cursor: Which AI Agent for Beginners in 2026? (comparison/tutorial)comparisonpoolside/laguna-s-2.18.2deepseek-v4-flash8.2mimo-v2.57.7⚖ poolside-laguna-s-2.12026-08-258.2/10
Date: 2026-08-25
Winner: poolside-laguna-s-2.1
Results:
🥇poolside/laguna-s-2.1openrouter8.2/10
🥈deepseek-v4-flashdeepseek8.2/10
🥉mimo-v2.5xiaomi7.7/10
Strengths
- Highest judge mean (8.18); accurate model/pricing coverage across passes
- Verdict-first H2, clean table, honest E-E-A-T block
- Strongest draft structure (FAQ H3s, 10 internal links, inline citations)
- Pass detail truncated from log tail
▶How to compare AI models with Hermes Agent — Beginner's Guide to Hermes Agent's Arena (subagent arena A/B/C pipeline tutorial)tutorialpoolside/laguna-s-2.19.2mimo-v2.59.1deepseek-v4-flash8.6⚖ poolside-laguna-s-2.12026-08-248.6/10
Date: 2026-08-24
Winner: poolside-laguna-s-2.1
Results:
🥇poolside/laguna-s-2.1openrouter9.2/10
🥈mimo-v2.5xiaomi9.1/10
🥉deepseek-v4-flashdeepseek8.6/10
Strengths
- Most polished structure with verdict-first opening and clean H2 flow
- Strong factual coverage, perfect sourcing scores across passes
▶Hermes Agent v0.20.5 release (Aug 19, 2026): keyless web search, Bot Mode group threads, CLI polish, cron memory - what's new and how to updatenewspoolside-laguna-s-2.19.0deepseek-v4-flash8.5mimo-v2.58.5⚖ poolside-laguna-s-2.12026-08-228.5/10
Date: 2026-08-22
Winner: poolside-laguna-s-2.1
Results:
🥇poolside-laguna-s-2.1openrouter9.0/10
🥈deepseek-v4-flashdeepseek8.5/10
🥉mimo-v2.5xiaomi8.5/10
Strengths
- Structurally complete: proper frontmatter, woven inline citations, internal links, 1042 words
- Consistent accuracy/sourcing (9-10) across passes
▶Beginner's Guide to Hermes Discord Bot 2026 (how to connect Hermes Agent to Discord: 8-step bot setup, gateway env config, troubleshooting)tutorialdeepseek-v4-flash7.4mimo-v2.56.7poolside-laguna-s-2.16.2⚖ deepseek-v4-flash2026-08-217.4/10
Date: 2026-08-21
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek7.4/10
🥈mimo-v2.5xiaomi6.7/10
🥉poolside-laguna-s-2.1openrouter6.2/10
Strengths
- Highest judge mean across all 3 iterations (r1 5.98 -> r2 7.43; r3 6.40 on near-identical content = qwen noise)
- Complete structure: E-E-A-T block, question-shaped H2, 40-60 word answers, FAQ with 3 question H3s, 8 internal + 12 external inline links
- Best iteration = r2 (6.72); consistent accuracy (7-8) across passes
▶How Hermes Agent Memory Works: Persistent Memory Internals Deep Dive (two-file frozen-snapshot model, FTS5 session search, memory providers, background review)deep-divemimo-v2.57.9deepseek-v4-flash7.8poolside-laguna-s-2.17.2⚖ mimo-v2.52026-08-207.9/10
Date: 2026-08-20
Winner: mimo-v2.5
Results:
🥇mimo-v2.5xiaomi7.9/10
🥈deepseek-v4-flashdeepseek7.8/10
🥉poolside-laguna-s-2.1openrouter7.2/10
Strengths
- Highest judge mean (7.93) across both iterations; accuracy 9 on every pass
- Complete structure: verdict-first opening, E-E-A-T block, architecture diagram, comparison table, FAQ, internal links
- Cleanest iteration-2 draft (r1 mean 7.55 -> r2 7.77)
- Strong accuracy/sourcing (8-9); no accuracy defects in QA review
▶Hermes Agent vs Cline: Which AI Coding Agent Should Beginners Use in 2026? (head-to-head comparison, 10-row table, verdict per beginner persona)comparisonmimo-v2.59.0poolside-laguna-s-2.18.8deepseek-v4-flash8.3⚖ mimo-v2.52026-08-189.0/10
Date: 2026-08-18
Winner: mimo-v2.5
Results:
🥇mimo-v2.5xiaomi9.0/10
🥈poolside-laguna-s-2.1openrouter8.8/10
🥉deepseek-v4-flashdeepseek8.3/10
Strengths
- Highest judge mean (8.98) with 10/10 accuracy and sourcing on every pass
- Complete structure: verdict-first opening, E-E-A-T block, 10-row table, question-shaped decision H2, FAQ, 5 internal links
- Canonical honesty phrasing present; all external sources verified HTTP 200
- Strong accuracy and sourcing (10s and 9s)
- Concise (1152 words) with clean structure
▶Hermes Agent vs OpenHands: Open-Source AI Agent Guide (head-to-head comparison, 9-dimension table)comparisonpoolside-laguna-s-2.19.1mimo-v2.58.9deepseek-v4-flash8.4⚖ poolside-laguna-s-2.12026-08-179.1/10
Date: 2026-08-17
Winner: poolside-laguna-s-2.1
Results:
🥇poolside-laguna-s-2.1openrouter9.1/10
🥈mimo-v2.5xiaomi8.9/10
🥉deepseek-v4-flashdeepseek8.4/10
Strengths
- Highest judge mean (9.07)
- All 10 external sources verified HTTP 200
- Complete structure: verdict-first opening, E-E-A-T block, 9-row table, question H2, FAQ, 4 internal links
- Strong sourcing (10s on passes 1 and 3)
- Clean verdict-first structure with bold recommendations
▶Hermes Gateway Internals: Routing, Fallback, and Failover (research deep-dive of gateway routing, authorization, and provider fallback machinery)deep-divedeepseek-v4-flash8.6poolside-laguna-s-2.18.5mimo-v2.58.4⚖ deepseek-v4-flash2026-08-178.6/10
Date: 2026-08-17
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek8.6/10
🥈poolside-laguna-s-2.1openrouter8.5/10
🥉mimo-v2.5xiaomi8.4/10
Strengths
- Highest judge mean (8.57), tightest pass spread (8.20-8.90)
- Complete structure: verdict-first opening, E-E-A-T block, 2 ASCII diagrams, keyword H2, FAQ
- All 8 external sources verified HTTP 200; 5 internal links to live routes
- Second-highest judge mean (8.53)
- Judge pass 3 fell back to deepseek-v4-flash-0731 after OpenRouter 429 (documented fallback)
- Noisy pass 1 (quality 4.0) — flagged spread 2.15, mean still passed
▶Beginner's Guide to Hermes Local Models 2026 (how to run Hermes Agent with local models — Ollama + llama.cpp setup for beginners)tutorialmimo-v2.58.7deepseek-v4-flash8.5poolside-laguna-s-2.18.2⚖ mimo-v2.52026-08-178.7/10
Date: 2026-08-17
Winner: mimo-v2.5
Results:
🥇mimo-v2.5xiaomi8.7/10
🥈deepseek-v4-flashdeepseek8.5/10
🥉poolside-laguna-s-2.1openrouter8.2/10
Strengths
- Highest judge mean (8.68)
- Perfect accuracy on all 3 passes (10/10)
- QA-verified: all external links HTTP 200, Hugging Face link added
- Second-highest judge mean (8.48)
- Complete structure: verdict-first, E-E-A-T block, keyword H2, FAQ
▶Hermes Agent Browser Use Mode Replaces 12 Browser Tools (new default browser harness in v0.20.1)newsdeepseek-v4-flash9.1mimo-v2.58.8poolside-laguna-s-2.18.6⚖ deepseek-v4-flash2026-08-159.1/10
Date: 2026-08-15
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek9.1/10
🥈mimo-v2.5xiaomi8.8/10
🥉poolside-laguna-s-2.1openrouter8.6/10
Strengths
- Highest judge mean (9.13)
- All facts sourced from verified 200 URLs with descriptive anchors
- Self-contained answer paragraph under every H2
▶Hermes Cloud Preview — Nous Research's managed cloud for Hermes Agent (announced Jul 8 2026, Desktop auto-discovery Jul 10, cold-start fix PR #86477 merged Aug 14)newspoolside-laguna-s-2.18.5deepseek-v4-flash8.4mimo-v2.58.1⚖ poolside-laguna-s-2.12026-08-158.5/10
Date: 2026-08-15
Winner: poolside-laguna-s-2.1
Results:
🥇poolside-laguna-s-2.1openrouter8.5/10
🥈deepseek-v4-flashdeepseek8.4/10
🥉mimo-v2.5xiaomi8.1/10
▶Automate GitHub with Hermes Agent: PR Reviews, Issue Triage, and Cron Reportstutorialdeepseek-v4-flash8.3mimo-v2.57.8poolside-laguna-s-2.17.0⚖ deepseek-v4-flash2026-08-148.3/10
Date: 2026-08-14
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek8.3/10
🥈mimo-v2.5xiaomi7.8/10
🥉poolside-laguna-s-2.1openrouter7.0/10
Strengths
- Highest judge score (8.25 mean after QA fixes)
- Full 14-section structure, E-E-A-T block with canonical honesty phrasing
- All external URLs verified HTTP 200
- Accurate, verified facts
- Good FAQ and internal links
▶Telegram Bot Setup for Hermes Agent: Chat From Your Phone (how to connect your AI agent to Telegram)tutorialdeepseek-v4-flash9.4mimo-v2.59.2poolside-laguna-s-2.18.8⚖ deepseek-v4-flash2026-08-149.4/10
Date: 2026-08-14
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flashdeepseek9.4/10
🥈mimo-v2.5xiaomi9.2/10
🥉poolside-laguna-s-2.1openrouter8.8/10
Strengths
- Highest judge mean (9.45)
- Troubleshooting table covering top failure modes
- All facts sourced from verified 200 URLs with descriptive anchors
- Accurate, verified facts
- Good inline citation density
▶How to Run Hermes Agent in Docker: A Beginner's Guide 2026tutorialdeepseek-v4-flash8.4mimo-v2.58.0poolside-laguna-s-2.10.0⚖ deepseek-v4-flash2026-08-108.4/10
Date: 2026-08-10
Winner: deepseek-v4-flash
Results:
🥇deepseek-v4-flash8.4/10
🥈mimo-v2.58.0/10
🥉poolside-laguna-s-2.10.0/10
Strengths
- Highest judge score (8.37 mean)
- Thorough (1265 words)
- All 8 required sections present
- Passed 8.0 threshold
- Accurate facts with verified sources
▶Hermes Agent v0.20.0 The Herald Release: Voice, A2A Protocol, and Desktop Platformcontentpoolside-laguna-s-2.19.1deepseek-v4-flash8.6mimo-v2.50.0⚖ poolside-laguna-s-2.12026-08-089.1/10
Date: 2026-08-08
Winner: poolside-laguna-s-2.1
Results:
#undefinedpoolside-laguna-s-2.1openrouter9.1/10
#undefineddeepseek-v4-flashdeepseek8.6/10
#undefinedmimo-v2.5xiaomi0.0/10
Strengths
- Cleanest quality scores across all 3 passes (9, 8, 10)
- Excellent accuracy consistency (9-10 across passes)
- Self-contained answers under each H2
- Longest draft (1425 words)
- Perfect accuracy (10/10 all passes)
- Good internal linking to sibling posts
Process Feedback
Automated analysis from arena-process-feedback.py. Tracks brief quality, model compliance, and recurring issues across matches.
Recurring Issues
brief_ambiguity x19hero_image x5empty x3hallucination x3file_overwrite x2
Hermes Agent Web Dashboard beginner setup guide (long-tail 'hermes agent web dashboard') — Monday tutorial guide; source = official dashboard docs + CLI reference + install/platform/Nous Portal docs, all HTTP 200 verified 2026-09-14Clean
- [INFO]Current brief format working well. Consider adding a constraint to test improvement.
Beginner's Guide to Hermes WhatsApp: Getting Started in 2026 (Baileys bridge + WhatsApp Business Cloud API setup) — Monday tutorial guideClean
- [INFO]Current brief format working well. Consider adding a constraint to test improvement.
Arena data last updated: 2026-09-14