Alex's Hermes Agent exemplifies the shift towards secure, controllable AI assistants

In his quest to build a personalised and secure AI assistant, Alex integrates Hermes Agent with local and cloud models, emphasising control, security, and practical deployment on consumer hardware, signalling a broader shift in AI agent development.

Rather than treating an AI assistant as a trusted appliance on his home network, Alex has built the first version of his Hermes Agent setup as a segregated workload: a dedicated virtual machine, a Telegram front end and a split between local and cloud models. The point, as he sets it out, is not simply to self-host a chatbot, but to create a controllable agent that can eventually interact with tools, retain context and help with recurring technical and personal tasks without being given unrestricted access from day one.

That choice of software matters. OpenRouter’s documentation describes Hermes Agent as an open-source, terminal-native coding and task agent from Nous Research with persistent memory, agent-created skills and a messaging gateway that works across more than 21 platforms, including Telegram, Discord, Slack, WhatsApp, Signal, SMS and Matrix. OpenRouter’s own tutorial adds that Hermes ships with more than 40 built-in tools for jobs such as web search, browser automation and image understanding, and can run not only on a local machine but through Docker, SSH, Daytona, Modal, Vercel Sandbox and Singularity. In other words, the Telegram bot in Alex’s design is only the surface; the underlying system is built to orchestrate models, memory and tools rather than just answer prompts.

The cloud side of that arrangement is handled through OpenRouter, which is central to Alex’s hybrid approach. According to OpenRouter’s tutorial, one key gives Hermes access to more than 400 models from over 70 providers, with a single activity dashboard for requests, costs and token usage and automatic failover when a provider returns server errors or rate limits. The same guide says Hermes generally needs a model with at least a 64K-token context window, because smaller windows can be swallowed by the system prompt and tool schemas before useful work even begins. For a basic setup, OpenRouter recommends ~anthropic/claude-sonnet-latest; for automatic model selection it points users to openrouter/auto; and for coding-focused work it offers openrouter/pareto-code, which routes to the cheapest model that still clears a chosen quality threshold. OpenRouter also says it offers more than 25 free models, making a low-cost hobby deployment plausible.

The local side is less about novelty than about control. Hermes’s FAQ says API calls go only to the configured provider and that the software itself does not collect telemetry, usage data or analytics. Conversations, memory and skills are stored locally under ~/.hermes/. The same documentation explains how to point Hermes at a local OpenAI-compatible endpoint such as Ollama, vLLM, llama.cpp server, SGLang or LocalAI by selecting a custom provider, with an example configuration using http://localhost:11434/v1 and a qwen3.5:27b model. It also makes clear that Hermes expects a 64,000-token context minimum and that the configured context length should match the server’s real limit. That detail is important for anyone trying to reproduce Alex’s local experiments on consumer hardware rather than assume any local model will work out of the box.

The practical mechanics are more configurable than Alex’s post initially suggests. Hermes’s configuration reference shows that ~/.hermes/config.yaml can define the default model, provider and base URL, allowing the same agent to target OpenRouter or a custom local endpoint. It lists local, ssh, docker, singularity, modal and daytona as terminal back ends, and sets agent.max_turns to 90 by default, described as the maximum number of tool-calling iterations in a conversation. The documentation recommends 20 to 30 turns for focused tasks and 50 to 100 for open exploration, a reminder that an agent’s usefulness and its token bill rise together. It also exposes reasoning levels from none to xhigh, alongside an approvals mode that defaults to manual, reinforcing Alex’s view that agent autonomy should be metered rather than assumed.

That security instinct becomes more significant when the conversation turns to standards such as the Model Context Protocol. The MCP specification published on 18 June 2025 defines three core primitives: prompts, resources and tools. It also states that users must explicitly consent to and understand data access and operations, that hosts must not transmit resource data elsewhere without user consent, and that tool invocation requires explicit approval because tools amount to arbitrary code execution. The specification says MCP is inspired by the Language Server Protocol, which helps explain why infrastructure-minded users see it as a route to a broader ecosystem. It also underlines the main tension in Alex’s project: as soon as an agent can read external content and operate tools, prompt injection and trust boundaries become design problems rather than abstract warnings.

The hardware constraint driving the rest of the build is equally concrete. Alex is trying to do serious local inference on a machine with an RTX 4090 and 24GB of VRAM, which makes quantisation a practical necessity rather than an academic curiosity. Hugging Face’s documentation on Activation-aware Weight Quantization, or AWQ, says the method preserves a small fraction of the weights most important to large language model performance while compressing a model to 4-bit precision with minimal degradation. That helps explain the trade-offs Alex is working through between capability, context length, performance and memory footprint. A local-first experiment of this kind is really a sizing exercise: how much model can fit, how responsive it remains and which tasks still need to be pushed to the cloud.

For now, the most interesting part of the project may be its restraint. Hermes’s own FAQ distinguishes memory, which stores facts about a user and their projects, from skills, which store repeatable procedures. It also notes that the messaging gateway can be exposed to multiple users through allowlists and DM pairing. Combined with OpenRouter’s claim that Hermes had processed more than 17 trillion tokens on its platform by June 2026 and ranked first on its app league table by token usage, that suggests Alex is building on software that is already being used heavily, not a toy framework. His proposed end uses, from route checking to homelab monitoring and automation, are deliberately ordinary. That may be the clearest sign of where this category is heading: away from spectacle, and towards agents that are only as trusted, connected and expensive as each job requires.