What tools and projects do you use for your local LLM chats and agents? I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.
What tools and projects do you use for your local LLM chats and agents? I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.
Hermes agent (actually two: one running Qwen3.8-27b locally via llama.cop on an RTX 3090 running Ubuntu server with access to my personal stuff as an assistant but no internet access to avoid personal info or secrets egress and another running Deepseek 4.1 Flash 1017/GPT-6.1 Sol - for tougher tasks - served via OpenRouter with full web access but no access to personal info and secrets). They both run in docker containers with limited access to host. I also use squid or caddy reverse proxies to protect against unauthorized web or network access. I am the bridge between them when needed.