"My worst fear is that it will do something." That's the honest reply in every homelab thread about AI agents, and it's the right instinct. An agent that can restart services can also delete the wrong volume. The answer isn't to never let it in. It's to let it in the way you'd let in a new colleague: with their own account, read-only for a while, and a backup before the first real change.
Here's the week I'd give it.
Before day 1: its own account
- Create a dedicated user on each box (
aior similar). Not root, not your account. - Give it an SSH key or, if the tool uses its own agent, install that agent under this user.
- Reach the boxes over the LAN or Tailscale/WireGuard only. No new open ports.
- Leave sudo off for now.
If something goes wrong, you'll know whose account did it, and you can lock it in one command.
Days 1–3: read-only
Let it look, not touch. These are the jobs where an AI saves real time without any risk:
- "Which containers restarted in the last 24 hours, and why?" (
docker ps,docker logs) - "How full are the disks, and what grew this week?"
- "Any SMART warnings?"
- "Read the last 200 lines of the Nextcloud log and tell me what the errors have in common."
Ask it to show the command and output behind every answer. You're checking two things: that it reads your setup correctly, and that its explanations match what you'd have concluded.
Day 4: snapshots before anything else
Before the first change, make undo cheap. Snapshot the VMs (Proxmox makes this one click), or the ZFS/Btrfs datasets, and confirm last night's backup actually restores. Keep the backup target out of the agent's reach: different user, different box, or read-only to it.
Days 5–6: small, reversible changes
Now give it sudo for specific commands, not everything (sudoers lets you list them): restart a named service, prune old images, rotate logs. Good first changes:
- "Prune dangling Docker images on the media box and tell me how much space came back."
- "Restart the Jellyfin container and check it answers on port 8096."
Make it say what it will do before it does it. Ask "what will this change and how do I undo it?" and read the answer.
Day 7: playbooks, not ad-hoc commands
For anything you'll repeat, have the AI write an Ansible playbook or a script that you review, commit and run yourself. The chat is great for investigating and one-offs; repeatable changes belong in something you can read and rerun.
Where a chat tool fits
Most homelab AI setups are one model plus SSH. That works. What I wanted on top was one chat that knows about all the boxes, so "which server needs attention?" gets one answer instead of five terminal windows.
I build Local Commander, so I'm biased. It puts a small agent on each machine; ChatGPT or Claude then works across all of them from one conversation. Per machine, you choose full access for that account or only selected folders, block commands you never want run, and revoke a machine instantly.
Limits, plainly
- No tool here is a sandbox. The agent can do what its account can do, which is why the account matters more than the tool.
- Every box needs the agent (and Node.js 20+), and the box has to be online.
- It works through commands and files, not a screen.
- The model can be wrong. The read-only days are there so you find out how often before it matters.
Your snapshots and backups are the real safety net. Everything above is about needing them less often.