Case Study

Distributed Swarm Orchestrator

Event-driven multi-agent command infrastructure orchestrating heavy scraping and systems tasks over private encrypted mesh networks.

Client: David the Gnomad Inc.
  • Python
  • Tailscale
  • Slack API
  • Systemd
  • devops
  • agents
  • mesh-network
  • distributed-systems

The Business Case

Running automation across a mix of home servers, laptops, and cloud nodes often leads to fragmented scripts and security risks. Distributed Swarm Orchestrator creates a secure, private communication network that links these machines into a coordinated swarm. Using a chat interface (Slack Socket Mode) as the controller, you can launch, monitor, and query scrapers, database indexes, and system backups across multiple devices.

System Architecture

The swarm maintains high availability and secure data links without exposing public API ports on the local network.

  ┌────────────────────────────────────────────────────────┐
  │                    Local Private Mesh                  │
  │                                                        │
  │  ┌──────────────┐       ┌──────────────┐               │
  │  │  Ryzen Rig   │ ◄───► │  Cloud VPS   │               │
  │  │ (Ollama LLM) │       │ (File Sync)  │               │
  │  └──────▲───────┘       └──────▲───────┘               │
  └─────────┼──────────────────────┼───────────────────────┘
            │                      │ (Tailscale Encrypted)
            ▼                      ▼
  ┌────────────────────────────────────────────────────────┐
  │                  Slack Command Router                  │
  │                   (Socket Mode API)                     │
  └────────────────────────────────────────────────────────┘

Engineering Highlights

1. Zero-Port Mesh Network

By using Tailscale to build a virtual private mesh, swarm nodes communicate over encrypted WireGuard tunnels. This design allows workers located behind double-NAT routers or firewalls to exchange commands securely without requiring public port forwarding.

2. Socket Mode Orchestration

By listening on the Slack Socket Mode API, worker agents interact with the controller without exposing HTTP server endpoints. Commands are parsed and distributed asynchronously, with system metrics and run logs streamed back in real-time.

3. Automated SOP Skill Libraries

Worker nodes pull markdown-defined Standard Operating Procedures (SOPs) from a central repo. When given a task, the agent parses the SOP, verifies the necessary command tools, and runs the script in a localized subprocess.