Spider Mon

spider-mon is a standalone command-line tool for querying Spider's self-monitoring metrics — the same data the Self-Monitoring GUI shows, exposed as JSON for shell scripting, baseline diffing, and AI-driven investigation. It is a sibling of the Spider CLI and shares the same ~/.spider/profiles.json and ~/.spider/tokens.json.

While spider (the CLI) returns captured traffic (HTTP, PSQL, TCP, packets), spider-mon returns Spider's own health and throughput — parsing pipeline activity, agent states, Redis/Elasticsearch internals, service CPU and latency, and service / GUI logs.

Install

Binaries are published for Linux, macOS and Windows on both amd64 and arm64 architectures at https://repository.floocus.com/bin/spider-mon-<os>-<arch>/<version>/spider-mon.xz (or spider-mon.exe.xz on Windows). Use latest for the most recent version, or a specific tag like 0.1.0 to pin.

Linux / macOS

Auto-detects OS and architecture:

OS=$(uname -s | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m); [ "$ARCH" = "x86_64" ] && ARCH=amd64; [ "$ARCH" = "aarch64" ] && ARCH=arm64
curl -fL "https://repository.floocus.com/bin/spider-mon-${OS}-${ARCH}/latest/spider-mon.xz" | unxz > /usr/local/bin/spider-mon
chmod +x /usr/local/bin/spider-mon

Requires xz-utils (Debian/Ubuntu: apt install xz-utils; macOS: brew install xz).

Windows

PowerShell, Windows 10+ (uses the bundled tar which handles .xz):

$arch = if ($Env:PROCESSOR_ARCHITECTURE -eq 'ARM64') { 'arm64' } else { 'amd64' }
Invoke-WebRequest "https://repository.floocus.com/bin/spider-mon-windows-$arch/latest/spider-mon.exe.xz" -OutFile spider-mon.exe.xz
tar -xf spider-mon.exe.xz
# Then move spider-mon.exe to a directory on your PATH.

Available platforms

OS	amd64	arm64
Linux	`spider-mon-linux-amd64/<v>/spider-mon.xz`	`spider-mon-linux-arm64/<v>/spider-mon.xz`
macOS	`spider-mon-darwin-amd64/<v>/spider-mon.xz`	`spider-mon-darwin-arm64/<v>/spider-mon.xz`
Windows	`spider-mon-windows-amd64/<v>/spider-mon.exe.xz`	`spider-mon-windows-arm64/<v>/spider-mon.exe.xz`

Binaries are statically linked (built with CGO_ENABLED=0) and have no runtime dependencies — the Linux builds run on glibc and musl (Alpine) alike.

Profiles

spider-mon reads the same profile file as the Spider CLI. If you've already configured a profile for spider, no extra setup is needed. Otherwise:

spider add profile prod \
  --api-url https://your-spider-instance \
  --auth-type service_account \
  --client-id <client_id> \
  --client-secret <client_secret>

Override the active profile per command with --profile <name>.

Quick start

# Snapshot all monitoring pages, summary only, last hour
spider-mon status --pretty

# Drill into one page — all graphs, summary only
spider-mon parsing

# Same page, only specific graphs, with time-series data
spider-mon parsing queues parsers --full --pretty

# Search the last hour of warn-and-above service logs
spider-mon search logs --pretty

# Errors only, just the Pg-* parsers, last 24 hours
spider-mon search logs --service Pg --level error --start -24h --size 200

Pages

Each page returns a {time, interval_min, ...graphs} JSON envelope. Pass graph names as positional arguments to keep only those graphs; pass --full to include the time-series array (otherwise only {avg, max, last} summaries are returned).

Page	What it shows
`health`	Probe statuses (no time range)
`parsing`	TCP / HTTP / PSQL parsing pipeline metrics
`services`	Service CPU, RAM, API latency, polling queues
`whisperers`	Whisperer agent states, CPU / RAM, queue overflow, latency
`gociphers`	Gocipher agent states, CPU / RAM, queue overflow, latency
`redis`	Redis instance internal view: keys, memory, commands/sec, CB latency
`elasticsearch`	Elasticsearch per-index content, size, load, CPU, CB latency
`infra`	Host nodes, Kubernetes workloads, Redis / Elasticsearch external view
`cb`	Circuit-breaker errors across whisperers, gociphers, services, datastores
`logs`	Error / warning log counts by service and by service / code
`status`	All pages in parallel — summary-only rollup, no positional args, no `--full`

Run <page> --help for that page's graph list.

Search (raw items)

Pages aggregate. search returns raw _source documents from one of the log indices. Output is {total, items[]} on stdout — same body shape regardless of subcommand.

Subcommand	Source
`search logs`	Service logs (Filebeat-shipped container stdout/stderr)
`search guilogs`	GUI logs (Network-View, Login, Self-Monitoring)

# Default: warn-and-above, last hour, newest first
spider-mon search logs

# All Pg-* parsers, errors only, last 24 hours, up to 200 items
spider-mon search logs --service Pg --level error --start -24h --size 200

# GUI errors from Network-View only
spider-mon search guilogs --level ERROR --app Net

# Combine with jq to project the fields you care about
spider-mon search logs --service Pg-Parser --level error \
  | jq -c '.items[] | {t:."@timestamp", code:.json.code, msg:.json.msg}'

Shared flags: --start (default -1h), --stop (default now), --query (extra Lucene clause), --size (default 50, no client-side cap — server paginates 99 items at a time).

--service and --app are prefix matches. On a mid-pagination failure the partial result is still emitted to stdout, with a warning on stderr and a non-zero exit.

Baselines

Snapshot a known-good state, then compare future runs against it.

# Snapshot the full status before an upgrade
spider-mon status --start -1h --save pre-upgrade

# After the upgrade, diff fresh metrics against the snapshot
spider-mon status --start -1h --compare pre-upgrade

# Slice a baseline to compare only a single graph
spider-mon parsing queues --start -1h --compare pre-upgrade

# Manage stored baselines
spider-mon baseline list
spider-mon baseline delete pre-upgrade

Baselines live at ~/.spider/baselines/ (override with SPIDER_BASELINES_DIR). --save always writes a full-page baseline; --compare slices it to match the live shape (canonical-order key set, series stripped if the live side ran in summary mode), then runs a threshold-filtered diff.

Common flags

All page commands accept:

--start — start time (ISO 8601 or relative like -1h, -24h, or now)
--stop — stop time (default: now)
--interval / -i — bucket size in minutes (0 = auto: ~50 buckets across the range)
--full — include time-series data (summary-only by default)
--save / --compare — baseline write / diff
--pretty — human-readable JSON output

Global:

--profile <name> — use a specific profile instead of the default

Repository

Gitlab

Install​

Linux / macOS​

Windows​

Available platforms​

Profiles​

Quick start​

Pages​

Search (raw items)​

Baselines​

Common flags​

Repository​