A shared “HIVE” that learns incident → fix patterns across a vendor’s whole deployment fleet from sanitized fingerprints, never raw logs — so the 500th site never rediscovers a bug the 1st already fixed.
A vendor ships one product (an HMS, ERP, GitLab-style platform) into hundreds of on-prem or regulated environments. Each deployment breaks alone; logs can’t cross the customer boundary, so Site A, B and C re-diagnose the same incident, and support cost and MTTR grow with fleet size.
Healthcare (Bahmni, Aarogya), financial and government vendors run identical software in walled-off environments.
Raw logs carry Aadhaar numbers, health IDs, patient records. Compliance keeps them in the deployment.
Without raw logs to compare, nothing carries the fix from one site’s incident to the next site’s identical one.
The hero diagram above is the full design. The key idea: the sanitization boundary and the matching boundary are the same line. Every deployment runs the same software, so the fields that enable cross-fleet matching — service, error class, version, fault taxonomy — are exactly the fields that carry no PII.
Seven independent methods — z-score, change-point, Elastic ML jobs, a supervised classifier and combinations — validated against the Elastic OpenTelemetry “Astronomy Shop” with injected faults.
Elastic ML CATEGORIZE strips variable tokens; Comprehend redacts what remains. Only the derived, client-agnostic fingerprint crosses.
Shared Elasticsearch index with ELSER + RRF hybrid retrieval, exposed as seven MCP tools. Same tools serve two SKUs.
FastAPI + React app that files the Jira ticket, provisions both Elastic↔Jira connectors and detects duplicates.
Watch one incident get reduced to the only thing allowed to leave the deployment. (Illustrative example.)
Jira is the UI and the state machine. Click a step (or let it auto-play). Payloads shown are illustrative.
A Detector flags a deployment. The Remediator files a Jira ticket that carries the sanitized fingerprint, and warns about likely duplicates found in the synced Jira index.
Fingerprints are matched against prior incidents across the whole fleet using hybrid retrieval (ELSER + keyword, fused with RRF) — same software everywhere, so service, error class and version line up.
Ranked fixes + runbooks + a confidence score and why it matched, posted back as a comment. Suggestion-only — nothing is applied automatically.
The engineer pings back with new info, corrects the agent, or confirms. Only a human moves the ticket to COMPLETE. Jira is the state machine — there is no custom workflow engine.
On validation the agent upserts a runbook and links it to the anomaly, incident, remediation, ticket and author. Each validated fix makes the next search better; later matches need less reasoning, not more.
Elastic has two unrelated “connectors”. Only the Kibana .jira action connector can create a Jira issue — and it is licence-gated to Gold, discontinued for new customers. So PROBE runs both paths behind one interface and picks at runtime.
| Content connector | Action connector | |
|---|---|---|
| Direction | Jira → Elasticsearch | Elasticsearch → Jira |
| Purpose | index tickets for search / dedupe | create tickets |
| Lives on | Elasticsearch | Kibana |
| Licence | Basic (free) | Gold |
BROWSE_PROJECTS + CREATE_ISSUESrespx standing in for ES, Kibana and JiraHonest numbers from DETECTORS.md: flags detected out of 11 injected faults on the demo fleet. The supervised classifier is not shown — it reports 55% test accuracy (a different metric). The two-tier v2/v3 reruns improved recall but were only partially re-measured.
The HIVE is a store plus these seven MCP tools.
Vendor imports the HIVE as an MCP tool into their own Agent Builder + LLM. Best reasoning and privacy posture; PROBE is infrastructure, not a product they adopt wholesale.
We run the agent; the vendor calls one turnkey diagnose(). More generic, faster to adopt, no agent stack needed.
| MCP tool | What it does |
|---|---|
match_incident | Find prior HIVE incidents with a matching symptom skeleton |
get_resolution | Fetch the validated fix / runbook for a matched incident |
blast_radius | PRs/commits in the version’s date window that likely caused it |
fleet_signal | Are other deployments seeing the same symptom right now? |
check_version_risk | Score a version by known incident history before rollout |
submit_outcome | Record whether a suggested fix actually worked |
upsert_runbook | Create / update the runbook a validated fix links to |
causal-changepoint-detection/’s heuristic ranking is the stand-in. Target cost ≈ $0.10 per incident and falling as the hive learns; suggestion-only, human-in-the-loop.