AI groups incoming alerts and analyzes the root cause first, then acts by risk level with approval and self-healing, and verifies the result; on-call engineers only confirm and decide.
The challenge
- In an alert storm one fault triggers dozens of alerts, and on-call engineers struggle to find the root cause quickly.
- Night shifts depend on the experience of a few senior engineers, and their troubleshooting is hard to capture.
- Automation scripts lack approval and verification, so nobody dares let them make real changes.
The solution
openaiops collects metrics, logs and alerts; Ruyi OpsBuddy analyzes and acts:
- Noise reduction: Alertmanager / webhook alerts are grouped by topology and time window, with the likely root cause and propagation chain.
- Diagnosis loop: detect → hypothesize → gather evidence → conclude → act (with approval) → verify → learn, with evidence at every step.
- Graded actions: write operations need approval by risk level (L0–L3), with multi-level approval chains and on-call schedules; results are verified automatically, and failures roll back or become tickets for people.
- Knowledge capture: feedback scores, knowledge gaps and experience distillation turn frequent troubleshooting paths into draft skills.
The value
- On-call engineers move from reading alerts one by one to confirming conclusions and approving actions.
- Investigations and conclusions are backed by evidence and can become tickets, group messages and diagnosis reports in one click.
- Without an LLM, the offline engine still handles skill search, alert grouping and root cause, inspections and verification.