Incident Tracking & History

Automatic incident logging

When your alert policies trigger, Sandglass creates an incident automatically. No manual logging, no missed events—every failure is captured with accurate timestamps and context.

• Incidents open automatically when alert thresholds are met
• Track which specific checks are failing within the incident
• Capture failure counts and status transitions
• Resolve incidents automatically when checks recover

Group-level incident management

Sandglass tracks incidents at the group level, not for individual checks. When multiple checks fail together, you see one cohesive incident instead of a flood of disconnected alerts.

This approach gives you a clearer picture of actual outages and makes it easier to understand scope and impact at a glance.

• Single incident for correlated failures across multiple checks
• See which check types are affected (HTTP, ping, port, content, heartbeat)
• Track partial vs complete outages
• Understand cascading failures and dependencies

Rich incident details

Every incident includes the information you need to understand what happened and how long it lasted.

• Status: Open or resolved at a glance
• Start time: Exact timestamp when the incident was detected
• Resolution time: When all checks recovered
• Failure count: Total number of consecutive failures
• Failing checks: Detailed list showing which specific checks are affected, their types, current status, and how long they've been failing

Incident history visibility

View your recent incident history directly in the dashboard. The incident tab shows the last 10 incidents for each group, giving you immediate visibility into reliability trends.

See at a glance which incidents are still open, how many have been resolved, and how frequently outages occur for each service.

Learning from patterns

Use your incident history to spot patterns and improve reliability over time.

• Identify services with frequent incidents that need architectural attention
• Measure incident duration to understand your mean time to recovery (MTTR)
• Correlate incidents with deployments or infrastructure changes
• Track failure count trends to see if stability is improving

Integrated with alerting

Incidents and alerts work together seamlessly. When an incident opens, your configured notification channels are triggered. When it resolves, recovery notifications go out automatically if you've enabled them.

This tight integration means your team always knows the current incident status without manually checking dashboards or parsing through disconnected alerts.

Built for reliability engineering

Sandglass incident tracking is designed for teams that take reliability seriously. Automatic creation, group-level aggregation, and rich context mean you spend less time documenting and more time improving.

Whether you're running a small side project or managing critical infrastructure for thousands of users, having a clear incident history helps you make better decisions about where to invest your time.

Getting started with incident tracking

Incident tracking happens automatically once you configure alert policies for your monitoring groups. Set up your notification channels, define your alert thresholds, and Sandglass handles the rest.

Every time a failure meets your alert criteria, an incident is created. Every time all checks recover, the incident is resolved. You get a complete reliability history without lifting a finger.