Incidents

correlation Package

Overview

Package correlation provides intelligent incident correlation and deduplication for the incident management platform.

This package implements the correlation system that automatically identifies relationships between incidents, detects duplicates, and reduces noise in incident management workflows. It uses pattern matching, similarity algorithms, and configurable rules to correlate incidents based on timing, content, service impact, and other contextual factors.

Key Features:

  • Intelligent duplicate incident detection with configurable similarity thresholds
  • Time-based correlation for related incidents occurring in proximity
  • Content-based correlation using text similarity and pattern matching
  • Service-based correlation for incidents affecting the same services
  • Configurable correlation rules and threshold management
  • Automated incident grouping with correlation reasoning
  • Comprehensive correlation result storage and audit trails
  • Integration with timeline service for correlation event logging

Architecture:

The correlation system follows a multi-stage analysis approach:

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ New Incidents   │───►│ Correlation      │───►│ Action          │
│ (Processing)    │    │ Analysis         │    │ Execution       │
└─────────────────┘    └──────────────────┘    └─────────────────┘

      │                        │                        │
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ Pattern         │    │ Similarity       │    │ Result Storage  │
│ Matching        │    │ Scoring          │    │ (Database)      │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Correlation Types:

  • Deduplication: Identifying exact or near-exact duplicate incidents
  • Correlation: Grouping related incidents for coordinated response
  • Suppression: Hiding redundant incidents to reduce noise
  • Merging: Combining duplicate incidents into single entities

Correlation Dimensions:

  • Time Correlation: Incidents occurring within configurable time windows
  • Service Correlation: Incidents affecting the same services or components
  • Content Correlation: Text similarity in titles, descriptions, and metadata
  • Pattern Correlation: Matching against known incident patterns and signatures

Example usage:

// Create correlation service
correlationService := correlation.NewService(db, timelineService)

// Process new incident for correlation
incident := &models.Incident{
	ID:          "INC-123",
	Title:       "Database Connection Timeout",
	Description: "Users unable to connect to primary database",
	Service:     "database-service",
	Severity:    models.SeveritySEV2,
	CreatedAt:   time.Now(),
}

result, err := correlationService.ProcessIncident(ctx, incident)
if err != nil {
	log.Fatal(err)
}

// Check correlation results
switch result.Action {
case ActionDeduplicate:
	fmt.Printf("Incident %s is a duplicate of %v\n",
		incident.ID, result.CorrelatedWith)
case ActionCorrelate:
	fmt.Printf("Incident %s is related to group %s\n",
		incident.ID, result.GroupID)
case ActionNone:
	fmt.Printf("No correlation found for incident %s\n", incident.ID)
}

// Get correlation statistics
stats, err := correlationService.GetCorrelationStats()
if err != nil {
	log.Fatal(err)
}
fmt.Printf("Correlation rate: %.2f%%, Deduplication rate: %.2f%%\n",
	stats["correlation_rate"], stats["deduplication_rate"])

Correlation Engine: The correlation engine processes incidents through multiple stages:

  1. Time Window Analysis: Check for incidents in temporal proximity
  2. Service Impact Analysis: Identify incidents affecting same services
  3. Content Similarity: Compare titles, descriptions, and metadata
  4. Pattern Matching: Match against known incident signatures
  5. Confidence Scoring: Calculate correlation confidence levels
  6. Action Determination: Decide appropriate correlation action

Configuration: Correlation behavior is controlled through configurable parameters:

  • Correlation Window: Time window for temporal correlation (default: 15 minutes)
  • Similarity Threshold: Minimum similarity score for correlation (default: 0.8)
  • Max Group Size: Maximum incidents per correlation group (default: 50)
  • Enable Flags: Toggle different correlation dimensions

Correlation Actions:

  • ActionNone: No correlation found, process normally
  • ActionDeduplicate: Mark as duplicate and close
  • ActionCorrelate: Add to correlation group for coordinated response
  • ActionSuppress: Suppress redundant incident to reduce noise
  • ActionMerge: Merge with existing incident (advanced deduplication)

Result Storage: All correlation decisions are stored with detailed context:

  • Correlation reasoning and confidence scores
  • Rule matches and similarity calculations
  • Incident fingerprints for future correlation
  • Group membership and relationship tracking
  • Audit trails for correlation decision analysis

Import Path: github.com/systmms/incidents/internal/correlation

Types

Action

Action defines what to do when a rule matches

{<nil> 3932 type 0 [0x1400021acc0] 0}

ActionType

ActionType defines the type of action to take

{<nil> 4109 type 0 [0x1400021adc0] 0}

Constants 

const ActionCorrelate const ActionDeduplicate const ActionSuppress const ActionMerge const ActionNotify

Adapter

Adapter implements the timeline.Correlator interface

{<nil> 157 type 0 [0x14000207800] 0}

Methods

NewAdapter

NewAdapter creates a new correlation adapter

{<nil> <nil> NewAdapter 0x140004a7240 <nil>}

Condition

Condition defines when a rule should trigger

{<nil> 3366 type 0 [0x1400021aa80] 0}

Config

Config holds correlation engine configuration

{<nil> 1962 type 0 [0x1400021a400] 0}

CorrelationGroup

CorrelationGroup represents a group of related incidents

{<nil> 5000 type 0 [0x1400021b1c0] 0}

CorrelationMatch

CorrelationMatch represents a potential correlation

{<nil> 11282 type 0 [0x1400023b180] 0}

CorrelationResult

CorrelationResult represents the result of correlation analysis

{<nil> 5772 type 0 [0x1400021b5c0] 0}

Engine

Engine manages correlation and deduplication of alerts and incidents

{<nil> 1711 type 0 [0x1400021a1c0] 0}

Methods

NewEngine

NewEngine creates a new correlation engine

{<nil> <nil> NewEngine 0x1400022ec80 <nil>}

Fingerprint

Fingerprint represents a unique signature for an incident/alert

{<nil> 4425 type 0 [0x1400021aec0] 0}

GroupStatus

GroupStatus defines the status of a correlation group

{<nil> 5488 type 0 [0x1400021b4c0] 0}

Constants 

const GroupStatusActive const GroupStatusMerged const GroupStatusSplit const GroupStatusResolved

Operator

Operator defines comparison operators for conditions

{<nil> 3563 type 0 [0x1400021abc0] 0}

Constants 

const OpEquals const OpContains const OpStartsWith const OpEndsWith const OpRegex const OpSimilarityGT const OpTimeWithin

Rule

Rule defines a correlation rule

{<nil> 2650 type 0 [0x1400021a6c0] 0}

Methods

getDefaultRules

getDefaultRules returns default correlation rules

{<nil> <nil> getDefaultRules 0x14000278a60 <nil>}

RuleType

RuleType defines the type of correlation rule

{<nil> 3094 type 0 [0x1400021a980] 0}

Constants 

const RuleTypeDeduplication const RuleTypeCorrelation const RuleTypeGrouping const RuleTypeSuppression

Service

Service provides comprehensive correlation analysis and incident relationship management.

The Service acts as the central coordination point for incident correlation, managing the correlation engine, result persistence, and correlation action execution. It provides intelligent correlation capabilities while maintaining detailed audit trails and comprehensive correlation group management.

Core Responsibilities:

  • Incident correlation analysis through the correlation engine
  • Correlation result persistence with detailed reasoning and context
  • Correlation action execution including deduplication, grouping, and suppression
  • Correlation group management and lifecycle tracking
  • Statistical analysis and correlation performance monitoring
  • Integration with timeline service for correlation event logging

The service operates as a critical component in reducing incident noise and improving response efficiency by automatically identifying related incidents and executing appropriate correlation actions based on configurable rules and thresholds.

{<nil> 28365 type 0 [0x14000708040] 0}

Methods

NewService

NewService creates a new correlation service with optimized default configuration.

This constructor initializes the correlation service with a comprehensive default configuration that balances correlation accuracy with performance. The service is immediately ready for correlation processing with industry-standard thresholds and all correlation dimensions enabled.

Default Configuration:

  • Correlation Window: 15 minutes for temporal proximity analysis
  • Similarity Threshold: 0.8 (80%) for high-confidence correlation
  • Max Group Size: 50 incidents per correlation group
  • All correlation dimensions enabled (time, service, content)

Parameters:

  • db: Database connection for correlation result storage and group management
  • timeline: Timeline service for correlation event logging and audit trails

Service Components: The service initializes with:

  • Correlation engine configured with optimized thresholds
  • Database integration for persistent correlation storage
  • Timeline integration for comprehensive audit logging
  • Default configuration suitable for most deployment scenarios

The correlation engine is configured to provide accurate correlation analysis while maintaining reasonable performance characteristics for high-volume incident processing environments.

Returns a fully configured Service ready for immediate correlation processing with comprehensive correlation capabilities enabled.

{<nil> <nil> NewService 0x140002795c0 <nil>}

Functions

calculateKeywordSimilarity

calculateKeywordSimilarity calculates similarity between keyword sets

{<nil> <nil> calculateKeywordSimilarity 0x14000241660 <nil>}

calculateTextSimilarity

calculateTextSimilarity calculates similarity between two text strings

{<nil> <nil> calculateTextSimilarity 0x14000240f40 <nil>}

extractKeywords

extractKeywords extracts important keywords from text

{<nil> <nil> extractKeywords 0x14000249ec0 <nil>}

Generated automatically from Go source code. Last updated: 2025-08-25T07:51:05-04:00

Edit this page on GitHub
Prev
connectors Package
Next
escalation Package