PagerDuty Bi-Directional Sync

Bi-directional synchronization between Incidents and PagerDuty enables real-time incident updates across both systems. Changes made in either system automatically propagate to the other within seconds.

Overview

The bi-directional sync provides:

  • Real-time Updates: Changes sync within 5 seconds via webhooks
  • Field Ownership: Control which system owns each field
  • Conflict Resolution: Automatic handling of simultaneous edits
  • Escalation Policy Integration: Sync on-call and escalation information

Prerequisites

  • PagerDuty account with admin access
  • API key with full access permissions
  • Webhook v3 configuration access
  • Network connectivity between systems

Configuration

Step 1: Create Integration

Use the CLI:

im integration configure \
  --name "Production PagerDuty" \
  --type pagerduty \
  --org acme-corp \
  --base-url "https://api.pagerduty.com" \
  --direction bidirectional \
  --token "your_pagerduty_api_token"

Or use the API:

curl -X POST http://localhost:8080/api/v1/integrations \
  -H "Content-Type: application/json" \
  -d '{
    "organization_id": "acme-corp",
    "integration_type": "pagerduty",
    "name": "Production PagerDuty",
    "base_url": "https://api.pagerduty.com",
    "credentials": {
      "token": "your_pagerduty_api_token"
    },
    "sync_direction": "bidirectional",
    "enabled": true
  }'

Step 2: Configure Webhook in PagerDuty

  1. Go to Integrations > Generic Webhooks (v3) in PagerDuty

  2. Click + New Webhook

  3. Configure:

    • Webhook URL: https://your-incidents-server/api/v1/webhooks/{integration_id}
    • Scope: Account
    • Events: Select incident events (triggered, acknowledged, resolved, etc.)
    • Custom Headers: Add signature header if using HMAC validation
  4. Copy the Signing Secret for webhook validation

Step 3: Configure Field Ownership

# Platform owns title and description
im integration field-ownership set prod-pd title --owner platform --priority 10
im integration field-ownership set prod-pd description --owner platform --priority 10

# PagerDuty owns escalation and assignment
im integration field-ownership set prod-pd escalation_policy --owner external --priority 5
im integration field-ownership set prod-pd assigned_to --owner external --priority 5

Field Mapping

Incidents Field PagerDuty Field Default Owner
title title Platform
description description Platform
severity urgency Platform
status status Platform
assigned_to assignments[0].assignee External
escalation_policy escalation_policy External

Urgency Mapping

Incidents Severity PagerDuty Urgency
SEV-1 High
SEV-2 High
SEV-3 Low
SEV-4 Low

Status Mapping

Incidents Status PagerDuty Status
open Triggered
mitigated Acknowledged
resolved Resolved
closed Resolved

Event Types

The sync handles all PagerDuty incident lifecycle events:

Event Direction Action
incident.triggered Inbound Create incident
incident.acknowledged Both Update status to mitigated
incident.resolved Both Update status to resolved
incident.reassigned Inbound Update assigned_to
incident.escalated Inbound Log escalation event
incident.priority_changed Both Update severity

Conflict Resolution

Automatic Strategies

# Last write wins
im integration configure prod-pd --conflict-strategy last-write-wins

# Ownership priority
im integration configure prod-pd --conflict-strategy ownership-priority

Manual Review

im integration configure prod-pd --conflict-strategy manual-review
im conflict list --integration prod-pd

Escalation Integration

Sync escalation policy information:

# Enable escalation sync
im integration configure prod-pd --sync-escalations true

# Map services to incident types
im integration service-mapping add prod-pd \
  --pd-service "P123ABC" \
  --incident-type "infrastructure"

On-Call Integration

Access current on-call information:

# Get current on-call for a schedule
im integration oncall prod-pd --schedule-id "PABC123"

Monitoring

# Check sync status
im integration sync-status prod-pd

# View linked incidents
im integration links prod-pd

# View dead letter queue
im integration dlq prod-pd

Rate Limiting

PagerDuty enforces rate limits:

  • 120 requests/minute for most endpoints
  • The sync automatically handles backoff

Configure rate limiting:

im integration configure prod-pd \
  --rate-limit 100 \
  --backoff-multiplier 2

Testing

# Test API connectivity
im integration test prod-pd

# Verify webhook signature validation
curl -X POST https://your-server/api/v1/webhooks/{id}/test

Troubleshooting

Webhook Signature Failures

  1. Verify the signing secret matches PagerDuty configuration
  2. Check that HMAC-SHA256 is being computed correctly
  3. Ensure request body isn’t being modified by proxies

Missing Events

  1. Confirm webhook scope includes all incident events
  2. Check PagerDuty webhook delivery logs
  3. Verify network connectivity

Assignment Sync Issues

  1. Ensure users exist in both systems
  2. Check user email matching configuration
  3. Verify API token has user read permissions