storage Package

Overview#

Package storage provides comprehensive database abstraction and management for the incident management platform.

This package implements the data persistence layer with support for multiple database backends, schema management, connection pooling, and performance optimization. It serves as the foundation for all data storage operations throughout the incident management system.

Key Features:

• Multi-database backend support (SQLite, PostgreSQL, MySQL)
• Automatic schema migration and versioning
• Connection pooling and performance optimization
• Transaction management with proper error handling
• Database-specific configuration and tuning
• Comprehensive logging and monitoring support
• Development and production environment optimization

Architecture:

The storage layer follows a database abstraction pattern with optimized implementations:

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ Application     │───►│ Storage Service  │───►│ Database Engine │
│ (Timeline/Auth) │    │ (Abstraction)    │    │ (SQLite/Postgres│
└─────────────────┘    └──────────────────┘    └─────────────────┘

      │                        │                        │

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ Connection Pool │    │ Schema Migration │    │ Performance     │
│ (Concurrency)   │    │ (Versioning)     │    │ (Optimization)  │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Database Support:

• SQLite: Development, testing, and small deployments
• PostgreSQL: Production deployments with high concurrency
• MySQL: Legacy integrations and specific requirements
• In-memory: Testing and temporary storage scenarios

Schema Management: The package provides automatic schema creation and migration capabilities, ensuring database structure remains consistent across deployments and versions. Schema changes are applied atomically with proper rollback support.

Example usage:

// Create database connection
db, err := storage.New("./incidents.db")
if err != nil {
	log.Fatal(err)
}
defer db.Close()

// Use in transaction
tx, err := db.BeginTx()
if err != nil {
	log.Fatal(err)
}
defer tx.Rollback()

// Perform operations
_, err = tx.Exec("INSERT INTO incidents (...) VALUES (...)")
if err != nil {
	return err
}

// Commit changes
if err := tx.Commit(); err != nil {
	return err
}

Performance Optimization: The storage layer includes several performance optimizations:

• Write-Ahead Logging (WAL) mode for SQLite
• Appropriate timeout configurations
• Index creation for common query patterns
• Connection pooling for concurrent access
• Query optimization and prepared statements

Configuration: Database behavior can be configured through connection strings and environment variables, allowing deployment-specific optimization without code changes. This includes connection limits, timeouts, and performance tuning parameters.

Import Path: github.com/systmms/incidents/internal/storage

Types#

DB#

DB represents a comprehensive database connection wrapper with enhanced functionality.

This structure provides a high-level interface to database operations while maintaining access to the underlying sql.DB for advanced use cases. It encapsulates connection management, transaction handling, and database-specific optimizations to provide a consistent interface across different deployment scenarios.

Key Features:

• Automatic connection pooling and lifecycle management
• Transaction support with proper error handling
• Database-specific performance optimizations
• Schema migration and versioning capabilities
• Comprehensive error handling and logging
• Support for both high-level and low-level operations

The DB struct maintains both private (conn) and public (DB) references to the underlying connection to support different access patterns while providing a clean abstraction for most use cases.

{<nil> 21621 type 0 [0x14000535cc0] 0}

Methods#

New#

New creates a new SQLite database connection with comprehensive configuration and optimization.

This constructor initializes a SQLite database connection with production-ready configuration, including performance optimizations, integrity constraints, and automatic schema migration. The resulting connection is ready for concurrent use in production environments.

Configuration Features:

• Write-Ahead Logging (WAL) mode for better concurrency
• Foreign key constraint enforcement for data integrity
• Optimized timeout settings for concurrent access
• Balanced synchronization mode for performance/durability trade-off
• Automatic schema creation and migration

Parameters:

• dataSourceName: Database file path (defaults to “./incidents.db” if empty)

Database Optimizations Applied:

• WAL Mode: Enables concurrent readers with single writer
• Foreign Keys: Ensures referential integrity across tables
• Busy Timeout: 5-second timeout for lock acquisition
• Normal Sync: Balanced durability and performance mode

Schema Management: The constructor automatically applies the current schema version, creating all necessary tables, indexes, and constraints. Migration is handled transparently, ensuring the database structure matches the application’s expectations.

Error Handling: Comprehensive error handling covers:

• Database file access and permission issues
• Configuration pragma failures
• Schema migration problems
• Connection establishment failures

Performance Characteristics: The configured database supports:

• Multiple concurrent readers
• Single writer with minimal blocking
• Efficient query execution through proper indexing
• Optimized transaction handling

Usage Examples: // Production database db, err := storage.New("/var/lib/incidents/incidents.db")

// Development database db, err := storage.New("") // Uses default “./incidents.db”

// In-memory database (testing) db, err := storage.New(":memory:")

Returns a configured DB instance ready for production use, or an error if initialization fails for any reason.

{<nil> <nil> New 0x1400054e960 <nil>}

Filter#

Filter represents a query filter

{<nil> 720 type 0 [0x140003e1900] 0}

PolicyAwareDB#

PolicyAwareDB wraps the base DB with policy enforcement

{<nil> 223 type 0 [0x140003e13c0] 0}

Methods#

NewPolicyAwareDB#

NewPolicyAwareDB creates a new policy-aware database wrapper

{<nil> <nil> NewPolicyAwareDB 0x140003db1c0 <nil>}

QueryBuilder#

QueryBuilder helps build SQL queries with dynamic filters

{<nil> 8898 type 0 [0x140004eba00] 0}

QueryOptions#

QueryOptions provides options for policy-aware queries

{<nil> 555 type 0 [0x140003e16c0] 0}

Functions#

Now#

Now returns the current time in UTC timezone for consistent timestamp operations.

This utility function provides a centralized way to generate timestamps throughout the application, ensuring consistent timezone handling and eliminating timezone-related bugs in incident timeline management.

Time Standardization:

• All timestamps are normalized to UTC
• Eliminates timezone conversion issues
• Provides consistent sorting and comparison
• Simplifies database storage and retrieval

Usage Examples: // Timeline event creation event.EventTime = storage.Now() event.CreatedAt = storage.Now()

// Session expiration session.ExpiresAt = storage.Now().Add(24 * time.Hour)

// Audit logging auditLog.Timestamp = storage.Now()

Database Integration: The UTC timestamps work seamlessly with database storage, avoiding timezone interpretation issues and ensuring consistent behavior across different server configurations and client locations.

Performance Considerations: This function uses the system clock and is suitable for most timestamp needs. For high-frequency timestamping or precision requirements, consider caching the current time for batch operations.

Returns the current time in UTC timezone as a time.Time value.

{<nil> <nil> Now 0x1400054f060 <nil>}

classifyIncidentData#

{<nil> <nil> classifyIncidentData 0x1400049c2e0 <nil>}

containsSensitiveInfo#

{<nil> <nil> containsSensitiveInfo 0x1400056e540 <nil>}

containsSensitivePatterns#

{<nil> <nil> containsSensitivePatterns 0x1400049c700 <nil>}

getTimelineRedactionFields#

{<nil> <nil> getTimelineRedactionFields 0x1400049cb60 <nil>}

incidentToAttributes#

{<nil> <nil> incidentToAttributes 0x14000499aa0 <nil>}

isSensitiveLabel#

{<nil> <nil> isSensitiveLabel 0x1400055d680 <nil>}

placeholders#

Helper to generate placeholders

{<nil> <nil> placeholders 0x140004e9d40 <nil>}

redactIncident#

redactIncident applies field redaction to an incident

{<nil> <nil> redactIncident 0x140001c91a0 <nil>}

redactTimelineEvent#

redactTimelineEvent applies field redaction to a timeline event

{<nil> <nil> redactTimelineEvent 0x140001c99c0 <nil>}

scanIncident#

scanIncident scans a row into an Incident

{<nil> <nil> scanIncident 0x14000164e20 <nil>}

scanIncidentRow#

scanIncidentRow scans a single row into an Incident

{<nil> <nil> scanIncidentRow 0x14000165ca0 <nil>}

scanTimelineEvent#

scanTimelineEvent scans a row into a TimelineEvent

{<nil> <nil> scanTimelineEvent 0x140001c8840 <nil>}

Generated automatically from Go source code. Last updated: 2025-08-25T07:51:05-04:00