Module 1: Azure Sentinel Architecture & Log Management

🔍 FUNDAMENTALS

Introduction to Cloud-Native SIEM

Before we dive into Azure Sentinel architecture, let's establish what we're building and why it matters.

📊

What is a SIEM Platform?

Security Information and Event Management (SIEM) centralizes security data from across your infrastructure. It ingests logs, correlates events, detects threats, and enables incident response at enterprise scale. Traditional SIEMs run on-premises; cloud-native SIEMs like Azure Sentinel run entirely in the cloud.

☁️

Why Cloud-Native Architecture?

Cloud-native means built for cloud from the ground up. Azure Sentinel benefits: no infrastructure to manage, automatic scaling for terabytes of logs daily, native Azure integration, global availability, and pay-per-use pricing. You focus on threat detection; Microsoft manages the platform.

🎯

Azure Sentinel's Advantage

Azure Sentinel combines SIEM (Security Information and Event Management) AND SOAR (Security Orchestration, Automation, and Response). Detect threats at scale. Automate response. Integrate with third-party tools. It's purpose-built for modern cloud security operations.

💡 SIEM vs. Traditional Firewalls

Firewalls block/allow traffic at network boundaries. They're reactive: "Did we see this IP before?" SIEMs analyze what happens AFTER traffic enters your environment. They're proactive: "Is this behavior suspicious? Is this user acting normally? Is this API call unusual?" Azure Sentinel is your SIEM brain for cloud-native threats.

🏗️ ARCHITECTURE

Azure Sentinel Architecture Overview

Understanding the building blocks: how data flows from sources to intelligence.

Azure Sentinel Data Flow Architecture

Security Data Sources
├─ Azure Services (Azure AD, Key Vault, Storage, SQL)
├─ On-Premises (Windows Event Logs, Syslog)
├─ Third-Party Apps (Office 365, AWS, Google Cloud)
├─ Network Devices (Firewalls, Proxies, IDS/IPS)
└─ Security Tools (Endpoint Detection, Antivirus)
│
↓
DATA CONNECTORS
├─ Azure Native Connectors
├─ API Connectors
├─ Common Event Format (CEF) Syslog
└─ REST API Ingestors
│
↓
LOG ANALYTICS WORKSPACE
├─ Ingestion and Parsing
├─ Field Normalization
├─ Storage in Tables
└─ Retention Management
│
↓
DETECTION & ANALYTICS
├─ Built-in Detection Rules
├─ KQL (Kusto Query Language) Queries
├─ ML-Based Anomalies
└─ Behavioral Analytics
│
↓
INCIDENTS & ALERTS
├─ Threat Alerts Generated
├─ Incidents Correlated
└─ Automated Response Triggered

Data Connectors: The Ingest Pipeline

Data connectors pull logs from sources. Azure Sentinel supports 200+ connectors: Azure AD (identity events), Storage (access logs), Firewalls (traffic logs), Endpoints (process execution), etc. Each connector knows how to parse its source format and deliver data to the workspace.

Log Analytics Workspace: The Core Store

The workspace is where ALL your log data lands. Think of it as a massive database optimized for security analytics. Data is stored in tables (CommonSecurityLog, SigninLogs, AuditLogs, etc.). You set retention (30 days free, up to 7 years paid). The workspace is your single source of truth for all security data.

Detection & Analytics: The Smart Layer

KQL queries and detection rules run against workspace data. Example: "Show me all failed login attempts in the last hour grouped by user." Azure Sentinel's built-in analytics detect known attack patterns. ML-powered anomaly detection identifies unusual behavior. When a threat is detected, an alert fires.

Incidents & Response: The Action Layer

Alerts are grouped into incidents. Security teams investigate incidents using the investigator graph (shows entity relationships). Playbooks automate response: disable user account, block IP, send Slack notification, create ticket. Response automation reduces response time from hours to minutes.

🔑 Key Architectural Concepts

Workspace Isolation: Each Azure Sentinel instance is a separate Log Analytics Workspace. Isolation by environment (dev/staging/prod) or business unit is standard practice.

Retention Tiers: Hot storage (0-90 days) is queryable in seconds. Cold storage (90+ days) is cheaper but slower. Plan retention by log type and compliance requirements.

Ingestion Costs: You pay per GB ingested (not stored). High-volume logs from servers can cost significantly. Filtering at source (ingest only what's needed) is critical for cost optimization.

📋 OPERATIONS

Log Management Strategy

Effective log management is the foundation of cloud security operations.

🔍

Importance of Visibility

You cannot defend what you cannot see. Comprehensive logging across Azure services, applications, and networks gives you 360° visibility. A misconfigured storage account or unauthorized API call leaves a log trail. Without logging, attacks go undetected.

⏱️

Retention Awareness

Decide what to keep and for how long. Short-term logs (30 days) support immediate threat detection. Long-term logs (1-3 years) support compliance audits and forensic investigations. Plan retention by log type: authentication logs may be 1 year, all API calls may be 90 days.

✅

Compliance Logging

Regulations (HIPAA, SOC 2, PCI-DSS, GDPR) mandate logging and retention. Azure Sentinel enables compliance: automated log collection, tamper-proof storage (immutable), audit trails of who accessed what. Compliance dashboards show your posture at a glance.

📊 Log Volume Reality Check

An average Azure environment with 500 users generates 50-100 GB of logs daily. 10,000 users = 1-2 TB daily. At $2.99/GB ingestion cost, that's $3k-6k monthly just for log ingestion. Strategy: Ingest high-value logs at full fidelity (Azure AD, security alerts). Sample or filter verbose logs (routine API calls, routine network traffic). This cuts costs 50-70% while keeping critical security data.

Enterprise SOC Best Practices

🎯

Centralized Monitoring

Single pane of glass across all accounts, regions, and subscriptions. All logs flow to one workspace. Eliminates blind spots. Makes correlation possible.

⚙️

Scaling Log Ingestion

Start small, scale strategically. Add data connectors incrementally. Monitor ingestion costs. Implement sampling for verbose sources. Auto-scale workspace as volume grows.

🔐

Role-Based Access

SOC analysts query security logs. Engineers query infrastructure logs. Admins manage workspace. Implement least-privilege access controls in Azure RBAC.

📈

Data Quality Monitoring

Validate that logs arrive on schedule. Check for parsing errors. Monitor for gaps. Alert if a data source stops sending. Data quality = trust in detections.

💾

Disaster Recovery

Workspace data is redundant across Azure regions. Configure export to long-term storage (Archive). Test recovery procedures annually.

📊

Cost Optimization

Monthly cost reports. Identify high-volume data sources. Implement sampling, filtering, or dedicated tables. Reserve capacity for predictable workloads (20-50% discount).

📚 REFERENCES