Monitoring, Logging & Runtime Protection | Module 3

Logging & Observability Fundamentals

📊

CloudWatch Concepts (High-Level)

CloudWatch is AWS's native monitoring and logging service. Lambda functions automatically write logs to CloudWatch Logs, which can be aggregated, searched, and analyzed.

CloudWatch Flow:

1. Lambda function executes and writes logs (print, console.log, etc.)
2. Logs automatically sent to CloudWatch Logs (JSON format)
3. CloudWatch Insights query logs in real-time
4. Metrics extracted from logs (errors, latency, etc.)
5. Alarms triggered based on metric thresholds
6. Notifications sent (email, SNS, PagerDuty, etc.)

CloudWatch Log Entry (JSON)

{
  "timestamp": 1704067200000,
  "message": "Processing order #12345",
  "requestId": "abc123def456",
  "duration_ms": 245,
  "function_version": "$LATEST"
}

👁️

Monitoring Execution Behavior

Track how Lambda functions execute in production. Monitor metrics that indicate normal behavior and anomalies.

✓
Invocation Count
Number of function executions over time. Sudden spikes may indicate DoS or malicious activity.
✓
Duration (Latency)
Execution time. Unusually long durations suggest resource exhaustion or code issues.
✓
Error Rate
% of invocations that fail. High error rates indicate compromises, bad input, or infrastructure issues.
✓
Memory Usage
RAM consumed. Excessive memory indicates memory leaks or malicious code execution.
✓
Throttling Events
When Lambda hits concurrency limits. May indicate infrastructure saturation or attack.

Key Monitoring Metrics

< 1s

Healthy Duration

< 1%

Target Error Rate

Normal ±20%

Expected Variance

< 100ms

Cold Start Target

Log Analysis Patterns (CloudWatch Insights Queries)

Find Errors in Last Hour

fields @timestamp, @message
| filter @message like /ERROR|Exception|failed/
| stats count() as error_count by @message
| sort error_count desc

Identify Slow Function Executions

fields @duration
| filter @duration > 5000
| stats count() as slow_executions, 
        avg(@duration) as avg_duration_ms
        by bin(5m)

Detect Unusual Access Patterns

fields @timestamp, @message, user_id
| filter @message like /database_access|api_call/
| stats count() as api_calls by user_id
| filter api_calls > 1000
| sort api_calls desc

Runtime Protection & Behavioral Detection

🔍

Behavioral Anomaly Detection

Detect when function behavior deviates from baseline. Attackers often exhibit different patterns than legitimate usage.

Unusual API Calls

Function suddenly calls APIs it never called before (iam:CreateRole, iam:AttachPolicy, ec2:DescribeInstances). Indicates compromise.

Data Access Spike

Function reads 100x more data from database in single execution. Suggests attacker exfiltrating data.

Geographic Anomalies

Function invoked from unusual IP ranges or regions. May indicate lateral movement or external attack.

Timing Anomalies

Function executes at unusual times (3 AM when typically runs 9-5). Suggests scheduled attack.

🚨

Alerting Awareness (High-Level)

Set up alerts for suspicious behavior. Timely notifications enable quick incident response.

✓
Threshold-Based Alerts
Alert when error rate > 5%, latency > 3 sec, or throttling events occur
✓
Anomaly Detection
Machine learning models baseline normal behavior. Alert on deviations.
✓
Log-Based Alerts
Alert on specific keywords: "ERROR", "Unauthorized", "CRITICAL", "breach"
✓
Multi-Channel Notifications
Send alerts to email, Slack, PagerDuty, SMS. Multiple channels ensure visibility.

Security Detection Patterns

1

Privilege Escalation Detection

Alert when function calls iam:CreateRole, iam:AttachUserPolicy, or sts:AssumeRole with unusual parameters. First sign of attack.

2

Data Exfiltration Detection

Monitor database query patterns. Alert when single execution reads > threshold records or exports to external S3 bucket.

3

Reverse Shell Detection

Monitor outbound network connections from Lambda. Alert on connections to known C2 (command-and-control) servers or unusual outbound ports.

4

Cryptomining Detection

Monitor CPU usage and process spawning. Alert on unusual process names (xmrig, monero, etc.) or extreme CPU utilization (>90% for extended periods).

5

Injection Attack Detection

Monitor logs for SQL syntax errors, command injection patterns, or XML parsing errors following suspicious input. Alert on repeated injection attempts.

Incident Response in Serverless

🔒

Containment Mindset

When a breach is detected, rapid containment prevents further damage. Don't panic—follow procedures methodically.

Immediate Priority: Stop the Bleeding

Assume function is compromised. Isolate it to prevent lateral spread to other systems. Revoke credentials immediately.

Preserve Evidence

Don't delete logs or function code. Save CloudWatch logs, X-Ray traces, and execution history for forensics and compliance.

Prevent Reinfection

Deploy patched function. Never revert to compromised version without verification. Update dependencies and fix vulnerabilities.

🚫

Access Revocation Awareness

Quickly remove attacker's access. Stop malicious activity before they cause more damage.

✓
Revoke Credentials
Delete Lambda's IAM role or detach all policies. Function can't make AWS API calls anymore.
✓
Disable Function
Set concurrency to 0. Prevents new invocations. Can revert once patched and verified.
✓
Rotate Secrets
Assume attacker stole database credentials from environment. Update all passwords in Secrets Manager.
✓
Isolate Dependent Services
If compromised function calls other systems, revoke cross-account roles, restrict database access, pause integrations.

Incident Response Playbook (7 Steps)

1

Alert & Assess

Security alert triggered (e.g., "IAM CreateRole detected from Lambda function"). Immediately verify alert is legitimate, not false positive.

2

Isolate & Contain

Set Lambda concurrency to 0. Disable execution. Function can't execute new invocations or make API calls. Containment immediate.

3

Gather Evidence

Export CloudWatch logs, X-Ray traces, and execution history. Download function code and dependencies. Save IAM role policies. Document timeline of events.

4

Investigate Root Cause

Analyze logs: when did compromise start? Which code or dependency introduced vulnerability? Check for supply chain attacks, injection vectors, or credential exposure.

5

Remediate & Deploy Patch

Fix vulnerable code, update dependencies, rotate secrets, tighten IAM role. Deploy new function version. Test thoroughly in staging first.

6

Re-enable & Monitor

Set Lambda concurrency back to normal. Re-enable event sources. Monitor closely for any signs of reinfection or lateral movement attempts.

7

Post-Mortem & Lessons Learned

Document incident, timeline, and response actions. Identify preventative measures for future (SAST scanning, dependency audits, better monitoring).

Enterprise Governance & Compliance

📋

Audit Logging

Maintain comprehensive logs of who changed what, when, and why. Essential for compliance, forensics, and audits.

✓
CloudTrail for API Changes
Logs all AWS API calls: who deployed new function, who modified role, who rotated secrets. Immutable audit trail.
✓
Function Execution Logs
CloudWatch Logs capture detailed execution info: inputs, outputs, errors, resource usage. Queryable for investigations.
✓
Change Management Logs
Track deployments: who, what version, approval chain, deployment status. Enable rollback if needed.
✓
Access Logs
Log all function invocations: requester identity, time, result. Detect suspicious access patterns.

⚖️

Compliance & Retention Strategy

Meet regulatory requirements. Retain logs for investigations and audits while managing costs.

Retention Periods

GDPR: 30 days (personal data), HIPAA: 6 years, SOC2: 1 year minimum. Define policy based on regulations.

Encryption at Rest

Logs stored in S3 with KMS encryption. Only authorized users can decrypt. Protects sensitive data in logs.

Immutability

Enable S3 Object Lock on audit logs. Prevent accidental or malicious deletion. Complies with regulatory requirements.

Redaction

Remove sensitive data from logs (PII, secrets, credit cards). Use CloudWatch Logs Insights to mask before storage.

Security Controls → Compliance Mapping

Security Control	Implementation (Lambda)	Compliance Standard
Access Control	IAM least-privilege roles, resource policies	SOC2, ISO27001, HIPAA
Encryption	KMS for secrets, HTTPS for transit, EBS encryption	GDPR, PCI-DSS, HIPAA
Audit Logging	CloudTrail, CloudWatch Logs, S3 logging	SOC2, HIPAA, FedRAMP
Monitoring & Detection	CloudWatch alarms, anomaly detection, X-Ray	SOC2, ISO27001
Incident Response	Playbooks, forensics, RCA process	ISO27001, NIST
Data Protection	Encryption keys, secret rotation, DLP rules	GDPR, PCI-DSS, CCPA
Change Management	Code reviews, staging deployments, version control	SOC2, ITIL, HIPAA

External Learning References

📊

AWS CloudWatch Documentation

Comprehensive guide to monitoring, logging, and alerting for AWS Lambda functions.

docs.aws.amazon.com/cloudwatch →

🔍

AWS X-Ray for Tracing

Distributed tracing service for analyzing Lambda execution and identifying performance bottlenecks and failures.

docs.aws.amazon.com/xray →

🚨

AWS CloudWatch Insights

Powerful log analysis and querying for investigating incidents and detecting anomalies.

docs.aws.amazon.com/CloudWatch Insights →

⚖️

NIST Cybersecurity Framework

Industry-standard framework for security controls: Identify, Protect, Detect, Respond, Recover.

www.nist.gov/cyberframework →

📋

AWS CloudTrail for Audit

Track API calls and account activity. Essential for compliance, security analysis, and incident investigation.

docs.aws.amazon.com/cloudtrail →

🔐

AWS Security Hub

Centralized security and compliance monitoring across AWS accounts and services.

docs.aws.amazon.com/securityhub →

Course Summary: Your Serverless Security Journey

🏗️

Module 1: Architecture

Serverless fundamentals, threat landscape, and enterprise risks

🔐

Module 2: Security Design

IAM hardening, secure functions, dependency management

👁️

Module 3: Runtime Defense

Monitoring, incident response, enterprise compliance

🎓

Verified Certificate

Industry-recognized security credential

You've mastered the complete serverless security lifecycle: from understanding threat models and designing secure functions with least-privilege IAM, to monitoring production systems and responding to incidents with enterprise-grade governance. You're now equipped to secure Lambda functions at scale.

🏆

Complete Course & Earn Your Certificate

Congratulations on completing all 3 modules of the Serverless Security Ops course! You've earned your Verified Cyber Security Certificate from MONEY MITRA NETWORK ACADEMY

✓ Unique Certificate ID with Blockchain Verification

✓ QR Code Link for Employer Verification

✓ LinkedIn Badge Ready for Your Profile

✓ Official Credential Recognizing Serverless Security Expertise

🎉 All 3 Modules Complete - Ready to Download Certificate!

Ready to Claim Your Achievement?

You've completed the entire Serverless Security Ops course and gained expert-level knowledge in Lambda security. Download your certificate, share your achievement on LinkedIn, and advance your cybersecurity career.

Certificate valid and verifiable indefinitely. Blockchain-backed credential with employer verification QR code.