MMNA
Money Mitra Network Academy
📚 MODULE 2 OF 3
HARDENING AI DEPLOYMENTS
Secure AI Pipeline & Prompt Defense Strategies
Learn production-hardened patterns for building secure AI pipelines. Master prompt defense strategies, implement access controls, secure model hosting, and protect against supply chain vulnerabilities. Design defense-in-depth architectures that protect LLM systems at enterprise scale.
Secure AI Pipeline Design
End-to-end security architecture for LLM deployments
Defense-in-Depth Pipeline Architecture
Data Validation
→
Model Loading
→
Inference
Input Filtering
→
Output Sanitization
→
User Response
1. Data Ingestion Validation
🔍 Input Source Verification
All data entering the pipeline must be verified for origin and integrity. Validate that data comes
from trusted sources. Implement checksums and digital signatures to ensure data hasn't been tampered
with during transmission. For API inputs, verify request signatures and validate request metadata
(API keys, rate limits, user identity).
📝 Schema & Type Validation
Define strict data schemas for all inputs. Enforce type checking (string, integer, list, etc.) and
validate data against schemas before processing. Reject malformed inputs that don't match expected
structure. This prevents injection attacks that rely on type confusion or unexpected data
structures.
🚫 Content Filtering
Implement multi-layer content validation:
- Size limits: Prevent buffer overflow and DoS attacks via oversized inputs
- Character whitelisting: Only allow expected character sets (alphanumeric, punctuation)
- Format validation: For structured data (JSON, CSV), validate against schema
- Encoding validation: Detect and reject suspicious encoding attempts (unicode tricks, base64 obfuscation)
⚡ Rate Limiting at Ingestion
Implement input rate limiting to prevent DoS and brute force attacks. Limit requests per user, per
IP, per API key. Monitor for suspicious patterns (rapid requests, bulk submissions). This protects
model extraction attempts that rely on high-volume API queries.
2. Secure Model Hosting
🔒 Model Isolation & Sandboxing
Host models in isolated execution environments separate from other systems. Use containerization
(Docker, Kubernetes) to sandbox model processes. Limit resource access: models should not have
direct access to filesystem, network, or other sensitive resources. This prevents compromised models
from affecting other systems.
🔐 Access Control & Authentication
Implement strict access control for model endpoints. Require authentication (API keys, OAuth tokens)
for all requests. Use mutual TLS (mTLS) for service-to-service communication. Implement role-based
access control (RBAC) to restrict which users/services can query specific models. Log all access
attempts.
🛡️ Encryption at Rest & in Transit
Encrypt models at rest using AES-256. Use HTTPS/TLS 1.3 for all network communication. Implement
encryption for model weights, configuration files, and any stored outputs. This protects
intellectual property and prevents model theft even if infrastructure is compromised.
🔄 Model Integrity Verification
Before loading any model, verify its integrity using cryptographic hashes (SHA-256). Maintain a
registry of approved model versions and their hashes. Reject any model that doesn't match approved
checksums. This prevents model poisoning and ensures only approved versions are deployed.
📊 Resource Monitoring & Limits
Monitor model resource consumption (CPU, memory, GPU). Set hard limits to prevent resource
exhaustion attacks. Detect anomalous resource usage that indicates attacks. Implement timeout
mechanisms to prevent hanging requests that could be used for DoS.
3. Access Control Strategies
API Key Management
Implement secure API key infrastructure. Rotate keys regularly (90-day rotation recommended).
Store keys securely using secrets management systems (HashiCorp Vault, AWS Secrets Manager).
Revoke compromised keys immediately. Monitor key usage for suspicious patterns. Implement key
scoping: each key should have minimal required permissions.
User Identity & Authentication
Use strong authentication mechanisms (OAuth 2.0, OpenID Connect). Implement multi-factor
authentication (MFA) for sensitive operations. Verify user identity before granting access.
Maintain audit logs of all authentication attempts. Use session tokens with expiration to
prevent token replay attacks.
Role-Based Access Control (RBAC)
Define clear roles with minimal necessary permissions. Separate concerns: data scientists can
develop models, but only authorized operators can deploy to production. Implement principle of
least privilege: grant minimal permissions needed for each role. Review and audit role
assignments regularly.
Network Segmentation
Isolate model endpoints from public internet. Use VPCs, firewalls, and network policies to
restrict access. Implement allowlists for IP addresses that can access model endpoints. Use API
gateways to centralize access control and monitoring. Prevent direct exposure of model endpoints
to untrusted networks.
Audit & Logging
Log all access attempts (successful and failed). Record who accessed what, when, and from where.
Use centralized logging systems (ELK stack, Splunk) for correlation and analysis. Implement
real-time alerting for suspicious patterns. Maintain immutable audit logs for compliance and
forensics.
Time-Based Access Control
Implement time-based restrictions for sensitive operations. Allow high-risk actions only during
business hours when human oversight is available. Restrict API key usage to specific time
windows. Implement IP-based geofencing to prevent access from unusual locations. Use
context-aware access control that considers multiple signals.
Prompt Security Awareness
Defense strategies for LLM manipulation attacks
1. Guardrail Concepts
System Prompt Hardening
System prompts are the foundational instructions that guide model behavior. Hardened system
prompts include: clear role definitions, explicit constraints, behavioral guardrails, and
failure modes. Use constitutional AI principles to teach models to refuse harmful requests.
Structure prompts to prevent prompt injection by clearly separating system instructions from
user input.
Prompt Boundaries & Delimiters
Use XML-style delimiters and clear separators between different prompt sections. Mark system
instructions, context, and user input distinctly. Example: <system>...</system>
<user>...</user>. This structural approach makes prompt injection harder by creating
clear semantic boundaries. Models can be trained to respect these boundaries and ignore
conflicting instructions in user input.
Instruction Hierarchy & Precedence
Establish clear instruction hierarchy: system instructions take absolute precedence over user
input. Implement explicit "do not override" rules. Use prompt structures that prevent user input
from contradicting system instructions. Teach models through training examples that system
constraints cannot be bypassed regardless of how user prompts are worded.
Behavioral Constraints & Fail-Safes
Define explicit behavioral constraints: "Do not answer questions about passwords", "Do not
process requests to modify data", "Do not reveal system information". Implement fail-safes: if a
request violates constraints, refuse clearly rather than attempting to process it. Create
graceful degradation: if requested functionality is restricted, explain why and offer
alternatives.
Iterative Prompt Refinement
Treat prompt engineering as an iterative security process. Continuously test prompts against
known attack patterns. Refine based on what adversarial users attempt. Maintain prompt
versioning to track evolution. Use red-teaming (internal adversarial testing) to find
vulnerabilities before they're exploited in production.
Few-Shot Examples & Behavior Shaping
Include few-shot examples in prompts that demonstrate desired behavior. Show examples of:
correct responses, rejection of harmful requests, handling of ambiguous queries. Models learn
from examples, so carefully chosen examples shape behavior toward security. Use positive
examples (showing what to do) rather than negative examples (showing attacks to avoid).
2. Input Sanitization Mindset
🛡️ Assumption: All User Input Is Hostile
Adopt a defensive security mindset: treat all user input as potentially malicious. This doesn't mean
being paranoid, but rather being methodical about validation. Implement defense-in-depth: multiple
validation layers ensure that if one fails, others catch the issue.
🔍 Validation Before Processing
Never pass user input directly to models without validation. Implement layered validation: type
checking → format validation → content filtering → semantic analysis. Each layer removes different
classes of attacks. Only after passing all validation layers should input reach the model.
📝 Encoding & Normalization
Normalize all input to consistent encoding (UTF-8). Remove or escape special characters that could
be used for injection. Convert to canonical form to prevent encoding-based attacks (zero-width
characters, unicode lookalikes, mixed scripts). This prevents attackers from using encoding tricks
to bypass filters.
🚫 Blocklist vs Allowlist Strategy
Avoid blocklists (blocking known bad inputs) because attackers constantly find new variations.
Instead, use allowlists (permitting only known-good inputs). For example: if you need email
addresses, validate against email regex; if you need numbers, convert to integer. Allowlists are
more secure because they're explicit about what's allowed.
🔐 Context-Aware Validation
Validation depends on context. For a search query, different validation applies than for an email
address or database query. Understand the intended use of input and validate accordingly. Combine
multiple validation techniques: length limits, character restrictions, format validation, and
semantic analysis for the specific use case.
⚠️ Graceful Error Handling
When validation fails, don't provide detailed error messages that hint at attack vectors. Generic
error messages ("Invalid input") are better than specific ones ("Input contains SQL injection
attempt"). Log detailed errors securely for debugging without exposing information to attackers.
Model Access Protection
Defending against extraction and abuse attacks
1. API Security Considerations
🔌 Endpoint Protection
Model APIs should not be directly exposed to the internet. Use API gateways (Kong, AWS API Gateway,
Azure API Management) to provide a security layer. Implement request validation at the gateway level
before requests reach models. Use WAF (Web Application Firewall) to block common attack patterns.
Implement mutual TLS for service-to-service communication.
📊 Response Filtering
Monitor and filter model outputs before returning to users. Scan for: confidential data leaks,
policy violations, harmful content. Implement output size limits to prevent data exfiltration. Use
secondary models or rules to detect if the model is revealing training data or system information.
This output layer is defense-in-depth after input validation.
🚨 Anomaly Detection on APIs
Monitor API query patterns for anomalies. Detect: unusual request frequencies (extraction attacks
often use high volumes), repeated similar queries (membership inference attacks), requests for
specific patterns (model extraction). Use statistical analysis to establish baselines and alert on
deviations. Implement automatic rate limiting increases when anomalies are detected.
🔐 Response Obfuscation
Consider adding noise to model outputs to prevent extraction attacks. Return only top-1 predictions
instead of confidence scores to all classes (makes surrogate model training harder). Add slight
randomness to rankings or confidence values. This doesn't affect normal user experience but
significantly increases attack complexity.
2. Rate Limiting & Identity Control
Multi-Level Rate Limiting
Implement rate limiting at multiple levels: per user (100 requests/hour), per IP (1000
requests/hour), per API key (5000 requests/hour). Use token bucket algorithms for smooth
enforcement. Distinguish between individual requests and batch operations. Rate limits should be
aggressive enough to prevent extraction attacks but allow legitimate usage.
Intelligent Rate Limiting
Adjust rate limits based on context. Trusted users/services can have higher limits. New accounts
or IPs should have conservative limits. Increase limits based on proven good behavior over time.
Decrease limits if suspicious patterns are detected. This adaptive approach balances security
with usability.
Request Pattern Monitoring
Monitor for suspicious patterns: repeated identical queries, queries designed to extract
specific information, queries trying different variations of the same input. Detect extraction
attack signatures: high volume, low diversity queries that suggest surrogate model training.
Classify queries as normal usage vs potential attacks.
Identity & Provenance Tracking
Track request provenance: which API key, which user, which IP, which device, which time. Use
this information to build trust scores. Queries from unknown sources/times/devices warrant
additional scrutiny. Implement step-up authentication for risky requests. Log complete
provenance for forensics and abuse investigation.
Abuse Detection & Response
Implement automated abuse detection that identifies suspicious accounts/IPs. Escalate detected
abuse to security team. Implement gradual responses: warnings for light abuse, rate limit
increases for moderate abuse, temporary suspension for severe abuse. Permanent bans only after
repeated violations and human review.
User Consent & Terms of Service
Require explicit user consent before API access. Include terms prohibiting reverse engineering,
model extraction, and unauthorized data collection. Make rate limits and usage policies
transparent. Include abuse clauses that allow account suspension for violations. Legal framework
supports technical controls.
AI Supply Chain Security
Protecting the model development and deployment pipeline
1. Model Version Control & Artifact Management
Model Lifecycle Security
1
Development Isolation
Models should be developed in isolated environments separate from production. Use separate infrastructure, repositories, and access controls for development. This prevents experimental or insecure models from leaking into production.
2
Version Control & Tagging
Track all model versions in version control systems (Git, DVC - Data Version Control). Tag production versions clearly. Maintain full commit history including what changed and who made changes. Use semantic versioning (major.minor.patch) for model versions to track compatibility.
3
Cryptographic Verification
Sign model artifacts with cryptographic signatures (digital signatures using private keys). Verify signatures before loading models to ensure authenticity. Maintain a trusted registry of approved model hashes. This prevents unauthorized model substitution or tampering.
4
Artifact Repository Security
Store model artifacts in secure repositories with access controls. Use immutable artifact storage where models cannot be modified after upload (only new versions can be created). Implement role-based access: developers can publish, but only authorized operators can promote to production. Audit all repository access.
5
Rollback Capability
Maintain previous model versions for rapid rollback if production model exhibits unexpected behavior. Test rollback procedures regularly. Implement canary deployments: deploy new models to small user subset before full rollout. This limits blast radius if models are compromised or perform poorly.
2. Dependency Integrity & Supply Chain Attacks
Dependency Scanning
Scan all dependencies (ML frameworks, libraries, tools) for known vulnerabilities. Use SBOM
(Software Bill of Materials) to track all components. Automated tools (Snyk, Dependabot) can
check for CVEs. Maintain updated dependencies to patch security issues. Never use packages with
unpatched critical vulnerabilities in production.
Package Integrity Verification
Verify cryptographic signatures of package downloads. Check package hashes against published
checksums. Use package managers that support signature verification (pip, npm, etc.). Be
cautious of packages from new or unvetted sources. Typosquatting attacks use similar package
names to distribute malicious code.
Supply Chain Monitoring
Monitor the security posture of upstream dependencies. Follow security advisories for packages
you use. Understand maintenance status of dependencies: abandoned packages are high-risk.
Subscribe to security mailing lists for critical packages. Implement automated alerts when
vulnerabilities are discovered in your dependency tree.
Build Pipeline Security
Secure your CI/CD pipeline to prevent injection of malicious code. Use secure build servers with
restricted access. Implement code review requirements before merging. Sign build artifacts. Use
trusted base images for container builds. Audit all CI/CD activities. Compromised build
pipelines are effective attack vectors.
Third-Party Model Security
When using pre-trained models (from HuggingFace, TensorFlow Hub, etc.), verify their source and
reputation. Use models from official/verified sources only. Check model documentation for any
warnings or known issues. Consider fine-tuning security implications: malicious fine-tuning data
could poison models. Run basic sanity tests on pre-trained models before use.
Vendor Risk Assessment
For third-party ML services, assess vendor security practices. Understand their data handling,
model security, and incident response. Require security certifications (SOC 2, ISO 27001).
Review vendor security documentation. Include security requirements in contracts. Regular audits
of vendor compliance with security commitments.
External Learning References
Official AI governance and security documentation
Ready for Module 3?
You've mastered secure pipeline design and prompt defense. Now learn production monitoring, governance frameworks, and incident response for enterprise AI systems.