Approver System
Last Updated: 2026-03-06 Source: innovation-sandbox-on-aws-approver Captured SHA:
cb27fa3
Executive Summary
The ISB Approver is an intelligent, event-driven lease approval service that transforms manual approval bottlenecks into an automated, score-based decision system for the Innovation Sandbox. It evaluates LeaseRequested events using a 19-rule scoring engine with configurable weights, AI-powered email analysis via Amazon Bedrock (Nova Micro), and UK government domain verification. Pre-approved users are identified via AWS Identity Center group membership through cross-account queries to the management account's Identity Store, replacing the previous hardcoded email allow-list. The system targets instant approval for 80%+ of legitimate requests, while escalating higher-risk requests to operators via Slack with full score breakdowns and one-click approve/deny actions.
Architecture Overview
The Approver operates as a single Lambda function subscribing to three event sources: LeaseRequested events and AccountCleanupSucceeded events from the ISB EventBridge bus, plus a 30-minute scheduled queue check. It implements an in-process state machine pattern (not AWS Step Functions) to orchestrate the decision flow, with SQS for out-of-hours delay queueing and DynamoDB for idempotency tracking and queue position management. Pre-approval checks use a cross-account STS role assumption to query the management account's AWS Identity Center Identity Store for group membership.
Component Architecture
Decision Flow
Identity Center Pre-Approval
Pre-approved users bypass normal scoring thresholds via membership in the ndx-IsbPreapprovedGroup Identity Center group. This replaced a hardcoded email allow-list (src/lib/allow-list.ts, now deleted) with a dynamic, operationally-managed group in AWS Identity Center. The decision is documented in ADR-001.
Source: src/services/identity-store.ts, docs/adr/001-identity-center-group-preapproval.md
How It Works
The Approver Lambda in the Hub account (568672915267) assumes a cross-account IAM role (ApproverIdentityCenterReadRole) in the management account (955063685555) via STS. Using the temporary credentials, it queries the Identity Store to find the user by email and check membership in the pre-approved group. The allow_list_override scoring rule applies a -100 bonus for pre-approved members, effectively guaranteeing approval since the escalation threshold is 20.
Pre-Approval Check Flow
Configuration
| Environment Variable | Value | Purpose |
|---|---|---|
IDENTITY_STORE_ID | d-9267e1e371 | Identity Store instance ID |
IDENTITY_CENTER_ROLE_ARN | Cross-account role ARN | Role in management account |
IDENTITY_CENTER_GROUP_ID | Group ID | ndx-IsbPreapprovedGroup group |
Resilience
- Fail-closed: If the Identity Store is unreachable or STS role assumption fails, the user is NOT pre-approved. The request proceeds through normal scoring and may be escalated for manual review.
- Credential caching: STS credentials are cached and refreshed when less than 5 minutes remain, avoiding redundant cross-account calls within a Lambda execution context.
- Latency: Approximately 200ms for STS AssumeRole + ListUsers + IsMemberInGroups (with cached credentials, subsequent checks are faster).
Operational Management
Pre-approved group membership is managed via the AWS Console or CLI -- no code changes or deployments required. See the runbook for operational procedures. All changes to group membership are auditable via CloudTrail.
Scoring Engine
The scoring engine evaluates 19 rules synchronously within a single Lambda invocation. Each rule returns a point value (positive for penalties, negative for bonuses). The composite score determines the decision: scores below 20 are auto-approved; scores of 20 or above are escalated for manual review.
Source: src/scoring/engine.ts, src/scoring/rules.ts
Penalty Rules (Increase Risk Score)
| Rule | Weight | Trigger |
|---|---|---|
expired_leases | +2 each | Expired lease in last 30 days |
budget_exceeded | +5 each | Budget exceeded in last 30 days |
first_time_user | +5 | No previous leases |
first_time_user_group_mailbox_compound | +20 | First lease + group mailbox |
cooldown_violation | +10 | Request within 1hr of previous lease end |
outside_target_audience | +50 | Non-local-government domain |
group_mailbox_detected | +20 | AI-detected group/shared mailbox |
org_recent_negative | +3 | Same-domain issues in last 30 days |
template_hopper | +2 | 3+ leases never repeating template |
end_of_window | +2 | Request in final 2 hours (5-7pm London) |
user_rate_limit | +5 per | Excess requests beyond 2/hour |
org_rate_limit | +3 | 5+ users from same org in last hour |
budget_amount | +1 per $10 | Higher budgets = more scrutiny |
duration_requested | +1 per 8hrs | Longer durations = more scrutiny |
Bonus Rules (Decrease Risk Score)
| Rule | Weight | Trigger |
|---|---|---|
allow_list_override | -100 | User in pre-approved Identity Center group (ndx-IsbPreapprovedGroup) |
verified_gov_domain | -5 | Domain in ukps-domains list |
familiar_template | -1 | Previously used template successfully |
manual_early_termination | -2 each | Responsible early termination |
org_clean_record | -2 | Domain clean for 90 days with 5+ leases |
In-Process State Machine
The approver implements an enum-based state machine within the Lambda function rather than using AWS Step Functions. This design keeps latency low (single Lambda invocation) while maintaining clear state transitions and audit trails.
Source: src/state-machine/types.ts, src/state-machine/orchestrator.ts, src/state-machine/handlers.ts
States: RECEIVED -> VALIDATING -> TIMING_CHECK -> ACCOUNT_AVAILABILITY_CHECK -> SCORING -> DECIDING -> Terminal state (APPROVED, DENIED, ESCALATED, DELAYED, ERROR)
Each state transition is recorded in a stateHistory array on the StateContext, providing a complete audit trail of the decision process including timestamps and durations.
Infrastructure (CDK)
The Approver deploys as a single CDK stack (ApproverStack) containing:
Source: cdk/lib/approver-stack.ts, cdk/config/environments.ts
| Resource | Purpose |
|---|---|
| Approver Lambda | Main decision engine (Node.js 20, TypeScript), with cross-account STS AssumeRole for Identity Center |
| Slack Approve Lambda | Handles Slack approve button clicks |
| Slack Deny Lambda | Handles Slack deny button clicks |
| DynamoDB: ApproverIdempotency | Lambda Powertools idempotency (TTL-based) |
| DynamoDB: ApproverQueuePosition | FIFO queue tracking with GSI for position ordering |
| S3: Domain List Bucket | Cached ukps-domains allowlist (1-hour TTL) |
| SQS: Delay Queue + DLQ | Out-of-hours request buffering |
| SNS: Notification Topic | Escalation notifications to Slack |
| AWS Chatbot: Slack Channel | Slack workspace integration |
| Chatbot Custom Actions | Approve/Deny buttons on Slack notifications |
| EventBridge Rules | LeaseRequested + AccountCleanupSucceeded on ISB bus |
| EventBridge Scheduler | 30-minute queue check schedule |
| CloudWatch Alarms | DLQ depth, error rate, latency, Slack action errors, SNS failures |
| CloudWatch Dashboard | Slack actions invocations, errors, duration |
Key Integration Points
ISB Core
- Inbound Events:
LeaseRequested,AccountCleanupSucceededfrom ISB EventBridge - Outbound: Direct Lambda invocation of ISB Leases Lambda for approve/deny actions
- Data Access: Reads ISB DynamoDB tables (Leases, Accounts) for user/org history
- ISB Client: Uses
@co-cddo/isb-clientv2.0.3 npm package for typed API calls
AWS Identity Center (Pre-Approval)
- Cross-account: Hub account Lambda assumes
ApproverIdentityCenterReadRolein management account (955063685555) via STS - Identity Store: Queries Identity Store (
d-9267e1e371) in us-west-2 for user lookup and group membership check - Group:
ndx-IsbPreapprovedGroup-- membership grants -100 scoring bonus (automatic approval) - Permissions:
identitystore:ListUsers,identitystore:IsMemberInGroups(granted via cross-account role) - Fail-closed: Any error in the Identity Store flow results in the user not being pre-approved; normal scoring proceeds
Amazon Bedrock
- Model: Amazon Nova Micro (
us.amazon.nova-micro-v1:0) - Purpose: Detect group/shared mailbox patterns in email addresses
- Resilience: Circuit breaker with 60-second recovery; falls back to rule-based heuristics
- Region: us-east-1
Slack Integration
- SNS -> AWS Chatbot -> Slack channel for escalation notifications
- Custom Actions with Approve/Deny buttons invoke dedicated Lambda functions
- Reference numbers (ISB-YYYY-NNNN) for tracking
- Deep links to ISB console for manual review
Business Hours
- Window: 7am-7pm London time, weekdays only
- UK bank holidays detected via gov.uk calendar API
- Out-of-hours requests queued to SQS with delay until next business day
- Queue expires after 5 business days with automatic denial
Technology Stack
| Component | Technology |
|---|---|
| Runtime | Node.js 20, TypeScript 5.7 (strict mode) |
| Infrastructure | AWS CDK v2.240+ |
| Build | esbuild (CJS output, externalize AWS SDK) |
| Testing | Vitest with 800+ tests |
| Validation | Zod schemas for event parsing |
| Logging | AWS Lambda Powertools structured JSON logging |
| Metrics | AWS Lambda Powertools custom CloudWatch metrics |
| Idempotency | AWS Lambda Powertools with DynamoDB backend |
| CI/CD | GitHub Actions with OIDC (no long-lived credentials) |
Observability
- Structured Logging: JSON CloudWatch logs with correlation IDs, score breakdowns, state transitions
- Custom Metrics: Decision counts, score distributions, per-rule trigger rates, Bedrock latency
- CloudWatch Alarms: DLQ depth >5, error rate >1%, p95 latency >5s, Slack action failures, SNS delivery failures
- Audit Trail: 7-year CloudWatch log retention (GDPR compliance)
- Dashboard: Slack actions invocations, errors, and duration graphs
Testing
The repository contains 800+ tests organized across:
test/scoring/- Scoring engine and individual rule teststest/state-machine/- Orchestrator and handler state transition teststest/lib/- Utility function tests (business hours, circuit breaker, domain verification, email analysis)test/handlers/- Slack approve/deny handler teststest/services/- AWS service integration tests (DynamoDB, SQS, Bedrock, SNS, Identity Store)cdk/test/- CDK infrastructure assertion tests
Source: test/ directory (28+ test files), cdk/test/ directory
Generated from source analysis of innovation-sandbox-on-aws-approver at SHA cb27fa3. See 00-repo-inventory.md for full inventory.