BigRAE Engine¶
Infrastructure lifecycle. Range provisioning, NGFW operations, terminal connections.
Responsibility¶
- Range lifecycle (provision, cancel, destroy)
- NGFW lifecycle (provision, deprovision, instance management)
- Subnet allocation
- Scenario configuration
- SSH terminal connections
- Container task orchestration (via cloud adapter)
Models¶
| Model | Purpose |
|---|---|
Request |
Provisioning request container (correlation via UUID) |
Range |
User's cyber range with lifecycle status and timestamps |
Instance |
Materialized VM instance with Pulumi/Terraform state |
App |
Materialized app (NGFW, etc.) with infrastructure state |
Subnet |
Allocated subnet with CIDR and routing configuration |
Service Interface¶
Range Operations¶
| Function | Purpose |
|---|---|
create_range(request_spec) |
Start provisioning from RequestSpec |
destroy_range_by_request(request_id) |
Destroy range by request UUID |
cancel_range_by_request(request_id) |
Cancel in-progress provisioning |
get_range_status(range_id) |
Get status, instances, progress |
pause_range(range_id) |
Pause range instances |
resume_range(range_id) |
Resume range instances |
Terminal Operations¶
| Function | Purpose |
|---|---|
connect_terminal(user, instance_uuid) |
Get SSH connection to instance |
get_rdp_connection_info(user, instance_uuid) |
Get Guacamole RDP connection |
NGFW Operations¶
| Function | Purpose |
|---|---|
create_ngfw(request_spec) |
Start NGFW provisioning |
destroy_ngfw(request_id) |
Destroy NGFW |
start_ngfw(request_id) |
Start stopped NGFW |
stop_ngfw(request_id) |
Stop running NGFW |
complete_ngfw_setup(request_id) |
Mark NGFW setup as complete |
Internal Services¶
Not exposed to MC. Used within Engine.
| Module | Purpose |
|---|---|
allocation |
Subnet index allocation with row locking |
scenarios |
Scenario validation and instance config |
serialization |
Range to DTO conversion |
ecs |
Container task execution (uses cloud adapter internally) |
ssh |
Async SSH connection management |
secrets |
Secret retrieval (uses cloud adapter internally) |
Event Handling¶
Engine receives events from the Provisioner via the message bus (SNS/SQS on AWS, Pub/Sub on GCP). The engine/handlers.py module processes these events:
def process_range_event(message):
"""Update Range model from provisioner events."""
# range.status.updated -> update Range.status, timestamps
# range.provisioned -> audit log (state written directly by provisioner)
Engine handlers update Engine models only. Mission Control handlers (not Engine) broadcast to WebSocket clients.
See BigRAE Platform for the full event flow diagram.
Range warm pool (#28, ADR-039-R11)¶
The warm pool keeps system-owned, pre-provisioned range generations that an initial
launch can atomically claim, reducing cold-start latency without bypassing capacity,
tenancy, or lifecycle controls. Its architecture is fixed by
docs/architecture/range-warm-pool-preflight-28.md and the optional
range-warm-activation/v1 capability in docs/architecture/provider-neutral-range-substrate.md.
Key seams:
- Ledger (claim authority).
engine.models.WarmRangeGenerationis the row-plus- constraint claim authority. Its state (provisioning,ready,claimed,activated,unhealthy,retiring,terminal) is private allocation state, deliberately distinct fromRange.Status: a warm-prepared range may be Engine-READYwhile the pool treats it as claimable only when the ledger row isready. - Atomic claim.
engine.services.claim_ready_generationtransitions one ready, exact-compatibility_digestgeneration toclaimedunderselect_for_update(skip_locked=True), giving one-winner semantics, with a partial-uniqueclaimed_by_request_idconstraint as the database backstop. The CMS launch path wraps the claim with the ownership rehome and commits before enqueuing activation. - Compatibility.
shared.warm_pool.compatibilitycomputes the canonical digest (registered package + lock digest + admitted placement/posture) that both the reconciler and the launch claim compare. Warm eligibility is RAES-native only (the ownership-neutraloperation_input). The loaded package and lock digests must match the selected compatibility key before a reservation is made. - Activation. The provisioner
raes-range activateoperation scrubs every pre-claim credential/VPN/access identity, realizes the claimant's fresh access, and negatively verifies the prior access is revoked (shared.warm_pool.activation_input, provisionerraes_gcp_activate*). GCE is warm-capable; AWS/GDC report the capability unsupported and cold-fall-back. - Reconciler.
cms.services.reconcile_warm_pool(workerreconcile_warm_pool) replenishes shortfalls, retires expired/incompatible/excess and unhealthy generations through canonicaldestroy, and releases capacity only after provider absence is observed. Engine reserves the generation, global and bucket counts, and capacity draw in one database transaction. Provider work follows that commit. An active policy requires the non-secretWARM_POOL_SERVICE_CREDENTIAL_UUIDreference. The referenced active service admission supplies the installation-scopedinstallation.launch_rangeauthority; no dummy human, personal workspace, or stored cloud key owns preparation. - Retained recovery. The production
reconcile_warm_pool --loopworker advances pages of at most 100 retained obligations, including disabled or removed buckets. The cursor wraps after the last page. A one-shot invocation covers only one page. Unknown cleanup retains the same resource generation and schedules another exact-origin destroy operation after a delay that grows from 60 seconds to one hour. Old operation inputs and results remain immutable. Capacity stays held until current-generation absence is verified. Activation timeout starts atclaimed_at, independent of the age of pool preparation. - Observability.
shared.warm_pool.metricspublishes gauges and claim outcomes to theShifter/WarmPoolnamespace; warm lifecycle events are audited under thewarm_*AuditActionvocabulary.
See identity and lifecycle contracts for the DFDs, lock order and recovery tests. Concurrent workers must use the same policy snapshot to enforce the same limits. Stop workers with an old configuration before treating a reduced limit as installation-wide.