Self-hosted model operations
Edge Model Service Runbook
Qualify and operate an edge model under device, power, connectivity, physical-access, privacy, update, and recovery constraints.
- Format
- Playbook
- Level
- Advanced
- Audience
- Developer, Operator, Leader
- Owner
- Project42 Editorial
- Review cadence
- Every 60 days
- Prerequisites
- A device, location, connectivity, power, and physical-access inventory; An optimized model and runtime candidate with license and provenance evidence; An update, fleet-observability, safe-degradation, and device-recovery owner
Design for the real device and environment
Record processor and accelerator, execution provider, memory, storage, thermal envelope, power source, operating system, secure boot or device-trust capabilities, sensors, local users, physical access, connectivity windows, clock behavior, and replacement process. Select a model and conversion that fit the device under sustained representative work, not only a short laboratory request.
Define which data stays on device, what telemetry may leave, how identity and policy work while disconnected, what features degrade safely, and which operations must stop. Local inference can reduce data movement, but it does not automatically protect exposed devices, local caches, removable storage, logs, model intellectual property, or downstream tool effects.
Qualify offline behavior and fleet lifecycle
Pin the original model revision and digest, conversion or quantization recipe, converted artifact digest, runtime and execution-provider versions, device policy, and application adapter. Evaluate the converted build itself for important slices, safety boundaries, numerical or quality drift, latency, energy, sustained heat, memory pressure, storage growth, reboot, clock error, damaged input, disconnected operation, and dependency failure.
Stage signed or otherwise integrity-verified updates to a small authorized cohort, preserve a compatible fallback, bound download and storage use, verify after reboot, and report fleet state without collecting unnecessary content. A device that misses updates beyond policy, loses trustworthy identity, cannot report critical state, or fails rollback must move to a defined limited or offline mode.
Task: [EDGE OUTCOME AND SAFE-DEGRADATION RULE]
Scope: [DEVICE CLASS, LOCATIONS, USERS, SENSORS, DATA, CONNECTIVITY, POWER, AND PHYSICAL ACCESS]
Permissions: [MODEL/LICENSE, DEVICE ENROLLMENT, DATA, UPDATE, TELEMETRY, AND RECOVERY AUTHORITY]
Exact build: [SOURCE MODEL/DIGEST, CONVERSION RECIPE, EDGE ARTIFACT/DIGEST, RUNTIME/PROVIDER, DEVICE POLICY, AND APP]
Offline contract: [IDENTITY, POLICY, CACHE, CLOCK, TELEMETRY BUFFER, EXPIRY, AND PROHIBITED ACTIONS]
Fleet limits: [MEMORY, STORAGE, THERMAL, ENERGY, REQUEST, RETRY, UPDATE COHORT, AND STALENESS]
Verification: [CONVERTED-BUILD EVAL, SUSTAINED LOAD, POWER/NETWORK LOSS, REBOOT, UPDATE, ROLLBACK, AND FLEET STATE]
Stop conditions: [UNTRUSTED ARTIFACT/DEVICE, CRITICAL QUALITY DRIFT, DATA LEAK, THERMAL/POWER LIMIT, EXPIRED POLICY, OR FAILED ROLLBACK]
Recovery: [ENTER SAFE MODE, ISOLATE DEVICE/COHORT, RECONCILE BUFFERED WORK, RESTORE PINNED BUILD, REENROLL OR REPLACE]Expected evidence and verification
Expected evidence includes the device and environment profile, source and converted artifact identities, license and provenance decision, conversion recipe, runtime compatibility, converted-build evaluation, sustained performance and power observations, offline and clock behavior, data and telemetry policy, update cohort evidence, rollback or replacement rehearsal, fleet owner, and review date.
Verify at least one loss-of-connectivity or power transition and one failed update. Stop or limit operation on identity loss, artifact mismatch, critical quality drift, prohibited data movement, unsafe temperature or power behavior, expired policy, or failed rollback. Isolate the device or cohort, preserve minimal evidence, reconcile buffered and external effects, restore the pinned build or reenroll a clean device, and verify the safe state before return.