System Architecture Overview
The platform operates across two compute tiers. The edge tier (Jetson Orin) runs continuous inference on camera, audio, and CAN telemetry streams — making decisions that require low latency, such as drowsiness detection and emergency braking alerts. The cloud tier (AWS) handles long-horizon analytics: driver profiling, route history, model retraining via SageMaker, and LLM-powered natural language coaching delivered back to the in-vehicle HMI.
All communication between tiers uses MQTT over TLS 1.3 via AWS IoT Core. The Jetson publishes telemetry topics at configurable rates (100 ms–5 s depending on data class) and subscribes to control topics for OTA model payloads and personalization config pushes.
Edge Inference Pipeline (Jetson Orin)
- Driver Monitoring System (DMS) — OpenCV + TFLite model detecting eye closure, head pose, yawn rate. Outputs drowsiness score 0–100 at 10 Hz.
- Object Detection — YOLO-based model on CUDA cores detects pedestrians, cyclists, road hazards with 30 Hz throughput.
- CAN Telemetry Parsing — Real-time vehicle data (speed, RPM, throttle, brake pressure) decoded from CAN frames at 500 kbps.
- Speech NLU — Wake-word detection + intent classification running locally, privacy-preserving (no audio leaves vehicle).
- Route Optimizer — Locally cached map data + GPS fusion for fuel-efficient route scoring without cloud round-trip latency.
MQTT Topic Architecture
LLM Integration — Cloud Layer
Vehicle telemetry, trip history, and driver behaviour scores are aggregated in DynamoDB and passed as structured context to a GPT-4o / Claude API call via AWS Lambda. The LLM generates natural-language coaching messages ("You've been driving for 3 hours — rest stop in 12 km") that are routed back to the Jetson HMI display via MQTT.
// Lambda: LLM context builder
const context = {
driver_fatigue_score: 68,
trip_duration_hrs: 2.9,
harsh_events_last_30min: 3,
fuel_efficiency_trend: "declining",
next_rest_stop_km: 12,
driver_profile: "commuter_aggressive"
};
const prompt = `You are an in-vehicle AI assistant.
Driver context: ${JSON.stringify(context)}
Generate a concise, actionable recommendation (max 20 words).`;
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: prompt }],
max_tokens: 40
});
// → "You seem fatigued. A rest stop is 12 km ahead. Consider taking a break."
OTA Model Update Pipeline
- Model Registry — Quantized TFLite models stored in S3 with versioning and SHA-256 integrity hashes.
- SageMaker Retraining — Fleet-wide aggregated telemetry used to retrain and improve DMS and route models monthly.
- Delta Updates — Only changed model weights transmitted to reduce bandwidth (typically 2–8 MB vs full 40 MB model).
- Rollback Guard — Jetson OTA agent validates model hash before activation; auto-rollback on inference failure.
Technology Stack
Security Considerations
- Device Identity — Each Jetson provisioned with unique X.509 certificate via AWS IoT Device Provisioning.
- Audio Privacy — All speech NLU processed on-device; no raw audio transmitted to cloud.
- Topic Isolation — IoT policies restrict each vehicle to publish/subscribe only to its own device shadow and topic hierarchy.
- Model Integrity — SHA-256 hash verification before any OTA model activation; signed by code-signing key in AWS KMS.