# COPILOT New System Design Summary

**Document Version:** 1.0
**Date:** 2026-02-03
**Status:** Design Complete - Ready for Implementation

---

## Executive Summary

This document describes the architecture for rewriting the COPILOT CCTV keyboard controller system with a new master/slave architecture. The new system eliminates the need for dedicated server hardware by enabling any keyboard to act as the coordination PRIMARY, while maintaining critical functionality (ViewerConnectLive) even when the coordination layer fails.

### Key Goals
- **Zero extra hardware** - PRIMARY runs on a keyboard (LattePanda Sigma)
- **Direct command path** - Commands go directly to servers, not through a coordinator
- **High availability** - Automatic failover with degraded mode support
- **Cross-platform** - Flutter-based keyboards (web/desktop)
- **State consistency** - Query-based alarm state, not just event subscription

---

## 1. Architecture Overview

### 1.1 Current System (COPILOT_D6)

```
┌─────────────────────────────────────────────────────────────────┐
│ COPILOT_D6 ARCHITECTURE                                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Keyboards (WPF) ──────► AppServer (Central) ──────► GeViSoft   │
│                          - SignalR Hub                          │
│                          - Lock Management                      │
│                          - Separate Hardware                    │
│                                                                  │
│  Problems:                                                       │
│  • Central AppServer = single point of failure                  │
│  • Extra hardware required                                       │
│  • All commands routed through central point                    │
│  • WPF = Windows-only                                           │
│  • Event-only alarm tracking (misses alarms if offline)         │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 1.2 New System Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│ NEW ARCHITECTURE                                                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐         │
│  │ Keyboard 1  │    │ Keyboard 2  │    │ Keyboard N  │         │
│  │ (PRIMARY)   │    │ (STANDBY)   │    │ (REGULAR)   │         │
│  │ Flutter     │    │ Flutter     │    │ Flutter     │         │
│  │ + Bridges   │    │ + Bridges   │    │ + Bridges   │         │
│  └──────┬──────┘    └──────┬──────┘    └──────┬──────┘         │
│         │                  │                  │                 │
│         │  WebSocket (coordination only)      │                 │
│         ├──────────────────┼──────────────────┤                 │
│         │                  │                  │                 │
│         │  Direct HTTP (ViewerConnectLive, PTZ)                 │
│         ▼                  ▼                  ▼                 │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐         │
│  │ GeViScope   │    │  G-Core     │    │ GeViServer  │         │
│  │ Servers     │    │  Servers    │    │ (PTZ only)  │         │
│  └─────────────┘    └─────────────┘    └─────────────┘         │
│                                                                  │
│  Benefits:                                                       │
│  • No extra hardware (PRIMARY runs on keyboard)                 │
│  • Direct commands = low latency (<100ms)                       │
│  • Automatic failover to STANDBY                                │
│  • ViewerConnectLive works even if PRIMARY dead                 │
│  • Query-based alarm state (never miss alarms)                  │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

---

## 2. Hardware Platform

### 2.1 LattePanda Sigma SBC (Inside Each Keyboard)

| Component | Specification |
|-----------|---------------|
| **CPU** | Intel Core i5-1340P (13th Gen), 12 cores/16 threads, up to 4.6 GHz |
| **RAM** | 16GB or 32GB LPDDR5-6400 (102.4 GB/s bandwidth) |
| **GPU** | Intel Iris Xe Graphics (80 EUs), up to 4x 4K displays |
| **Network** | Dual 2.5 GbE ports |
| **TDP** | Sustained 44W with proper cooling |
| **Size** | 102mm × 146mm |

### 2.2 Resource Usage (Per Keyboard)

| Process | RAM Usage | CPU Usage |
|---------|-----------|-----------|
| Flutter Keyboard App | ~200-400 MB | <5% |
| GeViScope Bridge (.NET 8) | ~100-150 MB | <2% |
| G-Core Bridge (.NET 8) | ~100-150 MB | <2% |
| GeViServer Bridge (.NET 8) | ~100-150 MB | <2% |
| OS + Background | ~2-3 GB | Variable |
| **Total** | **~3-4 GB** | **<20% typical** |

**Verdict:** LattePanda Sigma with 16GB RAM is highly suitable. 32GB provides future margin.

---

## 3. Component Responsibilities

### 3.1 PRIMARY Keyboard

The PRIMARY is elected from available keyboards based on priority. It handles:

| Responsibility | Description |
|----------------|-------------|
| **PTZ Lock Management** | Grant/revoke locks, enforce priority, handle timeouts |
| **Shared State** | Current video wall state (monitor → camera mappings) |
| **Sequence Engine** | Camera rotation sequences on monitors |
| **Alarm Coordination** | Aggregate alarm state, broadcast to keyboards |
| **Keyboard Registry** | Track which keyboards are online |
| **WebSocket Hub** | Broadcast state changes to all keyboards |

**What PRIMARY Does NOT Do:**
- Route ViewerConnectLive commands (keyboards send directly)
- Route PTZ commands (keyboards send directly)
- Act as single point of failure for critical operations

### 3.2 STANDBY Keyboard

- Receives all state updates from PRIMARY
- Maintains synchronized copy of shared state
- Monitors PRIMARY via heartbeat (every 2 seconds)
- Promotes to PRIMARY if heartbeat fails (6 seconds timeout)

### 3.3 REGULAR Keyboards

- Send commands directly to servers
- Request PTZ locks from PRIMARY
- Receive state broadcasts from PRIMARY
- Can operate in degraded mode without PRIMARY

### 3.4 C# Bridges (Per Keyboard)

| Bridge | Port | SDK | Purpose |
|--------|------|-----|---------|
| GeViScope Bridge | 7720 | GeViScope SDK (32-bit) | ViewerConnectLive, PTZ, alarms |
| G-Core Bridge | 7721 | G-Core SDK | ViewerConnectLive, PTZ, alarms |
| GeViServer Bridge | 7710 | GeViSoft SDK | PTZ fallback, state queries |

---

## 4. Command Flow

### 4.1 ViewerConnectLive (Camera → Monitor)

```
Keyboard                              Server
   │                                    │
   │  1. User selects camera + monitor  │
   │                                    │
   │  2. Route to correct server        │
   │     (based on camera ID mapping)   │
   │                                    │
   │  3. HTTP POST to local bridge      │
   │     POST localhost:7720/crossswitch│
   │         ─────────────────────────► │
   │                                    │
   │  4. Bridge calls SDK               │
   │     ViewerConnectLive(cam, mon)    │
   │         ─────────────────────────► │ GeViScope/G-Core
   │                                    │
   │  5. Server executes, fires event   │
   │     ViewerConnected notification   │
   │     ◄───────────────────────────── │
   │                                    │
   │  6. Update local state             │
   │  7. Notify PRIMARY (if online)     │
   │                                    │
```

**Key Point:** Commands go DIRECTLY to servers. PRIMARY is only notified for state tracking.

### 4.2 PTZ Control with Locking

```
Keyboard                    PRIMARY                    Server
   │                           │                          │
   │  1. Request PTZ lock      │                          │
   │  ────────────────────────►│                          │
   │                           │                          │
   │  2. PRIMARY checks:       │                          │
   │     - Is camera locked?   │                          │
   │     - Priority comparison │                          │
   │                           │                          │
   │  3. Lock granted/denied   │                          │
   │  ◄────────────────────────│                          │
   │                           │                          │
   │  4. If granted, send PTZ directly                    │
   │  ─────────────────────────────────────────────────►  │
   │                           │                          │
   │  5. Lock timeout (5 min)  │                          │
   │     Warning at 4 min      │                          │
   │  ◄────────────────────────│                          │
   │                           │                          │
```

### 4.3 PTZ Lock Configuration

| Setting | Value | Source |
|---------|-------|--------|
| Lock Timeout | 5 minutes | COPILOT_D6 LockExpirationConfig |
| Warning Before Expiry | 1 minute | COPILOT_D6 NotificationBeforeExpiration |
| Priority Levels | High, Low | COPILOT_D6 CameraLockOptions |

---

## 5. State Management

### 5.1 Video Wall State Verification

**Problem:** No direct "query current state" API in GeViScope/G-Core SDKs.

**Solution:** Event-based state tracking with startup query.

```
┌─────────────────────────────────────────────────────────────────┐
│ STATE VERIFICATION APPROACH                                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. STARTUP: Query current state                                │
│     - GetFirstVideoOutput / GetNextVideoOutput                  │
│     - sActualCamera field = currently displayed camera          │
│                                                                  │
│  2. REAL-TIME: Subscribe to notifications                       │
│     - ViewerConnected(Viewer, Channel, PlayMode)               │
│     - ViewerCleared(Viewer)                                     │
│     - ViewerSelectionChanged(Viewer, Channel, ...)             │
│                                                                  │
│  3. VERIFICATION: After sending command                         │
│     - Wait for ViewerConnected notification (500ms timeout)    │
│     - If received: confirm state                                │
│     - If timeout: flag potential failure                        │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 5.2 Alarm State Management

**Problem (D6):** Event-only tracking misses alarms if AppServer offline.

**Solution:** Query + Subscribe + Periodic Sync.

```
┌─────────────────────────────────────────────────────────────────┐
│ ALARM STATE APPROACH                                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. STARTUP: Query ALL active alarms                            │
│     - GeViSQ_GetFirstAlarm(activeOnly=true)                    │
│     - GeViSQ_GetNextAlarm() until no more                      │
│     - Populate local alarm state BEFORE accepting commands     │
│                                                                  │
│  2. REAL-TIME: Subscribe to events                              │
│     - EventStarted(EventID, TypeID) → Add alarm                │
│     - EventStopped(EventID) → Remove alarm                     │
│     - VCAlarmQueueNotification → Update monitor reservation    │
│                                                                  │
│  3. PERIODIC SYNC: Every 30 seconds                             │
│     - Re-query all active alarms                                │
│     - Compare with local state                                  │
│     - Add missing, remove stale                                 │
│     - Log discrepancies                                         │
│                                                                  │
│  RESULT: Never miss an alarm, even after restart/reconnect     │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 5.3 Alarm → Monitor Blocking

| Alarm State | Value | Monitor Effect |
|-------------|-------|----------------|
| `vasNewAlarm` | 0 | Monitor reserved, cannot switch |
| `vasPresented` | 1 | Alarm displayed, cannot switch |
| `vasStacked` | 2 | Alarm in queue, monitor may be switchable |
| `vasConfirmed` | 3 | Acknowledged, normal operation resumes |

---

## 6. Failover and Degraded Mode

### 6.1 Failover Sequence

```
┌─────────────────────────────────────────────────────────────────┐
│ FAILOVER SEQUENCE                                               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  NORMAL OPERATION:                                               │
│  • PRIMARY sends heartbeat every 2 seconds                      │
│  • STANDBY receives and acknowledges                            │
│                                                                  │
│  PRIMARY FAILURE DETECTED (3 missed heartbeats = 6 seconds):   │
│  1. STANDBY declares itself PRIMARY                             │
│  2. STANDBY broadcasts: "I am now PRIMARY"                     │
│  3. STANDBY starts accepting lock requests                     │
│  4. STANDBY starts sending heartbeats                          │
│  5. Next highest priority keyboard becomes STANDBY             │
│                                                                  │
│  SPLIT-BRAIN PREVENTION:                                        │
│  • Only one PRIMARY at a time (priority-based election)        │
│  • If old PRIMARY comes back, it defers to new PRIMARY         │
│  • Epoch/generation counter prevents stale PRIMARY takeover    │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 6.2 Degraded Mode

When BOTH PRIMARY and STANDBY are unavailable:

| Feature | Status | Notes |
|---------|--------|-------|
| **ViewerConnectLive** | ✅ Works | Direct to server |
| **PTZ Control** | ✅ Works | Direct to server |
| **PTZ Locking** | ❌ Unavailable | No coordinator |
| **Sequences** | ❌ Stopped | Runs on PRIMARY |
| **State Sync** | ❌ Unavailable | No broadcaster |
| **Alarm State** | ⚠️ Local only | Each keyboard tracks independently |

**Critical Path Guarantee:** Operators can always switch cameras and control PTZ, even in degraded mode.

---

## 7. Recorded Video Access

### 7.1 Playback Actions (Direct to GeViScope/G-Core)

| Action | Purpose | Parameters |
|--------|---------|------------|
| `ViewerConnect` | Play recorded video | Viewer, Channel, PlayMode |
| `ViewerPlayFromTime` | Seek to timestamp | Viewer, Channel, PlayMode, Time |
| `ViewerSetPlayMode` | Control speed/direction | Viewer, PlayMode, PlaySpeed |
| `ViewerJumpByTime` | Relative seek | Viewer, Channel, PlayMode, TimeInSec |
| `ViewerExportPicture` | Snapshot export | Viewer, FilePath |

### 7.2 Play Modes

| Mode | Description |
|------|-------------|
| `play forward` | Normal speed forward |
| `play backward` | Normal speed backward |
| `fast forward` / `fast backward` | High speed |
| `step forward` / `step backward` | Frame by frame |
| `play stop` | Pause |
| `play BOD` / `play EOD` | Beginning/End of database |
| `next detected motion` | Skip to motion event |

---

## 8. External Systems Integration

### 8.1 Alarm Sources (Digital Inputs)

```
External System → LAN I/O Device → DigitalInput Action → Event → Alarm
                  (MIO, IOI43)     Contact + State
```

**DigitalInputState Values:**
- `disLow` (0) - Contact open/inactive
- `disMiddle` (1) - Terminated (tamper)
- `disHigh` (2) - Contact closed/active

### 8.2 Logging (ELK Stack)

```
┌─────────────────────────────────────────────────────────────────┐
│ LOGGING ARCHITECTURE                                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Each Keyboard:                                                  │
│    Flutter App ──► /var/log/copilot/keyboard.jsonl             │
│    Bridges ──────► /var/log/copilot/bridge-*.jsonl             │
│         │                                                        │
│         ▼                                                        │
│    Filebeat (local) ──► Logstash ──► Elasticsearch ──► Kibana  │
│                                                                  │
│  Log Format: JSON Lines (structured)                            │
│  Retention: 30 days local, configurable in ELK                  │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 8.3 User Management (IAM Ready)

```
┌─────────────────────────────────────────────────────────────────┐
│ AUTHENTICATION ABSTRACTION                                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  interface IAuthProvider {                                       │
│    authenticate(credentials) → AuthResult                       │
│    getUser(userId) → User                                       │
│    getPermissions(userId) → List<Permission>                    │
│    logout(sessionId)                                            │
│  }                                                               │
│                                                                  │
│  Implementations:                                                │
│    Phase 1: LocalAuthProvider (JSON/SQLite)                     │
│    Phase 2: LdapAuthProvider (Active Directory)                 │
│    Phase 3: OidcAuthProvider (Keycloak, Azure AD, Okta)        │
│                                                                  │
│  Permission Model:                                               │
│    viewLive, viewRecorded, ptzControl, ptzControlHighPriority, │
│    switchMonitor, switchAlarmMonitor, manageSequences,         │
│    manageUsers, viewAuditLog, systemConfig                      │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

---

## 9. Migration Path

### Phase 0: Infrastructure (Week 1-2)
- Finalize C# bridges (GeViScope, G-Core, GeViServer)
- Add PLC notification subscriptions
- Test direct commands without GeViSoft routing
- Document server configurations

### Phase 1: Flutter Keyboard Core (Week 3-5)
- Keyboard layout UI
- Direct command execution (ViewerConnectLive, PTZ)
- Server routing logic
- State notification subscription
- Basic error handling

### Phase 2: Coordination Layer (Week 6-8)
- PRIMARY election mechanism
- PTZ lock management
- Shared state (video wall, locks, keyboards)
- WebSocket hub
- State sync from server notifications

### Phase 3: Advanced Features (Week 9-11)
- Sequence engine
- CrossSwitch rules
- Alarm handling
- User authentication

### Phase 4: Testing & Cutover (Week 12-14)
- Parallel operation testing
- Failover testing
- Load testing
- Operator training
- Production cutover

---

## 10. Key Design Decisions Summary

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Central Coordinator | PRIMARY keyboard | Zero extra hardware |
| Command Path | Direct to servers | Low latency, no bottleneck |
| State Verification | Event-based + startup query | GeViScope/G-Core have no query API |
| GeViSoft Usage | Minimal (PTZ only if needed) | User requirement |
| Failover | Priority-based STANDBY promotion | Automatic recovery |
| Degraded Mode | ViewerConnectLive works | Critical path must always work |
| PTZ Locks | 5-min timeout, priority levels | Match D6 behavior |
| Alarm State | Query + Subscribe + Sync | Never miss alarms |
| UI Technology | Flutter | Cross-platform |
| Hardware | LattePanda Sigma 16GB | Sufficient for bridges + app |
| Logging | Local JSON + Filebeat → ELK | Central Kibana dashboards |
| Auth | Abstracted, IAM-ready | Future AD/OIDC integration |

---

## 11. Port Assignments

| Service | Port | Protocol |
|---------|------|----------|
| GeViServer | 7700-7703 | Native SDK |
| GeViServer Bridge | 7710 | HTTP REST |
| GeViScope Bridge | 7720 | HTTP REST |
| G-Core Bridge | 7721 | HTTP REST |
| Python API (if used) | 8000 | HTTP REST |
| Flutter Web | 8081 | HTTP |
| PRIMARY WebSocket | 8090 | WebSocket |
| gRPC (SDK Bridge) | 50051 | gRPC |

---

## 12. Configuration Files

| File | Purpose | Location |
|------|---------|----------|
| `servers.json` | Bridge endpoints per keyboard | Per keyboard |
| `keyboards.json` | Keyboard priorities, roles | Shared |
| `sequences.json` | Camera rotation sequences | PRIMARY |
| `crossswitch-rules.json` | Logical → physical routing | Shared |
| `video-wall.json` | Monitor layout configuration | Shared |
| `auth.yaml` | Authentication provider config | Per keyboard |

---

## 13. Open Questions for Implementation

1. **Bridge Deployment:** Bridges run locally on each keyboard (recommended) vs. centralized
2. **PTZ via GeViServer:** Any PTZ that MUST go through GeViServer?
3. **Alarm Source:** External system integration details
4. **Sequence Persistence:** Where to store sequence state on PRIMARY failover?
5. **Log Retention:** Local retention period before ELK shipping

---

## 14. Risk Mitigation

| Risk | Mitigation |
|------|------------|
| PRIMARY failure | Automatic STANDBY promotion |
| Network partition | Degraded mode maintains critical functions |
| Missed alarms | Startup query + periodic sync |
| State desync | Event verification + periodic full sync |
| SDK incompatibility | C# bridges abstract SDK differences |
| Performance | LattePanda Sigma has significant headroom |

---

## 15. Success Criteria

- [ ] ViewerConnectLive latency <100ms (direct path)
- [ ] PTZ lock acquisition <50ms
- [ ] Failover time <10 seconds
- [ ] Zero missed alarms after keyboard restart
- [ ] Degraded mode maintains camera switching
- [ ] Support 10+ concurrent keyboards
- [ ] State consistency >99.9%

---

## Document History

| Version | Date | Author | Changes |
|---------|------|--------|---------|
| 1.0 | 2026-02-03 | Claude (AI Assistant) | Initial design document |

---

## References

- COPILOT_D6 Source: `C:\DEV\COPILOT_D6\App\`
- GeViScope SDK Docs: `C:\DEV\COPILOT\GeViScope_SDK_Docs\`
- G-Core SDK Docs: `C:\DEV\COPILOT\G-Core_SDK_Docs\`
- Existing Bridges: `C:\DEV\COPILOT\geviscope-bridge\`, `C:\DEV\COPILOT\gcore-bridge\`
- API Implementation: `C:\DEV\COPILOT\geutebruck-api\`
