Initial commit: COPILOT D6 Flutter keyboard controller

Flutter web app replacing legacy WPF CCTV surveillance keyboard controller.
Includes wall overview, section view with monitor grid, camera input,
PTZ control, alarm/lock/sequence BLoCs, and legacy-matching UI styling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
klas
2026-02-12 14:57:38 +01:00
commit 40143734fc
125 changed files with 65073 additions and 0 deletions

View File

@@ -0,0 +1,126 @@
---
title: "COPILOT D6 Legacy WPF Application"
description: "Complete reverse-engineered architecture documentation of the original COPILOT D6 CCTV keyboard controller system"
---
# COPILOT D6 Legacy Architecture
> Reverse-engineered from compiled .NET 7 assemblies (build 1.0.705, December 2023) using ILSpy decompilation. This documentation covers the complete architecture of the original WPF-based CCTV keyboard controller system.
## System Overview
The COPILOT D6 system is a **CCTV surveillance keyboard controller** used to manage video feeds from multiple camera servers (Geutebruck GeViScope, G-Core, GeViSoft) across a wall of physical monitors. Operators use a custom hardware keyboard with joystick to:
- Switch cameras to monitors (CrossSwitch)
- Control PTZ cameras (Pan/Tilt/Zoom) via joystick
- Manage camera prepositions (saved positions)
- Run camera sequences (automated cycling)
- View and manage alarms
- Playback recorded video
## Architecture at a Glance
```
COPILOT SYSTEM ARCHITECTURE
============================
┌──────────────────────────────────────────────────────────┐
│ COPILOT KEYBOARD (Hardware) │
│ ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ Numpad + │ │ 3-Axis │ │ Jog/Shuttle │ │
│ │ Function │ │ Joystick │ │ Wheel │ │
│ │ Keys │ │ (HID USB) │ │ │ │
│ │ [Serial] │ │ X/Y/Z Axes │ │ [Serial] │ │
│ └──────┬───────┘ └──────┬───────┘ └───────┬────────┘ │
│ │ │ │ │
│ └─────────────────┼───────────────────┘ │
│ Arduino Leonardo │
│ (USB Composite Device) │
└────────────────────────────┬──────────────────────────────┘
│ USB (Serial + HID)
┌────────────────────────────┼──────────────────────────────┐
│ LattePanda Sigma SBC │
│ ┌─────────────────────────┴──────────────────────────┐ │
│ │ Copilot.App.exe (WPF .NET 7) │ │
│ │ ┌──────────────────────────────────────────────┐ │ │
│ │ │ Copilot.Device Layer │ │ │
│ │ │ Serial Port ←→ Keyboard Keys/Jog/Shuttle │ │ │
│ │ │ HID Device ←→ Joystick X/Y/Z │ │ │
│ │ └──────────────────────┬───────────────────────┘ │ │
│ │ │ Events │ │
│ │ ┌──────────────────────┴───────────────────────┐ │ │
│ │ │ MainWindow (WPF) │ │ │
│ │ │ Routes input → current ViewModel │ │ │
│ │ └──────────────────────┬───────────────────────┘ │ │
│ │ │ │ │
│ │ ┌──────────────────────┴───────────────────────┐ │ │
│ │ │ SegmentViewModel (main screen) │ │ │
│ │ │ • PTZ control via joystick │ │ │
│ │ │ • Camera number entry │ │ │
│ │ │ • CrossSwitch execution │ │ │
│ │ │ • Camera lock management │ │ │
│ │ │ • Playback control (jog/shuttle) │ │ │
│ │ └──┬────────────────────────────────────┬──────┘ │ │
│ │ │ Direct SDK calls │ SignalR │ │
│ │ ┌──┴──────────────────────┐ ┌─────────┴──────┐ │ │
│ │ │ Camera Server │ │ AppServer │ │ │
│ │ │ Drivers │ │ Client │ │ │
│ │ │ • GeViScope SDK │ │ (SignalR Hub) │ │ │
│ │ │ • G-Core SDK │ │ │ │ │
│ │ │ • GeViSoft SDK │ │ │ │ │
│ │ └──┬──────────────────────┘ └────────┬───────┘ │ │
│ └─────┼──────────────────────────────────┼───────────┘ │
└────────┼──────────────────────────────────┼───────────────┘
│ Native SDK (TCP) │ HTTPS/WSS
▼ ▼
┌─────────────────────┐ ┌────────────────────────┐
│ Camera Servers │ │ Copilot AppServer │
│ • GeViScope │ │ (ASP.NET Core) │
│ 192.168.102.186 │ │ copilot.test.d6... │
│ • G-Core │ │ ┌──────────────────┐ │
│ 192.168.102.20 │ │ │ SignalR Hub │ │
│ │ │ │ • Camera Locks │ │
│ Each server has: │ │ │ • Sequences │ │
│ • Cameras │ │ │ • Config Sync │ │
│ • Monitors/Viewers │ │ │ • Viewer State │ │
│ • PTZ controllers │ │ │ • Alarm History │ │
│ • Alarm events │ │ └──────────────────┘ │
│ │ │ ┌──────────────────┐ │
│ │ │ │ SQLite Database │ │
│ │ │ │ • Lock state │ │
│ │ │ │ • Alarm history │ │
│ │ │ └──────────────────┘ │
│ │ │ ┌──────────────────┐ │
│ │ │ │ REST API │ │
│ │ │ │ • Auto-updates │ │
│ │ │ │ • Configuration │ │
│ │ │ │ • Blazor Admin UI │ │
│ │ │ └──────────────────┘ │
└─────────────────────┘ └────────────────────────┘
```
## Assembly Map
| Assembly | Type | Purpose |
|----------|------|---------|
| `Copilot.App.dll` | WPF Client | Main application - UI, ViewModels, navigation, input handling |
| `Copilot.Device.dll` | Client Library | Hardware abstraction - serial port, HID joystick, key mapping |
| `Copilot.Common.dll` | Shared Library | Configuration models, data protection, providers, hub interfaces |
| `Copilot.Common.Services.dll` | Shared Library | Driver providers, media channel service, viewer state management |
| `Copilot.Drivers.Common.dll` | Shared Library | Driver interfaces (`IMovementController`, `ICameraServerDriver`, etc.) |
| `Copilot.Drivers.GeViScope.dll` | Driver | GeViScope SDK wrapper - PLC actions for cameras/PTZ |
| `Copilot.Drivers.GCore.dll` | Driver | G-Core SDK wrapper - binary protocol for cameras/PTZ |
| `Copilot.Drivers.GeviSoft.dll` | Driver | GeViSoft SDK wrapper |
| `Copilot.AppServer.Client.dll` | Client Library | SignalR hub client, availability monitoring |
| `Copilot.AppServer.dll` | Server | ASP.NET Core server - API, SignalR hub, Blazor admin |
| `Copilot.AppServer.Database.dll` | Server Library | Entity Framework - SQLite models and repositories |
| `Copilot.Camea.Client.dll` | Server Library | Camea API integration (external alarm system) |
## Documentation Pages
- **[System Architecture](./architecture.md)** - Component diagrams, dependency graph, deployment model
- **[Hardware & Input](./hardware-input.md)** - Joystick, keyboard, serial/HID protocols
- **[PTZ Control Flow](./ptz-control.md)** - Joystick → Pan/Tilt/Zoom → SDK command pipeline
- **[Data Flows](./data-flows.md)** - CrossSwitch, Alarms, Sequences, Playback, Camera Lock
- **[Configuration](./configuration.md)** - JSON config files, monitor wall topology, function buttons
- **[AppServer](./appserver.md)** - SignalR hub, REST API, database, admin UI

View File

@@ -0,0 +1,21 @@
- **Overview**
- [Home](/)
- **Legacy Architecture**
- [System Architecture](architecture.md)
- [Hardware & Input](hardware-input.md)
- [PTZ Control Flow](ptz-control.md)
- **Legacy Workflows**
- [Data Flows](data-flows.md)
- [Configuration](configuration.md)
- [AppServer](appserver.md)
- **Architecture Review**
- [Critical Infrastructure Review](architecture-review.md)
- **Migration (WPF → Flutter)**
- [Migration Overview](migration-comparison.md)
- [Migration Guide](migration-guide.md)
- [Business Rules Reference](migration-business-rules.md)
- [Implementation Guide](migration-implementation.md)

View File

@@ -0,0 +1,190 @@
---
title: "AppServer"
description: "ASP.NET Core coordination server - SignalR hub, REST API, database, admin UI"
---
# AppServer (Copilot.AppServer)
The AppServer is a centralized coordination service that runs as an ASP.NET Core application (Windows Service capable). It does NOT handle video — only coordination between keyboards.
## Architecture
```mermaid
graph TB
subgraph "Copilot.AppServer.exe"
subgraph "SignalR Hub"
CamLock["Camera Lock<br/>Management"]
SeqMgr["Sequence<br/>Scheduler"]
CfgSync["Configuration<br/>Sync"]
ViewState["Viewer State<br/>Broadcasting"]
AlarmQ["Alarm Query<br/>Proxy"]
end
subgraph "REST API (v1)"
Updates["GET /api/v1/updates/{name}<br/>Auto-update manifest"]
CfgAPI["Configuration endpoints"]
end
subgraph "Blazor Admin UI"
AdminWWW["Web-based admin panel<br/>Configuration management"]
end
subgraph "Database"
SQLite["SQLite (copilot.db)<br/>• Camera locks<br/>• Lock history<br/>• Alarm cache"]
end
subgraph "External Integrations"
CameaClient["Camea API Client<br/>http://localhost:8081<br/>Alarm data source"]
end
end
K1["Keyboard 1"] -->|WSS| CamLock
K2["Keyboard 2"] -->|WSS| SeqMgr
K3["Keyboard 3"] -->|WSS| CfgSync
CamLock --> SQLite
AlarmQ --> CameaClient
AdminWWW --> CfgSync
```
## SignalR Hub Interface
The hub exposes methods grouped by function:
### Camera Locks
```
TryLockCamera(cameraId, copilotName, priority) → CameraLockResult
UnlockCamera(cameraId, copilotName)
RequestCameraLock(cameraId, copilotName, priority)
CameraLockConfirmTakeOver(cameraId, copilotName, confirm)
ResetCameraLockExpiration(cameraId, copilotName)
GetLockedCameraIds(copilotName) → IEnumerable<int>
```
### Camera Lock Notifications (Server → Client)
```
CameraLockNotify(notification) where notification has:
- NotificationType: Acquired | TakenOver | ConfirmTakeOver | Confirmed | Rejected | ExpireSoon | Unlocked
- CameraId: int
- CopilotName: string
```
### Sequences
```
Start(viewerId, sequenceId)
Stop(viewerId)
GetSequences(categoryId) → IEnumerable<SequenceMessage>
GetSequenceCategories() → IEnumerable<SequenceCategoryMessage>
GetRunningSequences() → IEnumerable<ViewerSequenceState>
```
### Sequence Notifications (Server → Client)
```
ViewerSequenceStateChanged(ViewerSequenceState)
```
### Configuration
```
GetConfigurationFile(filename) → ConfigurationFile
```
### Configuration Notifications (Server → Client)
```
ConfigurationFileChanged(ConfigurationFile)
```
### Alarms
```
GetCamerasWithAlarms() → HashSet<int>
GetAlarmsForCamera(cameraId, from, to) → IReadOnlyList<CameraAlarm>
```
## Database Schema (SQLite)
Managed via Entity Framework Core with code-first migrations:
```mermaid
erDiagram
CameraLock {
int CameraId PK
string OwnerName
string Priority
datetime ExpiresAt
datetime CreatedAt
}
CameraLockHistory {
int Id PK
int CameraId
string OwnerName
string Action
datetime Timestamp
}
AlarmCache {
int Id PK
int CameraId
int AlarmTypeId
string Name
datetime StartTime
datetime EndTime
}
```
## Lock Expiration System
```mermaid
graph TD
Lock["Lock Created<br/>(ExpiresAt = now + 5min)"] --> Timer["Expiration Timer"]
Timer -->|"T+4min<br/>(1min before expiry)"| Warn["Send ExpireSoon<br/>notification"]
Warn --> Reset{"PTZ Action?"}
Reset -->|Yes| ExtendLock["Reset ExpiresAt = now + 5min"]
ExtendLock --> Timer
Reset -->|No timeout| Expire["Send Unlocked<br/>notification"]
Expire --> Remove["Remove lock from DB"]
```
## Auto-Update System
The AppServer serves firmware and application updates:
```
GET /api/v1/updates/{copilotName}
→ Returns JSON array of available updates:
[
{
"Version": "1.0.706",
"Url": "https://copilot.test.d6.colsys.cz/updates/Copilot-1.0.706.zip",
"Changelog": "https://...",
"Mandatory": { "Value": true, "MinVersion": "1.0.700" },
"CheckSum": { "Value": "abc123...", "HashingAlgorithm": "SHA256" }
}
]
```
The WPF app uses `AutoUpdater.NET` to check on startup and apply updates.
## Deployment
```
Copilot.AppServer.exe
├── appsettings.json (main config)
├── configs/
│ ├── appsettings-copilot.json
│ ├── appsettings-camera-servers.json
│ ├── appsettings-monitor-wall.json
│ ├── appsettings-function-buttons.json
│ ├── appsettings-prepositions.json
│ ├── appsettings-sequences.json
│ └── appsettings-sequence-categories.json
├── copilot.db (SQLite database)
├── wwwroot/ (Blazor admin UI assets)
├── logs/
│ ├── copilot-appserver-YYYYMMDD.log
│ └── buffer/ (Elasticsearch buffer)
└── web.config (IIS hosting, if used)
```
Runs as:
- Windows Service (`Microsoft.Extensions.Hosting.WindowsServices`)
- Or standalone Kestrel server on HTTPS port 443
- Uses machine certificate store for TLS

View File

@@ -0,0 +1,399 @@
# Architecture Review: Legacy vs New — Critical Infrastructure Improvements
> Pre-implementation review. This system controls traffic/tunnel cameras in critical infrastructure. Every failure mode must be addressed. The system may run on Windows, Linux, or Android tablets in the future.
## 1. Side-by-Side Failure Mode Comparison
### 1.1 Camera Server Unreachable
| Aspect | Legacy (WPF) | New (Flutter) | Verdict |
|--------|-------------|---------------|---------|
| Detection | Driver `IsConnected` check every 2 seconds | HTTP timeout (5s) | Legacy better — faster detection |
| Recovery | `CameraServerDriverReconnectService` retries every 2s | **None** — user must click retry button | **Critical gap** |
| Partial failure | Skips disconnected drivers, other servers still work | Each bridge is independent — OK | Equal |
| State on reconnect | Reloads media channels, fires `DriverConnected` event | No state resync after reconnect | **Gap** |
### 1.2 Coordination Layer Down (AppServer / PRIMARY)
| Aspect | Legacy (WPF) | New (Flutter) | Verdict |
|--------|-------------|---------------|---------|
| Detection | SignalR built-in disconnect detection | Not implemented yet | Equal (both need this) |
| Recovery | SignalR auto-reconnect: 0s, 5s, 10s, 15s fixed delays | Not implemented yet | To be built |
| Degraded mode | CrossSwitch/PTZ work, locks/sequences don't | Same design — correct | Equal |
| State on reconnect | Hub client calls `GetLockedCameraIds()`, `GetRunningSequences()` | Not implemented yet | Must match |
### 1.3 Network Failure
| Aspect | Legacy (WPF) | New (Flutter) | Verdict |
|--------|-------------|---------------|---------|
| Detection | `NetworkAvailabilityWorker` polls every 5s (checks NIC status) | **None** — no network detection | **Critical gap** |
| UI feedback | `NetworkAvailabilityState` updates UI commands | Connection status bar (manual) | **Gap** |
| Recovery | Automatic — reconnect services activate when NIC comes back | **Manual only** — user clicks retry | **Critical gap** |
### 1.4 Bridge Process Crash
| Aspect | Legacy (WPF) | New (Flutter) | Verdict |
|--------|-------------|---------------|---------|
| Detection | N/A (SDK was in-process) | HTTP timeout → connection status false | OK |
| Recovery | N/A (app restarts) | **None** — bridge stays dead | **Critical gap** |
| Prevention | N/A | Process supervision needed | Must add |
### 1.5 Flutter App Crash
| Aspect | Legacy (WPF) | New (Flutter) | Verdict |
|--------|-------------|---------------|---------|
| Recovery | App restarts, reconnects in ~5s | App restarts, must reinitialize | Equal |
| State recovery | Queries AppServer for locks, sequences, viewer states | Queries bridges for monitor states, alarms | Equal |
| Lock state | Restored via `GetLockedCameraIds()` | Restored from coordination service | Equal |
## 2. Critical Improvements Required
### 2.1 Automatic Reconnection (MUST HAVE)
The legacy system reconnects automatically at every level. Our Flutter app does not. For tunnel/traffic camera control, an operator cannot be expected to click a retry button during an emergency.
**Required reconnection layers:**
```
Layer 1: Bridge Health Polling
Flutter → periodic GET /health to each bridge
If bridge was down and comes back → auto-reconnect WebSocket + resync state
Layer 2: WebSocket Auto-Reconnect
On disconnect → exponential backoff retry (1s, 2s, 4s, 8s, max 30s)
On reconnect → resync state from bridge
Layer 3: Coordination Auto-Reconnect
On PRIMARY disconnect → retry connection with backoff
After 6s → STANDBY promotion (if configured)
On reconnect to (new) PRIMARY → resync lock/sequence state
Layer 4: Network Change Detection
Monitor network interface status
On network restored → trigger reconnection at all layers
```
**Legacy equivalent:**
- Camera drivers: 2-second reconnect loop (`CameraServerDriverReconnectService`)
- SignalR: built-in auto-reconnect with `HubRetryPolicy` (0s, 5s, 10s, 15s)
- Network: 5-second NIC polling (`NetworkAvailabilityWorker`)
### 2.2 Process Supervision (MUST HAVE)
Every .NET process (bridges + coordination service) must auto-restart on crash. An operator should never have to SSH into a machine to restart a bridge.
| Platform | Supervision Method |
|----------|--------------------|
| Windows | Windows Service (via `Microsoft.Extensions.Hosting.WindowsServices`) or NSSM |
| Linux | systemd units with `Restart=always` |
| Docker | `restart: always` policy |
| Android tablet | Bridges run on server, not locally |
**Proposed process tree:**
```
LattePanda Sigma (per keyboard)
├── copilot-geviscope-bridge.service (auto-restart)
├── copilot-gcore-bridge.service (auto-restart)
├── copilot-geviserver-bridge.service (auto-restart)
├── copilot-coordinator.service (auto-restart, PRIMARY only)
└── copilot-keyboard.service (auto-restart, Flutter desktop)
or browser tab (Flutter web)
```
### 2.3 Health Monitoring Dashboard (SHOULD HAVE)
The operator must see at a glance what's working and what's not.
```
┌──────────────────────────────────────────────────────────┐
│ System Status │
│ ┌────────────┐ ┌────────────┐ ┌────────────────────┐ │
│ │ GeViScope │ │ G-Core │ │ Coordination │ │
│ │ ● Online │ │ ● Online │ │ ● PRIMARY active │ │
│ │ 12 cams │ │ 8 cams │ │ 2 keyboards │ │
│ │ 6 viewers │ │ 4 viewers │ │ 1 lock active │ │
│ └────────────┘ └────────────┘ └────────────────────┘ │
│ │
│ ⚠ G-Core bridge reconnecting (attempt 3/∞) │
└──────────────────────────────────────────────────────────┘
```
### 2.4 Command Retry with Idempotency (SHOULD HAVE)
Critical commands (CrossSwitch) should retry on transient failure:
```dart
Future<bool> viewerConnectLive(int viewer, int channel) async {
for (int attempt = 1; attempt <= 3; attempt++) {
try {
final response = await _client.post('/viewer/connect-live', ...);
if (response.statusCode == 200) return true;
} catch (e) {
if (attempt == 3) rethrow;
await Future.delayed(Duration(milliseconds: 200 * attempt));
}
}
return false;
}
```
PTZ commands should NOT retry (they're continuous — a stale retry would cause unexpected movement).
### 2.5 State Verification After Reconnection (MUST HAVE)
After any reconnection event, the app must not trust its cached state:
```
On bridge reconnect:
1. Query GET /monitors → rebuild monitor state
2. Query GET /alarms/active → rebuild alarm state
3. Re-subscribe WebSocket events
On coordination reconnect:
1. Query locks → rebuild lock state
2. Query running sequences → update sequence state
3. Re-subscribe lock/sequence change events
```
Legacy does this: `ViewerStatesInitWorker` rebuilds viewer state on startup/reconnect. `ConfigurationService.OnChangeAvailability` resyncs config when AppServer comes back.
## 3. Platform Independence Analysis
### 3.1 Current Platform Assumptions
| Component | Current Assumption | Future Need |
|-----------|-------------------|-------------|
| C# Bridges | Run locally on Windows (LattePanda) | Linux, Docker, remote server |
| Flutter App | Windows desktop or browser | Linux, Android tablet, browser |
| Coordination | Runs on PRIMARY keyboard (Windows) | Linux, Docker, any host |
| Hardware I/O | USB Serial + HID on local machine | Remote keyboard via network, or Bluetooth |
| Bridge URLs | `http://localhost:7720` | `http://192.168.x.y:7720` (already configurable) |
### 3.2 Architecture for Platform Independence
```mermaid
graph TB
subgraph "Deployment A: LattePanda (Current)"
LP_App["Flutter Desktop"]
LP_Bridge1["GeViScope Bridge"]
LP_Bridge2["G-Core Bridge"]
LP_Coord["Coordinator"]
LP_Serial["USB Serial/HID"]
LP_App --> LP_Bridge1
LP_App --> LP_Bridge2
LP_App --> LP_Coord
LP_Serial --> LP_App
end
subgraph "Deployment B: Android Tablet (Future)"
AT_App["Flutter Android"]
AT_BT["Bluetooth Keyboard"]
AT_App -->|"HTTP over WiFi"| Remote_Bridge1["Bridge on Server"]
AT_App -->|"HTTP over WiFi"| Remote_Bridge2["Bridge on Server"]
AT_App -->|"WebSocket"| Remote_Coord["Coordinator on Server"]
AT_BT --> AT_App
end
subgraph "Deployment C: Linux Kiosk (Future)"
LX_App["Flutter Linux"]
LX_Bridge1["GeViScope Bridge"]
LX_Bridge2["G-Core Bridge"]
LX_Coord["Coordinator"]
LX_Serial["USB Serial/HID"]
LX_App --> LX_Bridge1
LX_App --> LX_Bridge2
LX_App --> LX_Coord
LX_Serial --> LX_App
end
Remote_Bridge1 --> CS1["Camera Server 1"]
Remote_Bridge2 --> CS2["Camera Server 2"]
LP_Bridge1 --> CS1
LP_Bridge2 --> CS2
LX_Bridge1 --> CS1
LX_Bridge2 --> CS2
```
### 3.3 Key Design Rules for Platform Independence
1. **Flutter app never assumes bridges are on localhost.** Bridge URLs come from `servers.json`. Already the case.
2. **Bridges are deployable anywhere .NET 8 runs.** Currently Windows x86/x64. Must also build for Linux x64 and linux-arm64.
3. **Coordination service is just another network service.** Flutter app connects to it like a bridge — via configured URL.
4. **Hardware I/O is abstracted behind a service interface.** `KeyboardService` interface has platform-specific implementations:
- `NativeSerialKeyboardService` (desktop with USB)
- `WebSerialKeyboardService` (browser with Web Serial API)
- `BluetoothKeyboardService` (tablet with BT keyboard, future)
- `EmulatedKeyboardService` (development/testing)
5. **No platform-specific code in business logic.** All platform differences are in the service layer, injected via DI.
## 4. Coordination Service Design (Option B)
### 4.1 Service Overview
A minimal .NET 8 ASP.NET Core application (~400 lines) running on the PRIMARY keyboard:
```
copilot-coordinator/
├── Program.cs # Minimal API setup, WebSocket, endpoints
├── Services/
│ ├── LockManager.cs # Camera lock state (ported from legacy CameraLocksService)
│ ├── SequenceRunner.cs # Sequence execution (ported from legacy SequenceService)
│ └── KeyboardRegistry.cs # Track connected keyboards
├── Models/
│ ├── CameraLock.cs # Lock state model
│ ├── SequenceState.cs # Running sequence model
│ └── Messages.cs # WebSocket message types
└── appsettings.json # Lock timeout, heartbeat interval config
```
### 4.2 REST API
```
GET /health → Service health
GET /status → Connected keyboards, active locks, sequences
POST /locks/try {cameraId, keyboardId, priority} → Acquire lock
POST /locks/release {cameraId, keyboardId} → Release lock
POST /locks/takeover {cameraId, keyboardId, priority} → Request takeover
POST /locks/confirm {cameraId, keyboardId, confirm} → Confirm/reject takeover
POST /locks/reset {cameraId, keyboardId} → Reset expiration
GET /locks → All active locks
GET /locks/{keyboardId} → Locks held by keyboard
POST /sequences/start {viewerId, sequenceId} → Start sequence
POST /sequences/stop {viewerId} → Stop sequence
GET /sequences/running → Active sequences
WS /ws → Real-time events
```
### 4.3 WebSocket Events (broadcast to all connected keyboards)
```json
{"type": "lock_acquired", "cameraId": 5, "keyboardId": "KB1", "expiresAt": "..."}
{"type": "lock_released", "cameraId": 5}
{"type": "lock_expiring", "cameraId": 5, "keyboardId": "KB1", "expiresIn": 60}
{"type": "lock_takeover", "cameraId": 5, "from": "KB1", "to": "KB2"}
{"type": "sequence_started", "viewerId": 1001, "sequenceId": 3}
{"type": "sequence_stopped", "viewerId": 1001}
{"type": "keyboard_online", "keyboardId": "KB2"}
{"type": "keyboard_offline", "keyboardId": "KB2"}
{"type": "heartbeat"}
```
### 4.4 Failover (Configured STANDBY)
```
keyboards.json:
{
"keyboards": [
{"id": "KB1", "role": "PRIMARY", "coordinatorPort": 8090},
{"id": "KB2", "role": "STANDBY", "coordinatorPort": 8090}
]
}
```
- PRIMARY starts coordinator service on `:8090`
- STANDBY monitors PRIMARY's `/health` endpoint
- If PRIMARY unreachable for 6 seconds → STANDBY starts its own coordinator
- When old PRIMARY recovers → checks if another coordinator is running → defers (becomes STANDBY)
- Lock state after failover: **empty** (locks expire naturally in ≤5 minutes, same as legacy AppServer restart behavior)
## 5. Improvement Summary: Legacy vs New
### What the New System Does BETTER
| Improvement | Detail |
|-------------|--------|
| No central server hardware | Coordinator runs on keyboard, not separate machine |
| Alarm reliability | Query + Subscribe + Periodic sync (legacy had event-only + hourly refresh) |
| Direct command path | CrossSwitch/PTZ bypass coordinator entirely (legacy routed some through AppServer) |
| Multiplatform | Flutter + .NET 8 run on Windows, Linux, Android. Legacy was Windows-only WPF |
| No SDK dependency in UI | Bridges abstract SDKs behind REST. UI never touches native code |
| Independent operation | Each keyboard works standalone for critical ops. Legacy needed AppServer for several features |
| Deployable anywhere | Bridges + coordinator can run on any server, not just the keyboard |
### What the New System Must MATCH (Currently Missing)
| Legacy Feature | Legacy Implementation | New Implementation Needed |
|---------------|----------------------|---------------------------|
| Auto-reconnect to camera servers | 2-second periodic retry service | Bridge health polling + WebSocket auto-reconnect |
| Auto-reconnect to AppServer | SignalR built-in (0s, 5s, 10s, 15s) | Coordinator WebSocket auto-reconnect with backoff |
| Network detection | 5-second NIC polling worker | `connectivity_plus` package or periodic health checks |
| State resync on reconnect | `ViewerStatesInitWorker`, config resync on availability change | Query bridges + coordinator on any reconnect event |
| Graceful partial failure | `Parallel.ForEach` with per-driver try-catch | Already OK (each bridge independent) |
| Process watchdog | Windows Service | systemd / Windows Service / Docker restart policy |
| Media channel refresh | 10-minute periodic refresh | Periodic bridge status query |
### What the New System Should Do BETTER THAN Legacy
| Improvement | Legacy Gap | New Approach |
|-------------|-----------|--------------|
| Exponential backoff | Fixed delays (0, 5, 10, 15s) — no backoff | Exponential: 1s, 2s, 4s, 8s, max 30s with jitter |
| Circuit breaker | None — retries forever even if server is gone | After N failures, back off to slow polling (60s) |
| Command retry | None — single attempt | Retry critical commands (CrossSwitch) 3x with 200ms delay |
| Health visibility | Hidden in logs | Operator-facing status dashboard in UI |
| Structured logging | Basic ILogger | JSON structured logging → ELK (already in design) |
| Graceful degradation UI | Commands silently disabled | Clear visual indicator: "Degraded mode — locks unavailable" |
## 6. Proposed Resilience Architecture
```mermaid
graph TB
subgraph "Flutter App"
UI["UI Layer"]
BLoCs["BLoC Layer"]
RS["ReconnectionService"]
HS["HealthService"]
BS["BridgeService"]
CS["CoordinationClient"]
KS["KeyboardService"]
end
subgraph "Health & Reconnection"
RS -->|"periodic /health"| Bridge1["GeViScope Bridge"]
RS -->|"periodic /health"| Bridge2["G-Core Bridge"]
RS -->|"periodic /health"| Coord["Coordinator"]
RS -->|"on failure"| BS
RS -->|"on failure"| CS
HS -->|"status stream"| BLoCs
end
subgraph "Normal Operation"
BS -->|"REST commands"| Bridge1
BS -->|"REST commands"| Bridge2
BS -->|"WebSocket events"| Bridge1
BS -->|"WebSocket events"| Bridge2
CS -->|"REST + WebSocket"| Coord
end
BLoCs --> UI
KS -->|"Serial/HID"| BLoCs
```
**New services needed in Flutter app:**
| Service | Responsibility |
|---------|---------------|
| `ReconnectionService` | Polls bridge `/health` endpoints, auto-reconnects WebSocket, triggers state resync |
| `HealthService` | Aggregates health of all bridges + coordinator, exposes stream to UI |
| `CoordinationClient` | REST + WebSocket client to coordinator (locks, sequences, heartbeat) |
## 7. Action Items Before Implementation
- [ ] **Create coordination service** (.NET 8 minimal API, ~400 lines)
- [ ] **Add `ReconnectionService`** to Flutter app (exponential backoff, health polling)
- [ ] **Add `HealthService`** to Flutter app (status aggregation for UI)
- [ ] **Add `CoordinationClient`** to Flutter app (locks, sequences)
- [ ] **Fix WebSocket auto-reconnect** in `BridgeService`
- [ ] **Add command retry** for CrossSwitch (3x with backoff)
- [ ] **Add bridge process supervision** (systemd/Windows Service configs)
- [ ] **Add state resync** on every reconnect event
- [ ] **Build health status UI** component
- [ ] **Update `servers.json`** schema to include coordinator URL
- [ ] **Build for Linux** — verify .NET 8 bridges compile for linux-x64
- [ ] **Abstract keyboard input** behind `KeyboardService` interface with platform impls

View File

@@ -0,0 +1,268 @@
---
title: "System Architecture"
description: "Component diagrams, dependency graph, and deployment model"
---
# System Architecture
## Component Dependency Graph
```mermaid
graph TB
subgraph "Copilot.App.exe (WPF Client)"
App["App.cs<br/>.NET Generic Host"]
MW["MainWindow<br/>Input Router"]
SVM["SegmentViewModel<br/>Main Screen Logic"]
PVM["PlaybackViewModel"]
CSVM["CameraSearchViewModel"]
PreVM["PrepositionsViewModel"]
SeqVM["SequenceCategoriesViewModel<br/>SequencesViewModel"]
SvcVM["ServiceMenuViewModel"]
subgraph "App Services"
CCS["CameraControllerService"]
CLS["CameraLockService"]
CAS["CameraAlarmService"]
SeqS["SequenceService"]
FBS["FunctionButtonsService"]
PBS["PlaybackStateService"]
CfgS["ConfigurationService"]
NAS["NetworkAvailabilityState"]
end
subgraph "Navigation"
NS["NavigationService"]
NStore["NavigationStore"]
end
end
subgraph "Copilot.Device.dll"
CD["CopilotDevice"]
SP["SerialPortDataProvider"]
HID["JoystickHidDataProvider"]
end
subgraph "Copilot.Common.dll"
Config["Configuration System"]
Hub["ICopilotHub<br/>ICopilotHubEvents"]
Prov["ICopilotInfoProvider"]
end
subgraph "Copilot.Common.Services.dll"
MCS["MediaChannelService"]
VSS["ViewerStateService"]
CSDP["CameraServerDriverProvider"]
CenDP["CentralServerDriverProvider"]
end
subgraph "Copilot.Drivers.Common.dll"
IMC["IMovementController"]
ICSD["ICameraServerDriver"]
ICenD["ICentralServerDriver"]
IVC["IViewerController"]
end
subgraph "Copilot.Drivers.GeViScope.dll"
GscD["GeViScopeDriver"]
GscMC["GeViScopeMovementController"]
GscVC["GeviScopeViewerController"]
end
subgraph "Copilot.Drivers.GCore.dll"
GcD["GCoreDriver"]
GcMC["GCoreMovementController"]
GcVC["GCoreViewerController"]
end
subgraph "Copilot.AppServer.Client.dll"
ASC["AppServerClient<br/>(SignalR)"]
Avail["AvailabilityState"]
end
App --> MW
MW --> CD
MW --> SVM
SVM --> CCS
SVM --> CLS
SVM --> FBS
SVM --> CAS
CLS --> Hub
CAS --> Hub
SeqS --> Hub
CfgS --> Hub
CCS --> CSDP
CCS --> MCS
CSDP --> GscD
CSDP --> GcD
CenDP --> GscD
GscD --> GscMC
GscD --> GscVC
GcD --> GcMC
GcD --> GcVC
GscMC -.->|implements| IMC
GcMC -.->|implements| IMC
GscD -.->|implements| ICSD
GcD -.->|implements| ICSD
Hub --> ASC
```
## Service Lifetime & Registration
All services are registered via `Microsoft.Extensions.DependencyInjection` in `App.cs`:
```mermaid
graph LR
subgraph "Singletons (one instance)"
MainWindow
SegmentViewModel
NavigationStore
NavigationService
CameraLockService
PlaybackStateService
ConfigurationService
CameraAlarmService
CameraControllerService
PrepositionService
SequenceService
NetworkAvailabilityState
end
subgraph "Transient (new per request)"
CameraSearchViewModel
PlaybackViewModel
PrepositionsViewModel
SequenceCategoriesViewModel
SequencesViewModel
ServiceMenuViewModel
ErrorViewModel
CameraAlarmHistoryViewModel
end
subgraph "Hosted Services (background workers)"
StartupConfigurationCheckWorker
ViewerStatesInitWorker
CameraAlarmsUpdateWorker
NetworkAvailabilityWorker
SequenceService_Hosted["SequenceService<br/>(also IHostedService)"]
ProcessMonitorServiceHost
end
```
## Startup Sequence
```mermaid
sequenceDiagram
participant App as App.cs
participant Host as .NET Host
participant MW as MainWindow
participant Nav as NavigationService
participant Device as CopilotDevice
participant Hub as SignalR Hub
participant Config as ConfigurationService
App->>Host: Build & ConfigureServices
Host->>Host: Register all DI services
App->>Host: StartAsync()
Host->>Host: Start background workers
Note over Host: StartupConfigurationCheckWorker starts
Note over Host: ViewerStatesInitWorker starts
Note over Host: CameraAlarmsUpdateWorker starts
Note over Host: NetworkAvailabilityWorker starts
App->>Nav: Navigate<SegmentsPage>(wallId)
App->>MW: Show()
MW->>Device: Subscribe JoystickMoved
MW->>Device: Subscribe VirtualKeyDown/Up
alt Firmware outdated
MW->>Nav: Navigate<UpdateFirmwarePage>
end
App->>Config: UpdateAllConfigurationsIfChanged()
Config->>Hub: GetConfigurationFile() for each manager
Config->>Config: Write updated configs to disk
Note over App: Application Ready
```
## Navigation System
The app uses a custom stack-based navigation system (not WPF Navigation):
```mermaid
stateDiagram-v2
[*] --> SegmentsPage: Initial (select segment)
SegmentsPage --> SegmentPage: Select segment
SegmentPage --> CameraSearchPage: Search button
SegmentPage --> PrepositionsPage: Prepositions button
SegmentPage --> PlaybackPage: Playback button
SegmentPage --> SequenceCategoriesPage: Sequence button
SegmentPage --> CameraAlarmHistoryPage: Alarm history
SegmentPage --> ServiceMenuPage: Hold Backspace 3s
SegmentPage --> CameraLockExpirationView: Lock expiring
CameraSearchPage --> SegmentPage: Back
PrepositionsPage --> PrepositionAddPage: Add preset
PrepositionAddPage --> PrepositionsPage: Back
PrepositionsPage --> SegmentPage: Back
PlaybackPage --> SegmentPage: Back
SequenceCategoriesPage --> SequencesPage: Select category
SequencesPage --> SegmentPage: Start sequence
ServiceMenuPage --> UpdateFirmwarePage: Update firmware
```
## Two Communication Paths
The app communicates with two distinct backends simultaneously:
### Path 1: Direct SDK (Camera Operations)
```
App → CameraServerDriverProvider → GeViScope/GCore Driver → Native SDK → Camera Server
```
- **Used for:** CrossSwitch, PTZ, Playback, Viewer control
- **Latency:** < 50ms (direct TCP connection to camera server)
- **Reconnection:** Automatic, 2-second retry interval
### Path 2: SignalR Hub (Coordination)
```
App → CopilotHub (SignalR) → AppServer → Database/Logic → Response
```
- **Used for:** Camera locks, sequences, config sync, alarm history, viewer state
- **Latency:** ~100-200ms (HTTPS + server processing)
- **Reconnection:** Automatic via SignalR reconnection policy
```mermaid
graph LR
subgraph "App"
SVM["SegmentViewModel"]
end
subgraph "Path 1: Direct (low latency)"
Driver["GeViScope/GCore Driver"]
SDK["Native SDK (TCP)"]
end
subgraph "Path 2: Coordinated"
HubC["SignalR Client"]
HTTPS["HTTPS/WSS"]
end
subgraph "Camera Server"
CS["GeViScope / G-Core"]
end
subgraph "AppServer"
HubS["SignalR Hub"]
DB["SQLite DB"]
end
SVM -->|"CrossSwitch, PTZ,<br/>Playback"| Driver
Driver --> SDK --> CS
SVM -->|"Locks, Sequences,<br/>Config, Alarms"| HubC
HubC --> HTTPS --> HubS
HubS --> DB
```

View File

@@ -0,0 +1,253 @@
---
title: "Configuration"
description: "JSON configuration files, monitor wall topology, and function button mapping"
---
# Configuration System
## Configuration Files
All configuration is stored as JSON files, managed centrally on the AppServer and synced to each keyboard.
### appsettings.json (Main)
```json
{
"UseSoftwareRendering": true,
"AppServerConfiguration": {
"Uri": "https://copilot.test.d6.colsys.cz",
"Timeout": "00:00:01.5"
},
"ReconnectionConfiguration": {
"ReconnectionPeriod": "00:00:02"
},
"ViewerStates": {
"AppServerConnectionTimeout": "00:00:05"
}
}
```
| Setting | Value | Purpose |
|---------|-------|---------|
| `UseSoftwareRendering` | `true` | Forces WPF software rendering (no GPU) for LattePanda |
| `AppServerConfiguration.Uri` | HTTPS URL | Central AppServer for coordination |
| `AppServerConfiguration.Timeout` | 1.5s | HTTP request timeout |
| `ReconnectionPeriod` | 2s | Auto-reconnect interval to camera servers |
| `AppServerConnectionTimeout` | 5s | SignalR connection timeout |
### appsettings-copilot.json (Per-Keyboard)
```json
{
"CopilotConfig": {
"WallId": 2,
"CameraLockOptions": {
"Priority": "Low",
"Timeout": "00:05:00"
},
"CameraNumberOptions": {
"MaxLength": 6,
"Prefixes": ["500", "501", "502"],
"CancelEditTimeout": "00:05"
},
"ViewerStateThrottleInterval": "00:00:00.150",
"CentralServer": {
"IpAddress": "192.168.102.186",
"UserName": "sysadmin",
"Password": "masterkey"
},
"AllowPrepositionsInterval": {
"Min": 10,
"Max": 99
},
"PlaybackOptions": {
"Allowed": true,
"Speeds": [1, 2, 5, 15, 30, 100, 250]
},
"AlarmHistoryOptions": {
"CameraIdExtractionRegex": "^\\D*(?<CameraId>\\d{6})\\D*$",
"DefaultSearchInterval": "30.00:00:00"
}
}
}
```
| Setting | Purpose |
|---------|---------|
| `WallId` | Which monitor wall this keyboard controls |
| `CameraLockOptions.Priority` | PTZ lock priority (`Low` / `High`) |
| `CameraLockOptions.Timeout` | Lock auto-expiration (5 min) |
| `CameraNumberOptions.MaxLength` | Max digits for camera number (6) |
| `CameraNumberOptions.Prefixes` | Auto-prefixes for camera numbers |
| `CancelEditTimeout` | Auto-cancel camera number entry (5 sec) |
| `ViewerStateThrottleInterval` | Debounce viewer state updates (150ms) |
| `PlaybackOptions.Speeds` | Shuttle playback speeds (1x to 250x) |
| `AllowPrepositionsInterval` | Valid preposition range (10-99) |
### appsettings-camera-servers.json
```json
{
"CameraServerConfig": {
"CameraServers": [
{
"IpAddress": "192.168.102.20",
"UserName": "sysadmin",
"Password": "masterkey",
"CameraServerType": "GCore"
},
{
"IpAddress": "192.168.102.186",
"UserName": "sysadmin",
"Password": "masterkey",
"CameraServerType": "GeviScope"
}
]
}
}
```
Supported server types: `GeViScope`, `GCore`, `GeViSoft`
### appsettings-monitor-wall.json (Topology)
Defines the physical monitor wall layout:
```mermaid
graph TD
subgraph "Wall (WallId: 1, Name: HDŘÚ TSK/DIC)"
subgraph "Segment 1 (4 rows × 8 cols)"
M1["Monitor #01<br/>2×2 span<br/>VM: 151,152,153,154"]
M2["Monitor #02<br/>2×2 span<br/>VM: 155,156,157,158"]
M3["Monitor #03"]
M4["Monitor #04"]
M5["Monitor #05<br/>...etc"]
end
end
```
Hierarchy:
```
Wall
└── Segment (grid: RowCount × ColumnCount)
└── Monitor (physical screen, positioned at Row/Col with RowSpan/ColSpan)
└── VirtualMonitor (individual video feed, positioned within monitor grid)
```
A single physical monitor can display multiple virtual monitors (e.g., 2×2 quad view). Each VirtualMonitor has a unique `VirtualMonitorId` that maps to a `ViewerId` in the camera server.
### appsettings-function-buttons.json
Maps F1-F7 and Home keys to actions per wall:
```json
{
"FunctionButtonsConfig": {
"FunctionButtons": [
{
"WallId": 1,
"Buttons": [
{
"Button": "F1",
"Actions": [
{ "ViewerId": 1001, "ActionType": "CrossSwitch", "SourceId": 501111 },
{ "ViewerId": 1002, "ActionType": "CrossSwitch", "SourceId": 502345 }
]
},
{
"Button": "F4",
"Actions": [
{ "ViewerId": 1001, "ActionType": "SequenceStart", "SourceId": 258 }
]
}
]
}
]
}
}
```
Action types: `CrossSwitch` (switch camera to viewer), `SequenceStart` (start camera cycling)
### appsettings-sequences.json
```json
{
"SequencesConfig": {
"Sequences": [
{
"Id": 1,
"CategoryId": 1,
"Name": "Test-A",
"Interval": 1,
"MediaChannels": [500100, 500101, 500102, 500103]
}
]
}
}
```
## AppServer Configuration
```json
{
"ConnectionStrings": {
"SqliteConnection": "Data Source=copilot.db"
},
"CameaApiClient": {
"Uri": "http://localhost:8081",
"Timeout": "00:00:01.5",
"TimeZone": "Central Europe Standard Time"
},
"LockExpirationConfig": {
"LockExpirationTimeout": "00:05:00",
"NotificationBeforeExpiration": "00:01:00"
},
"Kestrel": {
"Endpoints": {
"Https": {
"Url": "https://*:443",
"Certificate": {
"Subject": "copilot.test.d6.colsys.cz",
"Store": "Root",
"Location": "LocalMachine"
}
}
}
}
}
```
## Network Topology
```
┌───────────────────────────────────────────────────────────┐
│ 192.168.102.0/24 Network │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ GeViScope Server │ │ G-Core Server │ │
│ │ 192.168.102.186 │ │ 192.168.102.20 │ │
│ │ sysadmin/master │ │ sysadmin/master │ │
│ └────────┬─────────┘ └────────┬────────┘ │
│ │ TCP (native SDK) │ TCP (native SDK) │
│ │ │ │
│ ┌────────┴──────────────────────┴────────┐ │
│ │ Keyboard SBC │ │
│ │ (LattePanda Sigma) │ │
│ │ Copilot.App.exe (WPF) │ │
│ └────────┬───────────────────────────────┘ │
│ │ HTTPS/WSS (SignalR) │
│ │ │
│ ┌────────┴───────────────────────────────┐ │
│ │ AppServer │ │
│ │ copilot.test.d6.colsys.cz:443 │ │
│ │ Copilot.AppServer.exe │ │
│ │ + SQLite DB + Camea API (:8081) │ │
│ └────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────┐ │
│ │ Elasticsearch (logging) │ │
│ │ localhost:9200 │ │
│ └────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────┘
```

View File

@@ -0,0 +1,139 @@
<mxfile host="app.diagrams.net" modified="2026-02-10T00:00:00.000Z" agent="Claude" version="24.0.0">
<diagram id="copilot-architecture" name="System Architecture">
<mxGraphModel dx="1422" dy="794" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1600" pageHeight="1200" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<!-- Title -->
<mxCell id="title" value="COPILOT D6 Legacy System Architecture" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=20;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="400" y="20" width="500" height="40" as="geometry" />
</mxCell>
<!-- Hardware Keyboard -->
<mxCell id="hw-group" value="COPILOT Keyboard Hardware" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;verticalAlign=top;fontStyle=1;fontSize=12;" vertex="1" parent="1">
<mxGeometry x="80" y="80" width="540" height="120" as="geometry" />
</mxCell>
<mxCell id="hw-keys" value="Button Panel&#xa;(Serial Port)&#xa;Numpad + F1-F7" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
<mxGeometry x="100" y="110" width="140" height="70" as="geometry" />
</mxCell>
<mxCell id="hw-joy" value="3-Axis Joystick&#xa;(USB HID)&#xa;X/Y/Z ±255" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="1">
<mxGeometry x="260" y="110" width="140" height="70" as="geometry" />
</mxCell>
<mxCell id="hw-shuttle" value="Jog/Shuttle&#xa;(Serial Port)&#xa;±7 positions" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
<mxGeometry x="420" y="110" width="140" height="70" as="geometry" />
</mxCell>
<mxCell id="hw-arduino" value="Arduino Leonardo (USB Composite)" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=10;fontColor=#999999;" vertex="1" parent="1">
<mxGeometry x="200" y="185" width="260" height="20" as="geometry" />
</mxCell>
<!-- USB Arrow -->
<mxCell id="usb-arrow" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;containSize=1;strokeWidth=2;" edge="1" parent="1" source="hw-group" target="device-layer">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="usb-label" value="USB (Serial + HID)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;" vertex="1" connectable="0" parent="usb-arrow">
<mxGeometry x="-0.2" relative="1" as="geometry"><mxPoint as="offset" /></mxGeometry>
</mxCell>
<!-- WPF App -->
<mxCell id="app-group" value="Copilot.App.exe (WPF .NET 7)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;verticalAlign=top;fontStyle=1;fontSize=12;" vertex="1" parent="1">
<mxGeometry x="40" y="240" width="620" height="440" as="geometry" />
</mxCell>
<!-- Device Layer -->
<mxCell id="device-layer" value="Copilot.Device&#xa;SerialPortDataProvider + JoystickHidDataProvider" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;" vertex="1" parent="1">
<mxGeometry x="120" y="270" width="460" height="40" as="geometry" />
</mxCell>
<!-- MainWindow -->
<mxCell id="mainwindow" value="MainWindow&#xa;(Input Router → Current ViewModel)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="120" y="330" width="460" height="40" as="geometry" />
</mxCell>
<!-- SegmentViewModel -->
<mxCell id="segmentvm" value="SegmentViewModel (Main Screen)&#xa;PTZ Control | CrossSwitch | Camera Lock | Playback | Sequences" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="80" y="390" width="540" height="50" as="geometry" />
</mxCell>
<!-- Services Row -->
<mxCell id="svc-camera" value="Camera&#xa;Controller&#xa;Service" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="60" y="460" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="svc-lock" value="Camera&#xa;Lock&#xa;Service" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="160" y="460" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="svc-alarm" value="Camera&#xa;Alarm&#xa;Service" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="260" y="460" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="svc-seq" value="Sequence&#xa;Service" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="360" y="460" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="svc-func" value="Function&#xa;Buttons&#xa;Service" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="460" y="460" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="svc-config" value="Config&#xa;Service" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="560" y="460" width="80" height="50" as="geometry" />
</mxCell>
<!-- Drivers Row -->
<mxCell id="drv-geviscope" value="GeViScope Driver&#xa;(GscPLCWrapper)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="60" y="540" width="130" height="40" as="geometry" />
</mxCell>
<mxCell id="drv-gcore" value="G-Core Driver&#xa;(GngPLCWrapper)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="200" y="540" width="130" height="40" as="geometry" />
</mxCell>
<mxCell id="drv-gevisoft" value="GeViSoft Driver" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="340" y="540" width="100" height="40" as="geometry" />
</mxCell>
<mxCell id="hub-client" value="SignalR Hub Client&#xa;(WSS)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="470" y="540" width="140" height="40" as="geometry" />
</mxCell>
<!-- App Arrows -->
<mxCell id="a1" style="edgeStyle=orthogonalEdgeStyle;rounded=0;" edge="1" parent="1" source="device-layer" target="mainwindow"><mxGeometry relative="1" as="geometry" /></mxCell>
<mxCell id="a2" style="edgeStyle=orthogonalEdgeStyle;rounded=0;" edge="1" parent="1" source="mainwindow" target="segmentvm"><mxGeometry relative="1" as="geometry" /></mxCell>
<!-- Camera Servers -->
<mxCell id="srv-geviscope" value="GeViScope Server&#xa;192.168.102.186&#xa;(Central + Camera)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="40" y="740" width="180" height="60" as="geometry" />
</mxCell>
<mxCell id="srv-gcore" value="G-Core Server&#xa;192.168.102.20&#xa;(Camera)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="240" y="740" width="160" height="60" as="geometry" />
</mxCell>
<!-- AppServer -->
<mxCell id="appserver" value="Copilot.AppServer.exe&#xa;(ASP.NET Core)&#xa;HTTPS :443 + SignalR Hub&#xa;SQLite DB + Camea API + Blazor Admin" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;fontStyle=1;" vertex="1" parent="1">
<mxGeometry x="440" y="730" width="240" height="80" as="geometry" />
</mxCell>
<!-- SDK Arrows -->
<mxCell id="sdk1" style="edgeStyle=orthogonalEdgeStyle;rounded=0;strokeWidth=2;strokeColor=#9673a6;" edge="1" parent="1" source="drv-geviscope" target="srv-geviscope"><mxGeometry relative="1" as="geometry" /></mxCell>
<mxCell id="sdk1-label" value="Native SDK&#xa;(TCP)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;" vertex="1" connectable="0" parent="sdk1"><mxGeometry relative="1" as="geometry"><mxPoint as="offset" /></mxGeometry></mxCell>
<mxCell id="sdk2" style="edgeStyle=orthogonalEdgeStyle;rounded=0;strokeWidth=2;strokeColor=#9673a6;" edge="1" parent="1" source="drv-gcore" target="srv-gcore"><mxGeometry relative="1" as="geometry" /></mxCell>
<mxCell id="sdk2-label" value="Native SDK&#xa;(TCP)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;" vertex="1" connectable="0" parent="sdk2"><mxGeometry relative="1" as="geometry"><mxPoint as="offset" /></mxGeometry></mxCell>
<mxCell id="hub-arrow" style="edgeStyle=orthogonalEdgeStyle;rounded=0;strokeWidth=2;strokeColor=#b85450;dashed=1;" edge="1" parent="1" source="hub-client" target="appserver"><mxGeometry relative="1" as="geometry" /></mxCell>
<mxCell id="hub-label" value="HTTPS/WSS&#xa;SignalR" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;" vertex="1" connectable="0" parent="hub-arrow"><mxGeometry relative="1" as="geometry"><mxPoint as="offset" /></mxGeometry></mxCell>
<!-- Legend -->
<mxCell id="legend" value="Legend" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#666666;verticalAlign=top;fontStyle=1;fontSize=11;" vertex="1" parent="1">
<mxGeometry x="720" y="80" width="200" height="160" as="geometry" />
</mxCell>
<mxCell id="leg1" value="Hardware" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#666666;fontSize=9;" vertex="1" parent="1">
<mxGeometry x="730" y="105" width="80" height="25" as="geometry" />
</mxCell>
<mxCell id="leg2" value="UI Layer" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;fontSize=9;" vertex="1" parent="1">
<mxGeometry x="730" y="135" width="80" height="25" as="geometry" />
</mxCell>
<mxCell id="leg3" value="Services" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=9;" vertex="1" parent="1">
<mxGeometry x="730" y="165" width="80" height="25" as="geometry" />
</mxCell>
<mxCell id="leg4" value="Drivers/SDK" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=9;" vertex="1" parent="1">
<mxGeometry x="730" y="195" width="80" height="25" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>

View File

@@ -0,0 +1,287 @@
---
title: "Data Flows"
description: "CrossSwitch, Alarms, Sequences, Playback, and Camera Lock workflows"
---
# Data Flows
## 1. CrossSwitch (Switch Camera to Monitor)
The primary operation — routing a camera feed to a physical monitor output.
```mermaid
sequenceDiagram
participant Op as Operator
participant VM as SegmentViewModel
participant CenD as CentralServerDriver
participant Srv as Camera Server
Op->>VM: Enter camera number (digits + Enter)
VM->>VM: Validate camera exists via MediaChannelService
alt Camera exists
alt Playback active on this viewer
VM->>VM: playbackStateService.SetPlaybackState(false)
end
alt Sequence active on this viewer
VM->>VM: hub.Stop(viewerId)
end
VM->>CenD: CrossSwitch(viewerId, cameraNumber)
CenD->>Srv: SDK CrossSwitch command
Note over Srv: Server switches video output<br/>and fires ViewerConnectionEvent
else Camera not found
VM->>VM: ShowError("MediaChannel does not exist", 2s)
end
```
### CrossSwitch Triggers
1. **Numeric entry** — Type camera number + Enter
2. **Plus/Minus keys** — Cycle to next/previous camera in sorted order
3. **Function buttons** — F1-F7 mapped to preconfigured CrossSwitch actions
4. **Home key** — When not in PTZ mode or playback, executes function button "Home"
### Camera Number Editing
```
State: Not Editing
├─ Press digit → BeginEdit(), append digit, start cancel timer
State: Editing (showing entered digits)
├─ Press digit → append digit, restart cancel timer
├─ Press Backspace → remove last digit, restart cancel timer
├─ Press Enter → validate, CrossSwitch if valid, EndEdit()
└─ Cancel timer fires → CancelEdit(), revert to original camera
```
Camera number can be up to 6 digits. Configurable prefixes (e.g., "500", "501", "502") are prepended automatically based on the selected prefix.
## 2. Camera Lock Workflow
Camera locks coordinate PTZ access across multiple keyboards via the AppServer SignalR hub.
```mermaid
sequenceDiagram
participant K1 as Keyboard 1 (Low priority)
participant Hub as AppServer Hub
participant DB as SQLite DB
participant K2 as Keyboard 2 (High priority)
K1->>Hub: TryLockCamera(camId, "K1", Low)
Hub->>DB: Check existing lock
DB-->>Hub: No lock exists
Hub->>DB: Create lock(camId, "K1", Low, expires=5min)
Hub-->>K1: LockResult(Acquired=true)
K1->>K1: IsCameraTelemetryActive = true
Note over K1: Operator controls PTZ...<br/>Each action calls ResetCameraLockExpiration()
K2->>Hub: TryLockCamera(camId, "K2", High)
Hub->>DB: Check existing lock → held by K1
Hub-->>K2: LockResult(Acquired=false, Owner="K1")
K2->>Hub: RequestCameraLock(camId, "K2", High)
Hub->>K1: CameraLockNotify(ConfirmTakeOver)
K1->>K1: Show dialog "K2 requests lock"
alt K1 confirms takeover
K1->>Hub: CameraLockConfirmTakeOver(confirm=true)
Hub->>K1: CameraLockNotify(Unlocked)
Hub->>K2: CameraLockNotify(Confirmed)
K2->>Hub: TryLockCamera(camId, "K2", High)
Hub-->>K2: LockResult(Acquired=true)
else K1 denies
K1->>Hub: CameraLockConfirmTakeOver(confirm=false)
Hub->>K2: CameraLockNotify(Rejected)
end
```
### Lock Priority Levels
Configured per keyboard in `appsettings-copilot.json`:
- `Low` — Standard operator
- `High` — Supervisor (can request takeover)
### Lock Expiration
- **Timeout:** 5 minutes (`LockExpirationConfig.LockExpirationTimeout`)
- **Warning:** 1 minute before expiration (`NotificationBeforeExpiration`)
- **Reset:** Every PTZ action or explicit reset call extends the timer
## 3. Alarm System
```mermaid
sequenceDiagram
participant Worker as CameraAlarmsUpdateWorker
participant CAS as CameraAlarmService
participant Hub as CopilotHub
participant AS as AppServer
participant Camea as Camea API
Note over Worker: Background worker (periodic)
Worker->>CAS: UpdateCameraToAlarmsMapping()
CAS->>Hub: GetCamerasWithAlarms()
Hub->>AS: Query
AS->>Camea: GET /api/alarms
Camea-->>AS: List of cameras with active alarms
AS-->>Hub: HashSet<int> cameraIds
Hub-->>CAS: Store in camerasWithAlarms
Note over CAS: UI can now check HasCameraAlarms(cameraId)
Note over Worker: Also on SegmentViewModel...
participant VM as SegmentViewModel
VM->>CAS: GetAlarmForCamera(camId, from, to)
CAS->>Hub: GetAlarmsForCamera(camId, from, to)
Hub->>AS: Query alarm history
AS-->>Hub: List<CameraAlarm>
Hub-->>CAS: Alarms (UTC → LocalTime)
CAS-->>VM: Sorted by StartTime desc
```
### Alarm-Related Behavior
- Screens with active alarms **block** camera number editing (CrossSwitch disabled)
- Alarm history page uses configurable regex to extract camera ID from alarm names: `^\D*(?<CameraId>\d{6})\D*$`
- Default search interval: 30 days
## 4. Sequence Management
Sequences automatically cycle cameras on a monitor at configurable intervals.
```mermaid
sequenceDiagram
participant Op as Operator
participant VM as SegmentViewModel
participant SeqS as SequenceService
participant Hub as SignalR Hub
participant AS as AppServer
Op->>VM: Press Sequence key
VM->>VM: Navigate to SequenceCategoriesPage
Op->>VM: Select category
VM->>SeqS: GetSequences(categoryId)
SeqS->>Hub: GetSequences(categoryId)
Hub-->>SeqS: List<SequenceMessage>
Op->>VM: Select sequence
VM->>SeqS: Start(viewerId, sequenceId)
SeqS->>Hub: Start(viewerId, sequenceId)
Note over AS: AppServer runs sequence timer
AS->>Hub: ViewerSequenceStateChanged
Hub->>SeqS: OnChanged(viewerSequenceState)
SeqS->>VM: Update UI (sequence indicator)
Note over AS: Timer fires every N seconds
AS->>AS: CrossSwitch next camera in list
```
### Sequence Configuration
```json
{
"Id": 1,
"CategoryId": 1,
"Name": "Test-A",
"Interval": 1, // seconds between switches
"MediaChannels": [500100, 500101, 500102, 500103]
}
```
Sequences are **run on the AppServer**, not on the keyboard. The keyboard only starts/stops them.
## 5. Playback Control
```mermaid
sequenceDiagram
participant Op as Operator
participant VM as SegmentViewModel
participant VC as IViewerController
participant Srv as Camera Server
Note over Op: Playback enters via Jog/Shuttle<br/>or Playback button
Op->>VM: Jog right (arrow key)
VM->>VM: IsPlaybackActive = true
VM->>VC: StepForwardAndStop()
VC->>Srv: Step forward one frame
Op->>VM: Shuttle right position 3
VM->>VM: TryGetShuttleSpeed(speeds, out speed, out forward)
Note over VM: position 3 → speeds[2] = 5
VM->>VC: FastForward(5)
VC->>Srv: Play at 5x speed
Op->>VM: Shuttle center
VM->>VC: Stop()
Op->>VM: Press Home key
VM->>VC: PlayLive()
VM->>VM: IsPlaybackActive = false
```
### Playback Controls
| Input | Action | SDK Method |
|-------|--------|------------|
| Jog Left | Step back 1 frame | `StepBackwardAndStop()` |
| Jog Right | Step forward 1 frame | `StepForwardAndStop()` |
| Shuttle Left 1-7 | Rewind at speed | `FastBackward(speed)` |
| Shuttle Right 1-7 | Fast forward at speed | `FastForward(speed)` |
| Shuttle Center | Pause | `Stop()` |
| Home | Return to live | `PlayLive()` |
## 6. Function Buttons
F1-F7 and Home keys execute preconfigured action lists per wall:
```mermaid
graph TD
Key["F1-F7 / Home Key Press"] --> FBS["FunctionButtonsService"]
FBS --> Config["Load from appsettings-function-buttons.json"]
Config --> Actions["Get actions for wallId + buttonKey"]
Actions --> ForEach["For each action:"]
ForEach -->|CrossSwitch| CS["CentralServerDriver.CrossSwitch(viewerId, sourceId)<br/>+ Minimize viewer"]
ForEach -->|SequenceStart| SS["SequenceService.Start(viewerId, sourceId)"]
```
Example: Pressing F1 on Wall 2 executes:
```json
{ "ViewerId": 12322, "ActionType": "CrossSwitch", "SourceId": 500123 }
```
This switches camera 500123 to viewer 12322.
## 7. Configuration Sync
Configuration files are managed centrally on the AppServer and synced to keyboards:
```mermaid
sequenceDiagram
participant AS as AppServer
participant Hub as SignalR Hub
participant CfgS as ConfigurationService
participant Disk as Local JSON Files
Note over CfgS: On startup
CfgS->>Hub: GetConfigurationFile(filename) for each manager
Hub-->>CfgS: ConfigurationFile (content + hash)
CfgS->>Disk: Compare hash, write if changed
Note over AS: Admin changes config via Blazor UI
AS->>Hub: ConfigurationFileChanged event
Hub->>CfgS: OnConfigurationFileChanged(file)
CfgS->>Disk: Write updated content
Note over CfgS: Also triggered when AppServer<br/>becomes available after disconnect
```
Configuration files synced:
- `appsettings-copilot.json` — Keyboard-specific settings
- `appsettings-camera-servers.json` — Camera server connections
- `appsettings-monitor-wall.json` — Monitor wall topology
- `appsettings-function-buttons.json` — Function button actions
- `appsettings-prepositions.json` — Camera prepositions
- `appsettings-sequences.json` — Sequence definitions
- `appsettings-sequence-categories.json` — Sequence categories

View File

@@ -0,0 +1,146 @@
---
title: "Hardware & Input System"
description: "COPILOT keyboard hardware, joystick, serial/HID protocols, and key mapping"
---
# Hardware & Input System
## Physical Hardware
The COPILOT keyboard is a custom-built control panel with three input subsystems:
```
┌──────────────────────────────────────────────────────────────────┐
│ COPILOT KEYBOARD │
│ │
│ ┌────────────────────────┐ ┌───────────────┐ ┌────────────┐ │
│ │ BUTTON PANEL │ │ JOYSTICK │ │ JOG/ │ │
│ │ │ │ │ │ SHUTTLE │ │
│ │ [Home] [F1-F7] │ │ ╱───╲ │ │ │ │
│ │ [0-9] [.] [*] │ │ │ ● │ │ │ ◄──●──► │ │
│ │ [+] [-] [(] [)] │ │ ╲───╱ │ │ │ │
│ │ [< >] │ │ 3-axis │ │ 7 speeds │ │
│ │ │ │ X/Y/Z │ │ per dir │ │
│ │ Serial (COM port) │ │ USB HID │ │ Serial │ │
│ └────────────────────────┘ └───────────────┘ └────────────┘ │
│ │
│ Arduino Leonardo (ATmega32U4) │
│ USB Composite: Serial + HID │
└───────────────────────────────┬──────────────────────────────────┘
│ USB
LattePanda SBC
```
## Communication Protocol
### Serial Port (Buttons + Jog/Shuttle)
Messages are newline-terminated strings with a single-character prefix:
| Prefix | Type | Example | Meaning |
|--------|------|---------|---------|
| `p` | Key Pressed | `pH\r\n` | Home key pressed |
| `r` | Key Released | `rH\r\n` | Home key released |
| `h` | Heartbeat | `h\r\n` | Keepalive (ignored) |
| `j` | Jog | `j+1\r\n` / `j-1\r\n` | Jog wheel step right/left |
| `s` | Shuttle | `s3\r\n` / `s-5\r\n` | Shuttle position (-7 to +7) |
| `v` | Version | `v2.1\r\n` | Firmware version response |
Startup: App sends byte `0xDC` to request firmware version.
### HID Device (Joystick)
| Property | Value |
|----------|-------|
| Vendor ID | `10959` (0x2ACF) |
| Product ID | `257` (0x0101) |
| Axes | X (pan), Y (tilt), Z (zoom) |
| Raw range | Scaled to **-255 to +255** |
| Buttons | Button1, Button2, Button3 |
```mermaid
graph TD
subgraph "HID Input Processing"
Raw["Raw HID Report"] --> Parser["DeviceItemInputParser"]
Parser --> Changed{"HasChanged?"}
Changed -->|Yes| Usage{"Usage Type"}
Usage -->|GenericDesktopX| PanEvt["JoystickX event<br/>(-255 to +255)"]
Usage -->|GenericDesktopY| TiltEvt["JoystickY event<br/>(-255 to +255)"]
Usage -->|GenericDesktopZ| ZoomEvt["JoystickZ event<br/>(-255 to +255)"]
Usage -->|Button1| B1["JoystickButton1<br/>Pressed/Released"]
Usage -->|Button2| B2["JoystickButton2<br/>Pressed/Released"]
Usage -->|Button3| Ignore["Ignored"]
end
```
## Key Mapping
### Hardware Keys → Virtual Keys
The serial protocol uses ASCII character codes:
| Serial Code | Char | Virtual Key | UI Function |
|-------------|------|-------------|-------------|
| 72 | `H` | Home | Home position / Jump to live |
| 65 | `A` | F1 | Function button 1 |
| 66 | `B` | F2 | Function button 2 |
| 67 | `C` | F3 | Function button 3 |
| 68 | `D` | F4 | Function button 4 |
| 69 | `E` | F5 | Function button 5 |
| 70 | `F` | F6 | Function button 6 |
| 71 | `G` | F7 | Function button 7 |
| 46 | `.` | Prefix | Camera number prefix cycle |
| 42 | `*` | FullScreen | Toggle monitor maximize |
| 41 | `)` | Sequence | Open sequence menu |
| 40 | `(` | Backspace | Delete last digit |
| 48 | `0` | D0 | Digit 0 |
| 55-51 | `7-3` | D1-D5 | Digits 1-5 (note: remapped!) |
| 49-51 | `1-3` | D7-D9 | Digits 7-9 (note: remapped!) |
| 52-54 | `4-6` | D4-D6 | Digits 4-6 |
| 60 | `<` | Minus | Previous camera / Focus far |
| 62 | `>` | Plus | Next camera / Focus near |
| 43 | `+` | Lock | Toggle camera PTZ lock |
| 45 | `-` | Enter | Confirm camera number |
**Important:** The digit key wiring is non-standard — D1 maps to ASCII 55 (`7`), not 49 (`1`). This is a hardware layout choice.
### Joystick Buttons
| Button | Virtual Key | Function |
|--------|-------------|----------|
| Button1 | JoystickButton1 | (Context-dependent) |
| Button2 | JoystickButton2 | Toggle camera PTZ lock |
### Development Keyboard Emulation
When no serial port is detected, the app falls back to standard keyboard input via `VirtualKeyboard.Emulate()`:
| PC Key | Virtual Key |
|--------|-------------|
| Home | Home |
| F1-F7 | F1-F7 |
| F10 | Prefix |
| F11 | FullScreen |
| F12 | Sequence |
| 0-9 / Numpad | D0-D9 |
| Backspace | Backspace |
| Enter | Enter |
| +/- | Plus/Minus |
| Left/Right arrow | JogLeft/JogRight |
| Shift+Left/Right | Shuttle (incremental) |
| Shift+Up | Shuttle center (reset) |
## Shuttle Speed Mapping
The shuttle wheel has 7 positions per direction. Speeds are configurable per deployment:
```
Position: -7 -6 -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 +6 +7
◄── backward ──────────── center ──────────── forward ──►
Config: [250,100, 30, 15, 5, 2, 1] ● [ 1, 2, 5, 15, 30,100,250]
```
Default playback speeds (from config): `[1, 2, 5, 15, 30, 100, 250]`
## Service Key (Hidden Menu)
Holding the **Backspace** key for 3 seconds opens the Service Menu page. This provides access to firmware update and diagnostic functions.

View File

@@ -0,0 +1,99 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>COPILOT D6 Legacy Architecture</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsify-themeable@0/dist/css/theme-simple-dark.css">
<style>
:root {
--base-font-size: 15px;
--theme-color: #3b82f6;
--sidebar-width: 260px;
--sidebar-background: #1e1e2e;
--sidebar-nav-link-color--active: #3b82f6;
}
.markdown-section pre {
background-color: #1e1e2e;
border-radius: 8px;
}
.markdown-section table {
display: table;
width: 100%;
}
.markdown-section h1 {
border-bottom: 2px solid #3b82f6;
padding-bottom: 8px;
}
.mermaid svg {
max-width: 100%;
}
.mermaid {
text-align: center;
margin: 1em 0;
}
</style>
</head>
<body>
<div id="app">Loading...</div>
<script src="https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.min.js"></script>
<script>
mermaid.initialize({
startOnLoad: false,
theme: 'dark',
securityLevel: 'loose',
flowchart: { useMaxWidth: true, htmlLabels: true },
sequence: { useMaxWidth: true }
});
window.$docsify = {
name: 'COPILOT D6',
repo: '',
loadSidebar: '_sidebar.md',
subMaxLevel: 3,
auto2top: true,
search: {
placeholder: 'Search docs...',
noData: 'No results.',
depth: 3
},
// Intercept mermaid code blocks at the markdown parser level
markdown: {
renderer: {
code: function(code, lang) {
if (lang && lang.toLowerCase() === 'mermaid') {
return '<div class="mermaid">' + code + '</div>';
}
return this.origin.code.apply(this, arguments);
}
}
},
plugins: [
function(hook) {
// Strip YAML frontmatter before markdown parsing
hook.beforeEach(function(content) {
return content.replace(/^---[\s\S]*?---\n*/m, '');
});
// Render mermaid diagrams after DOM update
hook.doneEach(function() {
var els = document.querySelectorAll('.mermaid');
if (els.length > 0) {
// Reset any previously processed elements
els.forEach(function(el) {
el.removeAttribute('data-processed');
});
mermaid.run({ nodes: Array.from(els) }).catch(function(e) {
console.error('Mermaid render error:', e);
});
}
});
}
]
}
</script>
<script src="https://cdn.jsdelivr.net/npm/docsify@4/lib/docsify.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/docsify@4/lib/plugins/search.min.js"></script>
</body>
</html>

View File

@@ -0,0 +1,126 @@
---
title: "COPILOT D6 Legacy WPF Application"
description: "Complete reverse-engineered architecture documentation of the original COPILOT D6 CCTV keyboard controller system"
---
# COPILOT D6 Legacy Architecture
> Reverse-engineered from compiled .NET 7 assemblies (build 1.0.705, December 2023) using ILSpy decompilation. This documentation covers the complete architecture of the original WPF-based CCTV keyboard controller system.
## System Overview
The COPILOT D6 system is a **CCTV surveillance keyboard controller** used to manage video feeds from multiple camera servers (Geutebruck GeViScope, G-Core, GeViSoft) across a wall of physical monitors. Operators use a custom hardware keyboard with joystick to:
- Switch cameras to monitors (CrossSwitch)
- Control PTZ cameras (Pan/Tilt/Zoom) via joystick
- Manage camera prepositions (saved positions)
- Run camera sequences (automated cycling)
- View and manage alarms
- Playback recorded video
## Architecture at a Glance
```
COPILOT SYSTEM ARCHITECTURE
============================
┌──────────────────────────────────────────────────────────┐
│ COPILOT KEYBOARD (Hardware) │
│ ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ Numpad + │ │ 3-Axis │ │ Jog/Shuttle │ │
│ │ Function │ │ Joystick │ │ Wheel │ │
│ │ Keys │ │ (HID USB) │ │ │ │
│ │ [Serial] │ │ X/Y/Z Axes │ │ [Serial] │ │
│ └──────┬───────┘ └──────┬───────┘ └───────┬────────┘ │
│ │ │ │ │
│ └─────────────────┼───────────────────┘ │
│ Arduino Leonardo │
│ (USB Composite Device) │
└────────────────────────────┬──────────────────────────────┘
│ USB (Serial + HID)
┌────────────────────────────┼──────────────────────────────┐
│ LattePanda Sigma SBC │
│ ┌─────────────────────────┴──────────────────────────┐ │
│ │ Copilot.App.exe (WPF .NET 7) │ │
│ │ ┌──────────────────────────────────────────────┐ │ │
│ │ │ Copilot.Device Layer │ │ │
│ │ │ Serial Port ←→ Keyboard Keys/Jog/Shuttle │ │ │
│ │ │ HID Device ←→ Joystick X/Y/Z │ │ │
│ │ └──────────────────────┬───────────────────────┘ │ │
│ │ │ Events │ │
│ │ ┌──────────────────────┴───────────────────────┐ │ │
│ │ │ MainWindow (WPF) │ │ │
│ │ │ Routes input → current ViewModel │ │ │
│ │ └──────────────────────┬───────────────────────┘ │ │
│ │ │ │ │
│ │ ┌──────────────────────┴───────────────────────┐ │ │
│ │ │ SegmentViewModel (main screen) │ │ │
│ │ │ • PTZ control via joystick │ │ │
│ │ │ • Camera number entry │ │ │
│ │ │ • CrossSwitch execution │ │ │
│ │ │ • Camera lock management │ │ │
│ │ │ • Playback control (jog/shuttle) │ │ │
│ │ └──┬────────────────────────────────────┬──────┘ │ │
│ │ │ Direct SDK calls │ SignalR │ │
│ │ ┌──┴──────────────────────┐ ┌─────────┴──────┐ │ │
│ │ │ Camera Server │ │ AppServer │ │ │
│ │ │ Drivers │ │ Client │ │ │
│ │ │ • GeViScope SDK │ │ (SignalR Hub) │ │ │
│ │ │ • G-Core SDK │ │ │ │ │
│ │ │ • GeViSoft SDK │ │ │ │ │
│ │ └──┬──────────────────────┘ └────────┬───────┘ │ │
│ └─────┼──────────────────────────────────┼───────────┘ │
└────────┼──────────────────────────────────┼───────────────┘
│ Native SDK (TCP) │ HTTPS/WSS
▼ ▼
┌─────────────────────┐ ┌────────────────────────┐
│ Camera Servers │ │ Copilot AppServer │
│ • GeViScope │ │ (ASP.NET Core) │
│ 192.168.102.186 │ │ copilot.test.d6... │
│ • G-Core │ │ ┌──────────────────┐ │
│ 192.168.102.20 │ │ │ SignalR Hub │ │
│ │ │ │ • Camera Locks │ │
│ Each server has: │ │ │ • Sequences │ │
│ • Cameras │ │ │ • Config Sync │ │
│ • Monitors/Viewers │ │ │ • Viewer State │ │
│ • PTZ controllers │ │ │ • Alarm History │ │
│ • Alarm events │ │ └──────────────────┘ │
│ │ │ ┌──────────────────┐ │
│ │ │ │ SQLite Database │ │
│ │ │ │ • Lock state │ │
│ │ │ │ • Alarm history │ │
│ │ │ └──────────────────┘ │
│ │ │ ┌──────────────────┐ │
│ │ │ │ REST API │ │
│ │ │ │ • Auto-updates │ │
│ │ │ │ • Configuration │ │
│ │ │ │ • Blazor Admin UI │ │
│ │ │ └──────────────────┘ │
└─────────────────────┘ └────────────────────────┘
```
## Assembly Map
| Assembly | Type | Purpose |
|----------|------|---------|
| `Copilot.App.dll` | WPF Client | Main application - UI, ViewModels, navigation, input handling |
| `Copilot.Device.dll` | Client Library | Hardware abstraction - serial port, HID joystick, key mapping |
| `Copilot.Common.dll` | Shared Library | Configuration models, data protection, providers, hub interfaces |
| `Copilot.Common.Services.dll` | Shared Library | Driver providers, media channel service, viewer state management |
| `Copilot.Drivers.Common.dll` | Shared Library | Driver interfaces (`IMovementController`, `ICameraServerDriver`, etc.) |
| `Copilot.Drivers.GeViScope.dll` | Driver | GeViScope SDK wrapper - PLC actions for cameras/PTZ |
| `Copilot.Drivers.GCore.dll` | Driver | G-Core SDK wrapper - binary protocol for cameras/PTZ |
| `Copilot.Drivers.GeviSoft.dll` | Driver | GeViSoft SDK wrapper |
| `Copilot.AppServer.Client.dll` | Client Library | SignalR hub client, availability monitoring |
| `Copilot.AppServer.dll` | Server | ASP.NET Core server - API, SignalR hub, Blazor admin |
| `Copilot.AppServer.Database.dll` | Server Library | Entity Framework - SQLite models and repositories |
| `Copilot.Camea.Client.dll` | Server Library | Camea API integration (external alarm system) |
## Documentation Pages
- **[System Architecture](./architecture.md)** - Component diagrams, dependency graph, deployment model
- **[Hardware & Input](./hardware-input.md)** - Joystick, keyboard, serial/HID protocols
- **[PTZ Control Flow](./ptz-control.md)** - Joystick → Pan/Tilt/Zoom → SDK command pipeline
- **[Data Flows](./data-flows.md)** - CrossSwitch, Alarms, Sequences, Playback, Camera Lock
- **[Configuration](./configuration.md)** - JSON config files, monitor wall topology, function buttons
- **[AppServer](./appserver.md)** - SignalR hub, REST API, database, admin UI

View File

@@ -0,0 +1,393 @@
# Business Rules Reference
> All business rules extracted from the decompiled legacy WPF source code. Each rule includes the source file and line reference for verification.
## 1. CrossSwitch Rules
**Source:** `SegmentViewModel.cs` lines 962-977
### Execution Flow
1. Stop any active playback on target viewer
2. Cancel any running sequence on target viewer
3. Call `CentralServerDriver.CrossSwitch(viewerId, cameraNumber)`
4. Log with `UserAction.CrossSwitch` for audit trail
5. Clear camera number edit state
### Preconditions
- Camera number must be valid (> 0)
- Screen must be selected
- No active alarm blocking the monitor (configurable)
- Network must be available for coordinated operations
### Camera Number Composition
```
Full Camera Number = Prefix + Entered Digits
Example: 500 (prefix) + 123 (digits) = 500123
```
- Prefix cycles through configured values (e.g., 500, 501, 502)
- Each prefix maps to a different camera server
- Prefix key (`VirtualKey.Prefix`) cycles to next prefix
### Edit Timeout
- **Source:** `SegmentViewModel.cs` lines 829-847
- Camera number entry has a configurable timeout (`CancelEditTimeout` from config)
- `DispatcherTimer` starts on first digit entry
- Timer resets on each subsequent digit
- On timeout: edit is cancelled, partial number discarded
- On Enter: edit is confirmed, CrossSwitch executed
## 2. Camera Lock Rules
**Source:** `CameraLockService.cs`, `SegmentViewModel.cs` lines 631-676
### Lock Lifecycle
```mermaid
stateDiagram-v2
[*] --> Unlocked
Unlocked --> LockRequested: User selects PTZ camera
LockRequested --> Locked: Lock acquired
LockRequested --> TakeoverDialog: Locked by another keyboard
TakeoverDialog --> TakeoverRequested: User confirms takeover
TakeoverDialog --> Unlocked: User cancels
TakeoverRequested --> Locked: Other keyboard confirms
TakeoverRequested --> Unlocked: Other keyboard rejects
Locked --> Unlocked: Manual unlock (Enter key)
Locked --> ExpiringSoon: 4 minutes elapsed
ExpiringSoon --> Locked: PTZ movement resets timer
ExpiringSoon --> Unlocked: 5 minutes elapsed (timeout)
```
### Lock Rules
| Rule | Detail | Source |
|------|--------|--------|
| Lock timeout | 5 minutes from acquisition | AppServer config |
| Expiry warning | 1 minute before timeout | `CameraLockNotificationType.ExpireSoon` |
| Reset on PTZ | Each joystick movement resets expiry timer | `SegmentViewModel:754, 941` |
| Priority levels | `Low` (default), `High` (override) | `CameraLockPriority` enum |
| Auto-unlock on CrossSwitch | Entering new camera number + Enter unlocks current | `SegmentViewModel:906-916` |
| Telemetry activation | `IsCameraTelemetryActive = true` only when locked | `SegmentViewModel:659` |
| Multiple locks | A keyboard can lock multiple cameras (HashSet) | `SegmentViewModel:631` |
| Restore on navigate | When entering segment, restore locks via `GetLockedCameraIds` | Startup logic |
### Takeover Protocol
1. Keyboard A holds lock on camera X
2. Keyboard B requests lock → gets `CameraLockResult.Acquired = false`
3. Keyboard B shows dialog: "Camera locked by [A]. Request takeover?"
4. If Yes → `RequestLock()` sends notification to Keyboard A
5. Keyboard A receives `ConfirmTakeOver` notification → shows dialog
6. If Keyboard A confirms → lock transferred to B
7. If Keyboard A rejects → B receives rejection, lock stays with A
### New Implementation (No AppServer)
- PRIMARY keyboard manages lock state in memory: `Map<int, CameraLock>`
- Lock operations via WebSocket messages to PRIMARY
- PRIMARY broadcasts lock changes to all connected keyboards
- **Degraded mode** (no PRIMARY): local-only lock tracking, no coordination
## 3. PTZ Control Rules
**Source:** `SegmentViewModel.cs` lines 739-769
### Joystick → PTZ Mapping
```
Joystick X axis → Pan (negative = left, positive = right)
Joystick Y axis → Tilt (negative = up, positive = down)
Joystick Z axis → Zoom (negative = out, positive = in)
```
### Speed Processing
| Step | Detail | Source |
|------|--------|--------|
| Raw HID value | Scaled to -255..+255 by HID driver | `CopilotDevice.HidDataProvider_DataReceived` |
| Deduplication | Only send if value changed from last sent | `ptzSpeeds` dictionary in `DoPtzAction` |
| Direction split | Negative = Left/Up/Out, Positive = Right/Down/In | Driver layer |
| Zero = Stop | Speed 0 sent as explicit stop command | Driver layer |
| No scaling | Raw value passed directly to SDK (0-255 absolute) | Confirmed in all drivers |
### Critical Finding: No Zoom-Proportional Speed
- **The app does NOT scale pan/tilt speed based on zoom level**
- Raw joystick value → directly to SDK → camera server handles proportional behavior
- This is a hardware/server feature, not an app feature
- **Flutter app should pass raw speed values unchanged**
### PTZ Preconditions
- `IsCameraTelemetryActive` must be true (camera must be locked)
- Camera must be PTZ-capable (checked via driver)
- Lock expiry resets on every joystick movement
### Bridge API for PTZ
```
POST /camera/pan {Camera: int, Direction: "left"|"right", Speed: 0-255}
POST /camera/tilt {Camera: int, Direction: "up"|"down", Speed: 0-255}
POST /camera/zoom {Camera: int, Direction: "in"|"out", Speed: 0-255}
POST /camera/stop {Camera: int}
```
## 4. Function Button Rules
**Source:** `FunctionButtonsService.cs` (78 lines)
### Configuration Structure
```json
{
"walls": {
"1": {
"F1": [
{"actionType": "CrossSwitch", "viewerId": 1001, "sourceId": 500001},
{"actionType": "CrossSwitch", "viewerId": 1002, "sourceId": 500002}
],
"F2": [
{"actionType": "SequenceStart", "viewerId": 1001, "sourceId": 5}
]
}
}
}
```
### Execution Rules
- Buttons F1-F7 mapped per wall ID
- Each button can trigger **multiple actions** (executed sequentially)
- Action types:
- `CrossSwitch` → calls `driver.CrossSwitch(viewerId, sourceId)`
- `SequenceStart` → calls `sequenceService.Start(viewerId, sourceId)`
- After CrossSwitch action: viewer is un-maximized
- Returns `false` if no actions configured for button+wall combination
### Key Input Mapping
```
VirtualKey.F1 through VirtualKey.F7 → ExecuteFunctionButtonActions(wallId, "F1"..."F7")
```
## 5. Preposition Rules
**Source:** `PrepositionService.cs` (114 lines)
### Operations
| Operation | Method | Requires AppServer? | Bridge Endpoint |
|-----------|--------|---------------------|-----------------|
| List prepositions | `GetPrepositionNames(mediaChannelId)` | No (local config) | — |
| Call up (move to) | `CallUpPreposition(mediaChannelId, prepositionId)` | No | `POST /camera/preset` |
| Save new | `SavePrepositionToAppServer(...)` | **Yes** (legacy) | Direct via bridge (new) |
| Delete | `DeletePrepositionToAppServer(...)` | **Yes** (legacy) | Direct via bridge (new) |
| Sync from server | `UpdatePrepositionsFromAppServer()` | **Yes** (legacy) | Not needed (local config) |
### Preconditions
- Camera must be PTZ-capable
- Camera must be locked (`IsCameraTelemetryActive == true`)
- Preposition IDs are per media channel, not per camera
### New Implementation
- Preposition names stored in `prepositions.json` locally
- Call-up via bridge: `POST /camera/preset {Camera, Preset}`
- Save/delete: PRIMARY keyboard coordinates name updates and broadcasts to other keyboards
- No AppServer dependency for basic call-up
## 6. Sequence Rules
**Source:** `SequenceService.cs` (77 lines)
### Sequence Model
```
SequenceCategory → contains multiple Sequences
Sequence → runs on a specific viewer, cycling through cameras
```
### Execution Rules
- Start: `sequenceService.Start(viewerId, sequenceId)`
- Stop: `sequenceService.Stop(viewerId)`
- Only one sequence per viewer at a time
- CrossSwitch on a viewer with active sequence → **stops sequence first**
- Sequences disabled during telemetry mode (`IsCameraTelemetryActive`)
- Requires AppServer availability (legacy) → PRIMARY availability (new)
### State Events
- `ViewerSequenceStateChanged` event updates viewer state
- `GetRunningSequences()` called on startup to restore state
### New Implementation
- PRIMARY keyboard runs sequence timer logic
- Sequence definitions stored in local config
- Start/stop via WebSocket messages to PRIMARY
- STANDBY can resume sequences after promotion (query bridge for current viewer state)
## 7. Playback Rules
**Source:** `PlaybackStateService.cs`, `SegmentViewModel.cs`
### Playback States (PlayMode enum)
| Value | Mode | Description |
|-------|------|-------------|
| 0 | Unknown | — |
| 1 | PlayStop | Paused |
| 2 | PlayForward | Normal speed forward |
| 3 | PlayBackward | Normal speed backward |
| 4 | FastForward | Fast forward |
| 5 | FastBackward | Fast backward |
| 6 | StepForward | Single frame forward |
| 7 | StepBackward | Single frame backward |
| 8 | PlayBOD | Jump to beginning of day |
| 9 | PlayEOD | Jump to end of day |
| 10 | QuasiLive | Near-live with delay |
| 11 | Live | Real-time live view |
### Jog/Shuttle Mapping
**Source:** `CopilotDevice.SerialPortDataProvider`
```
Jog wheel: Single-step events (j+1 or j-1)
→ StepForward / StepBackward
Shuttle wheel: Speed values -7 to +7
-7..-1 → FastBackward (increasing speed)
0 → PlayStop (pause)
+1..+7 → FastForward (increasing speed)
```
Shuttle speed mapping (from legacy `ShuttleConverter`):
```
±1 → speed 0.5x
±2 → speed 1x
±3 → speed 2x
±4 → speed 4x
±5 → speed 8x
±6 → speed 16x
±7 → speed 32x
```
### Playback Bridge Endpoints
```
POST /viewer/set-play-mode {Viewer, PlayMode, PlaySpeed}
POST /viewer/play-from-time {Viewer, Channel, PlayMode, Time}
POST /viewer/jump-by-time {Viewer, Channel, PlayMode, TimeInSec}
```
### Playback Rules
- CrossSwitch stops active playback before switching
- Playback state tracked per viewer in `PlaybackStateService`
- Entering playback mode: navigate to `PlaybackPage`
- Plus/Minus keys in playback: not camera navigation but focus control (speed ±128)
## 8. Keyboard Input Rules
**Source:** `MainWindow.cs` (217 lines), `SegmentViewModel.cs` (771-944)
### Input Routing Architecture
```
Hardware Input → CopilotDevice → MainWindow → CurrentViewModel (if implements handler)
Keyboard Emulation → PreviewKeyDown → VirtualKeyboard.Emulate → Same routing
```
### Key Mapping (SegmentViewModel)
| VirtualKey | Action | Condition |
|------------|--------|-----------|
| `Digit0-9` | Append to camera number | Begins edit mode |
| `Enter` | Confirm camera number → CrossSwitch | If editing: execute. If telemetry: unlock + execute |
| `Backspace` | Remove last digit | If editing |
| `Escape` | Cancel edit / deselect | If editing: cancel. Else: navigate back |
| `Prefix` | Cycle camera prefix | Cycles through configured prefixes |
| `Plus` | Next camera / Focus+ | If editing: ignored. If playback: focus speed +128 |
| `Minus` | Previous camera / Focus- | Similar to Plus |
| `Home` | Go to preposition / PlayLive | Context-dependent |
| `F1-F7` | Execute function button | Wall-specific actions |
| `Lock` / `JoyButton2` | Toggle camera lock | If locked: unlock. If unlocked: try lock |
| `Search` | Navigate to camera search | If no alarm on screen |
| `Prepositions` | Navigate to prepositions | If camera is PTZ + telemetry active |
| `Playback` | Navigate to playback page | If valid media channel |
| `Sequence` | Navigate to sequence categories | If screen selected + AppServer available |
| `AlarmHistory` | Navigate to alarm history | — |
| `FullScreen` | Toggle viewer maximize | If camera > 0 |
### Service Hotkey
- **Backspace held for 3 seconds** → Navigate to Service Menu
- Implemented via `DispatcherTimer` with 3000ms interval
- Timer starts on Backspace KeyDown, cancelled on KeyUp
### Keyboard Emulation Mode
- Activated when serial port not available (development)
- Uses `PreviewKeyDown` / `PreviewKeyUp` window events
- `VirtualKeyboard.Emulate()` converts `System.Windows.Input.Key` to `VirtualKey`
- Ignores key repeat (`e.IsRepeat` check)
## 9. Alarm Rules
**Source:** `CameraAlarmService.cs` (47 lines), `AlarmService` in Flutter
### Alarm States
```
vasNewAlarm → vasPresented → vasStacked → vasConfirmed → vasRemoved
```
### Alarm Blocking
- Active alarm on a camera → blocks CrossSwitch to that camera's monitor
- `HasCameraAlarms(cameraId)` returns true if unresolved alarm exists
- Alarm state displayed as red highlight on monitor tile
### Alarm Sync Strategy
1. **Startup:** Query all active alarms from bridges (`GET /alarms/active`)
2. **Real-time:** WebSocket events (`EventStarted`, `EventStopped`)
3. **Periodic:** Re-sync every 30 seconds (configurable)
4. **Hourly:** Full alarm mapping refresh (`CameraAlarmsUpdateWorker`)
### Alarm History
- `GetAlarmForCamera(cameraId, from, to)` → list of `CameraAlarm` records
- Times converted from UTC to local time for display
- Ordered by `StartTime` descending (newest first)
## 10. Monitor Wall Configuration Rules
**Source:** `MonitorWallConfiguration`, `SegmentModel`, `ScreenModel`
### Wall Structure
```
Wall → Segments → Monitors → Viewers
```
| Level | Example | Description |
|-------|---------|-------------|
| Wall | Wall 1 | The physical monitor wall installation |
| Segment | "Top Row" | A logical grouping of monitors |
| Monitor | Physical screen #1 | One physical display (can have 1 or 4 viewers) |
| Viewer | Viewer 1001 | One video stream slot on a monitor |
### Quad View
- A physical monitor can show 1 camera (single) or 4 cameras (quad)
- `PhysicalMonitor.IsQuadView` determines layout
- Quad view has 4 `viewerIds`, single has 1
### Screen Selection Rules
- Only one screen/viewer selected at a time
- Selection clears camera number edit state
- Selected viewer highlighted with cyan border
- CrossSwitch operates on selected viewer
## 11. Network Availability Rules
**Source:** `NetworkAvailabilityState`, `NetworkAvailabilityWorker`
### Availability States
- Camera server connection: tracked per server
- AppServer/PRIMARY connection: single state
- Combined state determines feature availability
### Feature Availability Matrix
| Feature | Camera Server Required | PRIMARY Required |
|---------|----------------------|------------------|
| CrossSwitch | Yes | No |
| PTZ control | Yes | No |
| Camera lock (coordinated) | No | **Yes** |
| Camera lock (local-only) | No | No |
| Preposition call-up | Yes | No |
| Preposition save/delete | Yes | **Yes** |
| Sequence start/stop | No | **Yes** |
| Alarm display | Yes | No |
| Config sync | No | **Yes** |
### Degraded Mode
When PRIMARY is unavailable:
- CrossSwitch and PTZ **always work** (direct to bridge)
- Lock shown as "local only" (no coordination between keyboards)
- Sequences cannot be started/stopped
- Config changes not synchronized

View File

@@ -0,0 +1,120 @@
---
title: "Legacy → Flutter Migration Comparison"
description: "Side-by-side comparison of legacy WPF architecture vs new Flutter system"
---
# Legacy → Flutter Migration Comparison
## Architecture Comparison
```mermaid
graph LR
subgraph "Legacy (WPF)"
L_HW["Hardware Keyboard<br/>Serial + HID"] --> L_App["Copilot.App.exe<br/>(WPF .NET 7)"]
L_App -->|"Native SDK<br/>(TCP direct)"| L_Srv["Camera Servers<br/>GeViScope / G-Core"]
L_App -->|"SignalR<br/>(HTTPS/WSS)"| L_AS["AppServer<br/>(ASP.NET Core)"]
L_AS --> L_DB["SQLite"]
end
subgraph "New (Flutter)"
N_HW["Hardware Keyboard<br/>Serial + HID"] --> N_App["Flutter App<br/>(Web/Desktop)"]
N_App -->|"REST HTTP<br/>(localhost)"| N_Bridge["C# Bridges<br/>(.NET 8)"]
N_Bridge -->|"Native SDK<br/>(TCP)"| N_Srv["Camera Servers<br/>GeViScope / G-Core"]
N_App -->|"WebSocket<br/>(PRIMARY)"| N_Coord["PRIMARY Keyboard<br/>(Coordination)"]
end
```
## Component Mapping
| Legacy (WPF) | New (Flutter) | Notes |
|---------------|---------------|-------|
| `Copilot.App.exe` | `copilot_keyboard` Flutter app | Full rewrite |
| `Copilot.Device.dll` | Web HID / Serial API | Browser APIs or native plugins |
| `Copilot.Drivers.GeViScope.dll` | GeViScope Bridge (:7720) | SDK stays in C#, exposed via REST |
| `Copilot.Drivers.GCore.dll` | G-Core Bridge (:7721) | SDK stays in C#, exposed via REST |
| `Copilot.Drivers.GeviSoft.dll` | GeViServer Bridge (:7710) | SDK stays in C#, exposed via REST |
| `Copilot.Drivers.Common.dll` | Bridge REST API contracts | Interfaces become HTTP endpoints |
| `Copilot.AppServer.exe` | PRIMARY keyboard + WebSocket hub | **No separate server** — runs on keyboard |
| `Copilot.AppServer.Database.dll` | In-memory state on PRIMARY | No SQLite needed |
| `Copilot.Common.Services.dll` | `BridgeService` + `StateService` | Dart services |
| SignalR Hub | WebSocket hub on PRIMARY | Simpler protocol |
| `IMovementController` | `POST /api/ptz/{action}` on bridge | REST instead of direct SDK |
| `ICameraServerDriver` | Bridge handles connection | App doesn't touch SDK |
| `ICentralServerDriver.CrossSwitch` | `POST /api/viewer/connect-live` | Via bridge REST |
## Key Architectural Differences
### 1. SDK Access Path
**Legacy:** App → Native SDK DLL → Camera Server (direct TCP)
```
Copilot.App → GeViScopeMovementController → GscPLCWrapper → TCP → Server
```
**New:** App → HTTP → C# Bridge → Native SDK → Camera Server
```
Flutter App → HTTP POST /api/ptz/pan → Bridge (.NET 8) → SDK → Server
```
**Why:** Flutter (especially web) cannot load native .NET DLLs. The C# bridges wrap the same SDKs behind a REST API.
### 2. Coordination Model
**Legacy:** Centralized AppServer (single point of coordination)
- All keyboards connect to one AppServer via SignalR
- AppServer manages locks, sequences, config, alarms
- AppServer failure = loss of coordination features
**New:** Distributed PRIMARY/STANDBY model
- Any keyboard can be PRIMARY (runs coordination logic)
- STANDBY monitors PRIMARY via heartbeat, auto-promotes after 6s
- No separate server hardware needed
- Critical operations (CrossSwitch, PTZ) work without PRIMARY
### 3. Configuration Management
**Legacy:** AppServer stores config → syncs to keyboards via SignalR
**New:** `servers.json` + `keyboards.json` + `crossswitch-rules.json` loaded from local files + PRIMARY sync
### 4. Alarm System
**Legacy:** AppServer → Camea API → SignalR → Keyboards
**New:** Each bridge can query alarms directly + periodic sync via PRIMARY
## Feature Parity Matrix
| Feature | Legacy Status | New Status | Priority |
|---------|--------------|------------|----------|
| CrossSwitch (camera → monitor) | Complete | Phase 1 | Critical |
| PTZ via joystick (Pan/Tilt/Zoom) | Complete | Phase 1 | Critical |
| Camera number entry (numpad) | Complete | Phase 1 | Critical |
| Camera lock (PTZ coordination) | Complete | Phase 2 | High |
| Prepositions (saved positions) | Complete | Phase 1 | High |
| Sequences (camera cycling) | Complete | Phase 3 | Medium |
| Function buttons (F1-F7) | Complete | Phase 1 | High |
| Playback (jog/shuttle) | Complete | Phase 3 | Medium |
| Alarm display | Complete | Phase 3 | Medium |
| Alarm history | Complete | Phase 3 | Low |
| Monitor wall segments | Complete | Phase 1 | High |
| Config sync from server | Complete | Phase 2 | Medium |
| Auto-update (firmware + app) | Complete | Phase 4 | Low |
| Service menu | Complete | Phase 4 | Low |
| Keyboard emulation (dev mode) | Complete | N/A (browser) | N/A |
## PTZ Speed Values: Compatible
The legacy app sends speed values **0-255** to the SDK. The new bridges should use the **same range** to maintain identical PTZ behavior. The zoom-proportional speed feature is provided by the camera/server infrastructure, not the app.
```
Legacy: Joystick HID (-255..+255) → PanRight(speed) → SDK → Server
New: Joystick HID (-255..+255) → POST /api/ptz/pan {speed} → Bridge → SDK → Server
```
Same speed values = same camera behavior.
## Risk Areas
1. **Joystick latency** — Legacy sends joystick events directly via in-process SDK call (~1ms). New path adds HTTP overhead (~5-20ms). Monitor for responsiveness.
2. **Lock coordination** — Legacy uses SignalR (battle-tested). New uses custom WebSocket protocol. Needs thorough testing.
3. **Sequence execution** — Legacy runs on AppServer (always-on). New runs on PRIMARY keyboard (could failover mid-sequence).
4. **Alarm reliability** — Legacy has Camea API integration on AppServer. New needs bridge-level alarm subscription.

View File

@@ -0,0 +1,220 @@
# Migration Guide: WPF → Flutter (No AppServer)
> This guide maps every component of the legacy WPF COPILOT D6 system to its Flutter equivalent. The key architectural change: **there is no centralized AppServer** — the PRIMARY keyboard handles coordination.
## Architecture Transformation
```mermaid
graph TB
subgraph "LEGACY Architecture"
direction TB
L_HW["Hardware Keyboard"] --> L_App["Copilot.App.exe<br/>(WPF .NET 7)"]
L_App -->|"Native SDK DLL<br/>(in-process)"| L_Cam["Camera Servers"]
L_App -->|"SignalR<br/>(HTTPS/WSS)"| L_AS["AppServer<br/>(separate machine)"]
L_AS --> L_DB["SQLite DB"]
end
subgraph "NEW Architecture"
direction TB
N_HW["Hardware Keyboard"] --> N_App["Flutter App<br/>(Web/Desktop)"]
N_App -->|"REST HTTP<br/>(localhost bridges)"| N_Bridge["C# Bridges<br/>(.NET 8)"]
N_Bridge -->|"Native SDK"| N_Cam["Camera Servers"]
N_App -->|"WebSocket<br/>(:8090)"| N_Primary["PRIMARY Keyboard<br/>(coordination)"]
end
```
## What Replaces the AppServer?
The legacy AppServer was a centralized ASP.NET Core server providing:
| AppServer Feature | New Approach | Where It Runs |
|-------------------|-------------|---------------|
| Camera locks (SignalR hub) | WebSocket hub + in-memory state | PRIMARY keyboard |
| Sequence execution | Sequence runner service | PRIMARY keyboard |
| Config file sync | Local JSON files + PRIMARY broadcast | Each keyboard |
| Alarm history (Camea API) | Bridge alarm endpoints + periodic sync | Each keyboard queries bridges |
| Viewer state tracking | Bridge WebSocket events | Each keyboard tracks locally |
| Auto-update | Out of scope (Phase 4+) | — |
| Admin UI (Blazor) | Not needed initially | — |
### PRIMARY/STANDBY Model
```mermaid
sequenceDiagram
participant K1 as Keyboard 1 (PRIMARY)
participant K2 as Keyboard 2 (STANDBY)
participant B as C# Bridges
participant C as Camera Servers
Note over K1,K2: Normal operation
K1->>B: Direct commands (CrossSwitch, PTZ)
K2->>B: Direct commands (CrossSwitch, PTZ)
K1->>K2: Heartbeat every 2s via WebSocket
K1->>K2: State sync (locks, sequences)
Note over K1,K2: PRIMARY failure
K2->>K2: No heartbeat for 6s
K2->>K2: Self-promote to PRIMARY
K2->>B: Continue direct commands
Note over K2: Lock/sequence state rebuilt from bridges
```
**Key principle:** Direct commands (CrossSwitch, PTZ) **always work** — they go straight to bridges. Only coordination features (locks, sequences) need the PRIMARY.
## Component Mapping: Legacy → Flutter
### Assembly-Level Mapping
| Legacy Assembly | Flutter Equivalent | Status |
|----------------|-------------------|--------|
| `Copilot.App.dll` (WPF UI + ViewModels) | `copilot_keyboard/` Flutter app | Partially built |
| `Copilot.Device.dll` (Serial + HID) | Web Serial API / `flutter_libserialport` | Not started |
| `Copilot.Drivers.GeViScope.dll` | GeViScope Bridge (:7720) REST API | **Complete** |
| `Copilot.Drivers.GCore.dll` | G-Core Bridge (:7721) REST API | **Complete** |
| `Copilot.Drivers.GeviSoft.dll` | GeViServer Bridge (:7710) REST API | Minimal |
| `Copilot.Drivers.Common.dll` | Bridge REST API contracts | **Complete** |
| `Copilot.Common.dll` | `config/`, `domain/entities/` | Partially built |
| `Copilot.Common.Services.dll` | `data/services/` (BridgeService, StateService) | Partially built |
| `Copilot.AppServer.Client.dll` | `CoordinationService` (WebSocket to PRIMARY) | Not started |
| `Copilot.AppServer.dll` | PRIMARY keyboard coordination logic | Not started |
| `Copilot.AppServer.Database.dll` | In-memory state on PRIMARY | Not started |
### Class-Level Mapping
| Legacy Class | Flutter Class | BLoC | Notes |
|-------------|--------------|------|-------|
| `SegmentViewModel` (1323 lines) | Split into multiple BLoCs | `WallBloc` + `PtzBloc` + `CameraBloc` | Core logic, needs decomposition |
| `MainWindow` (217 lines) | `MainScreen` + `KeyboardService` | — | Input routing |
| `CameraControllerService` (65 lines) | `BridgeService` | — | Already mapped to REST |
| `CameraLockService` (61 lines) | `LockService` (new) | `LockBloc` (new) | Via PRIMARY WebSocket |
| `SequenceService` (77 lines) | `SequenceService` (new) | `SequenceBloc` (new) | Via PRIMARY WebSocket |
| `ConfigurationService` (100 lines) | `AppConfig` + JSON files | — | Local files, no sync |
| `FunctionButtonsService` (78 lines) | `FunctionButtonService` (new) | `WallBloc` | Config-driven |
| `PrepositionService` (114 lines) | `PrepositionService` (new) | `PrepositionBloc` (new) | Via bridge REST |
| `CameraAlarmService` (47 lines) | `AlarmService` (exists) | `AlarmBloc` (exists) | Already implemented |
| `PlaybackStateService` (37 lines) | `PlaybackService` (new) | `PlaybackBloc` (new) | Via bridge REST |
| `CopilotDevice` | `KeyboardService` (new) | — | Web Serial + HID APIs |
| `NavigationService` | `GoRouter` | — | Already in pubspec |
## What's Already Built vs. What's Missing
### Already Built in Flutter
| Feature | Files | Status |
|---------|-------|--------|
| CrossSwitch (camera → monitor) | `WallBloc`, `BridgeService` | Working |
| PTZ control (Pan/Tilt/Zoom) | `PtzBloc`, `BridgeService` | Working |
| Camera number input (numpad) | `WallBloc`, `MainScreen` | Working |
| Camera prefix selection (500/501/502) | `WallBloc` | Working |
| Monitor state tracking | `MonitorBloc`, `StateService` | Working |
| Alarm monitoring | `AlarmBloc`, `AlarmService` | Working |
| Connection management | `ConnectionBloc`, `BridgeService` | Working |
| Wall grid UI (5 sections) | `WallOverview`, `SectionView` | Working |
| WebSocket event streaming | `BridgeService` | Working |
| Bridge health checks | `BridgeService` | Working |
| DI container (GetIt) | `injection_container.dart` | Working |
### Missing — Must Be Built
| Feature | Priority | Complexity | Legacy Reference |
|---------|----------|-----------|------------------|
| Camera lock system | **Critical** | High | `CameraLockService`, `SegmentViewModel:631-676` |
| PRIMARY/STANDBY coordination | **Critical** | High | Replaces `AppServer` |
| Hardware keyboard (Serial) | **Critical** | Medium | `CopilotDevice`, `SerialPortDataProvider` |
| Hardware joystick (HID) | **Critical** | Medium | `JoystickHidDataProvider` |
| Function buttons (F1-F7) | High | Low | `FunctionButtonsService` |
| Prepositions (saved positions) | High | Medium | `PrepositionService` |
| Camera number edit timeout | High | Low | `SegmentViewModel:829-847` |
| Playback mode (jog/shuttle) | Medium | Medium | `PlaybackStateService`, viewer controller |
| Sequences (camera cycling) | Medium | High | `SequenceService` |
| Keyboard emulation (dev mode) | Medium | Low | `MainWindow:60-104` |
| Service menu | Low | Low | Long-press Backspace 3s |
| Config hot-reload | Low | Medium | `ConfigurationService` |
| CrossSwitch rules engine | Medium | Medium | `crossswitch-rules.json` |
| Alarm history view | Low | Low | `CameraAlarmService.GetAlarmForCamera` |
## Implementation Phases
### Phase 2A: Camera Lock System (Critical)
The lock system is the most complex missing piece. In the legacy app:
1. User selects a PTZ camera → triggers lock attempt
2. If lock acquired → PTZ telemetry enabled
3. If locked by another keyboard → dialog: "Request takeover?"
4. Lock expires after 5 minutes unless reset by PTZ movement
5. 1-minute warning before expiration
6. Priority levels: Low (default), High (override)
**New implementation:**
- PRIMARY keyboard manages lock state in memory
- Keyboards send lock requests via WebSocket to PRIMARY
- PRIMARY broadcasts lock state changes to all keyboards
- In degraded mode (no PRIMARY): local-only locks (no coordination)
### Phase 2B: PRIMARY/STANDBY Coordination
```
WebSocket Hub on PRIMARY (:8090)
├── /ws/heartbeat → 2-second heartbeat
├── /ws/locks → Lock state sync
├── /ws/sequences → Sequence state sync
└── /ws/state → General state broadcast
```
### Phase 3: Hardware Input
Serial port protocol (from `CopilotDevice`):
- Prefix `p` = key press, `r` = key release
- Prefix `j` = jog value, `s` = shuttle value
- Prefix `v` = version, `h` = heartbeat
HID joystick:
- VendorId: 10959 (0x2ACF), ProductId: 257 (0x0101)
- 3 axes: X (pan), Y (tilt), Z (zoom)
- Value range: -255 to +255
### Phase 4: Sequences & Playback
- Sequences run on PRIMARY keyboard (not AppServer)
- Playback via bridge viewer control endpoints
- Jog/shuttle wheel maps to playback speed (-7 to +7)
## Bridge API Reference (Quick)
Both bridges expose identical endpoints:
```
POST /viewer/connect-live {Viewer, Channel} → CrossSwitch
POST /viewer/clear {Viewer} → Clear monitor
POST /camera/pan {Camera, Direction, Speed} → PTZ pan
POST /camera/tilt {Camera, Direction, Speed} → PTZ tilt
POST /camera/zoom {Camera, Direction, Speed} → PTZ zoom
POST /camera/stop {Camera} → Stop PTZ
POST /camera/preset {Camera, Preset} → Go to preset
GET /monitors → All monitor states
GET /alarms/active → Active alarms
GET /health → Bridge health
WS /ws/events → Real-time events
```
## Configuration Files
| File | Purpose | Legacy Equivalent |
|------|---------|-------------------|
| `servers.json` | Server connections, bridge URLs, ID ranges | `CameraServersConfiguration` |
| `keyboards.json` | Keyboard identity, PRIMARY/STANDBY role | `CopilotAppConfiguration` |
| `wall-config.json` | Monitor wall layout (sections, monitors, viewers) | `MonitorWallConfiguration` |
| `function-buttons.json` | F1-F7 actions per wall | `FunctionButtonsConfiguration` |
| `prepositions.json` | Camera preset names | `PrepositionsConfiguration` |
| `crossswitch-rules.json` | Routing rules for CrossSwitch | New (was in AppServer logic) |
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Joystick latency via HTTP bridge | Measure. Legacy: ~1ms (in-process). New: ~5-20ms (HTTP). Acceptable for PTZ. |
| Lock coordination without AppServer | PRIMARY WebSocket hub. Degraded mode = local-only locks. |
| Sequence failover mid-execution | PRIMARY tracks sequence state. STANDBY can resume from last known position. |
| Config drift between keyboards | PRIMARY broadcasts config changes. Startup sync on connect. |
| Browser HID/Serial support | Use Web HID API + Web Serial API. Desktop fallback via `flutter_libserialport`. |

View File

@@ -0,0 +1,701 @@
# Feature Implementation Guide
> Step-by-step guide for implementing each missing feature in the Flutter app. Each section references the legacy source code and provides the Flutter implementation approach.
## Implementation Priority Order
```mermaid
gantt
title Migration Implementation Timeline
dateFormat YYYY-MM-DD
axisFormat %b %d
section Phase 2 - Critical
Camera Lock System :p2a, 2026-02-15, 10d
PRIMARY/STANDBY Hub :p2b, 2026-02-15, 14d
Hardware Keyboard (Serial) :p2c, after p2a, 7d
Hardware Joystick (HID) :p2d, after p2a, 7d
section Phase 3 - High
Function Buttons F1-F7 :p3a, after p2c, 3d
Prepositions :p3b, after p2c, 5d
Camera Edit Timeout :p3c, after p2c, 2d
CrossSwitch Rules Engine :p3d, after p2c, 5d
section Phase 4 - Medium
Playback Mode :p4a, after p3b, 7d
Sequences :p4b, after p3b, 10d
Alarm History :p4c, after p3b, 3d
Service Menu :p4d, after p3b, 3d
```
---
## 1. Camera Lock System
**Legacy:** `CameraLockService.cs` + `SegmentViewModel.cs:631-676`
**Complexity:** High | **Priority:** Critical
### New Files to Create
```
lib/
├── data/services/
│ └── lock_service.dart # Lock operations via PRIMARY WebSocket
├── domain/entities/
│ └── camera_lock.dart # Lock state entity
└── presentation/blocs/lock/
├── lock_bloc.dart # Lock state management
├── lock_event.dart
└── lock_state.dart
```
### Entity: CameraLock
```dart
class CameraLock extends Equatable {
final int cameraId;
final String heldBy; // keyboard name
final CameraLockPriority priority;
final DateTime acquiredAt;
final DateTime expiresAt;
bool get isExpiringSoon =>
expiresAt.difference(DateTime.now()) < Duration(minutes: 1);
@override
List<Object?> get props => [cameraId, heldBy, priority, acquiredAt, expiresAt];
}
enum CameraLockPriority { low, high }
enum CameraLockResult { acquired, denied, takeoverRequested }
```
### Service: LockService
```dart
class LockService {
final CoordinationService _coordination;
// Lock operations (send to PRIMARY via WebSocket)
Future<CameraLockResult> tryLock(int cameraId, CameraLockPriority priority);
Future<void> unlock(int cameraId);
Future<void> requestTakeover(int cameraId);
Future<void> confirmTakeover(int cameraId, bool confirm);
Future<void> resetExpiration(int cameraId); // Called on every PTZ movement
// State streams
Stream<Map<int, CameraLock>> get lockStates; // All locks
Stream<LockNotification> get notifications; // Lock events for this keyboard
}
```
### BLoC: LockBloc
Key events:
- `TryLockCamera(cameraId)` → attempt lock, show dialog if denied
- `UnlockCamera(cameraId)` → release lock
- `ResetLockExpiration(cameraId)` → on PTZ movement
- `TakeoverRequested(cameraId, requestedBy)` → show confirmation dialog
- `LockStateUpdated(locks)` → from PRIMARY broadcast
Key state:
- `lockedCameras: Set<int>` — cameras locked by this keyboard
- `allLocks: Map<int, CameraLock>` — all locks across keyboards
- `pendingTakeover: int?` — camera with pending takeover dialog
- `isTelemetryActive: bool` — true when any camera locked
### Integration with Existing BLoCs
The `WallBloc` needs to check lock state before PTZ. Modify `PtzBloc`:
```dart
// Before sending PTZ command:
if (!lockBloc.state.lockedCameras.contains(cameraId)) {
// Camera not locked, try to lock first
lockBloc.add(TryLockCamera(cameraId));
return;
}
// Camera locked, proceed with PTZ
bridgeService.ptzPan(camera, direction, speed);
lockBloc.add(ResetLockExpiration(cameraId)); // Reset expiry timer
```
### Lock Expiry Timer (Client-Side)
```dart
// In LockBloc: start timer when lock acquired
Timer? _expiryWarningTimer;
void _onLockAcquired(int cameraId) {
_expiryWarningTimer?.cancel();
_expiryWarningTimer = Timer(Duration(minutes: 4), () {
emit(state.copyWith(expiryWarning: cameraId));
});
}
```
---
## 2. PRIMARY/STANDBY Coordination Hub
**Legacy:** Replaces `Copilot.AppServer.dll` + `Copilot.AppServer.Client.dll`
**Complexity:** High | **Priority:** Critical
### New Files to Create
```
lib/
├── data/services/
│ └── coordination_service.dart # WebSocket client to PRIMARY
├── coordination/
│ ├── primary_hub.dart # WebSocket server (when this keyboard is PRIMARY)
│ ├── election_service.dart # PRIMARY election logic
│ ├── heartbeat_service.dart # 2-second heartbeat
│ └── messages.dart # Shared message types
└── presentation/blocs/coordination/
├── coordination_bloc.dart
├── coordination_event.dart
└── coordination_state.dart
```
### Message Protocol
```dart
abstract class CoordinationMessage {
String get type;
Map<String, dynamic> toJson();
}
// Heartbeat
class HeartbeatMessage extends CoordinationMessage {
final String keyboardId;
final DateTime timestamp;
final bool isPrimary;
}
// Lock messages
class LockRequestMessage extends CoordinationMessage { ... }
class LockResponseMessage extends CoordinationMessage { ... }
class LockBroadcastMessage extends CoordinationMessage { ... }
// Sequence messages
class SequenceStartMessage extends CoordinationMessage { ... }
class SequenceStopMessage extends CoordinationMessage { ... }
class SequenceStateBroadcast extends CoordinationMessage { ... }
// State sync
class FullStateSyncMessage extends CoordinationMessage {
final Map<int, CameraLock> locks;
final Map<int, SequenceState> sequences;
}
```
### Election Protocol
```dart
class ElectionService {
// 1. On startup, try to connect to configured PRIMARY
// 2. If PRIMARY responds → become STANDBY
// 3. If PRIMARY doesn't respond within 6s → self-promote to PRIMARY
// 4. As PRIMARY: start WebSocket server on :8090
// 5. As STANDBY: send heartbeat checks every 2s
// 6. If 3 consecutive heartbeats missed (6s) → promote to PRIMARY
Stream<KeyboardRole> get role; // PRIMARY or STANDBY
}
enum KeyboardRole { primary, standby, standalone }
```
### Web Platform Consideration
For Flutter web, the WebSocket server (PRIMARY hub) cannot run in the browser. Options:
1. **Desktop build** on LattePanda → full support
2. **Web build** → can only be STANDBY (connects to PRIMARY via WebSocket client)
3. **Hybrid** → separate Dart CLI process for PRIMARY hub
---
## 3. Hardware Keyboard (Serial Port)
**Legacy:** `CopilotDevice.cs`, `SerialPortDataProvider.cs`
**Complexity:** Medium | **Priority:** Critical
### New Files
```
lib/
├── data/services/
│ └── keyboard_service.dart # Serial port + HID abstraction
├── domain/entities/
│ └── virtual_key.dart # Key enum (matches legacy VirtualKey)
```
### Serial Protocol
```
Baud rate: 115200
Data bits: 8, Stop bits: 1, Parity: None
Messages (ASCII, line-terminated):
p<keycode> → Key pressed (e.g., "p48" = digit 0)
r<keycode> → Key released (e.g., "r48" = digit 0)
j<value> → Jog wheel position (-1 or +1 per detent)
s<value> → Shuttle wheel position (-7 to +7)
v<version> → Firmware version response
h → Heartbeat from Arduino
```
### Web Serial API Implementation
```dart
// For Flutter Web:
class WebSerialKeyboardService implements KeyboardService {
// Uses dart:js_interop to call Web Serial API
// navigator.serial.requestPort()
// port.open({ baudRate: 115200 })
// port.readable.getReader() → parse messages
}
// For Flutter Desktop:
class NativeSerialKeyboardService implements KeyboardService {
// Uses flutter_libserialport package
// SerialPort(portName).open(read: true)
// Read stream → parse messages
}
```
### Key Enum (VirtualKey)
Map Arduino keycodes to VirtualKey enum (matches legacy):
```dart
enum VirtualKey {
digit0, digit1, digit2, digit3, digit4,
digit5, digit6, digit7, digit8, digit9,
enter, backspace, escape, prefix,
plus, minus, home,
f1, f2, f3, f4, f5, f6, f7,
lock, search, prepositions, playback,
sequence, alarmHistory, fullScreen,
joyButton1, joyButton2,
}
```
---
## 4. Hardware Joystick (HID)
**Legacy:** `JoystickHidDataProvider.cs`
**Complexity:** Medium | **Priority:** Critical
### HID Device Identification
```
VendorId: 0x2ACF (10959)
ProductId: 0x0101 (257)
```
### Web HID API Implementation
```dart
class WebHidJoystickService implements JoystickService {
// navigator.hid.requestDevice([{vendorId: 0x2ACF, productId: 0x0101}])
// device.open()
// device.addEventListener('inputreport', callback)
// Parse input report → extract X, Y, Z axes → scale to -255..+255
}
```
### Input Report Parsing
The Arduino Leonardo sends HID reports with 3 axes:
```
Report format (depends on HID descriptor):
Byte 0-1: X axis (16-bit signed) → Pan
Byte 2-3: Y axis (16-bit signed) → Tilt
Byte 4-5: Z axis (16-bit signed) → Zoom
Byte 6: Buttons (bitmask)
```
Scale raw values to -255..+255 range (same as legacy `GetScaledValue`).
### Deduplication
```dart
// Only emit if value changed (matches legacy DoPtzAction logic)
int _lastX = 0, _lastY = 0, _lastZ = 0;
void _onHidReport(int x, int y, int z) {
if (x != _lastX) { _lastX = x; _ptzBloc.add(PtzPanStart(x)); }
if (y != _lastY) { _lastY = y; _ptzBloc.add(PtzTiltStart(y)); }
if (z != _lastZ) { _lastZ = z; _ptzBloc.add(PtzZoomStart(z)); }
}
```
---
## 5. Function Buttons (F1-F7)
**Legacy:** `FunctionButtonsService.cs` (78 lines)
**Complexity:** Low | **Priority:** High
### New Files
```
lib/
├── config/
│ └── function_buttons_config.dart # Load function-buttons.json
├── data/services/
│ └── function_button_service.dart # Execute button actions
```
### Implementation
```dart
class FunctionButtonService {
final BridgeService _bridge;
final Map<String, Map<String, List<FunctionButtonAction>>> _config;
Future<bool> execute(String wallId, String buttonKey) async {
final actions = _config[wallId]?[buttonKey];
if (actions == null || actions.isEmpty) return false;
for (final action in actions) {
switch (action.type) {
case ActionType.crossSwitch:
await _bridge.viewerConnectLive(action.viewerId, action.sourceId);
case ActionType.sequenceStart:
await _sequenceService.start(action.viewerId, action.sourceId);
}
}
return true;
}
}
```
### Integration with WallBloc
Add to keyboard handler in `MainScreen`:
```dart
case VirtualKey.f1: functionButtonService.execute(wallId, 'F1');
case VirtualKey.f2: functionButtonService.execute(wallId, 'F2');
// ... through F7
```
---
## 6. Prepositions (Camera Presets)
**Legacy:** `PrepositionService.cs` (114 lines)
**Complexity:** Medium | **Priority:** High
### New Files
```
lib/
├── config/
│ └── prepositions_config.dart
├── data/services/
│ └── preposition_service.dart
└── presentation/
├── blocs/preposition/
│ ├── preposition_bloc.dart
│ ├── preposition_event.dart
│ └── preposition_state.dart
└── screens/
├── prepositions_screen.dart
└── preposition_add_screen.dart
```
### Service Implementation
```dart
class PrepositionService {
final BridgeService _bridge;
final PrepositionsConfig _config;
Map<int, String> getNames(int cameraId) =>
_config.getPrepositionNames(cameraId);
Future<void> callUp(int cameraId, int presetId) =>
_bridge.ptzPreset(cameraId, presetId);
Future<void> save(int cameraId, int presetId, String name) async {
_config.setPrepositionName(cameraId, presetId, name);
await _config.persist();
// If PRIMARY available: broadcast to other keyboards
}
Future<void> delete(int cameraId, int presetId) async {
_config.removePreposition(cameraId, presetId);
await _config.persist();
}
}
```
### UI Flow
```mermaid
stateDiagram-v2
[*] --> PrepositionList: User presses Prepositions key
PrepositionList --> CallUp: Select preset
CallUp --> PrepositionList: Camera moves to position
PrepositionList --> AddPreposition: Press Add button
AddPreposition --> EnterName: Move camera to position
EnterName --> PrepositionList: Save with name
PrepositionList --> [*]: Press Back
```
---
## 7. Camera Number Edit Timeout
**Legacy:** `SegmentViewModel.cs:829-847`
**Complexity:** Low | **Priority:** High
### Add to WallBloc
```dart
// In WallBloc
Timer? _editTimeoutTimer;
static const editTimeout = Duration(seconds: 5); // from config
void _onAddCameraDigit(AddCameraDigit event, Emitter<WallState> emit) {
_editTimeoutTimer?.cancel();
_editTimeoutTimer = Timer(editTimeout, () {
add(ClearCameraInput()); // Auto-cancel on timeout
});
// ... existing digit logic
}
void _onExecuteCrossSwitch(ExecuteCrossSwitch event, Emitter<WallState> emit) {
_editTimeoutTimer?.cancel();
// ... existing CrossSwitch logic
}
```
---
## 8. Playback Mode
**Legacy:** `PlaybackStateService.cs`, viewer controller
**Complexity:** Medium | **Priority:** Medium
### New Files
```
lib/
├── data/services/
│ └── playback_service.dart
├── presentation/
│ ├── blocs/playback/
│ │ ├── playback_bloc.dart
│ │ ├── playback_event.dart
│ │ └── playback_state.dart
│ └── screens/
│ └── playback_screen.dart
```
### Shuttle Speed Mapping
```dart
const shuttleSpeedMap = {
-7: -32.0, -6: -16.0, -5: -8.0, -4: -4.0,
-3: -2.0, -2: -1.0, -1: -0.5, 0: 0.0,
1: 0.5, 2: 1.0, 3: 2.0, 4: 4.0,
5: 8.0, 6: 16.0, 7: 32.0,
};
```
### Bridge Endpoints
```dart
class PlaybackService {
Future<void> setPlayMode(int viewer, PlayMode mode, double speed) =>
_bridge.post('/viewer/set-play-mode', {
'Viewer': viewer, 'PlayMode': mode.name, 'PlaySpeed': speed
});
Future<void> playFromTime(int viewer, int channel, PlayMode mode, DateTime time) =>
_bridge.post('/viewer/play-from-time', {
'Viewer': viewer, 'Channel': channel,
'PlayMode': mode.name, 'Time': time.toIso8601String()
});
Future<void> jumpByTime(int viewer, int channel, PlayMode mode, int seconds) =>
_bridge.post('/viewer/jump-by-time', {
'Viewer': viewer, 'Channel': channel,
'PlayMode': mode.name, 'TimeInSec': seconds
});
}
```
---
## 9. Sequence Execution
**Legacy:** `SequenceService.cs` (77 lines)
**Complexity:** High | **Priority:** Medium
### Architecture Decision
Sequences run on the PRIMARY keyboard (replaces AppServer):
```
Flutter App (any keyboard)
→ WebSocket message to PRIMARY
→ PRIMARY runs sequence timer
→ PRIMARY sends CrossSwitch commands to bridge
→ PRIMARY broadcasts state to all keyboards
```
### New Files
```
lib/
├── data/services/
│ └── sequence_service.dart # Client (sends to PRIMARY)
├── coordination/
│ └── sequence_runner.dart # Server (runs on PRIMARY)
├── presentation/
│ ├── blocs/sequence/
│ │ ├── sequence_bloc.dart
│ │ ├── sequence_event.dart
│ │ └── sequence_state.dart
│ └── screens/
│ ├── sequence_categories_screen.dart
│ └── sequences_screen.dart
```
### Sequence Runner (PRIMARY only)
```dart
class SequenceRunner {
final Map<int, Timer> _activeSequences = {}; // viewerId → timer
void start(int viewerId, Sequence sequence) {
stop(viewerId); // Stop existing sequence on this viewer
int stepIndex = 0;
_activeSequences[viewerId] = Timer.periodic(
Duration(seconds: sequence.dwellTime),
(_) {
final camera = sequence.cameras[stepIndex % sequence.cameras.length];
bridgeService.viewerConnectLive(viewerId, camera);
stepIndex++;
_broadcastState();
},
);
}
void stop(int viewerId) {
_activeSequences[viewerId]?.cancel();
_activeSequences.remove(viewerId);
_broadcastState();
}
}
```
---
## 10. Service Menu
**Legacy:** `MainWindow.cs` — Backspace held 3 seconds
**Complexity:** Low | **Priority:** Low
### Implementation
```dart
// In MainScreen or KeyboardService
Timer? _serviceKeyTimer;
void _onKeyDown(VirtualKey key) {
if (key == VirtualKey.backspace) {
_serviceKeyTimer = Timer(Duration(seconds: 3), () {
GoRouter.of(context).push('/service-menu');
});
}
}
void _onKeyUp(VirtualKey key) {
if (key == VirtualKey.backspace) {
_serviceKeyTimer?.cancel();
}
}
```
### Service Menu Options
- Restart application
- Restart hardware (LattePanda reboot)
- Shutdown hardware
- Show firmware version
- Show app version
- Network diagnostics
---
## Testing Strategy
### Unit Tests (per feature)
| Feature | Test Focus |
|---------|-----------|
| Lock system | Lock/unlock/takeover/expiry logic |
| Coordination | Election, heartbeat, failover |
| Keyboard input | Key mapping, edit timeout, service hotkey |
| Function buttons | Config loading, action execution |
| Prepositions | CRUD operations, name mapping |
| Playback | Shuttle speed mapping, mode transitions |
| Sequences | Start/stop/timer/failover |
### Integration Tests
| Test | Description |
|------|------------|
| CrossSwitch + Lock | Lock camera → PTZ → CrossSwitch → auto-unlock |
| Failover | PRIMARY stops → STANDBY promotes → locks preserved |
| Alarm blocking | Active alarm → CrossSwitch blocked → alarm resolved → CrossSwitch works |
| Sequence + CrossSwitch | Running sequence → CrossSwitch → sequence stopped |
| Playback + PTZ | Enter playback → switch to PTZ → playback stopped |
### Hardware Tests (Manual)
| Test | Verify |
|------|--------|
| Serial keyboard | All keys mapped correctly, no missed events |
| HID joystick | Axes correct direction, full range, smooth PTZ |
| Shuttle wheel | Speed mapping matches legacy behavior |
| Jog wheel | Single-step forward/backward |
| Long-press Backspace | Service menu opens after 3 seconds |
---
## File Summary: What to Create
| File | Type | Lines (est.) | Phase |
|------|------|-------------|-------|
| `lock_service.dart` | Service | 120 | 2A |
| `camera_lock.dart` | Entity | 40 | 2A |
| `lock_bloc.dart` + events/state | BLoC | 200 | 2A |
| `coordination_service.dart` | Service | 150 | 2B |
| `primary_hub.dart` | Server | 200 | 2B |
| `election_service.dart` | Service | 100 | 2B |
| `heartbeat_service.dart` | Service | 60 | 2B |
| `messages.dart` | Models | 120 | 2B |
| `coordination_bloc.dart` + events/state | BLoC | 150 | 2B |
| `keyboard_service.dart` | Service | 150 | 3 |
| `virtual_key.dart` | Enum | 50 | 3 |
| `joystick_service.dart` | Service | 100 | 3 |
| `function_button_service.dart` | Service | 60 | 3 |
| `function_buttons_config.dart` | Config | 40 | 3 |
| `preposition_service.dart` | Service | 80 | 3 |
| `prepositions_config.dart` | Config | 40 | 3 |
| `preposition_bloc.dart` + events/state | BLoC | 120 | 3 |
| `prepositions_screen.dart` | Screen | 100 | 3 |
| `playback_service.dart` | Service | 80 | 4 |
| `playback_bloc.dart` + events/state | BLoC | 120 | 4 |
| `playback_screen.dart` | Screen | 150 | 4 |
| `sequence_service.dart` | Service | 60 | 4 |
| `sequence_runner.dart` | Server | 100 | 4 |
| `sequence_bloc.dart` + events/state | BLoC | 120 | 4 |
| `sequence_screens.dart` | Screens | 150 | 4 |
| **Total** | | **~2,560** | |

View File

@@ -0,0 +1,43 @@
{
"$schema": "https://mintlify.com/schema.json",
"name": "COPILOT D6 Legacy Architecture",
"logo": {
"light": "/logo/light.svg",
"dark": "/logo/dark.svg"
},
"favicon": "/favicon.png",
"colors": {
"primary": "#2563EB",
"light": "#60A5FA",
"dark": "#1D4ED8"
},
"navigation": [
{
"group": "Overview",
"pages": ["index"]
},
{
"group": "Architecture",
"pages": [
"architecture",
"hardware-input",
"ptz-control"
]
},
{
"group": "Workflows",
"pages": [
"data-flows",
"configuration"
]
},
{
"group": "Infrastructure",
"pages": [
"appserver",
"migration-comparison"
]
}
],
"mermaid": {}
}

View File

@@ -0,0 +1,172 @@
---
title: "PTZ Control Flow"
description: "Complete joystick → Pan/Tilt/Zoom → SDK command pipeline with speed analysis"
---
# PTZ Control Flow
## End-to-End Pipeline
```mermaid
sequenceDiagram
participant HW as Joystick (HID)
participant Dev as CopilotDevice
participant MW as MainWindow
participant VM as SegmentViewModel
participant CCS as CameraControllerService
participant Drv as GeViScope/GCore Driver
participant SDK as Native SDK (PLC)
participant Srv as Camera Server
HW->>Dev: HID report (raw axis data)
Dev->>Dev: Scale to -255..+255
Dev->>MW: JoystickMoved(x, y, z)
MW->>MW: Dispatcher.Invoke (UI thread)
MW->>VM: IJoystickHandler.OnJoystickMoved(x, y, z)
alt IsCameraTelemetryActive (PTZ locked)
VM->>VM: DoPtzAction(x, Pan, controller.Pan)
VM->>VM: DoPtzAction(y, Tilt, controller.Tilt)
VM->>VM: DoPtzAction(z, Zoom, controller.Zoom)
Note over VM: DoPtzAction only sends if speed changed<br/>(deduplication via ptzSpeeds dictionary)
VM->>CCS: GetMovementController(cameraNumber)
CCS->>Drv: GetMovementControllerForChannel(id)
Drv-->>VM: IMovementController
alt x changed (Pan)
alt x > 0
VM->>Drv: PanRight(x)
else x < 0
VM->>Drv: PanLeft(abs(x))
else x == 0
VM->>Drv: PanStop()
end
end
Drv->>SDK: GscAct_PanRight(channelId, speed)
SDK->>Srv: TCP PLC command
VM->>VM: ResetCameraLockExpiration()
else Not locked
Note over VM: Joystick events ignored
end
```
## Speed Value Processing
### Layer 1: HID → Raw Value
```
Joystick physical position → HID report → GetScaledValue(-255.0, 255.0) → integer
```
The joystick reports continuously while being held. Each axis change fires independently.
### Layer 2: Deduplication (DoPtzAction)
```csharp
private void DoPtzAction(int? value, PtzAction ptzActionFlag, Action<int> ptzAction)
{
if (value.HasValue)
{
int speed = value.GetValueOrDefault();
// Only send if speed actually changed
if (!ptzSpeeds.TryGetValue(ptzActionFlag, out var prevSpeed) || speed != prevSpeed)
{
ptzSpeeds[ptzActionFlag] = speed;
ptzAction(speed); // Call Pan/Tilt/Zoom with raw value
}
}
}
```
**Key insight:** The `ptzSpeeds` dictionary prevents flooding the SDK with duplicate commands. Only changed values are forwarded.
### Layer 3: Direction Resolution (IMovementControllerExtensions)
```csharp
public static void Pan(this IMovementController controller, int speed)
{
if (speed > 0) controller.PanRight(speed);
if (speed < 0) controller.PanLeft(Math.Abs(speed));
if (speed == 0) controller.PanStop();
}
// Same pattern for Tilt and Zoom
```
### Layer 4: SDK Command
```csharp
// GeViScope example
public void PanRight(int speed)
{
using var channelId = new GscMediaChannelID(mediaChannelId);
using var action = new GscAct_PanRight(channelId, speed);
plcWrapper()?.SendAction(action);
}
```
The speed value (0-255) is passed directly to the native SDK with no transformation.
## Speed Range Summary
```
Joystick Position → Speed Value → SDK Parameter
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Full Left/Up → -255 → PanLeft(255) / TiltUp(255)
Half Left/Up → ~-128 → PanLeft(128) / TiltUp(128)
Dead Center → 0 → PanStop() / TiltStop()
Half Right/Down → ~+128 → PanRight(128) / TiltDown(128)
Full Right/Down → +255 → PanRight(255) / TiltDown(255)
Zoom twist out → -255 → ZoomOut(255)
Zoom center → 0 → ZoomStop()
Zoom twist in → +255 → ZoomIn(255)
```
## Zoom-Proportional Speed: NOT in App Code
**Critical finding:** The COPILOT WPF application does NOT implement zoom-proportional pan/tilt speed adjustment. The raw joystick value (-255 to +255) passes through all layers unmodified.
The "slower pan/tilt when zoomed in" behavior that operators observe is provided by one or more of:
1. **Camera hardware** — Most PTZ cameras (Axis, Bosch, Pelco) have built-in "proportional PTZ" that adjusts mechanical speed based on current zoom level
2. **GeViScope/G-Core server** — The video management software may apply speed scaling before sending commands to the physical camera
3. **PTZ protocol** — Some protocols (ONVIF, Pelco-D) support proportional speed as a protocol-level feature
**Implication for Flutter rewrite:** The new system will get this behavior "for free" as long as it sends the same speed values (0-255) to the same server/camera infrastructure.
## Focus Control
Focus is controlled via the `+` and `-` hardware keys (not the joystick):
```
Key Down (+) → FocusNear(128) (fixed speed)
Key Down (-) → FocusFar(128) (fixed speed)
Key Up → FocusStop()
```
The `DefaultFocusSpeed` constant is **128** (half of maximum).
## PTZ Lock Requirement
PTZ commands are only sent when `IsCameraTelemetryActive == true`, which requires:
1. The selected camera must be `Movable` (from MediaChannel metadata)
2. The operator must acquire a camera lock via the AppServer
3. The lock has a 5-minute timeout with a 1-minute warning
4. Each PTZ action resets the lock expiration timer
```mermaid
stateDiagram-v2
[*] --> Unlocked
Unlocked --> LockRequested: Press Lock/Button2
LockRequested --> Locked: Lock acquired
LockRequested --> ConfirmDialog: Lock held by other operator
ConfirmDialog --> TakeOverRequested: Confirm takeover
TakeOverRequested --> Locked: Takeover granted
TakeOverRequested --> Unlocked: Takeover denied
Locked --> ExpirationWarning: 4 minutes elapsed
ExpirationWarning --> Locked: Any PTZ action (resets timer)
ExpirationWarning --> Unlocked: 5 minutes elapsed (auto-unlock)
Locked --> Unlocked: Press Lock/Button2 (manual unlock)
Locked --> Unlocked: TakenOver by higher priority
```

View File

@@ -0,0 +1,34 @@
<!DOCTYPE html>
<html>
<head>
<title>Mermaid Test</title>
<style>body { background: #1e1e2e; color: #cdd6f4; font-family: sans-serif; padding: 2em; }</style>
</head>
<body>
<h1>Mermaid Rendering Test</h1>
<p>If you see a diagram below, mermaid is working:</p>
<pre class="mermaid">
graph LR
A[Flutter App] -->|REST| B[C# Bridge]
B -->|SDK| C[Camera Server]
</pre>
<p>Sequence diagram test:</p>
<pre class="mermaid">
sequenceDiagram
participant App
participant Bridge
participant Camera
App->>Bridge: POST /api/ptz/pan
Bridge->>Camera: SDK PanRight(speed)
Camera-->>Bridge: OK
Bridge-->>App: 200 OK
</pre>
<script src="https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.min.js"></script>
<script>
mermaid.initialize({ startOnLoad: true, theme: 'dark' });
</script>
</body>
</html>