COLSYS/COPILOT
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b2887b67db77cc632880da1b0d4ba4f79a37d653
COPILOT/Docs/NEW_SYSTEM_DESIGN_SUMMARY.md
klas 40143734fc Initial commit: COPILOT D6 Flutter keyboard controller 
Flutter web app replacing legacy WPF CCTV surveillance keyboard controller.
Includes wall overview, section view with monitor grid, camera input,
PTZ control, alarm/lock/sequence BLoCs, and legacy-matching UI styling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 14:57:38 +01:00

27 KiB
Raw Blame History

COPILOT New System Design Summary

Document Version: 1.0 Date: 2026-02-03 Status: Design Complete - Ready for Implementation

Executive Summary

This document describes the architecture for rewriting the COPILOT CCTV keyboard controller system with a new master/slave architecture. The new system eliminates the need for dedicated server hardware by enabling any keyboard to act as the coordination PRIMARY, while maintaining critical functionality (ViewerConnectLive) even when the coordination layer fails.

Key Goals

  • Zero extra hardware - PRIMARY runs on a keyboard (LattePanda Sigma)
  • Direct command path - Commands go directly to servers, not through a coordinator
  • High availability - Automatic failover with degraded mode support
  • Cross-platform - Flutter-based keyboards (web/desktop)
  • State consistency - Query-based alarm state, not just event subscription

1. Architecture Overview

1.1 Current System (COPILOT_D6)

┌─────────────────────────────────────────────────────────────────┐
│ COPILOT_D6 ARCHITECTURE                                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Keyboards (WPF) ──────► AppServer (Central) ──────► GeViSoft   │
│                          - SignalR Hub                          │
│                          - Lock Management                      │
│                          - Separate Hardware                    │
│                                                                  │
│  Problems:                                                       │
│  • Central AppServer = single point of failure                  │
│  • Extra hardware required                                       │
│  • All commands routed through central point                    │
│  • WPF = Windows-only                                           │
│  • Event-only alarm tracking (misses alarms if offline)         │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

1.2 New System Architecture

┌─────────────────────────────────────────────────────────────────┐
│ NEW ARCHITECTURE                                                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐         │
│  │ Keyboard 1  │    │ Keyboard 2  │    │ Keyboard N  │         │
│  │ (PRIMARY)   │    │ (STANDBY)   │    │ (REGULAR)   │         │
│  │ Flutter     │    │ Flutter     │    │ Flutter     │         │
│  │ + Bridges   │    │ + Bridges   │    │ + Bridges   │         │
│  └──────┬──────┘    └──────┬──────┘    └──────┬──────┘         │
│         │                  │                  │                 │
│         │  WebSocket (coordination only)      │                 │
│         ├──────────────────┼──────────────────┤                 │
│         │                  │                  │                 │
│         │  Direct HTTP (ViewerConnectLive, PTZ)                 │
│         ▼                  ▼                  ▼                 │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐         │
│  │ GeViScope   │    │  G-Core     │    │ GeViServer  │         │
│  │ Servers     │    │  Servers    │    │ (PTZ only)  │         │
│  └─────────────┘    └─────────────┘    └─────────────┘         │
│                                                                  │
│  Benefits:                                                       │
│  • No extra hardware (PRIMARY runs on keyboard)                 │
│  • Direct commands = low latency (<100ms)                       │
│  • Automatic failover to STANDBY                                │
│  • ViewerConnectLive works even if PRIMARY dead                 │
│  • Query-based alarm state (never miss alarms)                  │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

2. Hardware Platform

2.1 LattePanda Sigma SBC (Inside Each Keyboard)

Component Specification
CPU Intel Core i5-1340P (13th Gen), 12 cores/16 threads, up to 4.6 GHz
RAM 16GB or 32GB LPDDR5-6400 (102.4 GB/s bandwidth)
GPU Intel Iris Xe Graphics (80 EUs), up to 4x 4K displays
Network Dual 2.5 GbE ports
TDP Sustained 44W with proper cooling
Size 102mm × 146mm

2.2 Resource Usage (Per Keyboard)

Process RAM Usage CPU Usage
Flutter Keyboard App ~200-400 MB <5%
GeViScope Bridge (.NET 8) ~100-150 MB <2%
G-Core Bridge (.NET 8) ~100-150 MB <2%
GeViServer Bridge (.NET 8) ~100-150 MB <2%
OS + Background ~2-3 GB Variable
Total ~3-4 GB <20% typical

Verdict: LattePanda Sigma with 16GB RAM is highly suitable. 32GB provides future margin.

3. Component Responsibilities

3.1 PRIMARY Keyboard

The PRIMARY is elected from available keyboards based on priority. It handles:

Responsibility Description
PTZ Lock Management Grant/revoke locks, enforce priority, handle timeouts
Shared State Current video wall state (monitor → camera mappings)
Sequence Engine Camera rotation sequences on monitors
Alarm Coordination Aggregate alarm state, broadcast to keyboards
Keyboard Registry Track which keyboards are online
WebSocket Hub Broadcast state changes to all keyboards

What PRIMARY Does NOT Do:

  • Route ViewerConnectLive commands (keyboards send directly)
  • Route PTZ commands (keyboards send directly)
  • Act as single point of failure for critical operations

3.2 STANDBY Keyboard

  • Receives all state updates from PRIMARY
  • Maintains synchronized copy of shared state
  • Monitors PRIMARY via heartbeat (every 2 seconds)
  • Promotes to PRIMARY if heartbeat fails (6 seconds timeout)

3.3 REGULAR Keyboards

  • Send commands directly to servers
  • Request PTZ locks from PRIMARY
  • Receive state broadcasts from PRIMARY
  • Can operate in degraded mode without PRIMARY

3.4 C# Bridges (Per Keyboard)

Bridge Port SDK Purpose
GeViScope Bridge 7720 GeViScope SDK (32-bit) ViewerConnectLive, PTZ, alarms
G-Core Bridge 7721 G-Core SDK ViewerConnectLive, PTZ, alarms
GeViServer Bridge 7710 GeViSoft SDK PTZ fallback, state queries

4. Command Flow

4.1 ViewerConnectLive (Camera → Monitor)

Keyboard                              Server
   │                                    │
   │  1. User selects camera + monitor  │
   │                                    │
   │  2. Route to correct server        │
   │     (based on camera ID mapping)   │
   │                                    │
   │  3. HTTP POST to local bridge      │
   │     POST localhost:7720/crossswitch│
   │         ─────────────────────────► │
   │                                    │
   │  4. Bridge calls SDK               │
   │     ViewerConnectLive(cam, mon)    │
   │         ─────────────────────────► │ GeViScope/G-Core
   │                                    │
   │  5. Server executes, fires event   │
   │     ViewerConnected notification   │
   │     ◄───────────────────────────── │
   │                                    │
   │  6. Update local state             │
   │  7. Notify PRIMARY (if online)     │
   │                                    │

Key Point: Commands go DIRECTLY to servers. PRIMARY is only notified for state tracking.

4.2 PTZ Control with Locking

Keyboard                    PRIMARY                    Server
   │                           │                          │
   │  1. Request PTZ lock      │                          │
   │  ────────────────────────►│                          │
   │                           │                          │
   │  2. PRIMARY checks:       │                          │
   │     - Is camera locked?   │                          │
   │     - Priority comparison │                          │
   │                           │                          │
   │  3. Lock granted/denied   │                          │
   │  ◄────────────────────────│                          │
   │                           │                          │
   │  4. If granted, send PTZ directly                    │
   │  ─────────────────────────────────────────────────►  │
   │                           │                          │
   │  5. Lock timeout (5 min)  │                          │
   │     Warning at 4 min      │                          │
   │  ◄────────────────────────│                          │
   │                           │                          │

4.3 PTZ Lock Configuration

Setting Value Source
Lock Timeout 5 minutes COPILOT_D6 LockExpirationConfig
Warning Before Expiry 1 minute COPILOT_D6 NotificationBeforeExpiration
Priority Levels High, Low COPILOT_D6 CameraLockOptions

5. State Management

5.1 Video Wall State Verification

Problem: No direct "query current state" API in GeViScope/G-Core SDKs.

Solution: Event-based state tracking with startup query.

┌─────────────────────────────────────────────────────────────────┐
│ STATE VERIFICATION APPROACH                                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. STARTUP: Query current state                                │
│     - GetFirstVideoOutput / GetNextVideoOutput                  │
│     - sActualCamera field = currently displayed camera          │
│                                                                  │
│  2. REAL-TIME: Subscribe to notifications                       │
│     - ViewerConnected(Viewer, Channel, PlayMode)               │
│     - ViewerCleared(Viewer)                                     │
│     - ViewerSelectionChanged(Viewer, Channel, ...)             │
│                                                                  │
│  3. VERIFICATION: After sending command                         │
│     - Wait for ViewerConnected notification (500ms timeout)    │
│     - If received: confirm state                                │
│     - If timeout: flag potential failure                        │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

5.2 Alarm State Management

Problem (D6): Event-only tracking misses alarms if AppServer offline.

Solution: Query + Subscribe + Periodic Sync.

┌─────────────────────────────────────────────────────────────────┐
│ ALARM STATE APPROACH                                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. STARTUP: Query ALL active alarms                            │
│     - GeViSQ_GetFirstAlarm(activeOnly=true)                    │
│     - GeViSQ_GetNextAlarm() until no more                      │
│     - Populate local alarm state BEFORE accepting commands     │
│                                                                  │
│  2. REAL-TIME: Subscribe to events                              │
│     - EventStarted(EventID, TypeID) → Add alarm                │
│     - EventStopped(EventID) → Remove alarm                     │
│     - VCAlarmQueueNotification → Update monitor reservation    │
│                                                                  │
│  3. PERIODIC SYNC: Every 30 seconds                             │
│     - Re-query all active alarms                                │
│     - Compare with local state                                  │
│     - Add missing, remove stale                                 │
│     - Log discrepancies                                         │
│                                                                  │
│  RESULT: Never miss an alarm, even after restart/reconnect     │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

5.3 Alarm → Monitor Blocking

Alarm State Value Monitor Effect
vasNewAlarm 0 Monitor reserved, cannot switch
vasPresented 1 Alarm displayed, cannot switch
vasStacked 2 Alarm in queue, monitor may be switchable
vasConfirmed 3 Acknowledged, normal operation resumes

6. Failover and Degraded Mode

6.1 Failover Sequence

┌─────────────────────────────────────────────────────────────────┐
│ FAILOVER SEQUENCE                                               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  NORMAL OPERATION:                                               │
│  • PRIMARY sends heartbeat every 2 seconds                      │
│  • STANDBY receives and acknowledges                            │
│                                                                  │
│  PRIMARY FAILURE DETECTED (3 missed heartbeats = 6 seconds):   │
│  1. STANDBY declares itself PRIMARY                             │
│  2. STANDBY broadcasts: "I am now PRIMARY"                     │
│  3. STANDBY starts accepting lock requests                     │
│  4. STANDBY starts sending heartbeats                          │
│  5. Next highest priority keyboard becomes STANDBY             │
│                                                                  │
│  SPLIT-BRAIN PREVENTION:                                        │
│  • Only one PRIMARY at a time (priority-based election)        │
│  • If old PRIMARY comes back, it defers to new PRIMARY         │
│  • Epoch/generation counter prevents stale PRIMARY takeover    │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

6.2 Degraded Mode

When BOTH PRIMARY and STANDBY are unavailable:

Feature Status Notes
ViewerConnectLive ✅ Works Direct to server
PTZ Control ✅ Works Direct to server
PTZ Locking ❌ Unavailable No coordinator
Sequences ❌ Stopped Runs on PRIMARY
State Sync ❌ Unavailable No broadcaster
Alarm State ⚠️ Local only Each keyboard tracks independently

Critical Path Guarantee: Operators can always switch cameras and control PTZ, even in degraded mode.

7. Recorded Video Access

7.1 Playback Actions (Direct to GeViScope/G-Core)

Action Purpose Parameters
ViewerConnect Play recorded video Viewer, Channel, PlayMode
ViewerPlayFromTime Seek to timestamp Viewer, Channel, PlayMode, Time
ViewerSetPlayMode Control speed/direction Viewer, PlayMode, PlaySpeed
ViewerJumpByTime Relative seek Viewer, Channel, PlayMode, TimeInSec
ViewerExportPicture Snapshot export Viewer, FilePath

7.2 Play Modes

Mode Description
play forward Normal speed forward
play backward Normal speed backward
fast forward / fast backward High speed
step forward / step backward Frame by frame
play stop Pause
play BOD / play EOD Beginning/End of database
next detected motion Skip to motion event

8. External Systems Integration

8.1 Alarm Sources (Digital Inputs)

External System → LAN I/O Device → DigitalInput Action → Event → Alarm
                  (MIO, IOI43)     Contact + State

DigitalInputState Values:

  • disLow (0) - Contact open/inactive
  • disMiddle (1) - Terminated (tamper)
  • disHigh (2) - Contact closed/active

8.2 Logging (ELK Stack)

┌─────────────────────────────────────────────────────────────────┐
│ LOGGING ARCHITECTURE                                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Each Keyboard:                                                  │
│    Flutter App ──► /var/log/copilot/keyboard.jsonl             │
│    Bridges ──────► /var/log/copilot/bridge-*.jsonl             │
│         │                                                        │
│         ▼                                                        │
│    Filebeat (local) ──► Logstash ──► Elasticsearch ──► Kibana  │
│                                                                  │
│  Log Format: JSON Lines (structured)                            │
│  Retention: 30 days local, configurable in ELK                  │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

8.3 User Management (IAM Ready)

┌─────────────────────────────────────────────────────────────────┐
│ AUTHENTICATION ABSTRACTION                                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  interface IAuthProvider {                                       │
│    authenticate(credentials) → AuthResult                       │
│    getUser(userId) → User                                       │
│    getPermissions(userId) → List<Permission>                    │
│    logout(sessionId)                                            │
│  }                                                               │
│                                                                  │
│  Implementations:                                                │
│    Phase 1: LocalAuthProvider (JSON/SQLite)                     │
│    Phase 2: LdapAuthProvider (Active Directory)                 │
│    Phase 3: OidcAuthProvider (Keycloak, Azure AD, Okta)        │
│                                                                  │
│  Permission Model:                                               │
│    viewLive, viewRecorded, ptzControl, ptzControlHighPriority, │
│    switchMonitor, switchAlarmMonitor, manageSequences,         │
│    manageUsers, viewAuditLog, systemConfig                      │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

9. Migration Path

Phase 0: Infrastructure (Week 1-2)

  • Finalize C# bridges (GeViScope, G-Core, GeViServer)
  • Add PLC notification subscriptions
  • Test direct commands without GeViSoft routing
  • Document server configurations

Phase 1: Flutter Keyboard Core (Week 3-5)

  • Keyboard layout UI
  • Direct command execution (ViewerConnectLive, PTZ)
  • Server routing logic
  • State notification subscription
  • Basic error handling

Phase 2: Coordination Layer (Week 6-8)

  • PRIMARY election mechanism
  • PTZ lock management
  • Shared state (video wall, locks, keyboards)
  • WebSocket hub
  • State sync from server notifications

Phase 3: Advanced Features (Week 9-11)

  • Sequence engine
  • CrossSwitch rules
  • Alarm handling
  • User authentication

Phase 4: Testing & Cutover (Week 12-14)

  • Parallel operation testing
  • Failover testing
  • Load testing
  • Operator training
  • Production cutover

10. Key Design Decisions Summary

Decision Choice Rationale
Central Coordinator PRIMARY keyboard Zero extra hardware
Command Path Direct to servers Low latency, no bottleneck
State Verification Event-based + startup query GeViScope/G-Core have no query API
GeViSoft Usage Minimal (PTZ only if needed) User requirement
Failover Priority-based STANDBY promotion Automatic recovery
Degraded Mode ViewerConnectLive works Critical path must always work
PTZ Locks 5-min timeout, priority levels Match D6 behavior
Alarm State Query + Subscribe + Sync Never miss alarms
UI Technology Flutter Cross-platform
Hardware LattePanda Sigma 16GB Sufficient for bridges + app
Logging Local JSON + Filebeat → ELK Central Kibana dashboards
Auth Abstracted, IAM-ready Future AD/OIDC integration

11. Port Assignments

Service Port Protocol
GeViServer 7700-7703 Native SDK
GeViServer Bridge 7710 HTTP REST
GeViScope Bridge 7720 HTTP REST
G-Core Bridge 7721 HTTP REST
Python API (if used) 8000 HTTP REST
Flutter Web 8081 HTTP
PRIMARY WebSocket 8090 WebSocket
gRPC (SDK Bridge) 50051 gRPC

12. Configuration Files

File Purpose Location
servers.json Bridge endpoints per keyboard Per keyboard
keyboards.json Keyboard priorities, roles Shared
sequences.json Camera rotation sequences PRIMARY
crossswitch-rules.json Logical → physical routing Shared
video-wall.json Monitor layout configuration Shared
auth.yaml Authentication provider config Per keyboard

13. Open Questions for Implementation

  1. Bridge Deployment: Bridges run locally on each keyboard (recommended) vs. centralized
  2. PTZ via GeViServer: Any PTZ that MUST go through GeViServer?
  3. Alarm Source: External system integration details
  4. Sequence Persistence: Where to store sequence state on PRIMARY failover?
  5. Log Retention: Local retention period before ELK shipping

14. Risk Mitigation

Risk Mitigation
PRIMARY failure Automatic STANDBY promotion
Network partition Degraded mode maintains critical functions
Missed alarms Startup query + periodic sync
State desync Event verification + periodic full sync
SDK incompatibility C# bridges abstract SDK differences
Performance LattePanda Sigma has significant headroom

15. Success Criteria

  • ViewerConnectLive latency <100ms (direct path)
  • PTZ lock acquisition <50ms
  • Failover time <10 seconds
  • Zero missed alarms after keyboard restart
  • Degraded mode maintains camera switching
  • Support 10+ concurrent keyboards
  • State consistency >99.9%

Document History

Version Date Author Changes
1.0 2026-02-03 Claude (AI Assistant) Initial design document

References

  • COPILOT_D6 Source: C:\DEV\COPILOT_D6\App\
  • GeViScope SDK Docs: C:\DEV\COPILOT\GeViScope_SDK_Docs\
  • G-Core SDK Docs: C:\DEV\COPILOT\G-Core_SDK_Docs\
  • Existing Bridges: C:\DEV\COPILOT\geviscope-bridge\, C:\DEV\COPILOT\gcore-bridge\
  • API Implementation: C:\DEV\COPILOT\geutebruck-api\