COPILOT/Docs/plans/PHASE_1_FLUTTER_KEYBOARD.md

Phase 1: Flutter Keyboard Core Implementation Plan

Status: In Progress Duration: Week 3-5 Goal: Build the core Flutter keyboard app with direct command execution and state tracking

---

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                    Flutter Keyboard App                          │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐             │
│  │   UI Layer  │  │  BLoC Layer │  │ Data Layer  │             │
│  │  (Screens)  │◄─┤  (State)    │◄─┤ (Services)  │             │
│  └─────────────┘  └─────────────┘  └──────┬──────┘             │
│                                           │                     │
│  ┌────────────────────────────────────────┼─────────────────┐  │
│  │              Service Layer             │                 │  │
│  │  ┌─────────────┐  ┌─────────────┐  ┌───┴───────┐        │  │
│  │  │BridgeService│  │ StateService│  │AlarmService│        │  │
│  │  │ (HTTP+WS)   │  │ (Monitor+   │  │(Query+Track)│       │  │
│  │  └──────┬──────┘  │  Alarm)     │  └─────┬─────┘        │  │
│  └─────────┼─────────┴─────────────┴────────┼──────────────┘  │
└────────────┼────────────────────────────────┼──────────────────┘
             │                                │
             ▼                                ▼
    ┌────────────────┐               ┌────────────────┐
    │ GeViScope/GCore│               │  GeViServer    │
    │   Bridges      │               │    Bridge      │
    │ (7720/7721)    │               │    (7710)      │
    └────────────────┘               └────────────────┘

---

Phase 1 Tasks

Task 1.1: Project Setup ✅ COMPLETED

Priority: HIGH

Create new Flutter project with proper structure.

Sub-tasks:

• Create Flutter project: copilot_keyboard
• Set up directory structure (clean architecture)
• Add dependencies (BLoC, dio, web_socket_channel, etc.)
• Configure for Windows desktop target
• Create base config loading from servers.json

Directory Structure:

copilot_keyboard/
├── lib/
│   ├── main.dart
│   ├── app.dart
│   ├── config/
│   │   ├── app_config.dart
│   │   └── server_config.dart
│   ├── core/
│   │   ├── constants/
│   │   ├── errors/
│   │   └── utils/
│   ├── data/
│   │   ├── models/
│   │   ├── repositories/
│   │   └── services/
│   ├── domain/
│   │   ├── entities/
│   │   ├── repositories/
│   │   └── usecases/
│   └── presentation/
│       ├── blocs/
│       ├── screens/
│       └── widgets/
├── assets/
│   └── config/
├── test/
└── pubspec.yaml

---

Task 1.2: Bridge Service ✅ COMPLETED

Priority: HIGH

Create service to communicate with all bridges.

Sub-tasks:

• Create BridgeService class
• Implement HTTP client for REST calls
• Implement WebSocket client for event streaming
• Add connection management (connect/disconnect/reconnect)
• Route commands to correct bridge based on camera/monitor ID

Key Methods:

class BridgeService {
  // Connection
  Future<void> connect(ServerConfig server);
  Future<void> disconnect(String serverId);

  // Commands (routed to correct bridge)
  Future<void> viewerConnectLive(int viewer, int channel);
  Future<void> viewerClear(int viewer);
  Future<void> ptzPan(int camera, String direction, int speed);
  Future<void> ptzTilt(int camera, String direction, int speed);
  Future<void> ptzZoom(int camera, String direction, int speed);
  Future<void> ptzStop(int camera);
  Future<void> ptzPreset(int camera, int preset);

  // Event stream
  Stream<BridgeEvent> get eventStream;
}

---

Task 1.3: State Service ✅ COMPLETED

Priority: HIGH

Track monitor and alarm state from events.

Sub-tasks:

• Create StateService class
• Subscribe to bridge WebSocket events
• Track monitor states (viewer → camera mapping)
• Track alarm states (active alarms)
• Provide state streams for UI

Key Methods:

class StateService {
  // Monitor state
  Stream<Map<int, MonitorState>> get monitorStates;
  MonitorState? getMonitorState(int viewerId);

  // Alarm state
  Stream<List<AlarmState>> get activeAlarms;
  bool isMonitorBlocked(int viewerId);

  // Sync
  Future<void> syncFromBridges();
}

---

Task 1.4: Alarm Service (GeViServer Query) ✅ COMPLETED

Priority: HIGH

Query initial alarm state from GeViServer on startup.

Sub-tasks:

• Create AlarmService class
• Implement GeViServer bridge connection
• Query active alarms on startup using GetFirstAlarm/GetNextAlarm pattern
• Merge with event-based alarm tracking
• Periodic sync (every 30 seconds)

Key Methods:

class AlarmService {
  Future<List<AlarmInfo>> queryAllAlarms();
  Future<void> startPeriodicSync(Duration interval);
  void stopPeriodicSync();
}

---

Task 1.5: Keyboard Layout UI ✅ COMPLETED

Priority: HIGH

Build the main keyboard interface.

Sub-tasks:

• Create main keyboard screen layout
• Camera selection grid (numbered buttons)
• Monitor selection grid (numbered buttons)
• PTZ control panel (joystick or directional buttons)
• Preset buttons
• Status display (current camera on selected monitor)
• Alarm indicator panel

UI Components:

┌─────────────────────────────────────────────────────────────┐
│  COPILOT Keyboard                          [Status: Online] │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────────────────┐  ┌─────────────────────────┐  │
│  │     CAMERAS             │  │      MONITORS           │  │
│  │  [1] [2] [3] [4] [5]   │  │  [1] [2] [3] [4]       │  │
│  │  [6] [7] [8] [9] [10]  │  │  [5] [6] [7] [8]       │  │
│  │  ...                    │  │  [9!][10][11][12]      │  │
│  └─────────────────────────┘  └─────────────────────────┘  │
│                                                             │
│  ┌─────────────────────────┐  ┌─────────────────────────┐  │
│  │     PTZ CONTROL         │  │    PRESETS              │  │
│  │       [▲]               │  │  [1] [2] [3] [4]       │  │
│  │    [◄][●][►]            │  │  [5] [6] [7] [8]       │  │
│  │       [▼]               │  │                         │  │
│  │    [Z-] [Z+]            │  │                         │  │
│  └─────────────────────────┘  └─────────────────────────┘  │
│                                                             │
│  ┌─────────────────────────────────────────────────────┐   │
│  │  ACTIVE ALARMS                                       │   │
│  │  [!] Camera 5 - Motion Detected (10:30:15)          │   │
│  │  [!] Camera 12 - Door Contact (10:28:42)            │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

---

Task 1.6: BLoC Implementation ✅ COMPLETED

Priority: HIGH

Implement state management with BLoC pattern.

BLoCs to Create:

• ConnectionBloc - Bridge connection state
• CameraBloc - Camera selection and routing
• MonitorBloc - Monitor state and selection
• PtzBloc - PTZ control state
• AlarmBloc - Alarm state and display

---

Task 1.7: Server Routing Logic ✅ COMPLETED

Priority: HIGH

Route commands to correct bridge based on camera/monitor ranges.

Sub-tasks:

• Load server config from servers.json
• Implement camera-to-server mapping
• Implement monitor-to-server mapping
• Handle cross-server scenarios (camera on server A → monitor on server B)

Routing Rules:

class ServerRouter {
  // Find which server owns a camera
  ServerConfig? getServerForCamera(int cameraId);

  // Find which server owns a monitor
  ServerConfig? getServerForMonitor(int monitorId);

  // Get bridge URL for a server
  String getBridgeUrl(String serverId);
}

---

Task 1.8: Error Handling ✅ COMPLETED

Priority: MEDIUM

Implement basic error handling and recovery.

Sub-tasks:

• Connection error handling with retry
• Command timeout handling
• Offline/degraded mode detection
• User-friendly error messages
• Logging for debugging

---

Dependencies (pubspec.yaml)

dependencies:
  flutter:
    sdk: flutter

  # State Management
  flutter_bloc: ^8.1.6
  equatable: ^2.0.5

  # Networking
  dio: ^5.7.0
  web_socket_channel: ^3.0.1

  # Local Storage
  shared_preferences: ^2.3.3

  # Routing
  go_router: ^14.6.2

  # Dependency Injection
  get_it: ^8.0.2

  # Utilities
  json_annotation: ^4.9.0
  rxdart: ^0.28.0

dev_dependencies:
  flutter_test:
    sdk: flutter
  build_runner: ^2.4.13
  json_serializable: ^6.8.0
  bloc_test: ^9.1.7
  mocktail: ^1.0.4

---

Success Criteria

• App connects to all configured bridges on startup
• Camera button switches camera to selected monitor
• PTZ controls move the selected camera
• Monitor states update in real-time from WebSocket events
• Alarm states query from GeViServer on startup
• Alarms update in real-time from events
• Monitors with active alarms show visual indicator
• App works in degraded mode if bridges unavailable

---

Files to Create

copilot_keyboard/
├── lib/
│   ├── main.dart
│   ├── app.dart
│   ├── injection_container.dart
│   ├── config/
│   │   ├── app_config.dart
│   │   └── server_config.dart
│   ├── core/
│   │   ├── constants/
│   │   │   └── api_constants.dart
│   │   ├── errors/
│   │   │   └── failures.dart
│   │   └── utils/
│   │       └── logger.dart
│   ├── data/
│   │   ├── models/
│   │   │   ├── monitor_state_model.dart
│   │   │   ├── alarm_state_model.dart
│   │   │   └── bridge_event_model.dart
│   │   └── services/
│   │       ├── bridge_service.dart
│   │       ├── state_service.dart
│   │       └── alarm_service.dart
│   ├── domain/
│   │   └── entities/
│   │       ├── monitor_state.dart
│   │       ├── alarm_state.dart
│   │       └── server_config.dart
│   └── presentation/
│       ├── blocs/
│       │   ├── connection/
│       │   ├── camera/
│       │   ├── monitor/
│       │   ├── ptz/
│       │   └── alarm/
│       ├── screens/
│       │   └── keyboard_screen.dart
│       └── widgets/
│           ├── camera_grid.dart
│           ├── monitor_grid.dart
│           ├── ptz_control.dart
│           ├── preset_buttons.dart
│           └── alarm_panel.dart
└── pubspec.yaml

---

Next Phase Dependencies

Phase 1 creates the foundation for:

• Phase 2: Coordination layer (PRIMARY election, PTZ locks)
• Phase 3: Advanced features (sequences, CrossSwitch rules)