---
title: "Hardware & Input System"
description: "COPILOT keyboard hardware, joystick, serial/HID protocols, and key mapping"
---

# Hardware & Input System

## Physical Hardware

The COPILOT keyboard is a custom-built control panel with three input subsystems:

```
┌──────────────────────────────────────────────────────────────────┐
│                    COPILOT KEYBOARD                               │
│                                                                   │
│  ┌────────────────────────┐  ┌───────────────┐  ┌────────────┐  │
│  │     BUTTON PANEL        │  │   JOYSTICK    │  │  JOG/      │  │
│  │                         │  │               │  │  SHUTTLE   │  │
│  │  [Home] [F1-F7]        │  │    ╱───╲      │  │            │  │
│  │  [0-9] [.] [*]         │  │   │  ●  │     │  │   ◄──●──►  │  │
│  │  [+] [-] [(] [)]       │  │    ╲───╱      │  │            │  │
│  │  [< >]                  │  │   3-axis     │  │  7 speeds  │  │
│  │                         │  │  X/Y/Z       │  │  per dir   │  │
│  │  Serial (COM port)      │  │  USB HID     │  │  Serial    │  │
│  └────────────────────────┘  └───────────────┘  └────────────┘  │
│                                                                   │
│                    Arduino Leonardo (ATmega32U4)                   │
│                    USB Composite: Serial + HID                    │
└───────────────────────────────┬──────────────────────────────────┘
                                │ USB
                          LattePanda SBC
```

## Communication Protocol

### Serial Port (Buttons + Jog/Shuttle)

Messages are newline-terminated strings with a single-character prefix:

| Prefix | Type | Example | Meaning |
|--------|------|---------|---------|
| `p` | Key Pressed | `pH\r\n` | Home key pressed |
| `r` | Key Released | `rH\r\n` | Home key released |
| `h` | Heartbeat | `h\r\n` | Keepalive (ignored) |
| `j` | Jog | `j+1\r\n` / `j-1\r\n` | Jog wheel step right/left |
| `s` | Shuttle | `s3\r\n` / `s-5\r\n` | Shuttle position (-7 to +7) |
| `v` | Version | `v2.1\r\n` | Firmware version response |

Startup: App sends byte `0xDC` to request firmware version.

### HID Device (Joystick)

| Property | Value |
|----------|-------|
| Vendor ID | `10959` (0x2ACF) |
| Product ID | `257` (0x0101) |
| Axes | X (pan), Y (tilt), Z (zoom) |
| Raw range | Scaled to **-255 to +255** |
| Buttons | Button1, Button2, Button3 |

```mermaid
graph TD
    subgraph "HID Input Processing"
        Raw["Raw HID Report"] --> Parser["DeviceItemInputParser"]
        Parser --> Changed{"HasChanged?"}
        Changed -->|Yes| Usage{"Usage Type"}
        Usage -->|GenericDesktopX| PanEvt["JoystickX event<br/>(-255 to +255)"]
        Usage -->|GenericDesktopY| TiltEvt["JoystickY event<br/>(-255 to +255)"]
        Usage -->|GenericDesktopZ| ZoomEvt["JoystickZ event<br/>(-255 to +255)"]
        Usage -->|Button1| B1["JoystickButton1<br/>Pressed/Released"]
        Usage -->|Button2| B2["JoystickButton2<br/>Pressed/Released"]
        Usage -->|Button3| Ignore["Ignored"]
    end
```

## Key Mapping

### Hardware Keys → Virtual Keys

The serial protocol uses ASCII character codes:

| Serial Code | Char | Virtual Key | UI Function |
|-------------|------|-------------|-------------|
| 72 | `H` | Home | Home position / Jump to live |
| 65 | `A` | F1 | Function button 1 |
| 66 | `B` | F2 | Function button 2 |
| 67 | `C` | F3 | Function button 3 |
| 68 | `D` | F4 | Function button 4 |
| 69 | `E` | F5 | Function button 5 |
| 70 | `F` | F6 | Function button 6 |
| 71 | `G` | F7 | Function button 7 |
| 46 | `.` | Prefix | Camera number prefix cycle |
| 42 | `*` | FullScreen | Toggle monitor maximize |
| 41 | `)` | Sequence | Open sequence menu |
| 40 | `(` | Backspace | Delete last digit |
| 48 | `0` | D0 | Digit 0 |
| 55-51 | `7-3` | D1-D5 | Digits 1-5 (note: remapped!) |
| 49-51 | `1-3` | D7-D9 | Digits 7-9 (note: remapped!) |
| 52-54 | `4-6` | D4-D6 | Digits 4-6 |
| 60 | `<` | Minus | Previous camera / Focus far |
| 62 | `>` | Plus | Next camera / Focus near |
| 43 | `+` | Lock | Toggle camera PTZ lock |
| 45 | `-` | Enter | Confirm camera number |

**Important:** The digit key wiring is non-standard — D1 maps to ASCII 55 (`7`), not 49 (`1`). This is a hardware layout choice.

### Joystick Buttons
| Button | Virtual Key | Function |
|--------|-------------|----------|
| Button1 | JoystickButton1 | (Context-dependent) |
| Button2 | JoystickButton2 | Toggle camera PTZ lock |

### Development Keyboard Emulation

When no serial port is detected, the app falls back to standard keyboard input via `VirtualKeyboard.Emulate()`:

| PC Key | Virtual Key |
|--------|-------------|
| Home | Home |
| F1-F7 | F1-F7 |
| F10 | Prefix |
| F11 | FullScreen |
| F12 | Sequence |
| 0-9 / Numpad | D0-D9 |
| Backspace | Backspace |
| Enter | Enter |
| +/- | Plus/Minus |
| Left/Right arrow | JogLeft/JogRight |
| Shift+Left/Right | Shuttle (incremental) |
| Shift+Up | Shuttle center (reset) |

## Shuttle Speed Mapping

The shuttle wheel has 7 positions per direction. Speeds are configurable per deployment:

```
Position:  -7  -6  -5  -4  -3  -2  -1   0  +1  +2  +3  +4  +5  +6  +7
           ◄── backward ──────────── center ──────────── forward ──►
Config:   [250,100, 30, 15,  5,  2,  1]  ●  [ 1,  2,  5, 15, 30,100,250]
```

Default playback speeds (from config): `[1, 2, 5, 15, 30, 100, 250]`

## Service Key (Hidden Menu)

Holding the **Backspace** key for 3 seconds opens the Service Menu page. This provides access to firmware update and diagnostic functions.