COPILOT/Docs/legacy-architecture/hardware-input.md

title, description

title	description
Hardware & Input System	COPILOT keyboard hardware, joystick, serial/HID protocols, and key mapping

Hardware & Input System

Physical Hardware

The COPILOT keyboard is a custom-built control panel with three input subsystems:

┌──────────────────────────────────────────────────────────────────┐
│                    COPILOT KEYBOARD                               │
│                                                                   │
│  ┌────────────────────────┐  ┌───────────────┐  ┌────────────┐  │
│  │     BUTTON PANEL        │  │   JOYSTICK    │  │  JOG/      │  │
│  │                         │  │               │  │  SHUTTLE   │  │
│  │  [Home] [F1-F7]        │  │    ╱───╲      │  │            │  │
│  │  [0-9] [.] [*]         │  │   │  ●  │     │  │   ◄──●──►  │  │
│  │  [+] [-] [(] [)]       │  │    ╲───╱      │  │            │  │
│  │  [< >]                  │  │   3-axis     │  │  7 speeds  │  │
│  │                         │  │  X/Y/Z       │  │  per dir   │  │
│  │  Serial (COM port)      │  │  USB HID     │  │  Serial    │  │
│  └────────────────────────┘  └───────────────┘  └────────────┘  │
│                                                                   │
│                    Arduino Leonardo (ATmega32U4)                   │
│                    USB Composite: Serial + HID                    │
└───────────────────────────────┬──────────────────────────────────┘
                                │ USB
                          LattePanda SBC

Communication Protocol

Serial Port (Buttons + Jog/Shuttle)

Messages are newline-terminated strings with a single-character prefix:

Prefix	Type	Example	Meaning
p	Key Pressed	pH\r\n	Home key pressed
r	Key Released	rH\r\n	Home key released
h	Heartbeat	h\r\n	Keepalive (ignored)
j	Jog	j+1\r\n / j-1\r\n	Jog wheel step right/left
s	Shuttle	s3\r\n / s-5\r\n	Shuttle position (-7 to +7)
v	Version	v2.1\r\n	Firmware version response

Startup: App sends byte 0xDC to request firmware version.

HID Device (Joystick)

Property	Value
Vendor ID	10959 (0x2ACF)
Product ID	257 (0x0101)
Axes	X (pan), Y (tilt), Z (zoom)
Raw range	Scaled to -255 to +255
Buttons	Button1, Button2, Button3

graph TD
    subgraph "HID Input Processing"
        Raw["Raw HID Report"] --> Parser["DeviceItemInputParser"]
        Parser --> Changed{"HasChanged?"}
        Changed -->|Yes| Usage{"Usage Type"}
        Usage -->|GenericDesktopX| PanEvt["JoystickX event<br/>(-255 to +255)"]
        Usage -->|GenericDesktopY| TiltEvt["JoystickY event<br/>(-255 to +255)"]
        Usage -->|GenericDesktopZ| ZoomEvt["JoystickZ event<br/>(-255 to +255)"]
        Usage -->|Button1| B1["JoystickButton1<br/>Pressed/Released"]
        Usage -->|Button2| B2["JoystickButton2<br/>Pressed/Released"]
        Usage -->|Button3| Ignore["Ignored"]
    end

Key Mapping

Hardware Keys → Virtual Keys

The serial protocol uses ASCII character codes:

Serial Code	Char	Virtual Key	UI Function
72	H	Home	Home position / Jump to live
65	A	F1	Function button 1
66	B	F2	Function button 2
67	C	F3	Function button 3
68	D	F4	Function button 4
69	E	F5	Function button 5
70	F	F6	Function button 6
71	G	F7	Function button 7
46	.	Prefix	Camera number prefix cycle
42	*	FullScreen	Toggle monitor maximize
41	)	Sequence	Open sequence menu
40	(	Backspace	Delete last digit
48	0	D0	Digit 0
55-51	7-3	D1-D5	Digits 1-5 (note: remapped!)
49-51	1-3	D7-D9	Digits 7-9 (note: remapped!)
52-54	4-6	D4-D6	Digits 4-6
60	<	Minus	Previous camera / Focus far
62	>	Plus	Next camera / Focus near
43	+	Lock	Toggle camera PTZ lock
45	-	Enter	Confirm camera number

Important: The digit key wiring is non-standard — D1 maps to ASCII 55 (7), not 49 (1). This is a hardware layout choice.

Joystick Buttons

Button	Virtual Key	Function
Button1	JoystickButton1	(Context-dependent)
Button2	JoystickButton2	Toggle camera PTZ lock

Development Keyboard Emulation

When no serial port is detected, the app falls back to standard keyboard input via VirtualKeyboard.Emulate():

PC Key	Virtual Key
Home	Home
F1-F7	F1-F7
F10	Prefix
F11	FullScreen
F12	Sequence
0-9 / Numpad	D0-D9
Backspace	Backspace
Enter	Enter
+/-	Plus/Minus
Left/Right arrow	JogLeft/JogRight
Shift+Left/Right	Shuttle (incremental)
Shift+Up	Shuttle center (reset)

Shuttle Speed Mapping

The shuttle wheel has 7 positions per direction. Speeds are configurable per deployment:

Position:  -7  -6  -5  -4  -3  -2  -1   0  +1  +2  +3  +4  +5  +6  +7
           ◄── backward ──────────── center ──────────── forward ──►
Config:   [250,100, 30, 15,  5,  2,  1]  ●  [ 1,  2,  5, 15, 30,100,250]

Default playback speeds (from config): [1, 2, 5, 15, 30, 100, 250]

Service Key (Hidden Menu)

Holding the Backspace key for 3 seconds opens the Service Menu page. This provides access to firmware update and diagnostic functions.