COPILOT/Docs/legacy-architecture/ptz-control.md

title, description

title	description
PTZ Control Flow	Complete joystick → Pan/Tilt/Zoom → SDK command pipeline with speed analysis

PTZ Control Flow

End-to-End Pipeline

sequenceDiagram
    participant HW as Joystick (HID)
    participant Dev as CopilotDevice
    participant MW as MainWindow
    participant VM as SegmentViewModel
    participant CCS as CameraControllerService
    participant Drv as GeViScope/GCore Driver
    participant SDK as Native SDK (PLC)
    participant Srv as Camera Server

    HW->>Dev: HID report (raw axis data)
    Dev->>Dev: Scale to -255..+255
    Dev->>MW: JoystickMoved(x, y, z)
    MW->>MW: Dispatcher.Invoke (UI thread)
    MW->>VM: IJoystickHandler.OnJoystickMoved(x, y, z)

    alt IsCameraTelemetryActive (PTZ locked)
        VM->>VM: DoPtzAction(x, Pan, controller.Pan)
        VM->>VM: DoPtzAction(y, Tilt, controller.Tilt)
        VM->>VM: DoPtzAction(z, Zoom, controller.Zoom)

        Note over VM: DoPtzAction only sends if speed changed<br/>(deduplication via ptzSpeeds dictionary)

        VM->>CCS: GetMovementController(cameraNumber)
        CCS->>Drv: GetMovementControllerForChannel(id)
        Drv-->>VM: IMovementController

        alt x changed (Pan)
            alt x > 0
                VM->>Drv: PanRight(x)
            else x < 0
                VM->>Drv: PanLeft(abs(x))
            else x == 0
                VM->>Drv: PanStop()
            end
        end

        Drv->>SDK: GscAct_PanRight(channelId, speed)
        SDK->>Srv: TCP PLC command

        VM->>VM: ResetCameraLockExpiration()
    else Not locked
        Note over VM: Joystick events ignored
    end

Speed Value Processing

Layer 1: HID → Raw Value

Joystick physical position → HID report → GetScaledValue(-255.0, 255.0) → integer

The joystick reports continuously while being held. Each axis change fires independently.

Layer 2: Deduplication (DoPtzAction)

private void DoPtzAction(int? value, PtzAction ptzActionFlag, Action<int> ptzAction)
{
    if (value.HasValue)
    {
        int speed = value.GetValueOrDefault();
        // Only send if speed actually changed
        if (!ptzSpeeds.TryGetValue(ptzActionFlag, out var prevSpeed) || speed != prevSpeed)
        {
            ptzSpeeds[ptzActionFlag] = speed;
            ptzAction(speed);  // Call Pan/Tilt/Zoom with raw value
        }
    }
}

Key insight: The ptzSpeeds dictionary prevents flooding the SDK with duplicate commands. Only changed values are forwarded.

Layer 3: Direction Resolution (IMovementControllerExtensions)

public static void Pan(this IMovementController controller, int speed)
{
    if (speed > 0)  controller.PanRight(speed);
    if (speed < 0)  controller.PanLeft(Math.Abs(speed));
    if (speed == 0) controller.PanStop();
}
// Same pattern for Tilt and Zoom

Layer 4: SDK Command

// GeViScope example
public void PanRight(int speed)
{
    using var channelId = new GscMediaChannelID(mediaChannelId);
    using var action = new GscAct_PanRight(channelId, speed);
    plcWrapper()?.SendAction(action);
}

The speed value (0-255) is passed directly to the native SDK with no transformation.

Speed Range Summary

Joystick Position    →  Speed Value  →  SDK Parameter
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Full Left/Up         →  -255         →  PanLeft(255) / TiltUp(255)
Half Left/Up         →  ~-128        →  PanLeft(128) / TiltUp(128)
Dead Center          →  0            →  PanStop() / TiltStop()
Half Right/Down      →  ~+128        →  PanRight(128) / TiltDown(128)
Full Right/Down      →  +255         →  PanRight(255) / TiltDown(255)

Zoom twist out       →  -255         →  ZoomOut(255)
Zoom center          →  0            →  ZoomStop()
Zoom twist in        →  +255         →  ZoomIn(255)

Zoom-Proportional Speed: NOT in App Code

Critical finding: The COPILOT WPF application does NOT implement zoom-proportional pan/tilt speed adjustment. The raw joystick value (-255 to +255) passes through all layers unmodified.

The "slower pan/tilt when zoomed in" behavior that operators observe is provided by one or more of:

1. Camera hardware — Most PTZ cameras (Axis, Bosch, Pelco) have built-in "proportional PTZ" that adjusts mechanical speed based on current zoom level
2. GeViScope/G-Core server — The video management software may apply speed scaling before sending commands to the physical camera
3. PTZ protocol — Some protocols (ONVIF, Pelco-D) support proportional speed as a protocol-level feature

Implication for Flutter rewrite: The new system will get this behavior "for free" as long as it sends the same speed values (0-255) to the same server/camera infrastructure.

Focus Control

Focus is controlled via the + and - hardware keys (not the joystick):

Key Down (+)  →  FocusNear(128)   (fixed speed)
Key Down (-)  →  FocusFar(128)    (fixed speed)
Key Up        →  FocusStop()

The DefaultFocusSpeed constant is 128 (half of maximum).

PTZ Lock Requirement

PTZ commands are only sent when IsCameraTelemetryActive == true, which requires:

1. The selected camera must be Movable (from MediaChannel metadata)
2. The operator must acquire a camera lock via the AppServer
3. The lock has a 5-minute timeout with a 1-minute warning
4. Each PTZ action resets the lock expiration timer

stateDiagram-v2
    [*] --> Unlocked
    Unlocked --> LockRequested: Press Lock/Button2
    LockRequested --> Locked: Lock acquired
    LockRequested --> ConfirmDialog: Lock held by other operator
    ConfirmDialog --> TakeOverRequested: Confirm takeover
    TakeOverRequested --> Locked: Takeover granted
    TakeOverRequested --> Unlocked: Takeover denied
    Locked --> ExpirationWarning: 4 minutes elapsed
    ExpirationWarning --> Locked: Any PTZ action (resets timer)
    ExpirationWarning --> Unlocked: 5 minutes elapsed (auto-unlock)
    Locked --> Unlocked: Press Lock/Button2 (manual unlock)
    Locked --> Unlocked: TakenOver by higher priority