geutebruck-api/specs/002-geviset-file-format/spec.md

GeViSet File Format Reverse Engineering Specification

Version: 1.0 Date: 2024-12-12 Status: In Progress

Overview

This specification documents the reverse engineering effort to fully parse, understand, and manipulate the GeViSoft .set configuration file format. The goal is to enable programmatic reading, editing, and writing of GeViServer configurations, particularly action mappings.

Background

What is a .set File?

• Source: Exported from GeViSet application or read via GeViAPI_SetupClient_ReadSetup
• Purpose: Complete GeViServer configuration (cameras, alarms, action mappings, users, etc.)
• Format: Proprietary binary format with "GeViSoft Parameters" header
• Size: Typically 200-300 KB for production configurations
• Use Case: Backup, migration, and programmatic configuration management

Current State

What Works:

• ✅ Read .set file from GeViServer via SetupClient API
• ✅ Extract 64 action mappings from binary data
• ✅ Write back byte-for-byte identical (round-trip verified)
• ✅ Password encryption with GeViAPI_EncodeString

What's Missing:

• ❌ Full structure parsing (all sections, all items)
• ❌ Understanding of all data types and relationships
• ❌ Ability to add NEW action mappings
• ❌ JSON representation of complete structure
• ❌ Comprehensive Excel export/import

Requirements

Primary Request

"I want to do full reverse engineering - I need to parse the whole file and maybe to json format in the first phase and then we will revert this json or its parts to excel"

Key Requirements

1. Parse Entire File Structure

   • All sections (Alarms, Clients, GeViIO, Cameras, ActionMappings, etc.)
   • All configuration items (key-value pairs)
   • All rules and triggers
   • All metadata and relationships

2. JSON Serialization

   • Complete structure in JSON format
   • Human-readable and editable
   • Preserves all data and relationships
   • Round-trip safe (JSON → Binary → JSON)

3. Excel Export/Import

   • Export action mappings to Excel
   • User-friendly editing interface
   • Add new mappings
   • Delete existing mappings
   • Import back to JSON

4. Safety & Validation

   • Verify integrity before writing to server
   • Backup original configuration
   • Validate against schema
   • Error handling and recovery

Architecture

Data Flow

┌─────────────────────────────────────────────────────────────┐
│                    GeViServer                                │
│                         ↓                                    │
│              SetupClient API (ReadSetup)                     │
└─────────────────────────────────────────────────────────────┘
                          ↓
                    .set file (binary)
                    281,714 bytes
                          ↓
┌─────────────────────────────────────────────────────────────┐
│                  PHASE 1: Binary Parser                      │
│  - Parse header                                              │
│  - Parse all sections                                        │
│  - Parse all items                                           │
│  - Parse all rules                                           │
│  - Extract action mappings                                   │
└─────────────────────────────────────────────────────────────┘
                          ↓
                    JSON Structure
            (full configuration representation)
                          ↓
┌─────────────────────────────────────────────────────────────┐
│              PHASE 2: JSON Processing                        │
│  - Validate structure                                        │
│  - Transform for editing                                     │
│  - Extract sections                                          │
└─────────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────┐
│              PHASE 3: Excel Export                           │
│  - Convert action mappings to Excel                          │
│  - User edits in Excel                                       │
│  - Add/delete/modify mappings                                │
└─────────────────────────────────────────────────────────────┘
                          ↓
                    Excel file
                          ↓
┌─────────────────────────────────────────────────────────────┐
│              PHASE 4: Excel Import                           │
│  - Read Excel changes                                        │
│  - Validate data                                             │
│  - Update JSON structure                                     │
└─────────────────────────────────────────────────────────────┘
                          ↓
                    JSON Structure
                     (modified)
                          ↓
┌─────────────────────────────────────────────────────────────┐
│              PHASE 5: Binary Writer                          │
│  - Rebuild .set file from JSON                               │
│  - Maintain binary format                                    │
│  - Validate integrity                                        │
└─────────────────────────────────────────────────────────────┘
                          ↓
                    .set file (binary)
                          ↓
              SetupClient API (WriteSetup)
                          ↓
                    GeViServer

Binary Format Analysis

File Structure

.set file
├── Header
│   ├── 0x00 (optional null byte)
│   └── Pascal String: "GeViSoft Parameters" (0x07 <len> <data>)
│
├── Sections (multiple)
│   ├── Section Name (Pascal String)
│   ├── Items (key-value pairs)
│   │   ├── Key (Pascal String)
│   │   └── Value (typed)
│   │       ├── 0x01 = Boolean
│   │       ├── 0x04 = Integer (4 bytes)
│   │       └── 0x07 = String (Pascal)
│   │
│   └── Rules Subsection
│       ├── "Rules" marker (0x05 0x52 0x75 0x6C 0x65 0x73)
│       ├── Count/Metadata
│       └── Action Rules (multiple)
│           ├── Trigger Properties
│           │   └── .PropertyName = Boolean
│           ├── Main Action String
│           │   └── 0x07 0x01 0x40 <len_2bytes> <action_data>
│           └── Action Variations
│               ├── GscAction (GeViScope)
│               ├── GNGAction (G-Net-Guard)
│               └── GCoreAction (GCore)
│
└── Footer (metadata/checksums?)

Data Types Discovered

Marker	Type	Format	Example
0x01	Boolean	0x01	0x01 0x01 = true
0x04	Integer	0x04 <4-byte little-endian>	0x04 0x0A 0x00 0x00 0x00
0x07	String	0x07	0x07 0x0B "Description"
0x07 0x01 0x40	Action	0x07 0x01 0x40 <len_2bytes>	Action string format

Action String Format

Pattern: 07 01 40 <len_2bytes_LE> <action_text>

Example:

07 01 40 1C 00 47 53 43 20 56 69 65 77 65 72 43 6F 6E 6E 65 63 74 4C 69 76 65...
│  │  │  │  │  └─ "GSC ViewerConnectLive V <- C"
│  │  │  └──┴─ Length: 0x001C (28 bytes)
│  │  └─ 0x40 (action marker)
│  └─ 0x01 (subtype)
└─ 0x07 (string type)

Sections Found

From file analysis, sections include:

• Alarms: Alarm configurations
• Clients: Client connections
• GeViIO: Digital I/O configurations
• Cameras: Camera settings
• Description: Various descriptive entries
• IpHost: Network configurations
• ActionMappings: Trigger → Action rules (our focus)

JSON Schema

Complete Structure

{
  "$schema": "https://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "version": {
      "type": "string",
      "description": "Parser version"
    },
    "header": {
      "type": "string",
      "description": "File header (GeViSoft Parameters)"
    },
    "sections": {
      "type": "array",
      "items": {
        "$ref": "#/definitions/Section"
      }
    }
  },
  "definitions": {
    "Section": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string"
        },
        "items": {
          "type": "array",
          "items": {
            "$ref": "#/definitions/ConfigItem"
          }
        },
        "rules": {
          "type": "array",
          "items": {
            "$ref": "#/definitions/ActionRule"
          }
        }
      }
    },
    "ConfigItem": {
      "type": "object",
      "properties": {
        "key": {
          "type": "string"
        },
        "value": {
          "oneOf": [
            { "type": "boolean" },
            { "type": "integer" },
            { "type": "string" }
          ]
        },
        "type": {
          "enum": ["boolean", "integer", "string"]
        }
      }
    },
    "ActionRule": {
      "type": "object",
      "properties": {
        "id": {
          "type": "integer"
        },
        "triggers": {
          "type": "object",
          "additionalProperties": {
            "type": "boolean"
          }
        },
        "mainAction": {
          "type": "string"
        },
        "variations": {
          "type": "array",
          "items": {
            "$ref": "#/definitions/ActionVariation"
          }
        }
      }
    },
    "ActionVariation": {
      "type": "object",
      "properties": {
        "platform": {
          "enum": ["GSC", "GNG", "GCore"]
        },
        "actionString": {
          "type": "string"
        },
        "serverType": {
          "type": "string"
        },
        "serverName": {
          "type": "string"
        }
      }
    }
  }
}

Example JSON Output

{
  "version": "1.0",
  "header": "GeViSoft Parameters",
  "sections": [
    {
      "name": "ActionMappings",
      "items": [],
      "rules": [
        {
          "id": 1,
          "triggers": {
            "InputContact": true,
            "VideoInput": false
          },
          "mainAction": "AlternateContact(2, 1000, 500)",
          "variations": [
            {
              "platform": "GSC",
              "actionString": "GSC ViewerConnectLive V <- C_101027",
              "serverType": "GeViScope",
              "serverName": "GEVISCOPE"
            },
            {
              "platform": "GNG",
              "actionString": "GNG PanLeft_101027",
              "serverType": "",
              "serverName": ""
            }
          ]
        }
      ]
    },
    {
      "name": "Alarms",
      "items": [
        {
          "key": "AlarmCount",
          "value": 5,
          "type": "integer"
        },
        {
          "key": "Enabled",
          "value": true,
          "type": "boolean"
        }
      ],
      "rules": []
    }
  ]
}

Implementation Phases

Phase 1: Complete Binary Parser ✅

Goal: Parse entire .set file structure into memory

Components:

• ✅ Header parser
• 🚧 Section parser (all types)
• 🚧 Item parser (all data types)
• 🚧 Rules parser (complete structure)
• 🚧 Action variation parser

Status: Basic parser exists, needs enhancement for full structure

Phase 2: JSON Serialization 🚧

Goal: Convert parsed structure to JSON

Components:

• JSON serializer
• Schema validator
• Round-trip tester (Binary → JSON → Binary)

Deliverables:

• SetFileToJson converter
• JSON schema definition
• Validation tools

Phase 3: Excel Export 🚧

Goal: Export action mappings to Excel for editing

Components:

• Excel writer (EPPlus library)
• Action mapping table generator
• Template with formulas/validation

Excel Structure:

Sheet: ActionMappings
| Rule ID | Trigger Type | Trigger Param | Action 1 | Action 2 | Action 3 |
|---------|--------------|---------------|----------|----------|----------|
| 1       | InputContact | 3, false      | Alternate| Viewer   |          |
| 2       | VideoInput   | 4, true       | CrossSwi | VCChange |          |

Phase 4: Excel Import 🚧

Goal: Import edited Excel back to JSON

Components:

• Excel reader
• Validation engine
• Diff generator (show changes)
• JSON merger

Phase 5: Binary Writer 🚧

Goal: Rebuild .set file from JSON

Components:

• Binary writer
• Structure rebuilder
• Validation
• Backup mechanism

Critical: Must maintain binary compatibility!

Phase 6: Testing & Validation 🚧

Goal: Ensure safety and correctness

Test Cases:

1. Round-trip (Binary → JSON → Binary) = identical
2. Round-trip (Binary → JSON → Excel → JSON → Binary) = valid
3. Add new mapping → write → server accepts
4. Modify existing mapping → write → server accepts
5. Delete mapping → write → server accepts

Current Progress

Completed ✅

• SetupClient API integration
• Password encryption
• Basic binary parsing (64 action mappings extracted)
• Safe round-trip (byte-for-byte identical)
• File structure analysis
• Data type discovery

In Progress 🚧

• Complete section parsing
• Full rule structure parsing
• JSON serialization
• Excel export
• Binary writer for modifications

Pending 📋

• Excel import
• Add new mapping functionality
• API endpoints
• Documentation
• Production deployment

Technical Challenges

Challenge 1: Unknown Metadata Bytes

Problem: Many byte sequences whose purpose is unknown

Solution:

• Document all patterns found
• Test modifications to understand behavior
• Preserve unknown bytes during round-trip

Challenge 2: Complex Nested Structure

Problem: Sections contain items and rules, rules contain variations

Solution:

• Recursive parsing
• Clear data model hierarchy
• Offset tracking for debugging

Challenge 3: Binary Format Changes

Problem: Format may vary between GeViSoft versions

Solution:

• Version detection
• Support multiple format versions
• Graceful degradation

Challenge 4: Action String Syntax

Problem: Action strings have complex syntax (parameters, types, etc.)

Solution:

• Pattern matching
• Action string parser
• Validation against known action types

Safety Considerations

Before Writing to Server

1. ✅ Verify round-trip: Parse → Write → Compare = Identical
2. ✅ Backup original: Always keep copy of working config
3. ⚠️ Test in dev: Never test on production first
4. ⚠️ Validate structure: Check against schema
5. ⚠️ Incremental changes: Small changes, test frequently

Error Handling

• Validate before write
• Provide detailed error messages
• Support rollback
• Log all operations

Tools & Libraries

Development

• Language: C# / .NET 8.0
• Binary Parsing: Custom binary reader
• JSON: System.Text.Json
• Excel: EPPlus (for .xlsx)
• Testing: xUnit
• Logging: Serilog

Project Structure

GeViSetEditor/
├── GeViSetEditor.Core/
│   ├── Models/
│   │   ├── SetFileStructure.cs
│   │   ├── Section.cs
│   │   ├── ConfigItem.cs
│   │   ├── ActionRule.cs
│   │   └── ActionVariation.cs
│   ├── Parsers/
│   │   ├── SetFileBinaryParser.cs
│   │   ├── SectionParser.cs
│   │   └── RuleParser.cs
│   ├── Writers/
│   │   ├── SetFileBinaryWriter.cs
│   │   └── JsonWriter.cs
│   ├── Converters/
│   │   ├── JsonToExcel.cs
│   │   └── ExcelToJson.cs
│   └── Validators/
│       └── StructureValidator.cs
├── GeViSetEditor.CLI/
│   └── Commands/
│       ├── ParseCommand.cs
│       ├── ToJsonCommand.cs
│       ├── ToExcelCommand.cs
│       └── FromExcelCommand.cs
└── GeViSetEditor.Tests/
    ├── ParserTests.cs
    ├── RoundTripTests.cs
    └── ValidationTests.cs

Next Steps

Immediate (This Session)

1. ✅ Create specification document
2. ✅ Update git repository
3. 🚧 Implement complete binary parser
4. 🚧 Implement JSON serialization
5. 🚧 Test round-trip with JSON

Short Term (Next Session)

1. Excel export implementation
2. Excel import implementation
3. Add new mapping functionality
4. Comprehensive testing

Long Term

1. Web UI for configuration management
2. API endpoints
3. Multi-version support
4. Documentation and examples

References

• GeViSoft SDK Documentation
• SetupClient API Reference
• Existing .set file samples (TestMKS.set, setup_config_*.dat)
• Binary analysis notes
• Round-trip test results

Version History

Version	Date	Changes
1.0	2024-12-12	Initial specification

---

Status: Ready for full implementation Priority: High Complexity: High Timeline: 2-3 days estimated