# GeViSet File Format Reverse Engineering Specification **Version:** 1.0 **Date:** 2024-12-12 **Status:** In Progress ## Overview This specification documents the reverse engineering effort to fully parse, understand, and manipulate the GeViSoft `.set` configuration file format. The goal is to enable programmatic reading, editing, and writing of GeViServer configurations, particularly action mappings. ## Background ### What is a .set File? - **Source**: Exported from GeViSet application or read via `GeViAPI_SetupClient_ReadSetup` - **Purpose**: Complete GeViServer configuration (cameras, alarms, action mappings, users, etc.) - **Format**: Proprietary binary format with "GeViSoft Parameters" header - **Size**: Typically 200-300 KB for production configurations - **Use Case**: Backup, migration, and programmatic configuration management ### Current State **What Works:** - ✅ Read .set file from GeViServer via SetupClient API - ✅ Extract 64 action mappings from binary data - ✅ Write back byte-for-byte identical (round-trip verified) - ✅ Password encryption with `GeViAPI_EncodeString` **What's Missing:** - ❌ Full structure parsing (all sections, all items) - ❌ Understanding of all data types and relationships - ❌ Ability to add NEW action mappings - ❌ JSON representation of complete structure - ❌ Comprehensive Excel export/import ## Requirements ### Primary Request > "I want to do full reverse engineering - I need to parse the whole file and maybe to json format in the first phase and then we will revert this json or its parts to excel" ### Key Requirements 1. **Parse Entire File Structure** - All sections (Alarms, Clients, GeViIO, Cameras, ActionMappings, etc.) - All configuration items (key-value pairs) - All rules and triggers - All metadata and relationships 2. **JSON Serialization** - Complete structure in JSON format - Human-readable and editable - Preserves all data and relationships - Round-trip safe (JSON → Binary → JSON) 3. **Excel Export/Import** - Export action mappings to Excel - User-friendly editing interface - Add new mappings - Delete existing mappings - Import back to JSON 4. **Safety & Validation** - Verify integrity before writing to server - Backup original configuration - Validate against schema - Error handling and recovery ## Architecture ### Data Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ GeViServer │ │ ↓ │ │ SetupClient API (ReadSetup) │ └─────────────────────────────────────────────────────────────┘ ↓ .set file (binary) 281,714 bytes ↓ ┌─────────────────────────────────────────────────────────────┐ │ PHASE 1: Binary Parser │ │ - Parse header │ │ - Parse all sections │ │ - Parse all items │ │ - Parse all rules │ │ - Extract action mappings │ └─────────────────────────────────────────────────────────────┘ ↓ JSON Structure (full configuration representation) ↓ ┌─────────────────────────────────────────────────────────────┐ │ PHASE 2: JSON Processing │ │ - Validate structure │ │ - Transform for editing │ │ - Extract sections │ └─────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────┐ │ PHASE 3: Excel Export │ │ - Convert action mappings to Excel │ │ - User edits in Excel │ │ - Add/delete/modify mappings │ └─────────────────────────────────────────────────────────────┘ ↓ Excel file ↓ ┌─────────────────────────────────────────────────────────────┐ │ PHASE 4: Excel Import │ │ - Read Excel changes │ │ - Validate data │ │ - Update JSON structure │ └─────────────────────────────────────────────────────────────┘ ↓ JSON Structure (modified) ↓ ┌─────────────────────────────────────────────────────────────┐ │ PHASE 5: Binary Writer │ │ - Rebuild .set file from JSON │ │ - Maintain binary format │ │ - Validate integrity │ └─────────────────────────────────────────────────────────────┘ ↓ .set file (binary) ↓ SetupClient API (WriteSetup) ↓ GeViServer ``` ## Binary Format Analysis ### File Structure ``` .set file ├── Header │ ├── 0x00 (optional null byte) │ └── Pascal String: "GeViSoft Parameters" (0x07 ) │ ├── Sections (multiple) │ ├── Section Name (Pascal String) │ ├── Items (key-value pairs) │ │ ├── Key (Pascal String) │ │ └── Value (typed) │ │ ├── 0x01 = Boolean │ │ ├── 0x04 = Integer (4 bytes) │ │ └── 0x07 = String (Pascal) │ │ │ └── Rules Subsection │ ├── "Rules" marker (0x05 0x52 0x75 0x6C 0x65 0x73) │ ├── Count/Metadata │ └── Action Rules (multiple) │ ├── Trigger Properties │ │ └── .PropertyName = Boolean │ ├── Main Action String │ │ └── 0x07 0x01 0x40 │ └── Action Variations │ ├── GscAction (GeViScope) │ ├── GNGAction (G-Net-Guard) │ └── GCoreAction (GCore) │ └── Footer (metadata/checksums?) ``` ### Data Types Discovered | Marker | Type | Format | Example | |--------|---------|----------------------------------|----------------------------| | 0x01 | Boolean | 0x01 | 0x01 0x01 = true | | 0x04 | Integer | 0x04 <4-byte little-endian> | 0x04 0x0A 0x00 0x00 0x00 | | 0x07 | String | 0x07 | 0x07 0x0B "Description" | | 0x07 0x01 0x40 | Action | 0x07 0x01 0x40 | Action string format | ### Action String Format Pattern: `07 01 40 ` Example: ``` 07 01 40 1C 00 47 53 43 20 56 69 65 77 65 72 43 6F 6E 6E 65 63 74 4C 69 76 65... │ │ │ │ │ └─ "GSC ViewerConnectLive V <- C" │ │ │ └──┴─ Length: 0x001C (28 bytes) │ │ └─ 0x40 (action marker) │ └─ 0x01 (subtype) └─ 0x07 (string type) ``` ### Sections Found From file analysis, sections include: - **Alarms**: Alarm configurations - **Clients**: Client connections - **GeViIO**: Digital I/O configurations - **Cameras**: Camera settings - **Description**: Various descriptive entries - **IpHost**: Network configurations - **ActionMappings**: Trigger → Action rules (our focus) ## JSON Schema ### Complete Structure ```json { "$schema": "https://json-schema.org/draft-07/schema#", "type": "object", "properties": { "version": { "type": "string", "description": "Parser version" }, "header": { "type": "string", "description": "File header (GeViSoft Parameters)" }, "sections": { "type": "array", "items": { "$ref": "#/definitions/Section" } } }, "definitions": { "Section": { "type": "object", "properties": { "name": { "type": "string" }, "items": { "type": "array", "items": { "$ref": "#/definitions/ConfigItem" } }, "rules": { "type": "array", "items": { "$ref": "#/definitions/ActionRule" } } } }, "ConfigItem": { "type": "object", "properties": { "key": { "type": "string" }, "value": { "oneOf": [ { "type": "boolean" }, { "type": "integer" }, { "type": "string" } ] }, "type": { "enum": ["boolean", "integer", "string"] } } }, "ActionRule": { "type": "object", "properties": { "id": { "type": "integer" }, "triggers": { "type": "object", "additionalProperties": { "type": "boolean" } }, "mainAction": { "type": "string" }, "variations": { "type": "array", "items": { "$ref": "#/definitions/ActionVariation" } } } }, "ActionVariation": { "type": "object", "properties": { "platform": { "enum": ["GSC", "GNG", "GCore"] }, "actionString": { "type": "string" }, "serverType": { "type": "string" }, "serverName": { "type": "string" } } } } } ``` ### Example JSON Output ```json { "version": "1.0", "header": "GeViSoft Parameters", "sections": [ { "name": "ActionMappings", "items": [], "rules": [ { "id": 1, "triggers": { "InputContact": true, "VideoInput": false }, "mainAction": "AlternateContact(2, 1000, 500)", "variations": [ { "platform": "GSC", "actionString": "GSC ViewerConnectLive V <- C_101027", "serverType": "GeViScope", "serverName": "GEVISCOPE" }, { "platform": "GNG", "actionString": "GNG PanLeft_101027", "serverType": "", "serverName": "" } ] } ] }, { "name": "Alarms", "items": [ { "key": "AlarmCount", "value": 5, "type": "integer" }, { "key": "Enabled", "value": true, "type": "boolean" } ], "rules": [] } ] } ``` ## Implementation Phases ### Phase 1: Complete Binary Parser ✅ **Goal**: Parse entire .set file structure into memory **Components**: - ✅ Header parser - 🚧 Section parser (all types) - 🚧 Item parser (all data types) - 🚧 Rules parser (complete structure) - 🚧 Action variation parser **Status**: Basic parser exists, needs enhancement for full structure ### Phase 2: JSON Serialization 🚧 **Goal**: Convert parsed structure to JSON **Components**: - JSON serializer - Schema validator - Round-trip tester (Binary → JSON → Binary) **Deliverables**: - `SetFileToJson` converter - JSON schema definition - Validation tools ### Phase 3: Excel Export 🚧 **Goal**: Export action mappings to Excel for editing **Components**: - Excel writer (EPPlus library) - Action mapping table generator - Template with formulas/validation **Excel Structure**: ``` Sheet: ActionMappings | Rule ID | Trigger Type | Trigger Param | Action 1 | Action 2 | Action 3 | |---------|--------------|---------------|----------|----------|----------| | 1 | InputContact | 3, false | Alternate| Viewer | | | 2 | VideoInput | 4, true | CrossSwi | VCChange | | ``` ### Phase 4: Excel Import 🚧 **Goal**: Import edited Excel back to JSON **Components**: - Excel reader - Validation engine - Diff generator (show changes) - JSON merger ### Phase 5: Binary Writer 🚧 **Goal**: Rebuild .set file from JSON **Components**: - Binary writer - Structure rebuilder - Validation - Backup mechanism **Critical**: Must maintain binary compatibility! ### Phase 6: Testing & Validation 🚧 **Goal**: Ensure safety and correctness **Test Cases**: 1. Round-trip (Binary → JSON → Binary) = identical 2. Round-trip (Binary → JSON → Excel → JSON → Binary) = valid 3. Add new mapping → write → server accepts 4. Modify existing mapping → write → server accepts 5. Delete mapping → write → server accepts ## Current Progress ### Completed ✅ - [x] SetupClient API integration - [x] Password encryption - [x] Basic binary parsing (64 action mappings extracted) - [x] Safe round-trip (byte-for-byte identical) - [x] File structure analysis - [x] Data type discovery ### In Progress 🚧 - [ ] Complete section parsing - [ ] Full rule structure parsing - [ ] JSON serialization - [ ] Excel export - [ ] Binary writer for modifications ### Pending 📋 - [ ] Excel import - [ ] Add new mapping functionality - [ ] API endpoints - [ ] Documentation - [ ] Production deployment ## Technical Challenges ### Challenge 1: Unknown Metadata Bytes **Problem**: Many byte sequences whose purpose is unknown **Solution**: - Document all patterns found - Test modifications to understand behavior - Preserve unknown bytes during round-trip ### Challenge 2: Complex Nested Structure **Problem**: Sections contain items and rules, rules contain variations **Solution**: - Recursive parsing - Clear data model hierarchy - Offset tracking for debugging ### Challenge 3: Binary Format Changes **Problem**: Format may vary between GeViSoft versions **Solution**: - Version detection - Support multiple format versions - Graceful degradation ### Challenge 4: Action String Syntax **Problem**: Action strings have complex syntax (parameters, types, etc.) **Solution**: - Pattern matching - Action string parser - Validation against known action types ## Safety Considerations ### Before Writing to Server 1. ✅ **Verify round-trip**: Parse → Write → Compare = Identical 2. ✅ **Backup original**: Always keep copy of working config 3. ⚠️ **Test in dev**: Never test on production first 4. ⚠️ **Validate structure**: Check against schema 5. ⚠️ **Incremental changes**: Small changes, test frequently ### Error Handling - Validate before write - Provide detailed error messages - Support rollback - Log all operations ## Tools & Libraries ### Development - **Language**: C# / .NET 8.0 - **Binary Parsing**: Custom binary reader - **JSON**: System.Text.Json - **Excel**: EPPlus (for .xlsx) - **Testing**: xUnit - **Logging**: Serilog ### Project Structure ``` GeViSetEditor/ ├── GeViSetEditor.Core/ │ ├── Models/ │ │ ├── SetFileStructure.cs │ │ ├── Section.cs │ │ ├── ConfigItem.cs │ │ ├── ActionRule.cs │ │ └── ActionVariation.cs │ ├── Parsers/ │ │ ├── SetFileBinaryParser.cs │ │ ├── SectionParser.cs │ │ └── RuleParser.cs │ ├── Writers/ │ │ ├── SetFileBinaryWriter.cs │ │ └── JsonWriter.cs │ ├── Converters/ │ │ ├── JsonToExcel.cs │ │ └── ExcelToJson.cs │ └── Validators/ │ └── StructureValidator.cs ├── GeViSetEditor.CLI/ │ └── Commands/ │ ├── ParseCommand.cs │ ├── ToJsonCommand.cs │ ├── ToExcelCommand.cs │ └── FromExcelCommand.cs └── GeViSetEditor.Tests/ ├── ParserTests.cs ├── RoundTripTests.cs └── ValidationTests.cs ``` ## Next Steps ### Immediate (This Session) 1. ✅ Create specification document 2. ✅ Update git repository 3. 🚧 Implement complete binary parser 4. 🚧 Implement JSON serialization 5. 🚧 Test round-trip with JSON ### Short Term (Next Session) 1. Excel export implementation 2. Excel import implementation 3. Add new mapping functionality 4. Comprehensive testing ### Long Term 1. Web UI for configuration management 2. API endpoints 3. Multi-version support 4. Documentation and examples ## References - GeViSoft SDK Documentation - SetupClient API Reference - Existing .set file samples (TestMKS.set, setup_config_*.dat) - Binary analysis notes - Round-trip test results ## Version History | Version | Date | Changes | |---------|------------|--------------------------------------| | 1.0 | 2024-12-12 | Initial specification | --- **Status**: Ready for full implementation **Priority**: High **Complexity**: High **Timeline**: 2-3 days estimated