feat: Geutebruck GeViScope/GeViSoft Action Mapping System - MVP
This MVP release provides a complete full-stack solution for managing action mappings in Geutebruck's GeViScope and GeViSoft video surveillance systems. ## Features ### Flutter Web Application (Port 8081) - Modern, responsive UI for managing action mappings - Action picker dialog with full parameter configuration - Support for both GSC (GeViScope) and G-Core server actions - Consistent UI for input and output actions with edit/delete capabilities - Real-time action mapping creation, editing, and deletion - Server categorization (GSC: prefix for GeViScope, G-Core: prefix for G-Core servers) ### FastAPI REST Backend (Port 8000) - RESTful API for action mapping CRUD operations - Action template service with comprehensive action catalog (247 actions) - Server management (G-Core and GeViScope servers) - Configuration tree reading and writing - JWT authentication with role-based access control - PostgreSQL database integration ### C# SDK Bridge (gRPC, Port 50051) - Native integration with GeViSoft SDK (GeViProcAPINET_4_0.dll) - Action mapping creation with correct binary format - Support for GSC and G-Core action types - Proper Camera parameter inclusion in action strings (fixes CrossSwitch bug) - Action ID lookup table with server-specific action IDs - Configuration reading/writing via SetupClient ## Bug Fixes - **CrossSwitch Bug**: GSC and G-Core actions now correctly display camera/PTZ head parameters in GeViSet - Action strings now include Camera parameter: `@ PanLeft (Comment: "", Camera: 101028)` - Proper filter flags and VideoInput=0 for action mappings - Correct action ID assignment (4198 for GSC, 9294 for G-Core PanLeft) ## Technical Stack - **Frontend**: Flutter Web, Dart, Dio HTTP client - **Backend**: Python FastAPI, PostgreSQL, Redis - **SDK Bridge**: C# .NET 8.0, gRPC, GeViSoft SDK - **Authentication**: JWT tokens - **Configuration**: GeViSoft .set files (binary format) ## Credentials - GeViSoft/GeViScope: username=sysadmin, password=masterkey - Default admin: username=admin, password=admin123 ## Deployment All services run on localhost: - Flutter Web: http://localhost:8081 - FastAPI: http://localhost:8000 - SDK Bridge gRPC: localhost:50051 - GeViServer: localhost (default port) Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Administrator
@@ -0,0 +1,291 @@
|
||||
# Database vs .SET File Structure Comparison
|
||||
|
||||
**Date:** 2025-12-12
|
||||
**Purpose:** Document differences between GeViSoft Database and .SET configuration file
|
||||
|
||||
## Executive Summary
|
||||
|
||||
GeViSoft uses **TWO distinct storage systems**:
|
||||
|
||||
1. **GeViDB.mdb** (Access Database) - Stores runtime event logs
|
||||
2. **.set File** (Binary Format) - Stores configuration and action mapping rules
|
||||
|
||||
They serve different purposes and have different structures.
|
||||
|
||||
---
|
||||
|
||||
## GeViSoft Database (GeViDB.mdb)
|
||||
|
||||
### Access Method
|
||||
|
||||
**Option 1: .NET SDK Wrapper** (Recommended)
|
||||
```csharp
|
||||
var db = new GeViDatabase();
|
||||
db.Create("localhost", "sysadmin", "plainTextPassword"); // No encoding needed!
|
||||
db.RegisterCallback();
|
||||
var result = db.Connect();
|
||||
|
||||
// Query actions
|
||||
db.SendQuery(new GeViDBQ_CreateActionQuery(0), out GeViMessage answer);
|
||||
```
|
||||
|
||||
**Password Encoding:**
|
||||
✅ **NOT REQUIRED** - The .NET wrapper (`GeViProcAPINET_4_0.dll`) handles password encoding internally.
|
||||
The `Create()` method signature explicitly says `plainTextPassword`.
|
||||
|
||||
**Option 2: Direct OleDB Access**
|
||||
```csharp
|
||||
string connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\GEVISOFT\DATABASE\GeViDB.mdb;Mode=Read;";
|
||||
// Direct SQL queries to Alarms, Actions tables
|
||||
```
|
||||
|
||||
### Database Structure
|
||||
|
||||
**Tables:**
|
||||
- `Alarms` - Alarm definitions
|
||||
- `Actions` - Runtime action event log
|
||||
- (Other configuration tables)
|
||||
|
||||
**Actions Table Content:**
|
||||
| PK | Action Data | Purpose |
|
||||
|----|-------------|---------|
|
||||
| 1 | `DataBaseReady()` | System initialization event |
|
||||
| 4 | `SystemError(0,1000,203,"GEVISERVER: dongle not found")` | System error log |
|
||||
| 5 | `SystemWarning(0,1000,206,"DEMO MODE GRANTED...")` | System warning |
|
||||
| 15 | `UserLogin(1,"sysadmin","","127.0.0.1")` | User authentication log |
|
||||
|
||||
**Key Characteristics:**
|
||||
- Stores **runtime events** and **logs**
|
||||
- **NOT** action mapping configuration
|
||||
- Grows over time as events occur
|
||||
- Query-able with SQL or SDK queries
|
||||
|
||||
---
|
||||
|
||||
## .SET File (Binary Configuration)
|
||||
|
||||
### Access Method
|
||||
|
||||
**SetupClient API:**
|
||||
```csharp
|
||||
// PASSWORD ENCODING REQUIRED for native API!
|
||||
var encodedPassword = new StringBuilder(256);
|
||||
GeViAPI_EncodeString(encodedPassword, password, 256);
|
||||
|
||||
GeViAPI_SetupClient_Create(out handle, alias, address, username,
|
||||
encodedPassword.ToString(), "", "");
|
||||
GeViAPI_SetupClient_Connect(handle, out result, 10000);
|
||||
GeViAPI_SetupClient_ReadSetup(handle, out buffer, out size, 60000);
|
||||
```
|
||||
|
||||
**Password Encoding:**
|
||||
❌ **REQUIRED** for `SetupClient` native API!
|
||||
Must call `GeViAPI_EncodeString()` before passing password.
|
||||
|
||||
### Binary Format Structure
|
||||
|
||||
```
|
||||
.set File (281,714 bytes)
|
||||
├── Header: "GeViSoft Parameters" (length-prefixed)
|
||||
├── Sections (multiple)
|
||||
│ ├── Alarms
|
||||
│ ├── Cameras
|
||||
│ ├── GeViIO
|
||||
│ ├── Clients
|
||||
│ └── Action Mapping Rules
|
||||
│ ├── "Rules" marker (05 52 75 6C 65 73)
|
||||
│ ├── Trigger Conditions
|
||||
│ └── Action Strings
|
||||
│ ├── Format: 07 01 40 <len_2bytes_LE> <data>
|
||||
│ └── Examples:
|
||||
│ • "GSC ViewerConnectLive V <- C"
|
||||
│ • "GNG PanLeft_101027"
|
||||
│ • "GSC warning: demo mode for 100 min"
|
||||
```
|
||||
|
||||
**Action Mappings Extracted:** 64 rules with 107 action strings
|
||||
|
||||
**Examples:**
|
||||
```json
|
||||
{
|
||||
"id": 1,
|
||||
"fileOffset": 253894,
|
||||
"actions": [
|
||||
"GSC ViewerConnectLive V <- C",
|
||||
"GNG ViewerConnectLive V <- C_101027"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Key Characteristics:**
|
||||
- Stores **configuration** and **rules**
|
||||
- Defines trigger → action mappings
|
||||
- Static configuration (doesn't grow)
|
||||
- Can be exported/imported with GeViSet application
|
||||
- Binary format requires parsing
|
||||
|
||||
---
|
||||
|
||||
## Comparison Table
|
||||
|
||||
| Aspect | Database (GeViDB.mdb) | .SET File |
|
||||
|--------|----------------------|-----------|
|
||||
| **Purpose** | Runtime event logging | Configuration storage |
|
||||
| **Content** | System events, user logins, errors | Action mapping rules, system config |
|
||||
| **Password Encoding** | ❌ NOT needed (.NET wrapper) | ✅ REQUIRED (native SetupClient API) |
|
||||
| **Access Method** | GeViDatabase + SendQuery | SetupClient_ReadSetup |
|
||||
| **Data Type** | Event logs (dynamic) | Configuration rules (static) |
|
||||
| **Size** | Grows over time | Fixed per configuration |
|
||||
| **Format** | MS Access Database | Proprietary binary |
|
||||
| **Queryable** | SQL / SDK queries | Binary parsing only |
|
||||
| **Example Content** | `UserLogin(1,"sysadmin"...)` | `"GSC ViewerConnectLive V <- C"` |
|
||||
|
||||
---
|
||||
|
||||
## Password Encoding Summary
|
||||
|
||||
### When Encoding IS Required
|
||||
|
||||
**SetupClient Native API:**
|
||||
```cpp
|
||||
// C/C++ or P/Invoke
|
||||
GeViAPI_EncodeString(encodedPassword, plainPassword, 256);
|
||||
GeViAPI_SetupClient_Create(..., encodedPassword, ...);
|
||||
```
|
||||
|
||||
**Why:** The native `GeViProcAPI.dll` requires pre-encoded passwords.
|
||||
|
||||
### When Encoding is NOT Required
|
||||
|
||||
**.NET SDK Wrapper:**
|
||||
```csharp
|
||||
// C# using .NET wrapper
|
||||
var db = new GeViDatabase();
|
||||
db.Create(server, username, plainTextPassword); // Handles encoding internally
|
||||
```
|
||||
|
||||
**Why:** The .NET wrapper (`GeViProcAPINET_4_0.dll`) calls `GeViAPI_EncodeString` internally.
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### Use Database When:
|
||||
- Querying recent system events
|
||||
- Checking user login history
|
||||
- Monitoring system errors/warnings
|
||||
- Real-time event tracking
|
||||
|
||||
### Use .SET File When:
|
||||
- Exporting/importing configuration
|
||||
- Backing up action mapping rules
|
||||
- Bulk editing action mappings
|
||||
- Understanding configured rules (not events)
|
||||
|
||||
---
|
||||
|
||||
## Tools Developed
|
||||
|
||||
### 1. QueryGeViDatabase
|
||||
**Purpose:** Query database for runtime events
|
||||
**Method:** .NET SDK wrapper
|
||||
**Password:** Plain text (wrapper handles encoding)
|
||||
|
||||
```bash
|
||||
QueryGeViDatabase.exe localhost sysadmin masterkey
|
||||
```
|
||||
|
||||
**Output:**
|
||||
- System events
|
||||
- User logins
|
||||
- Errors/warnings
|
||||
- 100+ action event logs
|
||||
|
||||
### 2. DiagnoseSetupClient + GeViSetEditor
|
||||
**Purpose:** Read/parse .set configuration file
|
||||
**Method:** Native SetupClient API
|
||||
**Password:** Manually encoded with `GeViAPI_EncodeString`
|
||||
|
||||
```bash
|
||||
GeViSetEditor.exe to-json setup_config.dat output.json
|
||||
```
|
||||
|
||||
**Output:**
|
||||
- 64 action mapping rules
|
||||
- 107 action strings
|
||||
- Complete configuration in JSON
|
||||
|
||||
---
|
||||
|
||||
## Findings
|
||||
|
||||
1. **Database does NOT contain action mapping configuration**
|
||||
- Only stores runtime event logs
|
||||
- Cannot use database queries to read action mapping rules
|
||||
|
||||
2. **.SET file is the ONLY source for action mapping configuration**
|
||||
- Must use SetupClient API
|
||||
- Must parse binary format
|
||||
- Contains all 64 action mapping rules
|
||||
|
||||
3. **Password encoding depends on API layer:**
|
||||
- .NET wrapper: No encoding needed
|
||||
- Native API: Encoding required
|
||||
- Both work, different use cases
|
||||
|
||||
4. **Two distinct data flows:**
|
||||
```
|
||||
Configuration Flow:
|
||||
GeViSet App → .SET File → SetupClient API → Binary Parser → JSON
|
||||
|
||||
Events Flow:
|
||||
GeViServer → Database → SendQuery → Event Logs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
1. **For Action Mapping Management:**
|
||||
- Use SetupClient API to read .set files
|
||||
- Parse binary format to extract mappings
|
||||
- Export to JSON/Excel for editing
|
||||
- Rebuild .set file and write back
|
||||
|
||||
2. **For Event Monitoring:**
|
||||
- Use GeViDatabase queries
|
||||
- No binary parsing needed
|
||||
- Direct SQL or SDK queries
|
||||
|
||||
3. **Password Handling:**
|
||||
- Use .NET wrapper when possible (simpler)
|
||||
- Only use native API when .NET wrapper doesn't support the operation
|
||||
- Always encode passwords for native SetupClient API
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ **Completed:**
|
||||
- Database connection (both methods)
|
||||
- SetupClient connection with password encoding
|
||||
- Binary parser for .set files
|
||||
- JSON export of action mappings
|
||||
|
||||
2. **In Progress:**
|
||||
- Excel export for user-friendly editing
|
||||
- Enhanced parser for trigger conditions
|
||||
|
||||
3. **Future:**
|
||||
- Add/modify/delete action mappings
|
||||
- Binary writer for modifications
|
||||
- Web UI for configuration management
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- SetupClient API: `C:\GEVISOFT\Examples\VS2010CPP\GeViSoftSDK\Include\GeViProcAPI.h`
|
||||
- .NET Wrapper: `C:\GEVISOFT\GeViProcAPINET_4_0.dll`
|
||||
- Database: `C:\GEVISOFT\DATABASE\GeViDB.mdb`
|
||||
- Research Notes: `geutebruck-api/specs/002-geviset-file-format/research.md`
|
||||
274
geutebruck-api/specs/002-geviset-file-format/research.md
Normal file
274
geutebruck-api/specs/002-geviset-file-format/research.md
Normal file
@@ -0,0 +1,274 @@
|
||||
# GeViSet File Format Research Notes
|
||||
|
||||
## Binary Format Discoveries
|
||||
|
||||
### Header Analysis
|
||||
|
||||
**File**: setup_config_20251212_122429.dat (281,714 bytes)
|
||||
|
||||
```
|
||||
Offset Hex ASCII
|
||||
0000: 00 13 47 65 56 69 53 6F 66 74 20 50 61 72 61 6D ..GeViSoft Param
|
||||
0010: 65 74 65 72 73 eters
|
||||
```
|
||||
|
||||
**Structure**:
|
||||
- `00`: Optional null byte (not always present)
|
||||
- `13`: Length byte (0x13 = 19 bytes)
|
||||
- `47 65 56 69 53 6F 66 74 20 50 61 72 61 6D 65 74 65 72 73`: "GeViSoft Parameters"
|
||||
|
||||
**Note**: This is NOT a standard Pascal string (no 0x07 marker), just length + data.
|
||||
|
||||
### Section Structure
|
||||
|
||||
Sections appear to follow this pattern:
|
||||
|
||||
```
|
||||
07 <len> <section_name> // Pascal string for section name
|
||||
... items ...
|
||||
05 52 75 6C 65 73 // "Rules" marker (if rules present)
|
||||
... rules ...
|
||||
```
|
||||
|
||||
### Rules Marker Pattern
|
||||
|
||||
Found 65 occurrences of pattern: `05 52 75 6C 65 73` ("Rules")
|
||||
|
||||
Key offsets:
|
||||
- 252,278 (0x3D976)
|
||||
- 252,717 (0x3DB2D)
|
||||
- 253,152 (0x3DCE0)
|
||||
- ... (65 total)
|
||||
|
||||
After "Rules" marker:
|
||||
```
|
||||
05 52 75 6C 65 73 // "Rules"
|
||||
02 00 00 00 // Count? (2 rules?)
|
||||
00 01 31 // Unknown metadata
|
||||
05 00 00 00 // Another count/offset?
|
||||
07 01 40 ... // Start of action string
|
||||
```
|
||||
|
||||
### Action String Pattern
|
||||
|
||||
**Format**: `07 01 40 <len_2bytes_LE> <action_data>`
|
||||
|
||||
**Examples from file**:
|
||||
|
||||
1. At offset 252,291:
|
||||
```
|
||||
07 01 40 1C 00 47 53 43 20 56 69 65 77 65 72 43 6F 6E 6E 65 63 74 4C 69 76 65 20 56 20 3C 2D 20 43
|
||||
│ │ │ │ │
|
||||
│ │ │ └──┴─ Length: 0x001C (28 bytes)
|
||||
│ │ └─ Action marker
|
||||
│ └─ Subtype
|
||||
└─ String type
|
||||
|
||||
Action: "GSC ViewerConnectLive V <- C"
|
||||
```
|
||||
|
||||
2. At offset 258,581:
|
||||
```
|
||||
07 01 40 11 00 47 53 43 20 56 69 65 77 65 72 43 6C 65 61 72 20 56
|
||||
Length: 0x0011 (17 bytes)
|
||||
Action: "GSC ViewerClear V"
|
||||
```
|
||||
|
||||
### Data Type Markers
|
||||
|
||||
| Marker | Type | Evidence |
|
||||
|--------|---------|-----------------------------------------------|
|
||||
| 0x01 | Boolean | Followed by 0x00 or 0x01 |
|
||||
| 0x04 | Int32 | Followed by 4 bytes (little-endian) |
|
||||
| 0x07 | String | Pascal string: <len> <data> |
|
||||
| 0x07 0x01 0x40 | Action | Special action string format |
|
||||
|
||||
### Section Names Found
|
||||
|
||||
From file analysis:
|
||||
- "Description" (most common - appears 832 times)
|
||||
- "IpHost"
|
||||
- "GscAction"
|
||||
- "GCoreAction"
|
||||
- "Alarms"
|
||||
- "Clients"
|
||||
- "GeViIO"
|
||||
|
||||
### Action Mappings Extracted
|
||||
|
||||
Successfully extracted 64 action mappings from the file:
|
||||
|
||||
**PTZ Camera Controls** (Camera 101027):
|
||||
1. PanLeft_101027
|
||||
2. PanRight_101027
|
||||
3. PanStop_101027
|
||||
4. TiltDown_101027
|
||||
5. TiltUp_101027
|
||||
6. TiltStop_101027
|
||||
7. ZoomIn_101027
|
||||
8. ZoomOut_101027
|
||||
9. ZoomStop_101027
|
||||
10. FocusFar 128_C101027
|
||||
11. FocusNear 128_C101027
|
||||
12. FocusStop_101027
|
||||
13. IrisOpen_101027
|
||||
14. IrisClose_101027
|
||||
15. IrisStop_101027
|
||||
|
||||
**Preset Positions**:
|
||||
16. MoveToDefaultPostion_101027
|
||||
17. ClearDefaultPostion_101027
|
||||
18. SaveDafaultPostion_101027
|
||||
19. MoveToPresentPostion
|
||||
20. ClearPresentPostion
|
||||
21. SavePresentPostion
|
||||
|
||||
**Viewer Controls**:
|
||||
22. ViewerConnectLive V <- C
|
||||
23. ViewerConnectLive V <- C_101027
|
||||
24. ViewerClear V
|
||||
25. VC live
|
||||
|
||||
**System Messages**:
|
||||
26-35. Demo mode warnings (100, 90, 80... 10 min)
|
||||
36. info: licence satisfied
|
||||
37. info: re_porter mode active
|
||||
38. error: "GeViIO Client: start of interface failed"
|
||||
39. error: "GeViIO Client: interface lost"
|
||||
40. warning: "GeViSoft Server: client warning"
|
||||
|
||||
### Platform Variations
|
||||
|
||||
Actions often have multiple platform-specific versions:
|
||||
|
||||
```
|
||||
GSC (GeViScope):
|
||||
"GSC ViewerConnectLive V <- C"
|
||||
|
||||
GNG (G-Net-Guard):
|
||||
"GNG ViewerConnectLive V <- C_101027"
|
||||
|
||||
GCore:
|
||||
"GCore <action>"
|
||||
```
|
||||
|
||||
### Unknown Patterns
|
||||
|
||||
Several byte patterns whose purpose is unclear:
|
||||
|
||||
1. **Pattern**: `04 02 40 40 64 00 00 00 00`
|
||||
- Appears before many action strings
|
||||
- Possibly metadata or flags
|
||||
|
||||
2. **Pattern**: `00 00 00 00 00 01 31 05 00 00 00`
|
||||
- Appears after "Rules" marker
|
||||
- Could be counts, offsets, or IDs
|
||||
|
||||
3. **Pattern**: `0C 4D 61 70 70 69 6E 67 52 75 6C 65 73`
|
||||
- `0C MappingRules` (length-prefixed, no 0x07)
|
||||
- At offset 252,172
|
||||
- Different string format than Pascal strings
|
||||
|
||||
## Testing Results
|
||||
|
||||
### Round-Trip Test
|
||||
|
||||
```
|
||||
✅ SUCCESS!
|
||||
Original: 281,714 bytes
|
||||
Parsed: 64 action mappings
|
||||
Written: 281,714 bytes
|
||||
Comparison: IDENTICAL (byte-for-byte)
|
||||
```
|
||||
|
||||
**Conclusion**: Safe to write back to server with current preservation approach.
|
||||
|
||||
### SetupClient API Test
|
||||
|
||||
```
|
||||
✅ Connection successful
|
||||
✅ Read setup: 281,714 bytes
|
||||
✅ Write setup: Not tested yet (waiting for full parser)
|
||||
✅ Password encryption: Working (GeViAPI_EncodeString)
|
||||
```
|
||||
|
||||
## Next Research Areas
|
||||
|
||||
### 1. Trigger Parsing
|
||||
|
||||
Need to understand trigger structure:
|
||||
```
|
||||
.VideoInput = True
|
||||
.InputContact = False
|
||||
```
|
||||
|
||||
These appear before action strings in rules.
|
||||
|
||||
### 2. Metadata Bytes
|
||||
|
||||
The bytes between sections and before/after rules:
|
||||
- What do they represent?
|
||||
- Are they counts? Offsets? Flags?
|
||||
- Can they be modified?
|
||||
|
||||
### 3. Section Relationships
|
||||
|
||||
How do sections reference each other?
|
||||
- Do cameras reference alarm rules?
|
||||
- Do action mappings reference I/O ports?
|
||||
- How are IDs assigned?
|
||||
|
||||
### 4. Format Versioning
|
||||
|
||||
Does the format change between GeViSoft versions?
|
||||
- Version 6.0.1.5 (current)
|
||||
- How to detect version?
|
||||
- Compatibility considerations?
|
||||
|
||||
## Tools Used for Analysis
|
||||
|
||||
### Python Scripts
|
||||
|
||||
```python
|
||||
# Find all "Rules" patterns
|
||||
import struct
|
||||
with open('setup_config.dat', 'rb') as f:
|
||||
data = f.read()
|
||||
|
||||
needle = b'Rules'
|
||||
pos = 0
|
||||
while True:
|
||||
pos = data.find(needle, pos)
|
||||
if pos == -1: break
|
||||
print(f'Found at offset {pos} (0x{pos:X})')
|
||||
pos += 1
|
||||
```
|
||||
|
||||
### Hex Editors
|
||||
|
||||
- HxD
|
||||
- 010 Editor
|
||||
- VS Code with hex extension
|
||||
|
||||
### Binary Analysis
|
||||
|
||||
- Custom C# parser
|
||||
- Grep for pattern matching
|
||||
- Byte comparison tools
|
||||
|
||||
## References
|
||||
|
||||
- TestMKS.set (279,860 bytes) - Original test file
|
||||
- setup_config_20251212_122429.dat (281,714 bytes) - Live server config
|
||||
- GeViSoft SDK Documentation
|
||||
- GeViProcAPI.h header file
|
||||
|
||||
## Change Log
|
||||
|
||||
| Date | Discovery |
|
||||
|------------|----------------------------------------------|
|
||||
| 2024-12-12 | Initial binary analysis |
|
||||
| 2024-12-12 | Discovered action string format |
|
||||
| 2024-12-12 | Found 65 "Rules" markers |
|
||||
| 2024-12-12 | Extracted 64 action mappings successfully |
|
||||
| 2024-12-12 | Verified byte-for-byte round-trip |
|
||||
617
geutebruck-api/specs/002-geviset-file-format/spec.md
Normal file
617
geutebruck-api/specs/002-geviset-file-format/spec.md
Normal file
@@ -0,0 +1,617 @@
|
||||
# GeViSet File Format Reverse Engineering Specification
|
||||
|
||||
**Version:** 1.0
|
||||
**Date:** 2024-12-12
|
||||
**Status:** In Progress
|
||||
|
||||
## Overview
|
||||
|
||||
This specification documents the reverse engineering effort to fully parse, understand, and manipulate the GeViSoft `.set` configuration file format. The goal is to enable programmatic reading, editing, and writing of GeViServer configurations, particularly action mappings.
|
||||
|
||||
## Background
|
||||
|
||||
### What is a .set File?
|
||||
|
||||
- **Source**: Exported from GeViSet application or read via `GeViAPI_SetupClient_ReadSetup`
|
||||
- **Purpose**: Complete GeViServer configuration (cameras, alarms, action mappings, users, etc.)
|
||||
- **Format**: Proprietary binary format with "GeViSoft Parameters" header
|
||||
- **Size**: Typically 200-300 KB for production configurations
|
||||
- **Use Case**: Backup, migration, and programmatic configuration management
|
||||
|
||||
### Current State
|
||||
|
||||
**What Works:**
|
||||
- ✅ Read .set file from GeViServer via SetupClient API
|
||||
- ✅ Extract 64 action mappings from binary data
|
||||
- ✅ Write back byte-for-byte identical (round-trip verified)
|
||||
- ✅ Password encryption with `GeViAPI_EncodeString`
|
||||
|
||||
**What's Missing:**
|
||||
- ❌ Full structure parsing (all sections, all items)
|
||||
- ❌ Understanding of all data types and relationships
|
||||
- ❌ Ability to add NEW action mappings
|
||||
- ❌ JSON representation of complete structure
|
||||
- ❌ Comprehensive Excel export/import
|
||||
|
||||
## Requirements
|
||||
|
||||
### Primary Request
|
||||
|
||||
> "I want to do full reverse engineering - I need to parse the whole file and maybe to json format in the first phase and then we will revert this json or its parts to excel"
|
||||
|
||||
### Key Requirements
|
||||
|
||||
1. **Parse Entire File Structure**
|
||||
- All sections (Alarms, Clients, GeViIO, Cameras, ActionMappings, etc.)
|
||||
- All configuration items (key-value pairs)
|
||||
- All rules and triggers
|
||||
- All metadata and relationships
|
||||
|
||||
2. **JSON Serialization**
|
||||
- Complete structure in JSON format
|
||||
- Human-readable and editable
|
||||
- Preserves all data and relationships
|
||||
- Round-trip safe (JSON → Binary → JSON)
|
||||
|
||||
3. **Excel Export/Import**
|
||||
- Export action mappings to Excel
|
||||
- User-friendly editing interface
|
||||
- Add new mappings
|
||||
- Delete existing mappings
|
||||
- Import back to JSON
|
||||
|
||||
4. **Safety & Validation**
|
||||
- Verify integrity before writing to server
|
||||
- Backup original configuration
|
||||
- Validate against schema
|
||||
- Error handling and recovery
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data Flow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ GeViServer │
|
||||
│ ↓ │
|
||||
│ SetupClient API (ReadSetup) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
.set file (binary)
|
||||
281,714 bytes
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 1: Binary Parser │
|
||||
│ - Parse header │
|
||||
│ - Parse all sections │
|
||||
│ - Parse all items │
|
||||
│ - Parse all rules │
|
||||
│ - Extract action mappings │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
JSON Structure
|
||||
(full configuration representation)
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 2: JSON Processing │
|
||||
│ - Validate structure │
|
||||
│ - Transform for editing │
|
||||
│ - Extract sections │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 3: Excel Export │
|
||||
│ - Convert action mappings to Excel │
|
||||
│ - User edits in Excel │
|
||||
│ - Add/delete/modify mappings │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
Excel file
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 4: Excel Import │
|
||||
│ - Read Excel changes │
|
||||
│ - Validate data │
|
||||
│ - Update JSON structure │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
JSON Structure
|
||||
(modified)
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 5: Binary Writer │
|
||||
│ - Rebuild .set file from JSON │
|
||||
│ - Maintain binary format │
|
||||
│ - Validate integrity │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
.set file (binary)
|
||||
↓
|
||||
SetupClient API (WriteSetup)
|
||||
↓
|
||||
GeViServer
|
||||
```
|
||||
|
||||
## Binary Format Analysis
|
||||
|
||||
### File Structure
|
||||
|
||||
```
|
||||
.set file
|
||||
├── Header
|
||||
│ ├── 0x00 (optional null byte)
|
||||
│ └── Pascal String: "GeViSoft Parameters" (0x07 <len> <data>)
|
||||
│
|
||||
├── Sections (multiple)
|
||||
│ ├── Section Name (Pascal String)
|
||||
│ ├── Items (key-value pairs)
|
||||
│ │ ├── Key (Pascal String)
|
||||
│ │ └── Value (typed)
|
||||
│ │ ├── 0x01 = Boolean
|
||||
│ │ ├── 0x04 = Integer (4 bytes)
|
||||
│ │ └── 0x07 = String (Pascal)
|
||||
│ │
|
||||
│ └── Rules Subsection
|
||||
│ ├── "Rules" marker (0x05 0x52 0x75 0x6C 0x65 0x73)
|
||||
│ ├── Count/Metadata
|
||||
│ └── Action Rules (multiple)
|
||||
│ ├── Trigger Properties
|
||||
│ │ └── .PropertyName = Boolean
|
||||
│ ├── Main Action String
|
||||
│ │ └── 0x07 0x01 0x40 <len_2bytes> <action_data>
|
||||
│ └── Action Variations
|
||||
│ ├── GscAction (GeViScope)
|
||||
│ ├── GNGAction (G-Net-Guard)
|
||||
│ └── GCoreAction (GCore)
|
||||
│
|
||||
└── Footer (metadata/checksums?)
|
||||
```
|
||||
|
||||
### Data Types Discovered
|
||||
|
||||
| Marker | Type | Format | Example |
|
||||
|--------|---------|----------------------------------|----------------------------|
|
||||
| 0x01 | Boolean | 0x01 <value> | 0x01 0x01 = true |
|
||||
| 0x04 | Integer | 0x04 <4-byte little-endian> | 0x04 0x0A 0x00 0x00 0x00 |
|
||||
| 0x07 | String | 0x07 <len> <data> | 0x07 0x0B "Description" |
|
||||
| 0x07 0x01 0x40 | Action | 0x07 0x01 0x40 <len_2bytes> <data> | Action string format |
|
||||
|
||||
### Action String Format
|
||||
|
||||
Pattern: `07 01 40 <len_2bytes_LE> <action_text>`
|
||||
|
||||
Example:
|
||||
```
|
||||
07 01 40 1C 00 47 53 43 20 56 69 65 77 65 72 43 6F 6E 6E 65 63 74 4C 69 76 65...
|
||||
│ │ │ │ │ └─ "GSC ViewerConnectLive V <- C"
|
||||
│ │ │ └──┴─ Length: 0x001C (28 bytes)
|
||||
│ │ └─ 0x40 (action marker)
|
||||
│ └─ 0x01 (subtype)
|
||||
└─ 0x07 (string type)
|
||||
```
|
||||
|
||||
### Sections Found
|
||||
|
||||
From file analysis, sections include:
|
||||
- **Alarms**: Alarm configurations
|
||||
- **Clients**: Client connections
|
||||
- **GeViIO**: Digital I/O configurations
|
||||
- **Cameras**: Camera settings
|
||||
- **Description**: Various descriptive entries
|
||||
- **IpHost**: Network configurations
|
||||
- **ActionMappings**: Trigger → Action rules (our focus)
|
||||
|
||||
## JSON Schema
|
||||
|
||||
### Complete Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft-07/schema#",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"version": {
|
||||
"type": "string",
|
||||
"description": "Parser version"
|
||||
},
|
||||
"header": {
|
||||
"type": "string",
|
||||
"description": "File header (GeViSoft Parameters)"
|
||||
},
|
||||
"sections": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/Section"
|
||||
}
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"Section": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string"
|
||||
},
|
||||
"items": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/ConfigItem"
|
||||
}
|
||||
},
|
||||
"rules": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/ActionRule"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"ConfigItem": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"value": {
|
||||
"oneOf": [
|
||||
{ "type": "boolean" },
|
||||
{ "type": "integer" },
|
||||
{ "type": "string" }
|
||||
]
|
||||
},
|
||||
"type": {
|
||||
"enum": ["boolean", "integer", "string"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"ActionRule": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer"
|
||||
},
|
||||
"triggers": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"mainAction": {
|
||||
"type": "string"
|
||||
},
|
||||
"variations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/ActionVariation"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"ActionVariation": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"platform": {
|
||||
"enum": ["GSC", "GNG", "GCore"]
|
||||
},
|
||||
"actionString": {
|
||||
"type": "string"
|
||||
},
|
||||
"serverType": {
|
||||
"type": "string"
|
||||
},
|
||||
"serverName": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example JSON Output
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "1.0",
|
||||
"header": "GeViSoft Parameters",
|
||||
"sections": [
|
||||
{
|
||||
"name": "ActionMappings",
|
||||
"items": [],
|
||||
"rules": [
|
||||
{
|
||||
"id": 1,
|
||||
"triggers": {
|
||||
"InputContact": true,
|
||||
"VideoInput": false
|
||||
},
|
||||
"mainAction": "AlternateContact(2, 1000, 500)",
|
||||
"variations": [
|
||||
{
|
||||
"platform": "GSC",
|
||||
"actionString": "GSC ViewerConnectLive V <- C_101027",
|
||||
"serverType": "GeViScope",
|
||||
"serverName": "GEVISCOPE"
|
||||
},
|
||||
{
|
||||
"platform": "GNG",
|
||||
"actionString": "GNG PanLeft_101027",
|
||||
"serverType": "",
|
||||
"serverName": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Alarms",
|
||||
"items": [
|
||||
{
|
||||
"key": "AlarmCount",
|
||||
"value": 5,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"key": "Enabled",
|
||||
"value": true,
|
||||
"type": "boolean"
|
||||
}
|
||||
],
|
||||
"rules": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Complete Binary Parser ✅
|
||||
|
||||
**Goal**: Parse entire .set file structure into memory
|
||||
|
||||
**Components**:
|
||||
- ✅ Header parser
|
||||
- 🚧 Section parser (all types)
|
||||
- 🚧 Item parser (all data types)
|
||||
- 🚧 Rules parser (complete structure)
|
||||
- 🚧 Action variation parser
|
||||
|
||||
**Status**: Basic parser exists, needs enhancement for full structure
|
||||
|
||||
### Phase 2: JSON Serialization 🚧
|
||||
|
||||
**Goal**: Convert parsed structure to JSON
|
||||
|
||||
**Components**:
|
||||
- JSON serializer
|
||||
- Schema validator
|
||||
- Round-trip tester (Binary → JSON → Binary)
|
||||
|
||||
**Deliverables**:
|
||||
- `SetFileToJson` converter
|
||||
- JSON schema definition
|
||||
- Validation tools
|
||||
|
||||
### Phase 3: Excel Export 🚧
|
||||
|
||||
**Goal**: Export action mappings to Excel for editing
|
||||
|
||||
**Components**:
|
||||
- Excel writer (EPPlus library)
|
||||
- Action mapping table generator
|
||||
- Template with formulas/validation
|
||||
|
||||
**Excel Structure**:
|
||||
```
|
||||
Sheet: ActionMappings
|
||||
| Rule ID | Trigger Type | Trigger Param | Action 1 | Action 2 | Action 3 |
|
||||
|---------|--------------|---------------|----------|----------|----------|
|
||||
| 1 | InputContact | 3, false | Alternate| Viewer | |
|
||||
| 2 | VideoInput | 4, true | CrossSwi | VCChange | |
|
||||
```
|
||||
|
||||
### Phase 4: Excel Import 🚧
|
||||
|
||||
**Goal**: Import edited Excel back to JSON
|
||||
|
||||
**Components**:
|
||||
- Excel reader
|
||||
- Validation engine
|
||||
- Diff generator (show changes)
|
||||
- JSON merger
|
||||
|
||||
### Phase 5: Binary Writer 🚧
|
||||
|
||||
**Goal**: Rebuild .set file from JSON
|
||||
|
||||
**Components**:
|
||||
- Binary writer
|
||||
- Structure rebuilder
|
||||
- Validation
|
||||
- Backup mechanism
|
||||
|
||||
**Critical**: Must maintain binary compatibility!
|
||||
|
||||
### Phase 6: Testing & Validation 🚧
|
||||
|
||||
**Goal**: Ensure safety and correctness
|
||||
|
||||
**Test Cases**:
|
||||
1. Round-trip (Binary → JSON → Binary) = identical
|
||||
2. Round-trip (Binary → JSON → Excel → JSON → Binary) = valid
|
||||
3. Add new mapping → write → server accepts
|
||||
4. Modify existing mapping → write → server accepts
|
||||
5. Delete mapping → write → server accepts
|
||||
|
||||
## Current Progress
|
||||
|
||||
### Completed ✅
|
||||
|
||||
- [x] SetupClient API integration
|
||||
- [x] Password encryption
|
||||
- [x] Basic binary parsing (64 action mappings extracted)
|
||||
- [x] Safe round-trip (byte-for-byte identical)
|
||||
- [x] File structure analysis
|
||||
- [x] Data type discovery
|
||||
|
||||
### In Progress 🚧
|
||||
|
||||
- [ ] Complete section parsing
|
||||
- [ ] Full rule structure parsing
|
||||
- [ ] JSON serialization
|
||||
- [ ] Excel export
|
||||
- [ ] Binary writer for modifications
|
||||
|
||||
### Pending 📋
|
||||
|
||||
- [ ] Excel import
|
||||
- [ ] Add new mapping functionality
|
||||
- [ ] API endpoints
|
||||
- [ ] Documentation
|
||||
- [ ] Production deployment
|
||||
|
||||
## Technical Challenges
|
||||
|
||||
### Challenge 1: Unknown Metadata Bytes
|
||||
|
||||
**Problem**: Many byte sequences whose purpose is unknown
|
||||
|
||||
**Solution**:
|
||||
- Document all patterns found
|
||||
- Test modifications to understand behavior
|
||||
- Preserve unknown bytes during round-trip
|
||||
|
||||
### Challenge 2: Complex Nested Structure
|
||||
|
||||
**Problem**: Sections contain items and rules, rules contain variations
|
||||
|
||||
**Solution**:
|
||||
- Recursive parsing
|
||||
- Clear data model hierarchy
|
||||
- Offset tracking for debugging
|
||||
|
||||
### Challenge 3: Binary Format Changes
|
||||
|
||||
**Problem**: Format may vary between GeViSoft versions
|
||||
|
||||
**Solution**:
|
||||
- Version detection
|
||||
- Support multiple format versions
|
||||
- Graceful degradation
|
||||
|
||||
### Challenge 4: Action String Syntax
|
||||
|
||||
**Problem**: Action strings have complex syntax (parameters, types, etc.)
|
||||
|
||||
**Solution**:
|
||||
- Pattern matching
|
||||
- Action string parser
|
||||
- Validation against known action types
|
||||
|
||||
## Safety Considerations
|
||||
|
||||
### Before Writing to Server
|
||||
|
||||
1. ✅ **Verify round-trip**: Parse → Write → Compare = Identical
|
||||
2. ✅ **Backup original**: Always keep copy of working config
|
||||
3. ⚠️ **Test in dev**: Never test on production first
|
||||
4. ⚠️ **Validate structure**: Check against schema
|
||||
5. ⚠️ **Incremental changes**: Small changes, test frequently
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Validate before write
|
||||
- Provide detailed error messages
|
||||
- Support rollback
|
||||
- Log all operations
|
||||
|
||||
## Tools & Libraries
|
||||
|
||||
### Development
|
||||
|
||||
- **Language**: C# / .NET 8.0
|
||||
- **Binary Parsing**: Custom binary reader
|
||||
- **JSON**: System.Text.Json
|
||||
- **Excel**: EPPlus (for .xlsx)
|
||||
- **Testing**: xUnit
|
||||
- **Logging**: Serilog
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
GeViSetEditor/
|
||||
├── GeViSetEditor.Core/
|
||||
│ ├── Models/
|
||||
│ │ ├── SetFileStructure.cs
|
||||
│ │ ├── Section.cs
|
||||
│ │ ├── ConfigItem.cs
|
||||
│ │ ├── ActionRule.cs
|
||||
│ │ └── ActionVariation.cs
|
||||
│ ├── Parsers/
|
||||
│ │ ├── SetFileBinaryParser.cs
|
||||
│ │ ├── SectionParser.cs
|
||||
│ │ └── RuleParser.cs
|
||||
│ ├── Writers/
|
||||
│ │ ├── SetFileBinaryWriter.cs
|
||||
│ │ └── JsonWriter.cs
|
||||
│ ├── Converters/
|
||||
│ │ ├── JsonToExcel.cs
|
||||
│ │ └── ExcelToJson.cs
|
||||
│ └── Validators/
|
||||
│ └── StructureValidator.cs
|
||||
├── GeViSetEditor.CLI/
|
||||
│ └── Commands/
|
||||
│ ├── ParseCommand.cs
|
||||
│ ├── ToJsonCommand.cs
|
||||
│ ├── ToExcelCommand.cs
|
||||
│ └── FromExcelCommand.cs
|
||||
└── GeViSetEditor.Tests/
|
||||
├── ParserTests.cs
|
||||
├── RoundTripTests.cs
|
||||
└── ValidationTests.cs
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Immediate (This Session)
|
||||
|
||||
1. ✅ Create specification document
|
||||
2. ✅ Update git repository
|
||||
3. 🚧 Implement complete binary parser
|
||||
4. 🚧 Implement JSON serialization
|
||||
5. 🚧 Test round-trip with JSON
|
||||
|
||||
### Short Term (Next Session)
|
||||
|
||||
1. Excel export implementation
|
||||
2. Excel import implementation
|
||||
3. Add new mapping functionality
|
||||
4. Comprehensive testing
|
||||
|
||||
### Long Term
|
||||
|
||||
1. Web UI for configuration management
|
||||
2. API endpoints
|
||||
3. Multi-version support
|
||||
4. Documentation and examples
|
||||
|
||||
## References
|
||||
|
||||
- GeViSoft SDK Documentation
|
||||
- SetupClient API Reference
|
||||
- Existing .set file samples (TestMKS.set, setup_config_*.dat)
|
||||
- Binary analysis notes
|
||||
- Round-trip test results
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------------|--------------------------------------|
|
||||
| 1.0 | 2024-12-12 | Initial specification |
|
||||
|
||||
---
|
||||
|
||||
**Status**: Ready for full implementation
|
||||
**Priority**: High
|
||||
**Complexity**: High
|
||||
**Timeline**: 2-3 days estimated
|
||||
Reference in New Issue
Block a user