klas/tkb_timeshift
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Claude N8N toolkit with Docker mock API server

Browse Source 
- Added comprehensive N8N development tools collection
- Added Docker-containerized mock API server for testing
- Added complete documentation and setup guides
- Added mock API server with health checks and data endpoints
- Tools include workflow analyzers, debuggers, and controllers

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Docker Config Backup
2025-06-17 21:23:46 +02:00
parent c2d748f725
commit 8793ac4f59
26 changed files with 6632 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
claude_n8n/tools/mock_api_server/Dockerfile Normal file
View File

@@ -0,0 +1,24 @@
FROM python:3.9-slim
WORKDIR /app
# Install dependencies
RUN pip install flask werkzeug
# Create api_data directory
RUN mkdir -p api_data
# Copy the mock API server script
COPY mock_api_server.py .
# Copy API data files if they exist
COPY api_data/ ./api_data/
# Expose port 5000
EXPOSE 5000
# Set environment variables
ENV PYTHONUNBUFFERED=1
# Run the server
CMD ["python", "mock_api_server.py", "--host", "0.0.0.0", "--port", "5000", "--data-dir", "api_data"]

200
claude_n8n/tools/mock_api_server/README.md Normal file
View File

@@ -0,0 +1,200 @@
# Mock API Server for N8N Testing
A Docker-containerized REST API server that serves test data from JSON files for N8N workflow development and testing.
## Overview
This mock API server provides a consistent, controllable data source for testing N8N workflows without relying on external APIs. It serves data from JSON files and includes features like pagination, random data selection, and file upload capabilities.
## Files Structure
```
mock_api_server/
├── README.md # This documentation
├── Dockerfile # Docker image definition
├── docker-compose.yml # Docker Compose configuration
├── mock_api_server.py # Main Python Flask server (symlinked from ../mock_api_server.py)
└── api_data/ # JSON data files directory
├── matrix_messages.json # Matrix chat messages sample data
└── test_data.json # Simple test data
```
## Quick Start
### Using Docker Compose (Recommended)
```bash
cd /home/klas/claude_n8n/tools/mock_api_server
docker compose up -d
```
The server will be available at: `http://localhost:5002`
### Using Docker Build
```bash
cd /home/klas/claude_n8n/tools/mock_api_server
docker build -t mock-api-server .
docker run -d -p 5002:5000 -v $(pwd)/api_data:/app/api_data mock-api-server
```
### Using Python Directly
```bash
cd /home/klas/claude_n8n/tools
python mock_api_server.py --host 0.0.0.0 --port 5002 --data-dir mock_api_server/api_data
```
## API Endpoints
### Health Check
- **GET** `/health` - Server status and available endpoints
### Data Access
- **GET** `/data` - List all available data files
- **GET** `/data/<filename>` - Get data from specific file (`.json` extension optional)
- **GET** `/random/<filename>` - Get random item from array data
- **GET** `/paginated/<filename>?page=1&per_page=10` - Get paginated data
### Special Endpoints
- **GET** `/matrix` - Alias for `/data/matrix_messages.json`
- **POST** `/upload?filename=<name>` - Upload new JSON data file
### Query Parameters
- `page` - Page number for pagination (default: 1)
- `per_page` - Items per page (default: 10)
- `filename` - Target filename for upload (without .json extension)
## Example Usage
### Health Check
```bash
curl http://localhost:5002/health
```
### Get Test Data
```bash
curl http://localhost:5002/data/test_data.json
# Returns: {"message": "Hello from mock API", "timestamp": 1234567890, "items": [...]}
```
### Get Random Item
```bash
curl http://localhost:5002/random/test_data
# Returns random item from the test_data.json array
```
### Paginated Data
```bash
curl "http://localhost:5002/paginated/matrix_messages?page=1&per_page=5"
```
### Upload New Data
```bash
curl -X POST "http://localhost:5002/upload?filename=my_data" \
-H "Content-Type: application/json" \
-d '{"test": "value", "items": [1,2,3]}'
```
## Data Files
### Adding New Data Files
1. **Via File System:** Add `.json` files to the `api_data/` directory
2. **Via API:** Use the `/upload` endpoint to create new files
3. **Via Container:** Mount additional volumes or copy files into running container
### Data File Format
Files should contain valid JSON. The server supports:
- **Objects:** `{"key": "value", "items": [...]}`
- **Arrays:** `[{"id": 1}, {"id": 2}]`
### Sample Data Files
#### test_data.json
```json
{
"message": "Hello from mock API",
"timestamp": 1234567890,
"items": [
{"id": 1, "name": "Item 1"},
{"id": 2, "name": "Item 2"},
{"id": 3, "name": "Item 3"}
]
}
```
#### matrix_messages.json
Contains sample Matrix chat room messages with realistic structure for testing chat integrations.
## Configuration
### Environment Variables
- `PYTHONUNBUFFERED=1` - Enable real-time Python output in Docker
### Docker Compose Configuration
- **Host Port:** 5002
- **Container Port:** 5000
- **Volume Mount:** `./api_data:/app/api_data`
- **Restart Policy:** `unless-stopped`
### Health Check
Docker includes automatic health checking via curl to `/health` endpoint.
## Integration with N8N
### HTTP Request Node Configuration
```
Method: GET
URL: http://host.docker.internal:5002/data/test_data
```
### Webhook Testing
Use the mock API to provide consistent test data for webhook development and testing.
### Data Processing Workflows
Test data transformation nodes with predictable input from the mock API.
## Development
### Adding New Endpoints
Edit `mock_api_server.py` and add new Flask routes. The server will automatically restart in development mode.
### Debugging
Check container logs:
```bash
docker compose logs -f mock-api
```
### Stopping the Server
```bash
docker compose down
```
## Troubleshooting
### Port Already in Use
If port 5002 is occupied, edit `docker-compose.yml` and change the host port:
```yaml
ports:
- "5003:5000" # Change 5002 to 5003
```
### File Permissions
Ensure the `api_data` directory is writable for file uploads:
```bash
chmod 755 api_data/
```
### Container Not Starting
Check if all required files are present:
```bash
ls -la mock_api_server.py Dockerfile docker-compose.yml api_data/
```
## Related Files
- **Main Server Script:** `/home/klas/claude_n8n/tools/mock_api_server.py`
- **N8N Tools Directory:** `/home/klas/claude_n8n/tools/`
- **Original Development Files:** `/home/klas/mem0/.claude/` (can be removed after migration)

98
claude_n8n/tools/mock_api_server/api_data/matrix_messages.json Normal file
View File

@@ -0,0 +1,98 @@
[
{
"chunk": [
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@klas:matrix.klas.chat",
"content": {
"body": "The hybrid deduplication system is now working perfectly. We've successfully implemented content-based analysis that eliminates dependency on N8N workflow variables.",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1750017000000,
"unsigned": {
"membership": "join",
"age": 1000
},
"event_id": "$memory_1_recent_implementation_success",
"user_id": "@klas:matrix.klas.chat",
"age": 1000
},
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@developer:matrix.klas.chat",
"content": {
"body": "Key improvements include age-based filtering (30+ minutes), system message detection, and enhanced duplicate detection using content fingerprinting. The solution addresses the core issue where 10-message chunks were being reprocessed.",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1750017060000,
"unsigned": {
"membership": "join",
"age": 2000
},
"event_id": "$memory_2_technical_details",
"user_id": "@developer:matrix.klas.chat",
"age": 2000
},
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@ai_assistant:matrix.klas.chat",
"content": {
"body": "Memory retention has been significantly improved. The false duplicate detection that was causing 0.2-minute memory lifespans has been resolved through sophisticated content analysis and multiple validation layers.",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1750017120000,
"unsigned": {
"membership": "join",
"age": 3000
},
"event_id": "$memory_3_retention_improvement",
"user_id": "@ai_assistant:matrix.klas.chat",
"age": 3000
},
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@system_monitor:matrix.klas.chat",
"content": {
"body": "Test results: 2/2 scenarios passed. Valid recent messages are processed correctly, while old messages (1106+ minutes) are properly filtered. The enhanced deduplication is fully operational with robust duplicate detection.",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1750017180000,
"unsigned": {
"membership": "join",
"age": 4000
},
"event_id": "$memory_4_test_results",
"user_id": "@system_monitor:matrix.klas.chat",
"age": 4000
},
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@project_lead:matrix.klas.chat",
"content": {
"body": "Next phase: Monitor memory creation and consolidation patterns. The hybrid solution combines deterministic deduplication with AI-driven memory management for optimal performance and accuracy.",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1750017240000,
"unsigned": {
"membership": "join",
"age": 5000
},
"event_id": "$memory_5_next_phase",
"user_id": "@project_lead:matrix.klas.chat",
"age": 5000
}
],
"start": "t500-17000_0_0_0_0_0_0_0_0_0",
"end": "t505-17005_0_0_0_0_0_0_0_0_0"
}
]

18
claude_n8n/tools/mock_api_server/api_data/test_data.json Normal file
View File

@@ -0,0 +1,18 @@
{
"message": "Hello from mock API",
"timestamp": 1749928362092,
"items": [
{
"id": 1,
"name": "Item 1"
},
{
"id": 2,
"name": "Item 2"
},
{
"id": 3,
"name": "Item 3"
}
]
}

21
claude_n8n/tools/mock_api_server/docker-compose.yml Normal file
View File

@@ -0,0 +1,21 @@
version: '3.8'
services:
mock-api:
build: .
ports:
- "5002:5000"
volumes:
- ./api_data:/app/api_data
environment:
- PYTHONUNBUFFERED=1
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
volumes:
api_data:

439
claude_n8n/tools/mock_api_server/mock_api_server.py Normal file
View File

@@ -0,0 +1,439 @@
"""
Mock API Server for N8N Testing
This module provides a REST API server that serves data from text files.
N8N workflows can call this API to get consistent test data stored in files.
"""
import json
import logging
import os
import random
import threading
import time
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Any, Optional, Union
from flask import Flask, jsonify, request, Response
from werkzeug.serving import make_server
logger = logging.getLogger(__name__)
class MockAPIServer:
"""REST API server that serves data from text files."""
def __init__(self, data_dir: str = "api_data", host: str = "0.0.0.0", port: int = 5000):
"""
Initialize the mock API server.
Args:
data_dir: Directory containing data files
host: Host to bind the server to
port: Port to run the server on
"""
self.data_dir = Path(data_dir)
self.data_dir.mkdir(exist_ok=True)
self.host = host
self.port = port
self.app = Flask(__name__)
self.server = None
self.server_thread = None
self.is_running = False
# Setup routes
self._setup_routes()
# Create example data file if it doesn't exist
self._create_example_data()
def _setup_routes(self):
"""Setup API routes."""
@self.app.route('/health', methods=['GET'])
def health_check():
"""Health check endpoint."""
return jsonify({
'status': 'healthy',
'timestamp': datetime.now().isoformat(),
'data_dir': str(self.data_dir),
'available_endpoints': self._get_available_endpoints()
})
@self.app.route('/data/<path:filename>', methods=['GET'])
def get_data(filename):
"""Get data from a specific file."""
return self._serve_data_file(filename)
@self.app.route('/data', methods=['GET'])
def list_data_files():
"""List available data files."""
files = []
for file_path in self.data_dir.glob('*.json'):
files.append({
'name': file_path.stem,
'filename': file_path.name,
'size': file_path.stat().st_size,
'modified': datetime.fromtimestamp(file_path.stat().st_mtime).isoformat(),
'url': f"http://{self.host}:{self.port}/data/{file_path.name}"
})
return jsonify({
'files': files,
'count': len(files)
})
@self.app.route('/matrix', methods=['GET'])
def get_matrix_data():
"""Get Matrix chat data (example endpoint)."""
return self._serve_data_file('matrix_messages.json')
@self.app.route('/random/<path:filename>', methods=['GET'])
def get_random_data(filename):
"""Get random item from data file."""
data = self._load_data_file(filename)
if not data:
return jsonify({'error': 'File not found or empty'}), 404
if isinstance(data, list) and data:
return jsonify(random.choice(data))
else:
return jsonify(data)
@self.app.route('/paginated/<path:filename>', methods=['GET'])
def get_paginated_data(filename):
"""Get paginated data from file."""
page = int(request.args.get('page', 1))
per_page = int(request.args.get('per_page', 10))
data = self._load_data_file(filename)
if not data:
return jsonify({'error': 'File not found or empty'}), 404
if isinstance(data, list):
start_idx = (page - 1) * per_page
end_idx = start_idx + per_page
paginated_data = data[start_idx:end_idx]
return jsonify({
'data': paginated_data,
'page': page,
'per_page': per_page,
'total': len(data),
'total_pages': (len(data) + per_page - 1) // per_page
})
else:
return jsonify({
'data': data,
'page': 1,
'per_page': 1,
'total': 1,
'total_pages': 1
})
@self.app.route('/upload', methods=['POST'])
def upload_data():
"""Upload new data to a file."""
try:
filename = request.args.get('filename')
if not filename:
return jsonify({'error': 'filename parameter required'}), 400
data = request.get_json()
if not data:
return jsonify({'error': 'JSON data required'}), 400
filepath = self.data_dir / f"{filename}.json"
with open(filepath, 'w') as f:
json.dump(data, f, indent=2)
return jsonify({
'message': f'Data uploaded successfully to {filename}.json',
'filename': f"{filename}.json",
'size': filepath.stat().st_size
})
except Exception as e:
return jsonify({'error': str(e)}), 500
@self.app.errorhandler(404)
def not_found(error):
return jsonify({'error': 'Endpoint not found'}), 404
@self.app.errorhandler(500)
def internal_error(error):
return jsonify({'error': 'Internal server error'}), 500
def _serve_data_file(self, filename: str) -> Response:
"""Serve data from a specific file."""
# Add .json extension if not present
if not filename.endswith('.json'):
filename += '.json'
data = self._load_data_file(filename)
if data is None:
return jsonify({'error': f'File {filename} not found'}), 404
# Add some variation to timestamps if present
varied_data = self._add_timestamp_variation(data)
return jsonify(varied_data)
def _load_data_file(self, filename: str) -> Optional[Union[Dict, List]]:
"""Load data from a JSON file."""
filepath = self.data_dir / filename
if not filepath.exists():
logger.warning(f"Data file not found: {filepath}")
return None
try:
with open(filepath, 'r', encoding='utf-8') as f:
return json.load(f)
except Exception as e:
logger.error(f"Error loading data file {filepath}: {e}")
return None
def _add_timestamp_variation(self, data: Union[Dict, List]) -> Union[Dict, List]:
"""Add slight variations to timestamps to simulate real data."""
if isinstance(data, dict):
return self._vary_dict_timestamps(data)
elif isinstance(data, list):
return [self._vary_dict_timestamps(item) if isinstance(item, dict) else item for item in data]
else:
return data
def _vary_dict_timestamps(self, data: Dict) -> Dict:
"""Add variation to timestamps in a dictionary."""
varied_data = data.copy()
# Common timestamp fields to vary
timestamp_fields = ['timestamp', 'origin_server_ts', 'created_at', 'updated_at', 'time', 'date']
for key, value in varied_data.items():
if key in timestamp_fields and isinstance(value, (int, float)):
# Add random variation of ±5 minutes for timestamp fields
variation = random.randint(-300000, 300000) # ±5 minutes in milliseconds
varied_data[key] = value + variation
elif key == 'age' and isinstance(value, (int, float)):
# Add small random variation to age fields
variation = random.randint(-1000, 1000)
varied_data[key] = max(0, value + variation)
elif isinstance(value, dict):
varied_data[key] = self._vary_dict_timestamps(value)
elif isinstance(value, list):
varied_data[key] = [
self._vary_dict_timestamps(item) if isinstance(item, dict) else item
for item in value
]
return varied_data
def _get_available_endpoints(self) -> List[str]:
"""Get list of available API endpoints."""
endpoints = [
'/health',
'/data',
'/data/<filename>',
'/random/<filename>',
'/paginated/<filename>',
'/upload',
'/matrix'
]
# Add endpoints for each data file
for file_path in self.data_dir.glob('*.json'):
endpoints.append(f'/data/{file_path.name}')
return endpoints
def _create_example_data(self):
"""Create example data files if they don't exist."""
# Create matrix messages example
matrix_file = self.data_dir / 'matrix_messages.json'
if not matrix_file.exists():
matrix_example = [
{
"chunk": [
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@signal_37c02a2c-31a2-4937-88f2-3f6be48afcdc:matrix.klas.chat",
"content": {
"body": "Viděli jsme dopravní nehodu",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1749927752871,
"unsigned": {
"membership": "join",
"age": 668
},
"event_id": "$k2t8Uj8K9tdtXfuyvnxezL-Gijqb0Bw4rucgZ0rEOgA",
"user_id": "@signal_37c02a2c-31a2-4937-88f2-3f6be48afcdc:matrix.klas.chat",
"age": 668
},
{
"type": "m.room.message",
"room_id": "!xZkScMybPseErYMJDz:matrix.klas.chat",
"sender": "@signal_961d5a74-062f-4f22-88bd-e192a5e7d567:matrix.klas.chat",
"content": {
"body": "BTC fee je: 1 sat/vByte",
"m.mentions": {},
"msgtype": "m.text"
},
"origin_server_ts": 1749905152683,
"unsigned": {
"membership": "join",
"age": 22623802
},
"event_id": "$WYbz0dB8f16PxL9_j0seJbO1tFaSGiiWeDaj8yGEfC8",
"user_id": "@signal_961d5a74-062f-4f22-88bd-e192a5e7d567:matrix.klas.chat",
"age": 22623802
}
],
"start": "t404-16991_0_0_0_0_0_0_0_0_0",
"end": "t395-16926_0_0_0_0_0_0_0_0_0"
}
]
with open(matrix_file, 'w', encoding='utf-8') as f:
json.dump(matrix_example, f, indent=2, ensure_ascii=False)
logger.info(f"Created example matrix messages file: {matrix_file}")
# Create simple test data example
test_file = self.data_dir / 'test_data.json'
if not test_file.exists():
test_data = {
"message": "Hello from mock API",
"timestamp": int(time.time() * 1000),
"items": [
{"id": 1, "name": "Item 1"},
{"id": 2, "name": "Item 2"},
{"id": 3, "name": "Item 3"}
]
}
with open(test_file, 'w') as f:
json.dump(test_data, f, indent=2)
logger.info(f"Created example test data file: {test_file}")
def start(self, debug: bool = False, threaded: bool = True) -> bool:
"""
Start the API server.
Args:
debug: Enable debug mode
threaded: Run in a separate thread
Returns:
True if server started successfully
"""
try:
if self.is_running:
logger.warning("Server is already running")
return False
self.server = make_server(self.host, self.port, self.app, threaded=True)
if threaded:
self.server_thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.server_thread.start()
else:
self.server.serve_forever()
self.is_running = True
logger.info(f"Mock API server started on http://{self.host}:{self.port}")
logger.info(f"Health check: http://{self.host}:{self.port}/health")
logger.info(f"Data files: http://{self.host}:{self.port}/data")
return True
except Exception as e:
logger.error(f"Failed to start server: {e}")
return False
def stop(self):
"""Stop the API server."""
if self.server and self.is_running:
self.server.shutdown()
if self.server_thread:
self.server_thread.join(timeout=5)
self.is_running = False
logger.info("Mock API server stopped")
else:
logger.warning("Server is not running")
def add_data_file(self, filename: str, data: Union[Dict, List]) -> str:
"""
Add a new data file.
Args:
filename: Name of the file (without .json extension)
data: Data to store
Returns:
Path to the created file
"""
if not filename.endswith('.json'):
filename += '.json'
filepath = self.data_dir / filename
with open(filepath, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
logger.info(f"Added data file: {filepath}")
return str(filepath)
def get_server_info(self) -> Dict[str, Any]:
"""Get information about the server."""
return {
'host': self.host,
'port': self.port,
'is_running': self.is_running,
'data_dir': str(self.data_dir),
'base_url': f"http://{self.host}:{self.port}",
'health_url': f"http://{self.host}:{self.port}/health",
'data_files_count': len(list(self.data_dir.glob('*.json'))),
'available_endpoints': self._get_available_endpoints()
}
def create_mock_api_server(data_dir: str = "api_data", host: str = "0.0.0.0", port: int = 5000):
"""Create a mock API server instance."""
return MockAPIServer(data_dir, host, port)
# CLI functionality for standalone usage
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Mock API Server for N8N Testing")
parser.add_argument("--host", default="0.0.0.0", help="Host to bind to")
parser.add_argument("--port", type=int, default=5000, help="Port to bind to")
parser.add_argument("--data-dir", default="api_data", help="Directory for data files")
parser.add_argument("--debug", action="store_true", help="Enable debug mode")
args = parser.parse_args()
# Setup logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
# Create and start server
server = MockAPIServer(args.data_dir, args.host, args.port)
try:
print(f"Starting Mock API Server on http://{args.host}:{args.port}")
print(f"Data directory: {args.data_dir}")
print("Press Ctrl+C to stop")
server.start(debug=args.debug, threaded=False)
except KeyboardInterrupt:
print("\nShutting down server...")
server.stop()
print("Server stopped")