完善前端在线脚本接口

This commit is contained in:
靳中伟 2025-10-01 15:20:55 +08:00
parent 201877f0f0
commit 3d21b08ec5
33 changed files with 33559 additions and 33200 deletions

169
CLAUDE.md
View File

@ -1,169 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
VWED Task Module is a FastAPI-based Python system for managing and executing task workflows for AMR (Autonomous Mobile Robot) scheduling systems. The system provides a low-code configuration tool that allows users to design and configure complex robot task flows through a visual interface.
## Development Commands
### Application Startup
```bash
# Primary method - run the main application
python app.py
# Alternative method - using uvicorn directly
uvicorn app:app --host 0.0.0.0 --port 8000 --reload
```
### Database Operations
```bash
# Run database migrations
python scripts/run_migration.py
# Generate new migrations
python scripts/generate_migration.py
# Initialize database
python scripts/init_db.py
```
### Docker Deployment
```bash
# Build and run with Docker
docker build -t vwed-task:latest .
docker run -d -p 8000:8000 --name vwed-task-container vwed-task:latest
# Or use Docker Compose (recommended)
docker-compose up -d
```
## Architecture Overview
The system follows a layered architecture with clear separation of concerns:
### Core Layers
- **API Layer (`routes/`)**: FastAPI REST endpoints for client communication
- **Middleware Layer (`middlewares/`)**: Request logging, error handling, and cross-cutting concerns
- **Business Logic Layer (`services/`)**: Core business logic including task execution and scheduling
- **Data Layer (`data/`)**: ORM models, database session management, and data persistence
- **Component Layer (`components/`)**: Extensible component system for task workflow building
- **Configuration Layer (`config/`)**: System settings, database configuration, and error mappings
### Key Services
- **Task Execution Engine (`services/execution/`)**: Handles task lifecycle and execution
- `task_executor.py`: Main task execution controller
- `block_executor.py`: Individual task block execution
- `task_context.py`: Execution context and variable management
- `handlers/`: Specific handlers for different component types
- **Enhanced Scheduler (`services/enhanced_scheduler/`)**: High-performance async task scheduling
- **Intelligence Layer (`services/intelligence/`)**: AI-powered features (partial implementation)
### Data Models
Key models in `data/models/`:
- `taskdef.py`: Task definition and configuration
- `taskrecord.py`: Task execution records
- `blockrecord.py`: Individual block execution records
- `tasktemplate.py`: Reusable task templates
- `calldevice.py`: Device integration configurations
## Key Features
### Task Management
- Visual task flow designer with drag-and-drop interface
- Version control for task configurations
- Task templates for reusability
- Real-time execution monitoring
### Component System
Components are registered in `config/components/` and include:
- Foundation components (basic operations)
- Robot scheduling components
- HTTP request components
- Storage location management
- Progress tracking components
- Script execution components
### Device Integration
- Modbus protocol support (`modbusconfig.py`)
- Generic device calling interface (`calldevice.py`)
- Extensible communication protocols
## Development Guidelines
### Database Schema Changes
1. Modify models in `data/models/`
2. Generate migration: `python scripts/generate_migration.py`
3. Apply migration: `python scripts/run_migration.py`
### Adding New Components
1. Define component configuration in `config/components/`
2. Implement handler in `services/execution/handlers/`
3. Register component in the component system
### Task Execution Flow
1. Task definition stored in `taskdef` table
2. Execution creates `taskrecord` entry
3. Individual blocks create `blockrecord` entries
4. Context variables managed through `task_context.py`
5. Component handlers execute specific business logic
### Configuration Management
- Environment-specific settings in `config/settings.py`
- Database configuration in `config/database_config.py`
- Error messages centralized in `config/error_messages.py`
## Important Technical Details
### Async Task Scheduling
The system uses a custom enhanced scheduler (`services/enhanced_scheduler/`) that:
- Maintains worker pools for concurrent task execution
- Provides priority-based task queuing
- Handles task persistence and recovery
- Manages worker lifecycle
### Component Architecture
Components follow a registry pattern:
- Each component type has a handler class
- Handlers implement standard execution interface
- Components are configurable through JSON definitions
- Extensible for new component types
### Database Session Management
- Uses SQLAlchemy ORM with session management in `data/session.py`
- Supports both sync and async database operations
- Automatic connection pooling and cleanup
## Testing and Debugging
### Test Files
- Basic tests in `tests/` directory
- Test data and fixtures available
- Integration tests for API endpoints
### Logging
- Centralized logging configuration in `utils/logger.py`
- Application logs stored in `logs/` directory
- Structured logging for debugging task execution
### API Documentation
- Swagger UI available at `http://localhost:8000/docs`
- Comprehensive API documentation in `VWED任务模块接口文档/`
## Common Troubleshooting
### Database Issues
- Check database connection in `config/database_config.py`
- Verify database migrations are up to date
- Review logs in `logs/app.log`
### Task Execution Problems
- Monitor task status through API endpoints
- Check execution logs for specific error messages
- Verify component configurations are correct
### Performance Optimization
- Adjust scheduler worker counts in settings
- Monitor database connection pool usage
- Review task complexity and component efficiency

Binary file not shown.

34
app.py
View File

@ -38,29 +38,29 @@ async def lifespan(app: FastAPI):
logger.info("库区锁管理器已初始化")
# 启动设备处理服务和MQTT服务
# try:
# from services.online_script.device_handler_service import get_device_service
# device_service = get_device_service()
# await device_service.start_service()
# logger.info("设备处理服务已启动")
# except Exception as e:
# logger.error(f"启动设备处理服务失败: {e}", exc_info=True)
# # 即使MQTT服务启动失败也不中断应用启动
# logger.warning("设备处理服务启动失败,但应用将继续运行")
try:
from services.online_script.device_handler_service import get_device_service
device_service = get_device_service()
await device_service.start_service()
logger.info("设备处理服务已启动")
except Exception as e:
logger.error(f"启动设备处理服务失败: {e}", exc_info=True)
# 即使MQTT服务启动失败也不中断应用启动
logger.warning("设备处理服务启动失败,但应用将继续运行")
yield
# 应用程序关闭前的清理操作
logger.info("应用程序关闭中...")
# # 停止设备处理服务
# try:
# from services.online_script.device_handler_service import get_device_service
# device_service = get_device_service()
# await device_service.stop_service()
# logger.info("设备处理服务已停止")
# except Exception as e:
# logger.error(f"停止设备处理服务失败: {e}", exc_info=True)
# 停止设备处理服务
try:
from services.online_script.device_handler_service import get_device_service
device_service = get_device_service()
await device_service.stop_service()
logger.info("设备处理服务已停止")
except Exception as e:
logger.error(f"停止设备处理服务失败: {e}", exc_info=True)
# 停止增强版任务调度器
from services.enhanced_scheduler import scheduler

View File

@ -91,29 +91,74 @@ DG_ID = os.getenv("DG_ID", "e22cacb4-a580-45ba-949e-356f57fa1a43")
MQTT_TOPIC_CONFIG = {
# 实例ID配置
"instance_id": os.getenv("MQTT_INSTANCE_ID", "asbm2"),
# 支持的设备品牌配置
# 支持的设备品牌配置 - 按设备类型分类
"supported_brands": {
"huarui": {
"name": "华瑞",
"brand_suffix": "_IRAYPLE",
"oagv_version": "v2",
"uagv_version": "v2"
# 小车(vehicle)设备品牌
"vehicle": {
"huarui": {
"name": "华瑞",
"brand_suffix": "_IRAYPLE",
"oagv_version": "v2",
"uagv_version": "v2",
"device_type": "vehicle"
},
"seer": {
"name": "仙工",
"brand_suffix": "_SEER",
"oagv_version": "v2",
"uagv_version": "v2",
"device_type": "vehicle"
},
},
"seer": {
"name": "仙工",
"brand_suffix": "_SEER",
"oagv_version": "v2",
"uagv_version": "v2"
# 其他设备品牌(门、光电传感器、呼叫器等)
"other": {
"door": {
"name": "门控制器",
"brand_suffix": "_VWED",
"oagv_version": "v2",
"uagv_version": "2.0.0",
"device_type": "door"
},
"sensor": {
"name": "光电传感器",
"brand_suffix": "_SENSOR",
"oagv_version": "v2",
"uagv_version": "v2",
"device_type": "sensor"
},
# "caller": {
# "name": "呼叫器",
# "brand_suffix": "_CALLER",
# "oagv_version": "v2",
# "uagv_version": "v2",
# "device_type": "caller"
# },
# "lift": {
# "name": "电梯",
# "brand_suffix": "_LIFT",
# "oagv_version": "v2",
# "uagv_version": "v2",
# "device_type": "lift"
# },
# "conveyor": {
# "name": "输送带",
# "brand_suffix": "_CONVEYOR",
# "oagv_version": "v2",
# "uagv_version": "v2",
# "device_type": "conveyor"
# }
}
},
# 默认监听的指令类型
"default_command_types": {
"vehicle": ["order", "instantActions", "factsheet", "state"],
# vehicle类型设备默认监听的指令小车专有指令
"vehicle": ["order", "instantActions", "factsheet"],
# other类型设备默认监听的指令其他设备只支持即时动作
"other": ["instantActions"]
},
# 指令方向配置
"command_directions": {
"order": {"listen": "oagv", "forward": "uagv"},

File diff suppressed because it is too large Load Diff

View File

@ -1,730 +0,0 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
设备处理器快速入门示例
演示新的简化批量注册接口 - 自动生成MQTT topics
支持华瑞仙工等品牌小车的批量注册和自动消息转发
基于device_ids的通用设备注册方法
"""
def boot():
"""脚本启动函数 - 演示新的简化接口"""
print("=== 设备处理器快速入门示例 ===")
# 1. 最简单的自动透传(推荐用法)
example_1_auto_forward()
# 2. 自定义处理器
example_2_custom_handler()
# 3. 指定特定指令类型
example_3_specific_commands()
# 4. 仙工品牌示例
example_4_seer_vehicles()
# 5. 混合设备类型示例
example_5_mixed_device_types()
print("=== 设备处理器注册完成 ===")
def example_1_auto_forward():
"""示例1: 最简单的自动透传(推荐用法)"""
print("1. 批量注册华瑞小车自动透传")
# 批量注册华瑞小车系统自动生成topics和透传逻辑
VWED.device.register_and_run(
device_ids=["AGV001", "AGV002", "AGV003"],
device_type="vehicle",
brand_name="huarui",
script_id="example_script_001",
description="华瑞小车批量自动透传"
)
print("华瑞小车批量注册成功")
# 系统会自动:
# 监听: oagv/v2/VWED_001_IRAYPLE/+/order 等
# 转发: uagv/v2.0.0/IRAYPLE/AGV001/order 等
def example_2_custom_handler():
"""示例2: 自定义处理器"""
print("2. 自定义华瑞小车处理器")
def my_custom_handler(message):
"""自定义消息处理器"""
print(f"收到消息: {message.topic}")
print(f"载荷: {message.payload}")
# 可以修改消息内容
modified_payload = message.payload.copy()
modified_payload["processed_by"] = "my_script"
modified_payload["processed_at"] = VWED.util.now()
# 返回处理结果
return {
"forward": True, # 是否转发
"payload": modified_payload, # 修改后的载荷
"topics": message.target_topics # 转发到的topics
}
# 使用自定义处理器批量注册
VWED.device.register_and_run(
device_ids=["AGV001", "AGV002"],
device_type="vehicle",
brand_name="huarui",
handler=my_custom_handler,
script_id="example_script_002",
description="自定义华瑞小车处理器"
)
def example_3_specific_commands():
"""示例3: 指定特定指令类型"""
print("3. 指定特定指令类型")
def order_and_state_handler(message):
"""只处理order和state指令"""
command_type = message.topic.split('/')[-1]
if command_type == "order":
print(f"处理订单指令: {message.payload}")
elif command_type == "state":
print(f"处理状态消息: {message.payload}")
return {
"forward": True,
"payload": message.payload
}
# 只监听和转发order和state指令
VWED.device.register_and_run(
device_ids=["AGV001"],
device_type="vehicle",
brand_name="seer",
command_types=["order", "state"], # 只处理这两种指令
handler=order_and_state_handler,
)
def example_4_seer_vehicles():
"""示例4: 仙工品牌小车处理"""
print("4. 仙工品牌小车处理")
def seer_handler(message):
"""仙工小车处理器"""
print(f"仙工小车消息: {message.topic}")
# 根据消息类型进行不同处理
command_type = message.topic.split('/')[-1]
if command_type == "state":
# 状态消息特殊处理
battery_level = message.payload.get("batteryLevel", 100)
if battery_level < 20:
print(f"警告:小车电量低 {battery_level}%")
return {"forward": True, "payload": message.payload}
# 批量注册仙工小车处理器
VWED.device.register_and_run(
device_ids=["SEER_001", "SEER_002"],
device_type="vehicle",
brand_name="seer",
handler=seer_handler,
script_id="example_script_004",
description="仙工小车处理器"
)
def example_5_mixed_device_types():
"""示例5: 混合设备类型批量注册"""
print("5. 混合设备类型批量注册")
def mixed_handler(message):
"""混合设备处理器"""
print(f"混合设备处理器收到消息: {message.topic}")
device_id = message.topic.split('/')[-2] # 提取设备ID
print(f"设备 {device_id} 发送消息")
return {"forward": True, "payload": message.payload}
# 批量注册多个设备(包括小车和其他设备类型)
VWED.device.register_and_run(
device_ids=["AGV_001", "DOOR_001", "CALLER_001"],
device_type="vehicle", # 主要设备类型
brand_name="huarui",
handler=mixed_handler,
script_id="example_script_005",
description="混合设备批量处理器"
)
def register_door_handler():
"""注册门禁处理器"""
VWED.device.register_and_run(
device_ids=["door_demo_main"],
device_type="door",
handler=door_access_controller,
script_id="door_demo_001",
description="演示门禁访问控制器"
)
def register_caller_handler():
"""注册呼叫器处理器"""
VWED.device.register_and_run(
device_ids=["caller_demo_lobby"],
device_type="caller",
handler=caller_signal_processor,
script_id="caller_demo_001",
description="演示呼叫器信号处理器"
)
def register_sensor_handler():
"""注册传感器处理器"""
VWED.device.register_and_run(
device_ids=["sensor_demo_env"],
device_type="sensor",
handler=environmental_processor,
script_id="sensor_demo_001",
description="演示环境传感器处理器"
)
# ==================== 设备处理函数 ====================
def agv_command_processor(message):
"""AGV命令处理函数 - MQTT消息到达时自动执行"""
VWED.log.sync_info(f"AGV收到命令: {message.payload}")
command = message.payload.get("command")
if command == "move":
# 移动命令 - 添加安全检查
x = message.payload.get("x", 0)
y = message.payload.get("y", 0)
speed = message.payload.get("speed", 100)
# 限制最大速度为50
safe_speed = min(speed, 50)
# 构建安全移动命令
safe_command = {
"command": "move_safe",
"x": x,
"y": y,
"speed": safe_speed,
"safety_check": True,
"original_speed": speed,
"processed_by": "agv_demo_001",
"timestamp": VWED.util.now()
}
if speed != safe_speed:
VWED.log.sync_warning(f"AGV速度已限制: {speed} -> {safe_speed}")
VWED.log.sync_info(f"AGV安全命令生成: {safe_command}")
return {
"forward": True,
"payload": safe_command
}
elif command == "stop":
# 停止命令 - 直接转发
stop_command = {
"command": "emergency_stop",
"reason": message.payload.get("reason", "manual"),
"timestamp": VWED.util.now()
}
VWED.log.sync_info("AGV紧急停止命令")
return {
"forward": True,
"payload": stop_command
}
else:
# 未知命令
VWED.log.sync_warning(f"AGV未知命令: {command}")
return {
"forward": False,
"response": {
"status": "error",
"message": f"不支持的命令: {command}"
},
"response_topic": "demo/agv/response"
}
def huarui_agv_processor(message):
"""华睿AGV处理函数 - 支持华睿协议自动编码/解码"""
VWED.log.sync_info(f"华睿AGV收到消息: {message.payload}")
# 由于启用了auto_encode=True华睿协议已自动解码message.payload
# 这里收到的是标准格式的数据
action = message.payload.get("action")
if action == "status_update":
# 状态更新 - 添加业务逻辑处理
status = message.payload.get("status")
position = message.payload.get("position", {})
battery = message.payload.get("battery", 0)
VWED.log.sync_info(f"华睿AGV状态: {status}, 位置: {position}, 电量: {battery}%")
# 低电量预警
if battery < 20:
VWED.log.sync_warning(f"华睿AGV电量不足: {battery}%")
# 发送充电指令(会自动编码为华睿协议格式)
return {
"forward": True,
"payload": {
"action": "move",
"x": 0, # 充电桩位置
"y": 0,
"speed": 30,
"task_id": f"charge_{VWED.util.timestamp()}"
}
}
# 正常状态,无需特殊处理
return {"forward": False}
else:
VWED.log.sync_info(f"华睿AGV消息已处理: {action}")
return {"forward": False}
def seer_agv_processor(message):
"""仙工AGV处理函数 - 支持仙工协议自动编码/解码"""
VWED.log.sync_info(f"仙工AGV收到消息: {message.payload}")
# 仙工协议已自动解码,这里处理标准格式数据
action = message.payload.get("action")
if action == "status_update":
status = message.payload.get("status")
position = message.payload.get("position", {})
velocity = message.payload.get("velocity", 0)
VWED.log.sync_info(f"仙工AGV状态: {status}, 位置: {position}, 速度: {velocity}")
# 仙工特定的逻辑处理
if status == "error":
VWED.log.sync_error("仙工AGV出现错误发送重启指令")
# 发送重启指令(会自动编码为仙工协议格式)
return {
"forward": True,
"payload": {
"action": "stop",
"reason": "error_recovery"
}
}
return {"forward": False}
else:
VWED.log.sync_info(f"仙工AGV消息已处理: {action}")
return {"forward": False}
def door_access_controller(message):
"""门禁访问控制函数"""
card_id = message.payload.get("card_id")
door_id = message.payload.get("door_id", "main")
VWED.log.sync_info(f"门禁访问请求: 卡号={card_id}, 门={door_id}")
# 简单的白名单验证
whitelist = ["DEMO001", "DEMO002", "DEMO003"]
if card_id in whitelist:
# 授权开门
open_command = {
"action": "open",
"door_id": door_id,
"card_id": card_id,
"authorized": True,
"open_duration": 5, # 5秒
"timestamp": VWED.util.now()
}
VWED.log.sync_info(f"门禁访问已授权: {card_id}")
return {
"forward": True,
"payload": open_command
}
else:
# 拒绝访问
VWED.log.sync_warning(f"门禁访问被拒绝: {card_id}")
return {
"forward": False,
"response": {
"status": "denied",
"card_id": card_id,
"reason": "unauthorized",
"timestamp": VWED.util.now()
},
"response_topic": "demo/door/response"
}
def caller_signal_processor(message):
"""呼叫器信号处理函数"""
caller_id = message.payload.get("caller_id")
signal_type = message.payload.get("signal_type", "service")
location = message.payload.get("location", "unknown")
VWED.log.sync_info(f"呼叫信号: {caller_id} - {signal_type} @ {location}")
if signal_type == "emergency":
# 紧急呼叫
response = {
"type": "emergency_response",
"caller_id": caller_id,
"location": location,
"status": "received",
"response_team": "security",
"estimated_time": "2 minutes",
"message": "紧急响应团队已派遣",
"timestamp": VWED.util.now()
}
VWED.log.sync_error(f"紧急呼叫处理: {caller_id}")
elif signal_type == "service":
# 服务请求
response = {
"type": "service_response",
"caller_id": caller_id,
"location": location,
"status": "queued",
"queue_position": 1,
"estimated_time": "5 minutes",
"message": "服务请求已接收",
"timestamp": VWED.util.now()
}
VWED.log.sync_info(f"服务请求处理: {caller_id}")
else:
# 其他类型
response = {
"type": "info_response",
"caller_id": caller_id,
"status": "received",
"message": "请求已收到",
"timestamp": VWED.util.now()
}
return {
"forward": True,
"payload": response
}
def environmental_processor(message):
"""环境传感器处理函数"""
sensor_id = message.payload.get("sensor_id")
temperature = message.payload.get("temperature")
humidity = message.payload.get("humidity")
VWED.log.sync_info(f"环境数据: 传感器={sensor_id}, 温度={temperature}°C, 湿度={humidity}%")
control_commands = []
# 温度控制逻辑
if temperature is not None:
if temperature > 26:
control_commands.append({
"device": "ac",
"action": "cool",
"target_temp": 24,
"reason": f"温度过高: {temperature}°C"
})
VWED.log.sync_info(f"启动制冷: 目标温度24°C")
elif temperature < 20:
control_commands.append({
"device": "heater",
"action": "heat",
"target_temp": 22,
"reason": f"温度过低: {temperature}°C"
})
VWED.log.sync_info(f"启动加热: 目标温度22°C")
# 湿度控制逻辑
if humidity is not None and humidity > 70:
control_commands.append({
"device": "dehumidifier",
"action": "on",
"target_humidity": 60,
"reason": f"湿度过高: {humidity}%"
})
VWED.log.sync_info(f"启动除湿: 目标湿度60%")
if control_commands:
# 有控制指令需要发送
hvac_control = {
"sensor_id": sensor_id,
"controls": control_commands,
"timestamp": VWED.util.now()
}
return {
"forward": True,
"payload": hvac_control
}
else:
# 环境正常,不需要控制
VWED.log.sync_info("环境参数正常,无需调整")
return {"forward": False}
# ==================== 测试和管理函数 ====================
def test_device_handlers():
"""测试设备处理器功能"""
VWED.log.sync_info("开始测试设备处理器...")
# 测试AGV
VWED.device.sync_publish_message(
topic="demo/agv/command",
payload={
"command": "move",
"x": 100,
"y": 200,
"speed": 80
}
)
# 测试门禁
VWED.device.sync_publish_message(
topic="demo/door/access_request",
payload={
"card_id": "DEMO001",
"door_id": "main_entrance"
}
)
# 测试呼叫器
VWED.device.sync_publish_message(
topic="demo/caller/signal",
payload={
"caller_id": "LOBBY_01",
"signal_type": "service",
"location": "main_lobby"
}
)
# 测试传感器
VWED.device.sync_publish_message(
topic="demo/sensor/data",
payload={
"sensor_id": "ENV_001",
"temperature": 28,
"humidity": 75
}
)
VWED.log.sync_info("设备处理器测试完成")
def get_device_status():
"""获取所有设备处理器状态"""
handlers = VWED.device.get_running_handlers()
VWED.log.sync_info("=== 设备处理器状态 ===")
VWED.log.sync_info(f"设备总数: {handlers.get('device_count', 0)}")
VWED.log.sync_info(f"运行中: {handlers.get('running_devices', 0)}")
devices = handlers.get("devices", {})
for device_id, status in devices.items():
VWED.log.sync_info(f"设备 {device_id}: 消息处理 {status.get('total_messages', 0)}")
return handlers
# ==================== 工具函数 ====================
def stop_all_handlers():
"""停止所有设备处理器(用于测试)"""
handlers = ["agv_demo_001", "door_demo_main", "caller_demo_lobby", "sensor_demo_env"]
for device_id in handlers:
try:
VWED.device.stop_handler(device_id)
VWED.log.sync_info(f"已停止设备处理器: {device_id}")
except Exception as e:
VWED.log.sync_warning(f"停止设备处理器失败 {device_id}: {e}")
def show_device_types():
"""显示支持的设备类型"""
device_types = VWED.device.get_device_types()
VWED.log.sync_info(f"支持的设备类型: {', '.join(device_types)}")
return device_types
def show_device_brands():
"""显示支持的设备品牌"""
device_brands = VWED.device.get_device_brands()
VWED.log.sync_info(f"支持的设备品牌: {', '.join(device_brands)}")
return device_brands
def show_available_protocols():
"""显示所有可用的协议"""
protocols = VWED.device.get_protocols()
VWED.log.sync_info("=== 可用的设备协议 ===")
for protocol_key, protocol_info in protocols.items():
brand = protocol_info.get("brand", "unknown")
device_type = protocol_info.get("device_type", "unknown")
commands = protocol_info.get("supported_commands", [])
VWED.log.sync_info(f"协议: {protocol_key}")
VWED.log.sync_info(f" 品牌: {brand}")
VWED.log.sync_info(f" 设备类型: {device_type}")
VWED.log.sync_info(f" 支持指令: {', '.join(commands)}")
VWED.log.sync_info("")
return protocols
def test_protocol_encoding():
"""测试协议编码功能"""
VWED.log.sync_info("=== 测试协议编码功能 ===")
# 测试华睿协议编码
huarui_test = VWED.device.test_protocol_encoding(
protocol_key="huarui_vehicle",
test_command={
"action": "move",
"x": 100,
"y": 200,
"speed": 50,
"task_id": "test_001"
}
)
if huarui_test["success"]:
VWED.log.sync_info("华睿协议编码测试成功:")
VWED.log.sync_info(f" 原始指令: {huarui_test['original']}")
VWED.log.sync_info(f" 编码后: {huarui_test['encoded']}")
else:
VWED.log.sync_error(f"华睿协议编码测试失败: {huarui_test['error']}")
# 测试仙工协议编码
seer_test = VWED.device.test_protocol_encoding(
protocol_key="seer_vehicle",
test_command={
"action": "move",
"x": 150,
"y": 250,
"speed": 1500, # 仙工使用mm/s
"task_id": "test_002"
}
)
if seer_test["success"]:
VWED.log.sync_info("仙工协议编码测试成功:")
VWED.log.sync_info(f" 原始指令: {seer_test['original']}")
VWED.log.sync_info(f" 编码后: {seer_test['encoded']}")
else:
VWED.log.sync_error(f"仙工协议编码测试失败: {seer_test['error']}")
return {"huarui": huarui_test, "seer": seer_test}
def register_custom_protocol_example():
"""注册自定义协议示例"""
def my_encode(command):
"""自定义编码函数"""
return {
"header": {
"version": "1.0",
"timestamp": VWED.util.timestamp()
},
"body": {
"cmd_type": command.get("action", "unknown"),
"params": command
}
}
def my_decode(response):
"""自定义解码函数"""
body = response.get("body", {})
return {
"action": "status_update",
"data": body,
"decoded_at": VWED.util.now()
}
# 注册自定义协议
VWED.device.register_custom_protocol(
protocol_key="my_custom_agv",
brand="my_brand",
device_type="vehicle",
encode_func=my_encode,
decode_func=my_decode,
supported_commands=["move", "stop", "pause", "status"]
)
VWED.log.sync_info("自定义协议注册完成")
# 测试自定义协议
test_result = VWED.device.test_protocol_encoding(
protocol_key="my_custom_agv",
test_command={
"action": "move",
"x": 300,
"y": 400
}
)
if test_result["success"]:
VWED.log.sync_info("自定义协议测试成功:")
VWED.log.sync_info(f" 编码结果: {test_result['encoded']}")
else:
VWED.log.sync_error(f"自定义协议测试失败: {test_result['error']}")
return test_result

View File

@ -0,0 +1,567 @@
# 设备处理器模块文档Python 版本)
## 概述
设备处理器模块提供了基于 VDA5050 协议的设备管理框架,支持 MQTT 通信用于监听和处理各类设备AGV小车、门、电梯等的消息并自动转发到指定的 topic。
## 核心概念
### 设备类型 (DeviceType)
系统支持以下设备类型:
- `vehicle`: 小车/AGV
- `door`: 门
- `caller`: 呼叫器
- `lift`: 电梯
- `custom`: 自定义设备
### 设备品牌 (DeviceBrand)
系统支持以下品牌:
- `huarui`: 华睿
- `seer`: 仙工
- `hikrobot`: 海康机器人
- `standard`: 标准协议
- `custom`: 自定义品牌
### 指令类型 (CommandType)
根据设备类型不同,支持的指令类型:
- **小车设备**`order``state``factsheet``instantActions`
- **其他设备**`instantActions`
## API 方法
### 注册并运行设备处理器
```python
VWED.device.register_and_run(
device_ids: List[str],
device_type: str = "vehicle",
brand_name: str = "huarui",
command_type: str = None,
handler: Callable = None,
description: str = ""
)
```
**说明**:批量注册设备处理器,系统会自动:
1. 根据设备品牌和类型生成 MQTT topics
2. 订阅相应的 topics
3. 开始监听消息
4. 消息到达时自动调用处理函数
5. 自动转发处理结果
**参数**
- `device_ids`: List[str] - 设备ID列表必需
- `device_type`: str - 设备类型,默认 "vehicle"
- `brand_name`: str - 设备品牌,默认 "huarui"
- `command_type`: str - 指令类型(必需),如 "order"、"state"、"instantActions"
- `handler`: Callable - 消息处理函数(必需)
- `description`: str - 设备描述信息
**返回值**
- List[str] - 成功注册的设备ID列表
**处理函数格式**
处理函数接收一个 `DeviceMessage` 对象作为参数,并返回一个字典,指示如何处理消息。
```python
def handler(message):
"""
消息处理函数
参数:
message: DeviceMessage对象包含以下属性
- device_id: str - 设备ID
- device_type: DeviceType - 设备类型
- topic: str - 接收消息的topic
- payload: dict - 消息载荷(已解析为字典)
- timestamp: float - 消息时间戳
返回值: dict - 处理结果,支持以下字段:
{
"forward": bool, # 是否转发消息可选默认True
"payload": dict, # 要转发的消息载荷(可选,默认使用原始消息)
"response_topic": str, # 自定义响应topic可选
"response": dict # 自定义响应消息载荷可选需配合response_topic
}
"""
# 处理逻辑
return {"forward": True}
```
**返回值格式约束**
| 字段 | 类型 | 必需 | 说明 |
|------|------|------|------|
| forward | bool | 否 | 是否转发消息,默认 True |
| payload | dict | 否 | 转发的消息载荷,默认使用原始消息 |
| response_topic | str | 否 | 自定义响应的 topic |
| response | dict | 否 | 自定义响应的消息载荷,需配合 response_topic |
**注意事项**
- 返回值必须是字典类型
- `payload``response` 必须是字典类型
- 如果提供 `response`,必须同时提供 `response_topic`
### 使用示例
#### 示例1批量注册华瑞小车处理 order 指令
```python
def agv_order_processor(message):
"""处理AGV订单指令"""
VWED.log.info(f"收到订单: 设备 {message.device_id}")
VWED.log.info(f"订单内容: {message.payload}")
# 提取订单信息
order_id = message.payload.get("orderId")
nodes = message.payload.get("nodes", [])
# 可以在这里添加业务逻辑,比如:
# - 验证订单有效性
# - 记录到数据库
# - 通知其他系统
# 直接透传到设备
return {
"forward": True # 使用原始消息转发
}
# 注册处理器
VWED.device.register_and_run(
device_ids=["AGV001", "AGV002", "AGV003"],
device_type="vehicle",
brand_name="huarui",
command_type="order",
handler=agv_order_processor,
description="华瑞小车order指令处理器"
)
```
#### 示例2修改消息后转发
```python
def agv_state_processor(message):
"""处理AGV状态消息添加额外信息后转发"""
state_data = message.payload
# 添加额外的业务信息
enhanced_state = {
**state_data,
"processed_at": VWED.get_current_time(),
"system_status": "online",
"battery_warning": state_data.get("batteryState", {}).get("batteryCharge", 100) < 20
}
# 转发修改后的消息
return {
"forward": True,
"payload": enhanced_state
}
VWED.device.register_and_run(
device_ids=["AGV001", "AGV002"],
device_type="vehicle",
brand_name="huarui",
command_type="state",
handler=agv_state_processor,
description="AGV状态处理器增强版"
)
```
#### 示例3不转发仅记录
```python
def agv_monitor(message):
"""仅监控AGV消息不转发"""
# 记录到数据库
VWED.db.execute_update(
"INSERT INTO agv_logs (device_id, message, created_at) VALUES (?, ?, ?)",
[message.device_id, str(message.payload), VWED.get_current_time()]
)
# 不转发
return {"forward": False}
VWED.device.register_and_run(
device_ids=["AGV001"],
device_type="vehicle",
brand_name="huarui",
command_type="state",
handler=agv_monitor,
description="AGV监控仅记录"
)
```
#### 示例4发送自定义响应
```python
def door_controller(message):
"""门控制器,接收指令后返回确认"""
action = message.payload.get("instantActions", [])
if action:
action_type = action[0].get("actionType")
# 处理开门/关门指令
if action_type == "openDoor":
VWED.log.info(f"门 {message.device_id} 正在开启...")
result = "success"
elif action_type == "closeDoor":
VWED.log.info(f"门 {message.device_id} 正在关闭...")
result = "success"
else:
result = "unknown_action"
# 不转发原始消息,发送自定义响应
return {
"forward": False,
"response_topic": f"devices/door/{message.device_id}/response",
"response": {
"device_id": message.device_id,
"action": action_type,
"result": result,
"timestamp": VWED.get_current_time()
}
}
return {"forward": False}
VWED.device.register_and_run(
device_ids=["DOOR001", "DOOR002"],
device_type="door",
brand_name="huarui",
command_type="instantActions",
handler=door_controller,
description="门设备控制器"
)
```
#### 示例5条件转发
```python
def smart_forwarder(message):
"""根据条件决定是否转发"""
battery = message.payload.get("batteryState", {}).get("batteryCharge", 100)
# 电量低于20%时不转发,发送告警
if battery < 20:
VWED.log.warning(f"设备 {message.device_id} 电量过低: {battery}%")
# 发送告警消息到告警topic
return {
"forward": False,
"response_topic": "alerts/battery/low",
"response": {
"device_id": message.device_id,
"battery": battery,
"alert_level": "critical",
"timestamp": VWED.get_current_time()
}
}
# 正常转发
return {"forward": True}
VWED.device.register_and_run(
device_ids=["AGV001", "AGV002", "AGV003"],
device_type="vehicle",
brand_name="huarui",
command_type="state",
handler=smart_forwarder,
description="智能转发器(低电量告警)"
)
```
### 停止设备处理器
```python
VWED.device.stop_handler(device_id: str)
```
**说明**:停止指定设备处理器的运行
**参数**
- `device_id`: str - 设备ID
**示例**
```python
# 停止设备处理器
VWED.device.stop_handler("AGV001")
```
### 获取运行中的处理器列表
```python
VWED.device.get_running_handlers() -> Dict[str, Any]
```
**说明**:获取所有正在运行的设备处理器信息
**返回值**:包含所有设备处理器状态的字典
**示例**
```python
handlers = VWED.device.get_running_handlers()
VWED.log.info(f"运行中的处理器: {handlers}")
```
### 获取设备处理器状态
```python
VWED.device.get_handler_status(device_id: str) -> Dict[str, Any]
```
**说明**:获取指定设备处理器的运行状态
**参数**
- `device_id`: str - 设备ID
**返回值**:设备处理器状态信息
**示例**
```python
status = VWED.device.get_handler_status("AGV001")
VWED.log.info(f"设备状态: {status}")
```
### 主动发布MQTT消息
```python
await VWED.device.publish_message(topic: str, payload: Any)
```
**说明**:主动向指定 topic 发布 MQTT 消息(异步方法)
**参数**
- `topic`: str - MQTT topic
- `payload`: dict - 消息载荷
**示例**
```python
# 异步发布
await VWED.device.publish_message(
topic="factory/agv/command",
payload={"action": "move", "x": 100, "y": 200}
)
# 或使用同步方式
VWED.device.sync_publish_message(
topic="factory/agv/command",
payload={"action": "move", "x": 100, "y": 200}
)
```
### 获取支持的设备类型
```python
VWED.device.get_device_types() -> List[str]
```
**说明**:获取系统支持的所有设备类型
**返回值**:设备类型列表
**示例**
```python
types = VWED.device.get_device_types()
VWED.log.info(f"支持的设备类型: {types}")
# 输出: ['vehicle', 'door', 'caller', 'lift', 'conveyor', 'sensor', 'robot', 'camera', 'scanner', 'custom']
```
### 获取支持的设备品牌
```python
VWED.device.get_device_brands() -> List[str]
```
**说明**:获取系统支持的所有设备品牌
**返回值**:设备品牌列表
**示例**
```python
brands = VWED.device.get_device_brands()
VWED.log.info(f"支持的设备品牌: {brands}")
# 输出: ['huarui', 'seer', 'quicktron', 'geek', 'mushiny', 'flashhold', 'hikrobot', 'standard', 'custom']
```
## 完整应用示例
### 场景AGV车队管理系统
```python
# AGV订单处理器
def process_agv_order(message):
"""处理AGV订单"""
order_data = message.payload
order_id = order_data.get("orderId")
# 记录订单到数据库
VWED.db.execute_update(
"INSERT INTO agv_orders (device_id, order_id, data, created_at) VALUES (?, ?, ?, ?)",
[message.device_id, order_id, str(order_data), VWED.get_current_time()]
)
VWED.log.info(f"AGV {message.device_id} 收到订单 {order_id}")
# 透传订单到设备
return {"forward": True}
# AGV状态监控
def monitor_agv_state(message):
"""监控AGV状态并记录关键信息"""
state = message.payload
battery = state.get("batteryState", {}).get("batteryCharge", 100)
position = state.get("agvPosition", {})
errors = state.get("errors", [])
# 更新设备状态到数据库
VWED.db.execute_update(
"UPDATE agv_devices SET battery=?, position_x=?, position_y=?, last_update=? WHERE device_id=?",
[battery, position.get("x"), position.get("y"), VWED.get_current_time(), message.device_id]
)
# 处理错误
if errors:
for error in errors:
VWED.log.error(f"AGV {message.device_id} 错误: {error.get('errorDescription')}")
# 发送告警
VWED.device.sync_publish_message(
topic=f"alerts/agv/{message.device_id}/error",
payload={
"device_id": message.device_id,
"error": error,
"timestamp": VWED.get_current_time()
}
)
# 低电量告警
if battery < 20:
VWED.log.warning(f"AGV {message.device_id} 电量低: {battery}%")
VWED.device.sync_publish_message(
topic=f"alerts/agv/{message.device_id}/battery",
payload={
"device_id": message.device_id,
"battery": battery,
"level": "warning",
"timestamp": VWED.get_current_time()
}
)
# 转发状态
return {"forward": True}
# 注册华瑞AGV处理器
VWED.device.register_and_run(
device_ids=["HR_AGV001", "HR_AGV002", "HR_AGV003"],
device_type="vehicle",
brand_name="huarui",
command_type="order",
handler=process_agv_order,
description="华瑞AGV订单处理器"
)
VWED.device.register_and_run(
device_ids=["HR_AGV001", "HR_AGV002", "HR_AGV003"],
device_type="vehicle",
brand_name="huarui",
command_type="state",
handler=monitor_agv_state,
description="华瑞AGV状态监控"
)
# 注册仙工AGV处理器
VWED.device.register_and_run(
device_ids=["SEER_AGV001", "SEER_AGV002"],
device_type="vehicle",
brand_name="seer",
command_type="order",
handler=process_agv_order,
description="仙工AGV订单处理器"
)
VWED.device.register_and_run(
device_ids=["SEER_AGV001", "SEER_AGV002"],
device_type="vehicle",
brand_name="seer",
command_type="state",
handler=monitor_agv_state,
description="仙工AGV状态监控"
)
VWED.log.info("AGV车队管理系统已启动")
```
## 常见问题
### 1. 如何处理多个品牌的设备?
分别为每个品牌注册处理器,系统会自动根据品牌生成正确的 MQTT topic。
```python
# 华瑞设备
VWED.device.register_and_run(
device_ids=["HR_001", "HR_002"],
brand_name="huarui",
command_type="order",
handler=handler_func
)
# 仙工设备
VWED.device.register_and_run(
device_ids=["SEER_001", "SEER_002"],
brand_name="seer",
command_type="order",
handler=handler_func
)
```
### 2. 一个设备可以注册多个处理器吗?
可以,但需要处理不同的指令类型:
```python
# 处理order指令
VWED.device.register_and_run(
device_ids=["AGV001"],
command_type="order",
handler=order_handler
)
# 处理state指令
VWED.device.register_and_run(
device_ids=["AGV001"],
command_type="state",
handler=state_handler
)
```
### 3. 返回值必须是字典吗?
是的,必须返回字典类型,否则会记录错误日志并忽略该返回值。
### 4. 如何调试消息处理?
使用日志记录消息内容:
```python
def debug_handler(message):
VWED.log.info(f"收到消息:")
VWED.log.info(f" 设备ID: {message.device_id}")
VWED.log.info(f" Topic: {message.topic}")
VWED.log.info(f" 载荷: {message.payload}")
return {"forward": True}
```
## 注意事项
1. **处理函数必须返回字典**:返回值格式必须符合约定
2. **payload 必须是字典类型**:转发和响应的 payload 都必须是字典
3. **批量注册时所有设备使用相同的处理函数**:如果需要不同的处理逻辑,分别注册
4. **处理函数应该快速返回**:避免长时间阻塞,影响其他消息处理
5. **异常处理**:系统会自动捕获异常并记录日志,但建议在处理函数中添加适当的异常处理
6. **资源清理**:脚本停止时会自动清理所有注册的设备处理器

25660
logs/app.log

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

14529
logs/app.log.2025-09-30 Normal file

File diff suppressed because it is too large Load Diff

View File

@ -11,7 +11,7 @@ from fastapi.middleware.cors import CORSMiddleware
from middlewares.request_logger import register_middleware as register_request_logger
from middlewares.error_handlers import register_exception_handlers
from middlewares.response_alert_middleware import register_middleware as register_response_alert
# from middlewares.response_alert_middleware import register_middleware as register_response_alert
from config.settings import settings
def register_middlewares(app: FastAPI):
@ -33,8 +33,8 @@ def register_middlewares(app: FastAPI):
# 注册请求日志中间件
register_request_logger(app)
# 注册响应告警中间件
register_response_alert(app)
# # 注册响应告警中间件
# register_response_alert(app)
# 注册异常处理器
register_exception_handlers(app)

View File

@ -1,12 +1,16 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from pydantic.types import T
async def boot():
"""脚本启动函数"""
url = "opc.tcp://127.0.0.1:4840/freeopcua/server/"
result = VWED.websocket.send_msg_to_wsc_by_client_ip("i=2259", url)
print(result, "=====================")
VWED.device.register_and_run(
device_ids=["SIM-1"],
device_type="vehicle",
brand_name="huarui",
command_type="state",
handler=vda5050_vehicle_handler
)
def vda5050_vehicle_handler(message):
# print(message, "=====================")
pass

View File

@ -34,6 +34,7 @@ from services.enhanced_scheduler.periodic_task_manager import PeriodicTaskManage
from data.enum.task_record_enum import TaskStatus, SourceType
from data.enum.task_def_enum import EnableStatus, PeriodicTaskStatus, TaskStatusEnum
from utils.logger import get_logger
from utils.alert_sync import sync_warning_alert
# 获取日志记录器
logger = get_logger("services.enhanced_scheduler.task_scheduler")
@ -378,8 +379,20 @@ class EnhancedTaskScheduler:
# 如果所有工作线程都在工作中,添加警告信息
use_workers = [worker_id for worker_id, status in self.worker_manager.worker_status.items() if status.get("current_task") is not None]
if len(use_workers) == len(self.worker_manager.worker_status):
response["warning"] = f"警告:所有工作线程({len(self.worker_manager.worker_status)}个)都在忙碌中,任务可能需要等待较长时间"
warning_msg = f"警告:所有工作线程({len(self.worker_manager.worker_status)}个)都在忙碌中,任务可能需要等待较长时间"
response["warning"] = warning_msg
logger.warning(f"提交任务 {task_record_id} 时所有工作线程都在忙碌中,当前工作线程数: {len(self.worker_manager.worker_status)}")
# 推送警告告警到主系统
sync_warning_alert(
logger_name="services.enhanced_scheduler.task_scheduler",
warning_message="所有工作线程都在忙碌中,任务可能需要等待较长时间",
context_info={
"worker_count": len(self.worker_manager.worker_status),
"queue_size": self._get_queue_size(),
"task_record_id": task_record_id
}
)
return response
async def run_task(self, task_def_id: str, params: List[Dict[str, Any]] = None, parent_task_id: str = None,

View File

@ -1428,38 +1428,7 @@ class BlockExecutor:
# 其他基本类型直接返回
return data
# def _get_possible_output_keys(self, block_name: str) -> List[str]:
# """
# 根据块名称获取可能的输出键
#
# Args:
# block_name: 块名称
#
# Returns:
# List[str]: 可能的输出键列表
# """
# # 块类型与可能的输出变量名映射
# output_keys_map = {
# "CreateUuidBp": ["createUuid", "uuid"],
# "CurrentTimeStampBp": ["currentTimeStamp"],
# "TimestampBp": ["timestamp"],
# "StringToJsonObjectBp": ["jsonObject"],
# "StringToJsonArrayBp": ["jsonArray"],
# "JdbcQueryBp": ["jdbcQueryResult"],
# "HttpGetBp": ["httpResponse", "response"],
# "HttpPostBp": ["httpResponse", "response"],
# "ScriptBp": ["scriptResult"],
# # 添加更多块类型和对应的输出变量名
# }
#
# # 从块名中提取类型
# for block_type, keys in output_keys_map.items():
# if block_type in block_name:
# return keys
#
# # 如果没有找到匹配的块类型,返回空列表
# return []
#
async def _parse_task_inputs_reference(self, reference: str) -> Any:
"""
解析任务输入参数引用表达式支持嵌套路径解析 taskInputs.paramName taskInputs.param.nested.value
@ -1544,12 +1513,14 @@ class BlockExecutor:
return self._extract_nested_value(param_value, nested_path)
# 如果没有找到匹配的参数名
logger.warning(f"在任务输入参数中找不到参数: {param_name}")
return None
logger.error(f"在任务输入参数中找不到参数: {param_name}")
raise Exception(f"在任务输入参数中找不到参数: {param_name}")
# return None
except json.JSONDecodeError as e:
logger.error(f"解析input_params JSON失败: {str(e)}")
return None
raise Exception(f"在任务输入参数中找不到参数: {param_name}")
# return None
except Exception as e:
logger.error(f"获取任务输入参数 {reference} 失败: {str(e)}")

View File

@ -110,206 +110,6 @@ def get_block_handler(block_type: str) -> Optional[BlockHandler]:
"""
return _block_handlers.get(block_type)
# # =============================================子任务组件类========================================================
# # 子任务块处理器
# @register_handler("SubTaskBp")
# class SubTaskBlockHandler(BlockHandler):
# """子任务块处理器"""
#
# async def execute(
# self,
# block: Dict[str, Any],
# input_params: Dict[str, Any],
# context: TaskContext
# ) -> Dict[str, Any]:
# """执行子任务块"""
# from sqlalchemy import select, insert
# from data.models.taskdef import VWEDTaskDef
# from data.models.taskrecord import VWEDTaskRecord
# from data.session import get_async_session
# from services.execution.task_executor import TaskExecutor
# import uuid
# import asyncio
#
# try:
# # 获取子任务ID
# subtask_id = block.get("refTaskDefId")
#
# if not subtask_id:
# result = {
# "success": False,
# "message": "缺少子任务ID"
# }
# await self._record_task_log(block, result, context)
# return result
#
# # 获取执行参数(确保默认值)
# is_async = input_params.get("ifAsync", False) # 是否异步执行默认False
# specified_task_record_id = input_params.get("taskRecordId", None) # 指定的任务记录ID默认None
#
# # 从input_params中提取出子任务需要的参数
# # 排除控制参数
# control_params = ["ifAsync", "taskRecordId"]
# subtask_params = {}
# for key, value in input_params.items():
# if key not in control_params:
# subtask_params[key] = value
#
# logger.info(f"开始执行子任务: {subtask_id}, 异步执行: {is_async}, 指定ID: {specified_task_record_id}")
#
# # 确定任务记录ID
# task_record_id = None
# if specified_task_record_id:
# # 检查指定的任务记录是否存在
# async with get_async_session() as session:
# result = await session.execute(
# select(VWEDTaskRecord).where(VWEDTaskRecord.id == specified_task_record_id)
# )
# existing_record = result.scalars().first()
#
# if existing_record:
# # 如果指定的任务记录ID已存在则报错
# result = {
# "success": False,
# "message": f"指定的任务记录ID已存在: {specified_task_record_id}"
# }
# await self._record_task_log(block, result, context)
# return result
# else:
# # 使用指定的ID创建新任务
# task_record_id = specified_task_record_id
# logger.info(f"使用指定ID创建新任务记录: {task_record_id}")
#
# if not task_record_id:
# # 如果未指定ID或指定的ID不存在生成新ID
# task_record_id = str(uuid.uuid4())
# logger.info(f"使用生成的任务记录ID: {task_record_id}")
#
# # 获取根任务记录ID - 如果当前任务有父任务,则根任务是父任务的根任务,否则是当前任务
# root_task_record_id = context.get_variable("rootTaskRecordId", context.task_record_id)
#
# # 查询任务定义
# async with get_async_session() as session:
# result = await session.execute(
# select(VWEDTaskDef).where(VWEDTaskDef.id == subtask_id)
# )
# task_def = result.scalars().first()
#
# if not task_def:
# result = {
# "success": False,
# "message": f"找不到子任务定义: {subtask_id}"
# }
# await self._record_task_log(block, result, context)
# return result
#
# # 获取任务定义详情
# task_def_detail = task_def.detail
#
# # 创建任务记录 - 设置所有关键参数
# record_values = {
# "id": task_record_id,
# "def_id": subtask_id,
# "def_label": task_def.label,
# "def_version": task_def.version,
# "created_on": datetime.now(),
# "status": 0, # 初始状态
# "input_params": json.dumps(subtask_params, ensure_ascii=False),
# "parent_task_record_id": context.task_record_id, # 设置父任务ID
# "root_task_record_id": root_task_record_id, # 设置根任务ID
# "periodic_task": task_def.periodic_task if hasattr(task_def, "periodic_task") else 0, # 继承周期任务属性
# "task_def_detail": task_def_detail # 设置任务定义详情
# }
#
# # 执行插入
# await session.execute(
# insert(VWEDTaskRecord).values(record_values)
# )
# await session.commit()
#
# # 创建任务执行器
# executor = TaskExecutor(task_record_id)
#
# # 根据是否异步执行采取不同的处理方式
# if is_async:
# # 异步执行:启动任务但不等待结果
# # 初始化执行器
# init_success = await executor.initialize()
# if not init_success:
# result = {
# "success": False,
# "message": f"初始化子任务执行器失败: {executor.error_message}"
# }
# await self._record_task_log(block, result, context)
# return result
#
# # 创建异步任务但不等待
# asyncio.create_task(executor.execute())
#
# # 执行完成后记录日志
# result = {
# "success": True,
# "message": f"子任务已异步启动: {task_record_id}",
# "output": {
# "subtaskId": subtask_id,
# "taskRecordId": task_record_id,
# "async": True
# }
# }
# await self._record_task_log(block, result, context)
# return result
# else:
# # 同步执行:等待任务执行完成后返回结果
# # 初始化执行器
# init_success = await executor.initialize()
# if not init_success:
# result = {
# "success": False,
# "message": f"初始化子任务执行器失败: {executor.error_message}"
# }
# await self._record_task_log(block, result, context)
# return result
#
# # 执行子任务并等待结果
# execute_result = await executor.execute()
#
# # 处理子任务执行结果
# success = execute_result.get("success", False)
# message = execute_result.get("message", "子任务执行完成")
#
# # 记录子任务结果到上下文变量
# context.set_variable("subtaskResult", execute_result)
#
# # 获取子任务输出
# output = execute_result.get("result", {})
#
# # 将子任务输出合并到当前上下文变量
# if isinstance(output, dict):
# for key, value in output.items():
# variable_name = f"subtask_{key}"
# context.set_variable(variable_name, value)
#
# # 执行完成后记录日志
# result = {
# "success": success,
# "message": message,
# "output": {
# "subtaskId": subtask_id,
# "taskRecordId": task_record_id,
# "async": False,
# "output": output
# }
# }
# await self._record_task_log(block, result, context)
# return result
# except Exception as e:
# result = {
# "success": False,
# "message": f"子任务执行异常: {str(e)}"
# }
# await self._record_task_log(block, result, context)
# return result
# 根块处理器
@register_handler("RootBp")

View File

@ -25,6 +25,7 @@ from data.enum.task_record_enum import TaskStatus
from utils.logger import get_logger
from services.sync_service import set_task_in_progress, set_task_completed, set_task_failed, set_task_terminated, \
refresh_token_if_needed
from utils.alert_sync import sync_error_alert
# 获取日志记录器
logger = get_logger("services.execution.task_executor")
@ -219,16 +220,31 @@ class TaskExecutor:
site_list = [v.get("siteId") for k, v in dict(self.task_context.block_outputs).items()]
except Exception as e:
logger.error(f"任务 {self.task_record_id} 执行异常: {str(e)}")
# 推送错误告警到主系统
sync_error_alert(
logger_name="services.execution.task_executor",
error_message=f"任务 {self.task_record_id} 执行异常: {str(e)}",
level=logging.ERROR,
context_info={
"task_id": self.task_record_id,
"filename": "task_executor.py",
"lineno": 221,
"funcName": "execute"
}
)
# 更新任务状态为失败
async with get_async_session() as session:
await self._update_task_status(
session,
TaskStatus.FAILED,
session,
TaskStatus.FAILED,
f"任务执行异常: {str(e)}",
task_detail=self.task_def.detail
)
# 根据release_sites配置决定是否释放库位
if release_sites == 1 and site_list:
logger.info(f"具备自动释放库位锁定条件!释放一下库位:{site_list}")
await self._release_storage_locations(site_list)
@ -285,10 +301,24 @@ class TaskExecutor:
}
else:
error_msg = result.get("message", "任务执行失败")
if result.get("is_canceled", False):
await self._update_task_status(session, TaskStatus.CANCELED, error_msg, executor_time, task_detail=self.task_def.detail)
else:
# 推送错误告警到主系统
sync_error_alert(
logger_name="services.execution.task_executor",
error_message=f"任务 {self.task_record_id} 执行失败: {error_msg}",
level=logging.ERROR,
context_info={
"task_id": self.task_record_id,
"filename": "task_executor.py",
"lineno": 310,
"funcName": "execute",
"error_detail": result.get("error", {})
}
)
# 更新任务状态为失败
await self._update_task_status(session, TaskStatus.FAILED, error_msg, executor_time, task_detail=self.task_def.detail)
@ -320,7 +350,19 @@ class TaskExecutor:
site_list = [v.get("siteId") for k, v in dict(self.task_context.block_outputs).items() if v.get("siteId")]
except:
pass
# 推送错误告警到主系统
sync_error_alert(
logger_name="services.execution.task_executor",
error_message=f"任务 {self.task_record_id} 执行失败: {error_msg}",
level=logging.ERROR,
context_info={
"task_id": self.task_record_id,
"filename": "task_executor.py",
"lineno": 310,
"funcName": "execute",
"error_detail": result.get("error", {})
}
)
# 根据release_sites配置决定是否释放库位
if hasattr(self.task_def, 'release_sites') and self.task_def.release_sites == 1 and site_list:
logger.info(f"具备自动释放库位锁定条件!释放一下库位:{site_list}")

View File

@ -115,36 +115,7 @@ class VWEDDeviceModule:
brand_info = f" (品牌: {brand_name})" if brand_name else ""
self.logger.info(f"设备处理器批量注册成功: {len(registered_device_ids)}{device_type.value} 设备{brand_info}")
return registered_device_ids
def register_single_device(self, device_id: str, device_type: Union[str, DeviceType],
listen_topics: List[str], forward_topics: List[str] = None,
handler: Callable = None, description: str = "",
device_brand: str = None, protocol_key: str = None,
auto_encode: bool = True, **kwargs):
"""兼容旧接口:注册单个设备处理器
这是为了兼容旧版本代码而保留的接口建议使用新的register_and_run方法
"""
if handler is None:
raise ValueError("handler参数不能为空")
if not listen_topics:
raise ValueError("listen_topics参数不能为空")
# 这里可以实现旧版本的直接topic注册逻辑
# 为了简化,我们可以发出警告并建议用户升级到新接口
self.logger.warning(f"使用了已弃用的register_single_device接口建议升级到新的register_and_run接口")
# 简单实现:调用新接口
return self.register_and_run(
device_ids=[device_id],
device_type=device_type,
brand_name=device_brand or "huarui",
handler=handler,
description=description,
**kwargs
)
def stop_handler(self, device_id: str):
"""停止指定设备处理器的运行"""
self.device_service.stop_handler(device_id)
@ -226,101 +197,4 @@ class VWEDDeviceModule:
def get_device_brands(self) -> List[str]:
"""获取支持的设备品牌列表"""
return [db.value for db in DeviceBrand]
def get_protocols(self) -> Dict[str, Dict[str, Any]]:
"""获取所有可用的协议"""
protocol_registry = get_protocol_registry()
return protocol_registry.list_protocols()
def register_custom_protocol(self, protocol_key: str, brand: str, device_type: str,
encode_func: Callable, decode_func: Callable,
supported_commands: List[str] = None):
"""注册自定义设备协议
使用方式
def my_encode(command):
# 自定义编码逻辑
return {"custom_format": command}
def my_decode(response):
# 自定义解码逻辑
return {"decoded": response}
VWED.device.register_custom_protocol(
protocol_key="my_brand_vehicle",
brand="my_brand",
device_type="vehicle",
encode_func=my_encode,
decode_func=my_decode,
supported_commands=["move", "stop"]
)
"""
protocol_registry = get_protocol_registry()
protocol_registry.register_custom_protocol(
protocol_key, brand, device_type,
encode_func, decode_func, supported_commands
)
self.logger.info(f"自定义协议已注册: {protocol_key} ({brand}_{device_type})")
def test_protocol_encoding(self, protocol_key: str, test_command: Dict[str, Any]) -> Dict[str, Any]:
"""测试协议编码功能
使用方式
result = VWED.device.test_protocol_encoding(
protocol_key="huarui_vehicle",
test_command={"action": "move", "x": 100, "y": 200}
)
"""
protocol_registry = get_protocol_registry()
protocol = protocol_registry.get_protocol(protocol_key)
if not protocol:
return {
"success": False,
"error": f"协议不存在: {protocol_key}"
}
try:
encoded = protocol.encode_command(test_command)
return {
"success": True,
"original": test_command,
"encoded": encoded,
"protocol": protocol_key
}
except Exception as e:
return {
"success": False,
"error": f"编码失败: {str(e)}",
"protocol": protocol_key
}
def create_message_filter(self, conditions: Dict[str, Any]) -> Callable:
"""创建消息过滤器
使用方式
filter_func = VWED.device.create_message_filter({
"device_type": "vehicle",
"action": "move"
})
def my_handler(message):
if filter_func(message):
# 处理符合条件的消息
pass
"""
def filter_func(message):
payload = message.payload if hasattr(message, 'payload') else message
for key, expected_value in conditions.items():
if key == "device_type":
actual_value = message.device_type.value if hasattr(message, 'device_type') else None
else:
actual_value = payload.get(key)
if actual_value != expected_value:
return False
return True
return filter_func

View File

@ -18,10 +18,11 @@ logger = logging.getLogger(__name__)
class VWEDS7Module:
"""
S7 PLC通信模块类
"""
def __init__(self):
pass
"""
def __init__(self, script_id: str):
self.script_id = script_id
logger.debug(f"初始化 S7 模块脚本ID: {script_id}")
def _get_rack_slot(self, plc_type: str) -> tuple:
"""

View File

@ -382,121 +382,158 @@ def get_protocol_registry() -> DeviceProtocolRegistry:
class TopicManager:
"""MQTT Topic管理器"""
def __init__(self):
self.topic_config = MQTT_TOPIC_CONFIG
self.instance_id = self.topic_config["instance_id"]
self.supported_brands = self.topic_config["supported_brands"]
self.command_directions = self.topic_config["command_directions"]
# 扁平化品牌配置便于查找
self._flat_brands = {}
for category in ["vehicle", "other"]:
if category in self.supported_brands:
self._flat_brands.update(self.supported_brands[category])
def validate_brand(self, brand_name: str) -> bool:
"""验证品牌是否支持"""
return brand_name.lower() in self.supported_brands
return brand_name.lower() in self._flat_brands
def get_supported_brands(self, category: str = None) -> List[str]:
"""获取支持的品牌列表
Args:
category: 设备类型分类 ("vehicle", "other", None 返回所有)
Returns:
品牌名称列表
"""
if category is None:
# 返回所有品牌
return list(self._flat_brands.keys())
elif category in self.supported_brands:
# 返回指定分类的品牌
return list(self.supported_brands[category].keys())
else:
logger.warning(f"未知的设备类型分类: {category}")
return []
def get_brand_config(self, brand_name: str) -> Optional[Dict[str, Any]]:
"""获取品牌配置
Args:
brand_name: 品牌名称
Returns:
品牌配置字典如果不存在返回None
"""
return self._flat_brands.get(brand_name.lower())
def get_supported_brands(self) -> List[str]:
"""获取支持的品牌列表"""
return list(self.supported_brands.keys())
def generate_listen_topics(self, device_ids: List[str], brand_name: str,
def generate_listen_topics(self, device_ids: List[str], brand_name: str,
command_types: List[str] = None, device_type: DeviceType = DeviceType.VEHICLE) -> List[str]:
"""生成监听topics
Args:
device_ids: 设备ID列表
brand_name: 品牌名称
command_types: 指令类型列表如果为None则使用默认指令
device_type: 设备类型
Returns:
监听topic列表
"""
if not self.validate_brand(brand_name):
raise ValueError(f"不支持的品牌: {brand_name},支持的品牌: {self.get_supported_brands()}")
brand_config = self.supported_brands[brand_name.lower()]
brand_config = self.get_brand_config(brand_name)
if not brand_config:
raise ValueError(f"无法获取品牌配置: {brand_name}")
brand_suffix = brand_config["brand_suffix"]
oagv_version = brand_config["oagv_version"]
if command_types is None:
if device_type == DeviceType.VEHICLE:
command_types = self.topic_config["default_command_types"]["vehicle"]
else:
command_types = self.topic_config["default_command_types"]["other"]
listen_topics = []
for command_type in command_types:
if command_type not in self.command_directions:
logger.warning(f"未知的指令类型: {command_type}")
continue
direction_config = self.command_directions[command_type]
listen_queue = direction_config["listen"]
if listen_queue == "oagv":
# oagv格式: oagv/v2/{instanceId}_IRAYPLE/{deviceId}/order
# oagv格式: oagv/v2/{instanceId}_BRAND/{deviceId}/order
for device_id in device_ids:
topic = f"oagv/{oagv_version}/{self.instance_id}{brand_suffix}/{device_id}/{command_type}"
listen_topics.append(topic)
elif listen_queue == "uagv":
# uagv格式: uagv/v2.0.0/IRAYPLE/{deviceId}/state
# uagv格式: uagv/v2.0.0/BRAND/{deviceId}/state
uagv_version = brand_config["uagv_version"]
brand_name_only = brand_suffix[1:] # 去掉前缀下划线
for device_id in device_ids:
topic = f"uagv/{uagv_version}/{brand_name_only}/{device_id}/{command_type}"
listen_topics.append(topic)
return listen_topics
def generate_forward_topics(self, device_ids: List[str], brand_name: str,
def generate_forward_topics(self, device_ids: List[str], brand_name: str,
command_types: List[str] = None, device_type: DeviceType = DeviceType.VEHICLE) -> List[str]:
"""生成转发topics
Args:
device_ids: 设备ID列表
brand_name: 品牌名称
command_types: 指令类型列表如果为None则使用默认指令
device_type: 设备类型
Returns:
转发topic列表
"""
if not self.validate_brand(brand_name):
raise ValueError(f"不支持的品牌: {brand_name},支持的品牌: {self.get_supported_brands()}")
brand_config = self.supported_brands[brand_name.lower()]
brand_config = self.get_brand_config(brand_name)
if not brand_config:
raise ValueError(f"无法获取品牌配置: {brand_name}")
brand_suffix = brand_config["brand_suffix"]
uagv_version = brand_config["uagv_version"]
oagv_version = brand_config["oagv_version"]
if command_types is None:
if device_type == DeviceType.VEHICLE:
command_types = self.topic_config["default_command_types"]["vehicle"]
else:
command_types = self.topic_config["default_command_types"]["other"]
forward_topics = []
for command_type in command_types:
if command_type not in self.command_directions:
logger.warning(f"未知的指令类型: {command_type}")
continue
direction_config = self.command_directions[command_type]
forward_queue = direction_config["forward"]
if forward_queue == "uagv":
# uagv格式: uagv/v2.0.0/IRAYPLE/{deviceId}/order
# uagv格式: uagv/v2.0.0/BRAND/{deviceId}/order
brand_name_only = brand_suffix[1:] # 去掉前缀下划线
for device_id in device_ids:
topic = f"uagv/{uagv_version}/{brand_name_only}/{device_id}/{command_type}"
forward_topics.append(topic)
elif forward_queue == "oagv":
# oagv格式: oagv/v2/{instanceId}_IRAYPLE/{deviceId}/state
# oagv格式: oagv/v2/{instanceId}_BRAND/{deviceId}/state
for device_id in device_ids:
topic = f"oagv/{oagv_version}/{self.instance_id}{brand_suffix}/{device_id}/{command_type}"
forward_topics.append(topic)
return forward_topics
def extract_vehicle_id_from_topic(self, topic: str) -> Optional[str]:
@ -605,7 +642,29 @@ class DeviceHandlerRegistry:
self.system_listen_topics: Dict[str, str] = {} # topic -> handler_id
self.custom_device_handlers: Dict[str, str] = {} # device_id -> handler_id
self.is_system_listening: bool = False
# 设备消息队列和处理协程 - 按设备ID分流
self.device_queues: Dict[str, asyncio.Queue] = {} # device_id -> queue
self.device_workers: Dict[str, asyncio.Task] = {} # device_id -> worker_task
self.max_queue_size_per_device = 1000 # 每个设备的队列容量(可配置)
self.device_queue_stats: Dict[str, Dict[str, Any]] = {} # 队列统计信息
def set_max_queue_size(self, max_size: int):
"""设置每个设备的最大队列容量
Args:
max_size: 最大队列容量建议500-2000
"""
if max_size < 10:
logger.warning("队列大小太小建议至少设置为10")
max_size = 10
elif max_size > 10000:
logger.warning("队列大小太大建议不超过10000")
max_size = 10000
self.max_queue_size_per_device = max_size
logger.info(f"设备队列最大容量已设置为: {max_size}")
def register_device_handler(self, config: DeviceHandlerConfig, processor: Callable):
"""注册并运行设备处理器"""
device_id = config.device_id
@ -688,6 +747,21 @@ class DeviceHandlerRegistry:
del self.custom_device_handlers[device_id]
logger.info(f"设备 {device_id} 已恢复系统级监听")
# 清理设备队列和worker协程
if device_id in self.device_workers:
worker_task = self.device_workers[device_id]
if not worker_task.done():
worker_task.cancel()
del self.device_workers[device_id]
logger.info(f"已取消设备 {device_id} 的worker协程")
if device_id in self.device_queues:
del self.device_queues[device_id]
logger.info(f"已清理设备 {device_id} 的消息队列")
if device_id in self.device_queue_stats:
del self.device_queue_stats[device_id]
# 清理注册信息
del self.device_handlers[device_id]
del self.message_processors[device_id]
@ -695,7 +769,7 @@ class DeviceHandlerRegistry:
del self.active_devices[device_id]
if device_id in self.processing_stats:
del self.processing_stats[device_id]
logger.info(f"停止并注销设备处理器: {device_id}")
def _subscribe_topics(self, topics: List[str]):
@ -786,12 +860,12 @@ class DeviceHandlerRegistry:
# 从topic中提取品牌信息进行验证
if len(specific_parts) >= 3:
topic_brand_instance = specific_parts[2]
# 检查品牌是否匹配
brand_config = self.topic_manager.supported_brands.get(brand_name.lower())
# 检查品牌是否匹配(使用新的品牌配置获取方法)
brand_config = self.topic_manager.get_brand_config(brand_name)
if brand_config:
brand_suffix = brand_config["brand_suffix"]
# 对于oagv格式检查实例+品牌后缀是否匹配
if specific_parts[0] == "oagv":
expected_instance_brand = f"{self.topic_manager.instance_id}{brand_suffix}"
@ -802,7 +876,7 @@ class DeviceHandlerRegistry:
brand_name_only = brand_suffix[1:] if brand_suffix.startswith('_') else brand_suffix
if topic_brand_instance == brand_name_only:
return True
return False
def _unsubscribe_topics(self, topics: List[str], device_id: str):
@ -860,6 +934,151 @@ class DeviceHandlerRegistry:
def get_device_processor(self, device_id: str) -> Optional[Callable]:
"""获取设备处理器"""
return self.message_processors.get(device_id)
def _extract_device_id_from_topic(self, topic: str) -> Optional[str]:
"""快速从topic中提取设备ID优化版本"""
# oagv/v2/instance_BRAND/AGV001/state
# uagv/v2.0.0/BRAND/AGV001/state
parts = topic.split('/')
if len(parts) >= 5:
# 倒数第二个部分通常是设备ID
device_id = parts[-2]
# 过滤通配符
if device_id != "+":
return device_id
return None
async def _create_device_worker(self, device_id: str):
"""为设备创建独立的处理协程"""
if device_id in self.device_workers and not self.device_workers[device_id].done():
return # 已存在
# 创建队列
self.device_queues[device_id] = asyncio.Queue(maxsize=self.max_queue_size_per_device)
# 初始化统计信息
self.device_queue_stats[device_id] = {
"total_enqueued": 0,
"total_processed": 0,
"dropped_messages": 0,
"current_queue_size": 0,
"worker_started_at": datetime.now().isoformat()
}
# 创建处理协程
self.device_workers[device_id] = asyncio.create_task(
self._device_message_worker(device_id)
)
logger.info(f"为设备 {device_id} 创建了独立消息处理协程")
async def _device_message_worker(self, device_id: str):
"""设备消息处理协程 - 独立处理该设备的所有消息"""
logger.info(f"设备 {device_id} 消息处理协程已启动")
queue = self.device_queues[device_id]
while True:
try:
# 从队列取出消息
message_data = await queue.get()
try:
# 处理单条消息
await self._process_single_device_message(
device_id=device_id,
topic=message_data["topic"],
payload=message_data["payload"]
)
# 更新统计
if device_id in self.device_queue_stats:
self.device_queue_stats[device_id]["total_processed"] += 1
self.device_queue_stats[device_id]["current_queue_size"] = queue.qsize()
except Exception as e:
logger.error(f"处理设备 {device_id} 消息异常: {e}", exc_info=True)
finally:
queue.task_done()
except asyncio.CancelledError:
logger.info(f"设备 {device_id} 消息处理协程被取消")
break
except Exception as e:
logger.error(f"设备 {device_id} 处理协程异常: {e}", exc_info=True)
await asyncio.sleep(0.1) # 避免异常风暴
async def _process_single_device_message(self, device_id: str, topic: str,
payload: Union[str, bytes, Dict[str, Any]]):
"""处理单个设备的单条消息原来的handle_mqtt_message逻辑"""
try:
# 解析payload
if isinstance(payload, (str, bytes)):
payload_data = json.loads(payload)
else:
payload_data = payload
# 获取处理该topic的设备列表
handler_ids = self.get_handlers_for_topic(topic)
logger.debug(f"处理topic {topic} 的handler_ids: {handler_ids}")
# 分离自定义处理器和系统级处理器
custom_handlers = [hid for hid in handler_ids if not hid.startswith("system_")]
system_handlers = [hid for hid in handler_ids if hid.startswith("system_")]
# 优先处理自定义处理器
if custom_handlers:
logger.debug(f"使用自定义处理器: {custom_handlers}")
for handler_id in custom_handlers:
config = self.get_device_config(handler_id)
if config and config.enabled:
# 创建设备消息
message = DeviceMessage(
device_id=handler_id,
device_type=config.device_type,
topic=topic,
payload=payload_data,
timestamp=time.time(),
source_topic=topic,
target_topics=config.forward_topics
)
# 处理消息
await self._process_device_message(message)
# 如果没有自定义处理器,使用系统级处理器
elif system_handlers:
logger.debug(f"使用系统级处理器: {system_handlers}")
for handler_id in system_handlers:
# 系统级处理器从topic中提取设备ID
extracted_device_id = self.topic_manager.extract_vehicle_id_from_topic(topic)
if extracted_device_id:
# 根据收到的topic自动生成转发topic
forward_topic = self._generate_forward_topic_from_listen_topic(topic)
target_topics = [forward_topic] if forward_topic else []
# 创建系统级设备消息
message = DeviceMessage(
device_id=extracted_device_id,
device_type=DeviceType.VEHICLE, # 系统级默认为vehicle
topic=topic,
payload=payload_data,
timestamp=time.time(),
source_topic=topic,
target_topics=target_topics
)
# 使用系统级处理器处理
processor = self.message_processors.get(handler_id)
if processor:
await self._process_system_message(handler_id, message, processor)
except json.JSONDecodeError as e:
logger.error(f"MQTT消息JSON解析失败: {e}, topic: {topic}, payload: {payload}")
except Exception as e:
logger.error(f"处理设备消息异常: device_id={device_id}, topic={topic}, error={e}",
exc_info=True)
def _handle_mqtt_message(self, topic: str, payload: str):
"""处理MQTT消息"""
@ -886,74 +1105,42 @@ class DeviceHandlerRegistry:
await self.handle_mqtt_message(topic, payload)
async def handle_mqtt_message(self, topic: str, payload: Union[str, bytes, Dict[str, Any]]):
"""处理MQTT消息"""
"""处理MQTT消息 - 快速分发到设备队列"""
logger.debug(f"收到MQTT消息: topic={topic}")
try:
# 解析payload
if isinstance(payload, (str, bytes)):
payload_data = json.loads(payload)
else:
payload_data = payload
# 获取处理该topic的设备列表
handler_ids = self.get_handlers_for_topic(topic)
logger.debug(f"处理topic {topic} 的handler_ids: {handler_ids}")
# 分离自定义处理器和系统级处理器
custom_handlers = [hid for hid in handler_ids if not hid.startswith("system_")]
system_handlers = [hid for hid in handler_ids if hid.startswith("system_")]
# 优先处理自定义处理器,如果有自定义处理器就跳过系统级处理器
if custom_handlers:
logger.debug(f"使用自定义处理器: {custom_handlers}")
for handler_id in custom_handlers:
# 用户自定义处理器
config = self.get_device_config(handler_id)
if config and config.enabled:
# 创建设备消息
message = DeviceMessage(
device_id=handler_id,
device_type=config.device_type,
topic=topic,
payload=payload_data,
timestamp=time.time(),
source_topic=topic,
target_topics=config.forward_topics
)
# 立即处理消息
await self._process_device_message(message)
else:
# 只有在没有自定义处理器时才使用系统级处理器
logger.debug(f"使用系统级处理器: {system_handlers}")
for handler_id in system_handlers:
# 系统级处理器从topic中提取设备ID
device_id = self.topic_manager.extract_vehicle_id_from_topic(topic)
if device_id:
# 根据收到的topic自动生成转发topic
forward_topic = self._generate_forward_topic_from_listen_topic(topic)
target_topics = [forward_topic] if forward_topic else []
# 创建系统级设备消息
message = DeviceMessage(
device_id=device_id,
device_type=DeviceType.VEHICLE, # 系统级默认为vehicle
topic=topic,
payload=payload_data,
timestamp=time.time(),
source_topic=topic,
target_topics=target_topics
)
# 使用系统级处理器处理
processor = self.message_processors.get(handler_id)
if processor:
await self._process_system_message(handler_id, message, processor)
except json.JSONDecodeError as e:
logger.error(f"MQTT消息JSON解析失败: {e}, topic: {topic}, payload: {payload}")
# 快速提取设备ID不做复杂处理
device_id = self._extract_device_id_from_topic(topic)
if not device_id:
logger.warning(f"无法从topic中提取设备ID: {topic}")
return
# 确保该设备有处理队列
if device_id not in self.device_queues:
await self._create_device_worker(device_id)
# 快速入队不阻塞使用put_nowait避免await
try:
message_data = {
"topic": topic,
"payload": payload,
"timestamp": time.time()
}
self.device_queues[device_id].put_nowait(message_data)
# 更新队列统计
if device_id in self.device_queue_stats:
self.device_queue_stats[device_id]["total_enqueued"] += 1
self.device_queue_stats[device_id]["current_queue_size"] = self.device_queues[device_id].qsize()
except asyncio.QueueFull:
logger.warning(f"设备 {device_id} 消息队列已满({self.max_queue_size_per_device}),丢弃消息")
# 记录丢弃统计
if device_id in self.device_queue_stats:
self.device_queue_stats[device_id]["dropped_messages"] += 1
except Exception as e:
logger.error(f"处理MQTT消息失败: {e}", exc_info=True)
logger.error(f"分发消息异常: topic={topic}, error={e}", exc_info=True)
async def _process_system_message(self, handler_id: str, message: DeviceMessage, processor: Callable):
"""处理系统级消息"""
@ -1014,7 +1201,6 @@ class DeviceHandlerRegistry:
result = await processor(message)
else:
result = processor(message)
# 处理成功
stats["success_count"] += 1
stats["last_processed"] = datetime.now().isoformat()
@ -1046,43 +1232,90 @@ class DeviceHandlerRegistry:
logger.error(f"设备消息处理失败: {device_id}, 错误: {e}", exc_info=True)
async def _handle_message_result(self, original_message: DeviceMessage, result: Dict[str, Any]):
"""处理消息处理结果"""
# print(original_message)
"""处理消息处理结果
用户自定义处理函数返回值格式约束
{
"forward": bool, # 是否转发消息可选默认True
"payload": dict, # 要转发的消息载荷(可选,默认使用原始消息)
"response_topic": str, # 自定义响应topic可选
"response": dict # 自定义响应消息载荷可选需配合response_topic
}
示例1 - 直接透传
return {"forward": True} # 使用原始消息转发到默认topics
示例2 - 修改后转发
return {
"forward": True,
"payload": {"status": "processed", "data": modified_data}
}
示例3 - 不转发
return {"forward": False}
示例4 - 发送自定义响应
return {
"forward": False,
"response_topic": "custom/response/topic",
"response": {"result": "success"}
}
"""
try:
config = self.device_handlers.get(original_message.device_id)
if not config:
logger.warning(f"未找到设备配置: {original_message.device_id}")
return
# 验证返回值格式
if not isinstance(result, dict):
logger.error(f"处理器返回值格式错误,必须是字典类型,实际类型: {type(result)}")
return
# 检查是否需要转发
if result.get("forward", True):
should_forward = result.get("forward", True)
if should_forward:
# 确定转发的payload
forward_payload = result.get("payload", original_message.payload)
# 直接使用转发payload不进行编码
encoded_payload = forward_payload
# 验证payload格式
if not isinstance(forward_payload, dict):
logger.error(f"转发payload格式错误必须是字典类型实际类型: {type(forward_payload)}")
return
# 确定转发的topics
forward_topics = config.forward_topics
if isinstance(forward_topics, str):
forward_topics = [forward_topics]
# 转发到目标topics
if forward_topics:
for topic in forward_topics:
self._publish_mqtt_message(topic, encoded_payload)
self._publish_mqtt_message(topic, forward_payload)
logger.debug(f"消息已转发: {original_message.device_id} -> {topic}")
logger.debug(f"转发载荷: {forward_payload}")
else:
logger.warning(f"未配置转发topics设备: {original_message.device_id}")
# 处理自定义响应
if "response" in result:
response_topic = result.get("response_topic")
response_payload = result["response"]
# 直接使用响应payload不进行编码
if response_topic:
self._publish_mqtt_message(response_topic, response_payload)
logger.debug(f"响应已发送: {original_message.device_id} -> {response_topic}")
# 验证响应格式
if not response_topic:
logger.error("提供了response但未指定response_topic")
return
if not isinstance(response_payload, dict):
logger.error(f"响应payload格式错误必须是字典类型实际类型: {type(response_payload)}")
return
self._publish_mqtt_message(response_topic, response_payload)
logger.debug(f"响应已发送: {original_message.device_id} -> {response_topic}")
logger.debug(f"响应载荷: {response_payload}")
except Exception as e:
logger.error(f"处理消息结果失败: {e}", exc_info=True)
@ -1153,14 +1386,37 @@ class DeviceHandlerRegistry:
"last_heartbeat": self.active_devices[device_id]["last_heartbeat"],
"message_count": self.active_devices[device_id]["message_count"]
})
# 添加队列统计信息
if device_id in self.device_queue_stats:
stats["queue_stats"] = self.device_queue_stats[device_id].copy()
# 添加实时队列大小
if device_id in self.device_queues:
stats["queue_stats"]["current_queue_size"] = self.device_queues[device_id].qsize()
return stats
else:
# 计算总队列统计
total_enqueued = sum(s.get("total_enqueued", 0) for s in self.device_queue_stats.values())
total_processed = sum(s.get("total_processed", 0) for s in self.device_queue_stats.values())
total_dropped = sum(s.get("dropped_messages", 0) for s in self.device_queue_stats.values())
total_queue_size = sum(q.qsize() for q in self.device_queues.values())
return {
"device_count": len(self.device_handlers),
"running_devices": len([d for d in self.active_devices.values()
"running_devices": len([d for d in self.active_devices.values()
if d["status"] == "running"]),
"total_topics": len(self.topic_mappings),
"devices": {device_id: self.get_device_statistics(device_id)
"queue_summary": {
"total_workers": len(self.device_workers),
"total_queues": len(self.device_queues),
"total_enqueued": total_enqueued,
"total_processed": total_processed,
"total_dropped": total_dropped,
"current_total_queue_size": total_queue_size,
"max_queue_size_per_device": self.max_queue_size_per_device
},
"devices": {device_id: self.get_device_statistics(device_id)
for device_id in self.device_handlers.keys()}
}
@ -1186,7 +1442,6 @@ class DeviceHandlerRegistry:
# 生成监听所有设备的topics使用通配符+
system_topics = self._generate_system_listen_topics()
# 创建系统级透传处理器
system_handler = self._create_system_forward_handler()
@ -1223,30 +1478,78 @@ class DeviceHandlerRegistry:
logger.info(f"系统级监听已启动监听topics: {system_topics}")
def _generate_system_listen_topics(self) -> List[str]:
"""生成系统级监听所有设备的topics"""
"""生成系统级监听所有设备的topics包括vehicle和other设备"""
topics = []
# 遍历所有支持的品牌
for brand_name in self.topic_manager.get_supported_brands():
brand_config = self.topic_manager.supported_brands[brand_name.lower()]
# 获取默认监听的指令类型
default_vehicle_commands = self.topic_manager.topic_config["default_command_types"]["vehicle"]
default_other_commands = self.topic_manager.topic_config["default_command_types"]["other"]
# 1. 为vehicle类型设备生成监听topics
logger.info("生成vehicle设备系统监听topics...")
vehicle_brands = self.topic_manager.get_supported_brands(category="vehicle")
for brand_name in vehicle_brands:
brand_config = self.topic_manager.get_brand_config(brand_name)
if not brand_config:
continue
brand_suffix = brand_config["brand_suffix"]
oagv_version = brand_config["oagv_version"]
uagv_version = brand_config["uagv_version"]
# 为每种指令类型生成监听topic使用+通配符匹配所有设备)
for command_type, direction_config in self.topic_manager.command_directions.items():
# 只生成vehicle默认监听列表中的指令类型
if command_type not in default_vehicle_commands:
logger.debug(f"[Vehicle] 跳过非默认监听指令类型: {command_type}")
continue
listen_queue = direction_config["listen"]
if listen_queue == "oagv":
# oagv格式: oagv/v2/{instanceId}_BRAND/+/command
topic = f"oagv/{oagv_version}/{self.topic_manager.instance_id}{brand_suffix}/+/{command_type}"
topics.append(topic)
logger.debug(f"[Vehicle] 添加监听topic: {topic}")
elif listen_queue == "uagv":
# uagv格式: uagv/v2.0.0/BRAND/+/command
brand_name_only = brand_suffix[1:] # 去掉前缀下划线
topic = f"uagv/{uagv_version}/{brand_name_only}/+/{command_type}"
topics.append(topic)
logger.debug(f"[Vehicle] 添加监听topic: {topic}")
# 2. 为other类型设备生成监听topics
logger.info("生成other设备系统监听topics...")
other_brands = self.topic_manager.get_supported_brands(category="other")
for brand_name in other_brands:
brand_config = self.topic_manager.get_brand_config(brand_name)
if not brand_config:
continue
brand_suffix = brand_config["brand_suffix"]
oagv_version = brand_config["oagv_version"]
uagv_version = brand_config["uagv_version"]
for command_type, direction_config in self.topic_manager.command_directions.items():
# 只生成other默认监听列表中的指令类型
if command_type not in default_other_commands:
logger.debug(f"[Other] 跳过非默认监听指令类型: {command_type}")
continue
listen_queue = direction_config["listen"]
if listen_queue == "oagv":
# oagv格式: oagv/v2/{instanceId}_BRAND/+/command
topic = f"oagv/{oagv_version}/{self.topic_manager.instance_id}{brand_suffix}/+/{command_type}"
topics.append(topic)
logger.debug(f"[Other] 添加监听topic: {topic}")
elif listen_queue == "uagv":
# uagv格式: uagv/v2.0.0/BRAND/+/command
brand_name_only = brand_suffix[1:] # 去掉前缀下划线
topic = f"uagv/{uagv_version}/{brand_name_only}/+/{command_type}"
topics.append(topic)
logger.debug(f"[Other] 添加监听topic: {topic}")
logger.info(f"系统监听topics生成完成{len(topics)}")
return topics
def _create_system_forward_handler(self) -> Callable:
@ -1313,14 +1616,15 @@ class DeviceHandlerRegistry:
# logger.debug(f"指令 {command_type} 的转发方向: {forward_queue}")
# 解析品牌信息 - 改进品牌匹配逻辑
# 解析品牌信息 - 改进品牌匹配逻辑(支持新的分类配置)
brand_name = None
# logger.debug(f"开始匹配品牌当前brand_instance: {brand_instance}")
for brand, config in self.topic_manager.supported_brands.items():
# 遍历所有品牌(使用扁平化后的字典)
for brand, config in self.topic_manager._flat_brands.items():
brand_suffix = config["brand_suffix"]
# logger.debug(f"检查品牌 {brand}: suffix={brand_suffix}")
if queue_type == "oagv":
# oagv格式: oagv/v2/{instanceId}_BRAND/{deviceId}/command
# brand_instance 应该是 "asbm2_IRAYPLE" 这样的格式
@ -1336,13 +1640,16 @@ class DeviceHandlerRegistry:
brand_name = brand
# logger.debug(f"成功匹配uagv品牌: {brand}")
break
if not brand_name:
logger.warning(f"无法识别品牌信息: queue_type={queue_type}, brand_instance={brand_instance}")
logger.debug(f"可用的品牌配置: {list(self.topic_manager.supported_brands.keys())}")
logger.debug(f"可用的品牌配置: {list(self.topic_manager._flat_brands.keys())}")
return None
brand_config = self.topic_manager.get_brand_config(brand_name)
if not brand_config:
logger.error(f"无法获取品牌配置: {brand_name}")
return None
brand_config = self.topic_manager.supported_brands[brand_name]
# logger.debug(f"使用品牌配置: {brand_name} -> {brand_config}")
# 生成转发topic
@ -1438,157 +1745,26 @@ class DeviceHandlerService:
await self.registry.stop_mqtt_service()
logger.info("设备处理服务已停止")
def register_and_run(self, device_ids: List[str], device_type: Union[str, DeviceType] = DeviceType.VEHICLE,
brand_name: str = "huarui", command_type: str = None,
handler: Callable = None, script_id: str = "", description: str = "",
protocol_type: Union[str, ProtocolType] = ProtocolType.VDA5050,
**kwargs):
"""注册并运行设备处理器(通用设备注册方法)
Args:
device_ids: 设备ID列表必需- 支持批量注册多个设备
device_type: 设备类型默认vehicle- 支持所有设备类型
brand_name: 设备品牌huarui或seer默认huarui
command_type: 指令类型必需- 只能处理一种指令类型"order""state"
handler: 消息处理器函数必需
script_id: 脚本ID必需
description: 描述信息
protocol_type: 协议类型默认VDA5050
**kwargs: 其他配置参数
Examples:
# 注册多辆小车
register_and_run(
device_ids=["AGV001", "AGV002", "AGV003"],
device_type="vehicle",
brand_name="huarui",
handler=my_handler,
script_id="script_001"
)
# 注册多个门设备
register_and_run(
device_ids=["DOOR_001", "DOOR_002"],
device_type="door",
brand_name="standard",
handler=door_handler,
script_id="script_002"
)
"""
# 参数验证
if not device_ids or len(device_ids) == 0:
raise ValueError("device_ids参数是必需的且不能为空")
if handler is None:
raise ValueError("handler参数是必需的")
if not script_id:
raise ValueError("script_id参数是必需的")
if not command_type:
raise ValueError("command_type参数是必需的且不能为空")
# 验证command_type是否为有效的指令类型
valid_commands = ["order", "state", "factsheet", "instantActions"]
if command_type not in valid_commands:
raise ValueError(f"command_type参数错误'{command_type}' 不是有效的指令类型。支持的类型有:{', '.join(valid_commands)}")
# 确保device_type是DeviceType枚举
if isinstance(device_type, str):
device_type = DeviceType(device_type)
# 验证品牌
if not self.registry.topic_manager.validate_brand(brand_name):
supported_brands = self.registry.topic_manager.get_supported_brands()
raise ValueError(f"不支持的品牌: {brand_name},支持的品牌: {supported_brands}")
# 处理协议类型
if isinstance(protocol_type, str):
try:
protocol_type = ProtocolType(protocol_type)
except ValueError:
logger.warning(f"未知的协议类型: {protocol_type}, 使用VDA5050协议")
protocol_type = ProtocolType.VDA5050
# 设置品牌从brand_name转换
if brand_name.lower() == "huarui":
final_device_brand = DeviceBrand.HUARUI
elif brand_name.lower() == "seer":
final_device_brand = DeviceBrand.SEER
elif brand_name.lower() == "standard":
final_device_brand = DeviceBrand.STANDARD
else:
final_device_brand = DeviceBrand.CUSTOM
# 过滤kwargs中的兼容性参数
filtered_kwargs = {k: v for k, v in kwargs.items()
if k not in ['protocol_key', 'auto_encode', 'listen_topics', 'forward_topics']}
logger.info(f"批量注册设备: {len(device_ids)}{device_type.value} 设备,指令类型: {command_type}")
# 为每个设备创建单独的处理器注册每个设备有独立的监听和转发topics
registered_device_ids = []
for device_id in device_ids:
# 为当前设备生成独立的监听topics只监听自己对应的topic
device_listen_topics = self.registry.topic_manager.generate_listen_topics(
[device_id], brand_name, [command_type], device_type
)
# 为当前设备生成独立的转发topics只转发到自己对应的topic
device_forward_topics = self.registry.topic_manager.generate_forward_topics(
[device_id], brand_name, [command_type], device_type
)
logger.info(f"设备 {device_id} - 监听topics: {device_listen_topics}")
logger.info(f"设备 {device_id} - 转发topics: {device_forward_topics}")
# 创建配置
config = DeviceHandlerConfig(
device_id=device_id,
device_type=device_type,
listen_topics=device_listen_topics, # 监听topics必须独立
forward_topics=device_forward_topics, # 转发topics必须独立
handler_function=handler.__name__,
script_id=script_id,
description=description or f"{final_device_brand.value} {device_type.value} 设备处理器",
device_brand=final_device_brand,
protocol_type=protocol_type,
**filtered_kwargs
)
self.registry.register_device_handler(config, handler)
registered_device_ids.append(device_id)
logger.info(f"批量注册完成: {len(registered_device_ids)}个设备已注册")
return registered_device_ids
def register_default_vehicle_handler(self, vehicle_ids: List[str], brand_name: str,
handler: Callable, script_id: str,
description: str = "") -> List[str]:
"""注册默认的小车处理器(最简化接口)
Args:
vehicle_ids: 监听的小车ID列表
brand_name: 小车品牌huarui或seer
handler: 消息处理器函数
script_id: 脚本ID
description: 描述信息
Returns:
注册的设备ID列表
"""
return self.register_and_run(
device_ids=vehicle_ids,
device_type=DeviceType.VEHICLE,
brand_name=brand_name,
handler=handler,
script_id=script_id,
description=description or f"默认{brand_name}小车处理器,监听{len(vehicle_ids)}辆车"
)
def stop_handler(self, device_id: str):
"""停止指定设备处理器"""
self.registry.unregister_device_handler(device_id)
def set_device_queue_size(self, max_size: int):
"""设置设备消息队列的最大容量
Args:
max_size: 最大队列容量建议500-2000
- 如果消息处理速度快可以设置小一些(如500)
- 如果消息处理速度慢或消息频率高可以设置大一些(如2000)
- 队列满了会自动丢弃新消息
Example:
device_service = get_device_service()
device_service.set_device_queue_size(1500) # 设置为1500
"""
self.registry.set_max_queue_size(max_size)
def get_running_handlers(self) -> Dict[str, Any]:
"""获取所有正在运行的设备处理器"""

View File

@ -105,7 +105,6 @@ class MultiScriptEngine:
else:
return {"success": False, "error": "必须提供script_id或script_path参数"}
# 2. 验证脚本文件存在
# print("full_script_path::::::::::::::::::", full_script_path, "============")
if not os.path.exists(full_script_path):
return {"success": False, "error": f"脚本文件不存在: {script_path}"}

View File

@ -1,262 +1,122 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
获取密集库位处理器并发问题修复演示
展示优化前后的区别
"""
import asyncio
#!/usr/bin/env python3
import json
import time
import random
from typing import Dict, Any, List
import threading
import paho.mqtt.client as mqtt
from datetime import datetime
# ----------------- 公共 MQTT 配置 -----------------
MQTT_BROKER = "192.168.189.97" # 换成实际 IP
MQTT_PORT = 1883
KEEPALIVE = 60
QOS = 1
class MockDatabase:
"""模拟数据库"""
def __init__(self):
# 模拟5个空闲库位
self.storage_sites = {
f"A00{i}": {
"is_locked": False,
"is_occupied": False,
"last_access": datetime.now()
}
for i in range(1, 6)
}
self.operation_log = []
async def query_available_sites(self) -> List[str]:
"""查询可用库位"""
await asyncio.sleep(0.01) # 模拟数据库查询延迟
available = [
site_id for site_id, info in self.storage_sites.items()
if not info["is_locked"] and not info["is_occupied"]
# -------------------------------------------------
# 1. order 指令模板(发往 uagv/v2/SEER/{sn}/order
# -------------------------------------------------
ORDER_TMPL = {
"headerId": 103852,
"timestamp": "",
"version": "2.0.0",
"manufacturer": "SEER",
"serialNumber": "ROHDL04",
"orderId": "1972692682446491649",
"orderUpdateId": 81076,
"nodes": [
{"nodeId": "LM128", "nodeDescription": "LM128", "released": True,
"nodePosition": {"x": -24.226, "y": 1.071, "mapId": "ROH1122_0804"}},
{"nodeId": "LM129", "nodeDescription": "LM129", "released": True,
"nodePosition": {"x": -29.226, "y": 1.071, "mapId": "ROH1122_0804"}},
{"nodeId": "LM108", "nodeDescription": "LM108", "released": True,
"nodePosition": {"x": -34.221, "y": 1.093, "mapId": "ROH1122_0804"}}
],
"edges": [
{"edgeId": "LM128-LM129", "sequenceId": 216377, "released": True,
"startNodeId": "LM128", "endNodeId": "LM129", "rotationAllowed": False,
"trajectory": {"degree": 1.0, "knotVector": [0, 0, 1, 1],
"controlPoints": [{"x": -25.89, "y": 1.071},
{"x": -27.559, "y": 1.071}]}},
{"edgeId": "LM129-LM108", "sequenceId": 216378,
"startNodeId": "LM129", "endNodeId": "LM108", "rotationAllowed": False}
]
}
ORDER_TOPIC = f"oagv/v2/asbm2_IRAYPLE/{ORDER_TMPL['serialNumber']}/order"
# -------------------------------------------------
# 2. instantActions 指令模板(发往 oagv/v2/1_VWED/{sn}/instantActions
# -------------------------------------------------
INSTANT_TMPL = {
"headerId": 103854,
"timestamp": "",
"version": "2.0.0",
"serialNumber": "172.31.57.151-10067-17",
"instantActions": [{
"actionType": "writeValue",
"actionId": "2fea9ad2-277f-4db3-b81d-d85454af111a",
"actionDescription": "action parameters",
"blockingType": "HARD",
"actionParameters": [
{"key": "registerName", "value": {"string_value": "set"}},
{"key": "command", "value": {"string_value": "cmd:close"}}
]
return sorted(available)
async def lock_site_old_way(self, site_id: str, task_id: str) -> bool:
"""旧方式:分别查询后锁定(有竞争条件)"""
# 先查询
available_sites = await self.query_available_sites()
if site_id not in available_sites:
return False
# 模拟网络延迟和处理时间
await asyncio.sleep(random.uniform(0.01, 0.05))
# 尝试锁定(可能已被其他请求抢占)
if self.storage_sites[site_id]["is_locked"]:
return False
self.storage_sites[site_id]["is_locked"] = True
self.operation_log.append(f"[OLD] Task {task_id} locked {site_id}")
return True
async def lock_site_new_way(self, site_id: str, task_id: str) -> bool:
"""新方式:原子性锁定(无竞争条件)"""
# 原子操作:检查并锁定
if not self.storage_sites[site_id]["is_locked"] and not self.storage_sites[site_id]["is_occupied"]:
self.storage_sites[site_id]["is_locked"] = True
self.operation_log.append(f"[NEW] Task {task_id} locked {site_id}")
return True
return False
def reset(self):
"""重置数据库状态"""
for site_info in self.storage_sites.values():
site_info["is_locked"] = False
self.operation_log.clear()
}]
}
INSTANT_TOPIC = f"oagv/v2/asbm2_VWED/{INSTANT_TMPL['serialNumber']}/instantActions"
# -------------------------------------------------
# 工具:获取当前时间字符串
# -------------------------------------------------
def now_str():
return datetime.utcnow().strftime("%Y-%m-%d %H:%M:%S")
async def simulate_old_approach(db: MockDatabase, num_tasks: int) -> Dict[str, Any]:
"""模拟旧的实现方式"""
print(f"\n=== 模拟旧方式:查询后锁定(存在竞争条件)===")
async def old_task(task_id: int):
# 查询可用库位
available_sites = await db.query_available_sites()
if not available_sites:
return {"task_id": task_id, "success": False, "reason": "no_sites"}
# 选择第一个可用库位
target_site = available_sites[0]
# 尝试锁定
success = await db.lock_site_old_way(target_site, f"task_{task_id}")
return {
"task_id": task_id,
"success": success,
"site_id": target_site if success else None,
"reason": "success" if success else "race_condition"
}
# 重置数据库
db.reset()
start_time = time.time()
# 并发执行任务
tasks = [old_task(i) for i in range(num_tasks)]
results = await asyncio.gather(*tasks)
end_time = time.time()
# 分析结果
successful = [r for r in results if r["success"]]
failed = [r for r in results if not r["success"]]
# 检查是否有重复分配
allocated_sites = [r["site_id"] for r in successful if r["site_id"]]
duplicates = len(allocated_sites) - len(set(allocated_sites))
print(f"执行时间: {end_time - start_time:.3f}s")
print(f"成功任务: {len(successful)}")
print(f"失败任务: {len(failed)}")
print(f"重复分配: {duplicates} 个库位被重复分配")
if duplicates > 0:
print("❌ 发现并发竞争问题:有库位被重复分配!")
else:
print("✓ 未发现重复分配")
print("操作日志:")
for log in db.operation_log[:10]: # 只显示前10条
print(f" {log}")
return {
"total_tasks": num_tasks,
"successful": len(successful),
"failed": len(failed),
"duplicates": duplicates,
"execution_time": end_time - start_time
}
# -------------------------------------------------
# 线程函数:循环发 order
# -------------------------------------------------
def publish_order(client):
while True:
try:
ORDER_TMPL["timestamp"] = now_str()
payload = json.dumps(ORDER_TMPL, separators=(',', ':'))
client.publish(ORDER_TOPIC, payload, qos=QOS)
print(f"[ORDER] {now_str()} -> {ORDER_TOPIC}")
except Exception as e:
print(f"[ORDER-ERROR] {e}")
time.sleep(0.5)
# -------------------------------------------------
# 线程函数:循环发 instantActions
# -------------------------------------------------
def publish_instant(client):
while True:
try:
INSTANT_TMPL["timestamp"] = now_str()
payload = json.dumps(INSTANT_TMPL, separators=(',', ':'))
client.publish(INSTANT_TOPIC, payload, qos=QOS)
print(f"[INSTANT] {now_str()} -> {INSTANT_TOPIC}")
except Exception as e:
print(f"[INSTANT-ERROR] {e}")
time.sleep(0.5)
async def simulate_new_approach(db: MockDatabase, num_tasks: int) -> Dict[str, Any]:
"""模拟新的实现方式"""
print(f"\n=== 模拟新方式:原子性获取(无竞争条件)===")
async def new_task(task_id: int):
# 查询可用库位
available_sites = await db.query_available_sites()
if not available_sites:
return {"task_id": task_id, "success": False, "reason": "no_sites"}
# 按优先级尝试原子性锁定每个库位
for site_id in available_sites:
success = await db.lock_site_new_way(site_id, f"task_{task_id}")
if success:
return {
"task_id": task_id,
"success": True,
"site_id": site_id,
"reason": "success"
}
return {"task_id": task_id, "success": False, "reason": "all_occupied"}
# 重置数据库
db.reset()
start_time = time.time()
# 并发执行任务
tasks = [new_task(i) for i in range(num_tasks)]
results = await asyncio.gather(*tasks)
end_time = time.time()
# 分析结果
successful = [r for r in results if r["success"]]
failed = [r for r in results if not r["success"]]
# 检查是否有重复分配
allocated_sites = [r["site_id"] for r in successful if r["site_id"]]
duplicates = len(allocated_sites) - len(set(allocated_sites))
print(f"执行时间: {end_time - start_time:.3f}s")
print(f"成功任务: {len(successful)}")
print(f"失败任务: {len(failed)}")
print(f"重复分配: {duplicates} 个库位被重复分配")
if duplicates > 0:
print("❌ 仍然存在并发竞争问题!")
else:
print("✓ 成功解决并发竞争问题")
print("操作日志:")
for log in db.operation_log[:10]: # 只显示前10条
print(f" {log}")
return {
"total_tasks": num_tasks,
"successful": len(successful),
"failed": len(failed),
"duplicates": duplicates,
"execution_time": end_time - start_time
}
# -------------------------------------------------
# 主入口
# -------------------------------------------------
def main():
client = mqtt.Client()
client.connect(MQTT_BROKER, MQTT_PORT, KEEPALIVE)
client.loop_start()
# threading.Thread(target=publish_order, args=(client,), daemon=True).start()
threading.Thread(target=publish_instant, args=(client,), daemon=True).start()
async def run_comparison_demo():
"""运行对比演示"""
print("🔧 获取密集库位处理器并发问题修复演示")
print("=" * 60)
db = MockDatabase()
num_concurrent_tasks = 10 # 10个并发任务竞争5个库位
print(f"测试场景: {num_concurrent_tasks} 个并发任务竞争 {len(db.storage_sites)} 个库位")
# 测试旧方式
old_results = await simulate_old_approach(db, num_concurrent_tasks)
# 测试新方式
new_results = await simulate_new_approach(db, num_concurrent_tasks)
# 对比结果
print(f"\n=== 对比结果 ===")
print(f"旧方式 - 成功: {old_results['successful']}, 失败: {old_results['failed']}, 重复分配: {old_results['duplicates']}")
print(f"新方式 - 成功: {new_results['successful']}, 失败: {new_results['failed']}, 重复分配: {new_results['duplicates']}")
if old_results['duplicates'] > 0 and new_results['duplicates'] == 0:
print("🎉 优化成功!新方式完全解决了并发竞争问题")
elif old_results['duplicates'] == new_results['duplicates'] == 0:
print(" 在当前测试条件下,两种方式都没有出现竞争问题")
print(" 可能需要增加并发压力或调整延迟来复现问题")
else:
print("⚠️ 优化可能不够完善,需要进一步检查")
# 性能对比
performance_diff = ((new_results['execution_time'] - old_results['execution_time']) / old_results['execution_time']) * 100
if performance_diff > 0:
print(f"📊 性能: 新方式比旧方式慢 {performance_diff:.1f}%")
else:
print(f"📊 性能: 新方式比旧方式快 {abs(performance_diff):.1f}%")
async def stress_test():
"""压力测试"""
print(f"\n=== 压力测试 ===")
print("高并发场景下测试并发安全性...")
db = MockDatabase()
num_tasks = 50 # 50个并发任务
print(f"压力测试: {num_tasks} 个高并发任务")
# 只测试新方式的压力承受能力
results = await simulate_new_approach(db, num_tasks)
success_rate = (results['successful'] / results['total_tasks']) * 100
print(f"成功率: {success_rate:.1f}%")
print(f"平均每任务耗时: {results['execution_time']/results['total_tasks']*1000:.2f}ms")
if results['duplicates'] == 0:
print("✅ 压力测试通过:无并发竞争问题")
else:
print("❌ 压力测试失败:仍存在并发竞争问题")
# 主线程保持存活
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
print("--- 用户中断 ---")
finally:
client.loop_stop()
client.disconnect()
if __name__ == "__main__":
asyncio.run(run_comparison_demo())
asyncio.run(stress_test())
main()

View File

@ -301,12 +301,167 @@ def get_alert_sync_service() -> AlertSyncService:
def sync_alert_from_record(record: logging.LogRecord) -> bool:
"""
从日志记录同步告警外部调用接口
Args:
record: 日志记录
Returns:
是否成功
"""
service = get_alert_sync_service()
return service.sync_alert(record)
return service.sync_alert(record)
def sync_error_alert(logger_name: str, error_message: str, level: int = logging.ERROR,
context_info: Dict[str, Any] = None) -> bool:
"""
直接推送错误告警到主系统不通过日志系统
Args:
logger_name: logger名称用于分类错误类型
error_message: 错误消息
level: 日志级别默认ERROR
context_info: 上下文信息可选如任务ID文件名行号等
Returns:
是否成功推送
Example:
sync_error_alert(
logger_name="services.execution.task_executor",
error_message="任务执行异常: 连接超时",
level=logging.ERROR,
context_info={"task_id": "123", "filename": "task_executor.py", "lineno": 220}
)
"""
service = get_alert_sync_service()
if not service.enabled:
return False
# 检查日志级别
if level < service.min_level:
return False
try:
# 生成告警码
alert_code = service._generate_alert_code(logger_name, error_message)
# 确定告警类型
alert_type = service._determine_alert_type(logger_name)
# 映射告警级别
alert_level = service.LEVEL_MAPPING.get(level, 2)
# 生成告警名称
alert_name = service._generate_alert_name(logger_name, level)
# 构建描述信息
description = error_message
if context_info:
if context_info.get('filename') and context_info.get('lineno'):
description += f" [文件: {context_info['filename']}:{context_info['lineno']}]"
if context_info.get('funcName'):
description += f" [函数: {context_info['funcName']}()]"
if context_info.get('task_id'):
description += f" [任务ID: {context_info['task_id']}]"
# 添加其他上下文信息
for key, value in context_info.items():
if key not in ['filename', 'lineno', 'funcName', 'task_id']:
description += f" [{key}: {value}]"
# 构建告警数据
alert_data = {
"type": alert_type,
"level": alert_level,
"code": alert_code,
"name": alert_name,
"description": description,
"solution": "请检查相关日志文件获取详细的异常堆栈信息,并根据异常类型进行排查"
}
# 添加到队列
service.alert_queue.put_nowait(alert_data)
return True
except Exception as e:
# 避免告警同步本身产生错误影响正常流程
print(f"错误告警推送失败: {e}")
return False
def sync_warning_alert(logger_name: str, warning_message: str,
context_info: Dict[str, Any] = None) -> bool:
"""
直接推送警告告警到主系统不通过日志系统
Args:
logger_name: logger名称用于分类告警类型
warning_message: 警告消息
context_info: 上下文信息可选如工作线程数队列大小等
Returns:
是否成功推送
Example:
sync_warning_alert(
logger_name="services.enhanced_scheduler.task_scheduler",
warning_message="所有工作线程都在忙碌中",
context_info={"worker_count": 5, "queue_size": 10}
)
"""
service = get_alert_sync_service()
if not service.enabled:
return False
# 警告级别固定为WARNING
level = logging.WARNING
# 检查日志级别
if level < service.min_level:
return False
try:
# 生成告警码
alert_code = service._generate_alert_code(logger_name, warning_message)
# 确定告警类型
alert_type = service._determine_alert_type(logger_name)
# 映射告警级别
alert_level = service.LEVEL_MAPPING.get(level, 2)
# 生成告警名称
alert_name = service._generate_alert_name(logger_name, level)
# 构建描述信息
description = warning_message
if context_info:
if context_info.get('worker_count') is not None:
description += f" [工作线程数: {context_info['worker_count']}]"
if context_info.get('queue_size') is not None:
description += f" [队列大小: {context_info['queue_size']}]"
if context_info.get('task_record_id'):
description += f" [任务ID: {context_info['task_record_id']}]"
# 添加其他上下文信息
for key, value in context_info.items():
if key not in ['worker_count', 'queue_size', 'task_record_id']:
description += f" [{key}: {value}]"
# 构建告警数据
alert_data = {
"type": alert_type,
"level": alert_level,
"code": alert_code,
"name": alert_name,
"description": description,
"solution": "建议增加工作线程数量或优化任务执行效率,避免任务积压"
}
# 添加到队列
service.alert_queue.put_nowait(alert_data)
return True
except Exception as e:
# 避免告警同步本身产生错误影响正常流程
print(f"警告告警推送失败: {e}")
return False