ferdzo/iotDashboard
1
0
Fork 0
mirror of https://github.com/ferdzo/iotDashboard.git synced 2026-04-05 09:06:26 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Added auth, environment brief, docker for db_migrations,frontend,backend.

Browse Source
This commit is contained in:
Andrej Mickov
2025-12-15 23:40:34 +01:00
parent 3ab81fad8c
commit 1a5bef277d
36 changed files with 1059 additions and 132 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
services/device_manager/.dockerignore
View File

@@ -6,4 +6,5 @@ __pycache__/
*.pyd
*.crl
*.crt
*.pem
*.pem
*.sh

34
services/device_manager/gen_ca.sh Normal file
View File

@@ -0,0 +1,34 @@
#!/bin/bash
set -e
CERT_DIR="certs"
SERVER_IP="${1:-localhost}"
mkdir -p "$CERT_DIR"
openssl genrsa -out "$CERT_DIR/ca.key" 4096
openssl req -new -x509 -days 3650 -key "$CERT_DIR/ca.key" -out "$CERT_DIR/ca.crt" \
-subj "/C=MK/ST=State/L=City/O=IoTDashboard/OU=DeviceManager/CN=IoT Device CA"
openssl genrsa -out "$CERT_DIR/server.key" 4096
openssl req -new -key "$CERT_DIR/server.key" -out "$CERT_DIR/server.csr" \
-subj "/C=MK/ST=State/L=City/O=IoTDashboard/OU=MQTT/CN=$SERVER_IP"
cat > "$CERT_DIR/server.ext" << EOF
subjectAltName = @alt_names
[alt_names]
IP.1 = $SERVER_IP
DNS.1 = localhost
EOF
openssl x509 -req -in "$CERT_DIR/server.csr" -CA "$CERT_DIR/ca.crt" -CAkey "$CERT_DIR/ca.key" \
-CAcreateserial -out "$CERT_DIR/server.crt" -days 365 -sha256 -extfile "$CERT_DIR/server.ext"
rm "$CERT_DIR/server.csr" "$CERT_DIR/server.ext" "$CERT_DIR/ca.srl"
chmod 600 "$CERT_DIR/ca.key" "$CERT_DIR/server.key"
chmod 644 "$CERT_DIR/ca.crt" "$CERT_DIR/server.crt"
echo "Certificates created:"
echo " CA: $CERT_DIR/ca.crt"
echo " Server: $CERT_DIR/server.crt (valid for $SERVER_IP)"

57
services/device_manager/generate_ca.sh
View File

@@ -1,34 +1,39 @@
#!/bin/bash
# Script to generate a Certificate Authority (CA) for IoT devices
CERT_DIR="certs"
CA_KEY="$CERT_DIR/ca.key"
CA_CERT="$CERT_DIR/ca.crt"
set -e
CERT_DIR="certs"
DOMAINS="${@:-localhost}"
# Create certs directory
mkdir -p "$CERT_DIR"
echo "Generating CA Certificate Authority..."
echo "Generating CA..."
openssl genrsa -out "$CERT_DIR/ca.key" 4096
openssl req -new -x509 -days 3650 -key "$CERT_DIR/ca.key" -out "$CERT_DIR/ca.crt" \
-subj "/C=MK/ST=State/L=City/O=IoTDashboard/OU=DeviceManager/CN=IoT Device CA"
# Generate CA private key (4096-bit RSA)
openssl genrsa -out "$CA_KEY" 4096
echo "Generated CA private key: $CA_KEY"
echo "Generating server certificate..."
openssl genrsa -out "$CERT_DIR/server.key" 4096
openssl req -new -key "$CERT_DIR/server.key" -out "$CERT_DIR/server.csr" \
-subj "/C=MK/ST=State/L=City/O=IoTDashboard/OU=MQTT/CN=${1:-localhost}"
# Generate CA certificate (valid for 10 years)
openssl req -new -x509 -days 3650 -key "$CA_KEY" -out "$CA_CERT" \
-subj "/C=US/ST=State/L=City/O=IoTDashboard/OU=DeviceManager/CN=IoT Device CA"
echo "Generated CA certificate: $CA_CERT"
echo "subjectAltName = @alt_names" > "$CERT_DIR/server.ext"
echo "[alt_names]" >> "$CERT_DIR/server.ext"
# Set secure permissions
chmod 600 "$CA_KEY"
chmod 644 "$CA_CERT"
INDEX=1
for DOMAIN in $DOMAINS; do
if [[ $DOMAIN =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "IP.$INDEX = $DOMAIN" >> "$CERT_DIR/server.ext"
else
echo "DNS.$INDEX = $DOMAIN" >> "$CERT_DIR/server.ext"
fi
INDEX=$((INDEX + 1))
done
echo ""
echo "CA Certificate Authority created successfully!"
echo ""
echo "CA Certificate Details:"
openssl x509 -in "$CA_CERT" -noout -text | grep -A 2 "Subject:"
echo ""
echo "Valid from:"
openssl x509 -in "$CA_CERT" -noout -startdate
echo "Valid until:"
openssl x509 -in "$CA_CERT" -noout -enddate
openssl x509 -req -in "$CERT_DIR/server.csr" -CA "$CERT_DIR/ca.crt" -CAkey "$CERT_DIR/ca.key" \
-CAcreateserial -out "$CERT_DIR/server.crt" -days 365 -sha256 -extfile "$CERT_DIR/server.ext"
rm "$CERT_DIR/server.csr" "$CERT_DIR/server.ext" "$CERT_DIR/ca.srl"
chmod 600 "$CERT_DIR/ca.key" "$CERT_DIR/server.key"
chmod 644 "$CERT_DIR/ca.crt" "$CERT_DIR/server.crt"
echo "Done! Server cert valid for: $DOMAINS"

1
services/gpt_service/gpt_service.py
View File

@@ -446,7 +446,6 @@ Keep all text concise: summary under 50 words, each item under 20 words.""",
context = "\n\n".join(context_sections)
# Build briefing-specific prompts
prompts = {
"schedule": f"""You are a smart wellness coach for office workers (software engineers, tech/finance professionals).
Generate a Schedule Briefing focused on calendar and activity optimization.

5
services/gpt_service/main.py
View File

@@ -15,12 +15,10 @@ gpt_service = None
async def lifespan(app: FastAPI):
"""Lifespan event handler for startup and shutdown."""
global gpt_service
# Startup
logger.info("Initializing GPT Service...")
gpt_service = GPTService()
logger.info("GPT Service initialized successfully")
yield
# Shutdown (cleanup if needed)
logger.info("Shutting down GPT Service...")
@@ -96,10 +94,8 @@ async def analyze_telemetry(request: AnalyzeRequest):
logger.info(f"Analyzing {len(request.telemetry_data)} telemetry points with prompt_type={request.prompt_type}")
# Convert Pydantic models to dicts for GPTService
telemetry_dicts = [point.model_dump() for point in request.telemetry_data]
# Call GPT service analysis
analysis_result = await gpt_service.analyze(
telemetry_data=telemetry_dicts,
device_info=request.device_info,
@@ -133,7 +129,6 @@ async def generate_daily_briefing(request: DailyBriefingRequest):
logger.info(f"Generating {request.briefing_type} briefing")
# Convert calendar events to dicts
calendar_events = None
if request.calendar_events:
calendar_events = [event.model_dump() for event in request.calendar_events]

99
services/mqtt_ingestion/README.md
View File

@@ -1,3 +1,98 @@
# MQTT Ingestion service
# MQTT Ingestion Service
This service ingests all the data from the MQTT topics and sends the data to Redis
Subscribes to MQTT topics and writes telemetry data to Redis streams for downstream processing.
## Purpose
This service acts as the bridge between MQTT devices and the data pipeline. It:
- Connects to Mosquitto MQTT broker
- Subscribes to device topics using wildcard pattern
- Parses incoming messages
- Writes structured data to Redis stream
## Architecture
```
MQTT Broker (port 8883)
|
v
+-------------------+
| mqtt_ingestion |
| - MQTT subscriber |
| - Topic parser |
| - Redis writer |
+-------------------+
|
v
Redis Stream: mqtt:ingestion
```
## Topic Format
Devices publish to: `devices/{device_id}/{metric}`
Examples:
- `devices/a1b2c3d4/temperature` - Temperature reading
- `devices/a1b2c3d4/humidity` - Humidity reading
- `devices/a1b2c3d4/heart_rate` - Health metric
The service subscribes to `devices/#` to receive all device messages.
## Redis Stream Format
Each message written to `mqtt:ingestion` contains:
| Field | Type | Description |
|-------|------|-------------|
| device_id | string | 8-character device identifier |
| metric | string | Metric name (temperature, humidity, etc.) |
| value | string | Metric value (stored as string) |
| timestamp | string | ISO 8601 timestamp |
Example:
```json
{
"device_id": "a1b2c3d4",
"metric": "temperature",
"value": "23.5",
"timestamp": "2025-01-15T10:30:00.000Z"
}
```
## Configuration
Environment variables (`.env` file):
| Variable | Description | Default |
|----------|-------------|---------|
| MQTT_HOST | MQTT broker hostname | localhost |
| MQTT_PORT | MQTT broker port | 8883 |
| REDIS_HOST | Redis hostname | localhost |
| REDIS_PORT | Redis port | 6379 |
## Running
```bash
cd services/mqtt_ingestion
uv sync
uv run main.py
```
## Key Files
| File | Purpose |
|------|---------|
| `main.py` | Entry point, service initialization |
| `src/mqtt_client.py` | MQTT connection and subscription logic |
| `src/redis_writer.py` | Redis stream writing |
## Error Handling
- Invalid topics (not matching `devices/{id}/{metric}`) are logged and dropped
- Connection failures trigger automatic reconnection
- Redis write failures are logged (messages may be lost)
## Integration Points
- **Upstream**: Mosquitto MQTT broker with mTLS
- **Downstream**: Redis stream consumed by db_write service