Basic data ingestion and analytics

README.md

@@ -12,12 +12,67 @@ npm run web
 
 Open **http://localhost:3000**
 
+Visit **http://localhost:3000/analytics.html** for historical data and performance analytics.
+
+## TimescaleDB Setup
+
+The application uses TimescaleDB for storing time-series data (vehicle positions, arrivals, delays).
+
+### Start the database:
+
+```bash
+cd infrastructure
+docker compose up -d
+```
+
+### Configure environment:
+
+Create a `.env` file (or use the defaults):
+
+```bash
+POSTGRES_HOST=localhost
+POSTGRES_PORT=5432
+POSTGRES_DB=iot_data
+POSTGRES_USER=postgres
+POSTGRES_PASSWORD=example
+```
+
+The database will automatically:
+- Create hypertables for efficient time-series queries
+- Set up compression and retention policies (90 days)
+- Build continuous aggregates for hourly metrics
+- Index data for fast queries
+
+### Analytics Features:
+
+- **Vehicle Position History**: Track individual buses over time
+- **Delay Analysis**: On-time performance, average delays, patterns
+- **Hourly Patterns**: See when buses are typically late/early
+- **Route Statistics**: Reliability scores, service quality metrics
+- **Stop Performance**: Compare delays across different stops
+
+### Background Tracker:
+
+For continuous data collection without keeping the web interface open:
+
+```bash
+npm run track
+```
+
+This automatically tracks these popular routes every 30 seconds:
+- Routes: 2, 4, 5, 7, 15, 21, 22, 24
+- Private routes: 12П, 19П, 22П, 45П, 52П, 54П, 61П, 9П
+
+Data is stored in TimescaleDB for historical analysis. The tracker runs indefinitely until stopped with Ctrl+C.
+
 ## Features
 
 - **Fully Modular Web Interface**: Select any stop and route via UI controls or URL parameters
 - **Dynamic Tracking**: Change stops/routes without restarting the server
 - Interactive map with live vehicle positions
 - Real-time arrivals with delays
+- **Time-Series Data Storage**: Historical tracking with TimescaleDB
+- **Analytics Dashboard**: Delay statistics, hourly patterns, performance metrics
 - 5-second auto-refresh (web), 10-second (terminal)
 - CLI arguments for terminal tracker
 - Configurable defaults via [config.ts](config.ts)
@@ -32,6 +87,7 @@ npm run find -- --route "7"        # Find route IDs by number/name
 npm run web                # Web interface at http://localhost:3000
 npm run tracker            # Terminal interface (default)
 npm run tracker -- --stop 1571 --route 125  # Custom stop/route
+npm run track              # Background tracker for popular routes (30s intervals)
 npm start                  # Same as web
 ```
 
@@ -67,12 +123,34 @@ npm run tracker -- --help
 
 ### API Endpoints
 
+**This Application's API:**
+- Complete docs: **[API-DOCUMENTATION.md](API-DOCUMENTATION.md)**
+- Interactive docs: http://localhost:3000/api-docs.html (when server is running)
+- OpenAPI spec: **[openapi.yaml](openapi.yaml)**
+
+**Upstream ModeShift GTFS API:**
+- Documentation: **[UPSTREAM-API-DOCUMENTATION.md](UPSTREAM-API-DOCUMENTATION.md)**
+- Provider: ModeShift (Skopje public transport data)
+
+#### Quick Reference
+
 Query parameters for custom tracking:
 
 ```
 GET /api/config?stopId=1571&routeId=125
 GET /api/arrivals?stopId=1571&routeId=125
 GET /api/vehicles?routeId=125
+GET /api/stops                              # All stops
+GET /api/routes                             # All routes
+
+# Historical Data APIs
+GET /api/stats/db                           # Database statistics
+GET /api/history/vehicle/:vehicleId?hours=24
+GET /api/history/route/:routeId/vehicles?hours=24
+GET /api/history/stop/:stopId/arrivals?routeId=125&hours=24
+GET /api/stats/route/:routeId/delays?hours=24
+GET /api/stats/stop/:stopId/delays?hours=24
+GET /api/stats/route/:routeId/hourly?days=7
 ```
 
 ## Configuration
@@ -101,11 +179,17 @@ export const config: AppConfig = {
       terminal: 10000, // 10 seconds
     },
     minutesAhead: 90,
-  },
-};
-```
-
-## Structure
+  }, + analytics)
+├── bus-tracker-json.ts   # Terminal tracker (CLI args)
+├── lib/
+│   ├── gtfs.ts           # GTFS loader
+│   └── database.ts       # TimescaleDB time-series storage
+├── public/
+│   ├── index.html        # Live tracker UI
+│   └── analytics.html    # Analytics dashboard
+├── infrastructure/
+│   └── compose.yml       # TimescaleDB Docker setup
+└── gtfs/ ure
 
 ```
 bus/
@@ -116,8 +200,11 @@ bus/
 ├── bus-tracker-json.ts   # Terminal tracker (CLI args)
 ├── lib/gtfs.ts          # GTFS loader
 ├── public/index.html    # Frontend (modular UI)
-└── gtfs/                # Static data (auto-downloaded)
-```
+└─**TimescaleDB (PostgreSQL)** for time-series data
+- Leaflet.js + OpenStreetMap
+- Chart.js for analytics visualizations
+- GTFS + GTFS-RT Protocol Buffers
+- Docker Compose for database
 
 ## Stack