ferdzo/openjsp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Basic data ingestion and analytics

Browse Source
This commit is contained in:
Andrej Mickov
2026-02-09 01:10:52 +01:00
parent 699c826bf0
commit afd195dab9
12 changed files with 3203 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

556
UPSTREAM-API-DOCUMENTATION.md Normal file
View File

@@ -0,0 +1,556 @@
# ModeShift GTFS API Documentation
Documentation for the upstream ModeShift GTFS API used by this Skopje Bus Tracker application.
## Table of Contents
- [Overview](#overview)
- [OpenAPI Specification](#openapi-specification)
- [Base URL](#base-url)
- [Authentication](#authentication)
- [Endpoints](#endpoints)
- [GTFS-RT Trip Updates](#gtfs-rt-trip-updates)
- [Public Vehicles (JSON)](#public-vehicles-json)
- [Nearby Stop Times](#nearby-stop-times)
- [Data Models](#data-models)
- [Usage Examples](#usage-examples)
---
## Overview
The ModeShift API provides real-time transit data for Skopje public transport based on the **GTFS (General Transit Feed Specification)** and **GTFS-Realtime** standards.
**Provider:** ModeShift
**API Type:** GTFS + GTFS-Realtime
**Data Format:** JSON and Protocol Buffers
**Coverage:** Skopje, North Macedonia
---
## OpenAPI Specification
A complete **OpenAPI 3.0** specification for the ModeShift API is available at [upstream-openapi.yaml](upstream-openapi.yaml).
This machine-readable specification includes:
- All upstream API endpoints
- Request/response schemas
- Parameter validation
- Examples for each operation
You can view it interactively using:
- [Swagger Editor](https://editor.swagger.io/) - Paste the YAML content
- [Redoc](https://redocly.github.io/redoc/) - For a clean documentation view
- VS Code with OpenAPI extensions
---
## Base URL
```
https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e
```
The base URL includes a tenant/agency identifier (`9814b106-2afe-47c8-919b-bdec6a5e521e`) specific to Skopje's transit system.
---
## Authentication
Currently, the API appears to be publicly accessible without authentication tokens. Requests can be made directly to the endpoints.
---
## Endpoints
### GTFS-RT Trip Updates
Get real-time trip updates in GTFS-Realtime format (Protocol Buffers).
**Endpoint:** `GET /transport/gtfsrt/tripupdates.pb`
**Full URL:**
```
https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/gtfsrt/tripupdates.pb
```
**Response Format:** Protocol Buffers (binary)
**Content-Type:** `application/octet-stream`
**Description:**
Returns a GTFS-Realtime FeedMessage containing TripUpdate entities. This follows the official [GTFS-Realtime specification](https://gtfs.org/realtime/).
**Response Structure (Protobuf):**
```protobuf
message FeedMessage {
FeedHeader header = 1;
repeated FeedEntity entity = 2;
}
message FeedEntity {
required string id = 1;
optional TripUpdate trip_update = 3;
}
message TripUpdate {
optional TripDescriptor trip = 1;
optional VehicleDescriptor vehicle = 3;
repeated StopTimeUpdate stop_time_update = 2;
}
```
**Example Usage (JavaScript):**
```javascript
import GtfsRealtimeBindings from 'gtfs-realtime-bindings';
const response = await fetch(
'https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/gtfsrt/tripupdates.pb'
);
const buffer = await response.arrayBuffer();
const feed = GtfsRealtimeBindings.transit_realtime.FeedMessage.decode(
new Uint8Array(buffer)
);
for (const entity of feed.entity) {
if (entity.tripUpdate) {
console.log('Trip:', entity.tripUpdate.trip.tripId);
console.log('Route:', entity.tripUpdate.trip.routeId);
console.log('Vehicle:', entity.tripUpdate.vehicle?.id);
}
}
```
**Key Fields:**
- `trip.tripId`: GTFS trip identifier
- `trip.routeId`: GTFS route identifier
- `vehicle.id`: Vehicle identification number
- `vehicle.label`: Public-facing vehicle number
- `stopTimeUpdate[]`: Array of stop time predictions
---
### Public Vehicles (JSON)
Get current positions and status of all active vehicles in JSON format.
**Endpoint:** `GET /transport/public/vehicles`
**Full URL:**
```
https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/public/vehicles
```
**Response Format:** JSON
**Description:**
Returns an array of all currently active vehicles with their real-time GPS positions, speed, bearing, and status information.
**Response Structure:**
```json
[
{
"id": 12345,
"identificationNumber": "MK-SK-1234",
"inventoryNumber": 1234,
"positionLatitude": 41.9981,
"positionLongitude": 21.4254,
"positionBearing": 87.5,
"positionSpeed": 15.3,
"positionModifiedAt": "2026-02-07T14:27:15Z",
"status": 2
},
{
"id": 12346,
"identificationNumber": "MK-SK-5678",
"inventoryNumber": 5678,
"positionLatitude": 42.0012,
"positionLongitude": 21.4301,
"positionBearing": 180.0,
"positionSpeed": 0.0,
"positionModifiedAt": "2026-02-07T14:27:10Z",
"status": 1
}
]
```
**Fields:**
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Internal database identifier for the vehicle |
| `identificationNumber` | string | Vehicle license plate or identification (e.g., "MK-SK-1234") |
| `inventoryNumber` | integer | Public-facing bus number shown on the vehicle |
| `positionLatitude` | number | Current GPS latitude (WGS84) |
| `positionLongitude` | number | Current GPS longitude (WGS84) |
| `positionBearing` | number | Direction of travel in degrees (0-360, 0=North, 90=East) |
| `positionSpeed` | number | Current speed in km/h |
| `positionModifiedAt` | string | ISO 8601 timestamp of last position update |
| `status` | integer | Vehicle status code (1=stopped, 2=in transit) |
**Status Codes:**
- `1` - Vehicle is stopped
- `2` - Vehicle is in transit
**Example Usage:**
```bash
curl "https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/public/vehicles"
```
**Use Cases:**
- Display all vehicles on a map
- Match vehicles to routes using trip update data
- Track real-time vehicle movements
---
### Nearby Stop Times
Get arrival predictions for stops near a specific location.
**Endpoint:** `GET /transport/planner/stops/nearbyTimes`
**Full URL:**
```
https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/planner/stops/nearbyTimes
```
**Query Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `latitude` | number | Yes | Search center latitude (WGS84) |
| `longitude` | number | Yes | Search center longitude (WGS84) |
| `radius` | integer | Yes | Search radius in meters |
**Response Format:** JSON
**Description:**
Returns stops within the specified radius along with their upcoming arrivals, including real-time predictions.
**Response Structure:**
```json
[
{
"id": 1563,
"code": "363",
"name": "KARPOS II - STIV NAUMOV",
"latitude": 41.97964,
"longitude": 21.41283,
"patterns": [
{
"routeId": 125,
"routeShortName": "7",
"routeLongName": "Centar - GTC - Karpos 4 - Karposh 2",
"headsign": "Karpos 2",
"stopTimes": [
{
"serviceDay": 1738972800,
"scheduledArrival": 52380,
"realtimeArrival": 52500,
"arrivalDelay": 120,
"realtime": true,
"realtimeState": "UPDATED",
"headsign": "Karpos 2"
},
{
"serviceDay": 1738972800,
"scheduledArrival": 53280,
"realtimeArrival": 53280,
"arrivalDelay": 0,
"realtime": true,
"realtimeState": "SCHEDULED",
"headsign": "Karpos 2"
}
]
}
]
}
]
```
**Fields:**
**Stop Level:**
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | GTFS stop ID |
| `code` | string | Public stop code |
| `name` | string | Stop name |
| `latitude` | number | Stop latitude |
| `longitude` | number | Stop longitude |
| `patterns[]` | array | Array of route patterns serving this stop |
**Pattern Level:**
| Field | Type | Description |
|-------|------|-------------|
| `routeId` | integer | GTFS route ID |
| `routeShortName` | string | Public route number (e.g., "7") |
| `routeLongName` | string | Full route name |
| `headsign` | string | Direction/destination |
| `stopTimes[]` | array | Array of upcoming arrivals |
**Stop Time Level:**
| Field | Type | Description |
|-------|------|-------------|
| `serviceDay` | integer | Unix timestamp (seconds) of service day start (midnight) |
| `scheduledArrival` | integer | Seconds since midnight for scheduled arrival time |
| `realtimeArrival` | integer | Seconds since midnight for predicted arrival time |
| `arrivalDelay` | integer | Delay in seconds (positive=late, negative=early) |
| `realtime` | boolean | Whether prediction is based on real-time data |
| `realtimeState` | string | State: `SCHEDULED`, `UPDATED`, `CANCELED` |
| `headsign` | string | Trip headsign (destination) |
**Calculating Arrival Time:**
```javascript
const arrival = new Date(
(stopTime.serviceDay + stopTime.realtimeArrival) * 1000
);
```
**Example Usage:**
```bash
# Get arrivals near American College
curl "https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/planner/stops/nearbyTimes?latitude=42.00437&longitude=21.39687&radius=50"
```
**Use Cases:**
- Display upcoming arrivals at a stop
- Show real-time delay information
- Calculate minutes until arrival
- Filter by specific routes
---
## Data Models
### GTFS Stop
```typescript
interface Stop {
id: number; // GTFS stop_id (as integer)
code: string; // Public stop code
name: string; // Stop name
latitude: number; // WGS84 latitude
longitude: number; // WGS84 longitude
patterns: Pattern[]; // Routes serving this stop
}
```
### Route Pattern
```typescript
interface Pattern {
routeId: number; // GTFS route_id (as integer)
routeShortName: string; // Public route number
routeLongName: string; // Full route name
headsign: string; // Direction/destination
stopTimes: StopTime[]; // Upcoming arrivals
}
```
### Stop Time
```typescript
interface StopTime {
serviceDay: number; // Unix timestamp of service day start
scheduledArrival: number; // Seconds since midnight (scheduled)
realtimeArrival: number; // Seconds since midnight (predicted)
arrivalDelay: number; // Delay in seconds
realtime: boolean; // Based on real-time data
realtimeState: string; // SCHEDULED, UPDATED, CANCELED
headsign: string; // Trip headsign
}
```
### Vehicle
```typescript
interface Vehicle {
id: number; // Internal ID
identificationNumber: string; // License plate
inventoryNumber: number; // Bus number
positionLatitude: number; // GPS latitude
positionLongitude: number; // GPS longitude
positionBearing: number; // Direction (0-360°)
positionSpeed: number; // Speed in km/h
positionModifiedAt: string; // ISO 8601 timestamp
status: number; // 1=stopped, 2=in_transit
}
```
---
## Usage Examples
### Complete Real-Time Tracker Flow
```javascript
// 1. Get a specific stop's arrivals
const stopLat = 41.97964;
const stopLon = 21.41283;
const radius = 50;
const nearbyResponse = await fetch(
`https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/planner/stops/nearbyTimes?latitude=${stopLat}&longitude=${stopLon}&radius=${radius}`
);
const stops = await nearbyResponse.json();
// 2. Process arrivals for a specific route
const targetRouteId = 125;
const now = new Date();
for (const stop of stops) {
for (const pattern of stop.patterns) {
if (pattern.routeId === targetRouteId) {
for (const stopTime of pattern.stopTimes) {
const arrivalTime = new Date(
(stopTime.serviceDay + stopTime.realtimeArrival) * 1000
);
const minutesUntil = Math.floor(
(arrivalTime.getTime() - now.getTime()) / 60000
);
console.log(`Route ${pattern.routeShortName} - ${minutesUntil} min`);
console.log(`Delay: ${stopTime.arrivalDelay} seconds`);
}
}
}
}
// 3. Get all vehicle positions
const vehiclesResponse = await fetch(
'https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/public/vehicles'
);
const vehicles = await vehiclesResponse.json();
// 4. Get trip updates to match vehicles to routes
const tripUpdatesResponse = await fetch(
'https://www.modeshift.app/api/v1/9814b106-2afe-47c8-919b-bdec6a5e521e/transport/gtfsrt/tripupdates.pb'
);
const buffer = await tripUpdatesResponse.arrayBuffer();
const feed = GtfsRealtimeBindings.transit_realtime.FeedMessage.decode(
new Uint8Array(buffer)
);
// 5. Build vehicle-to-route mapping
const vehicleRouteMap = new Map();
for (const entity of feed.entity) {
if (entity.tripUpdate?.vehicle?.id && entity.tripUpdate?.trip?.routeId) {
vehicleRouteMap.set(
entity.tripUpdate.vehicle.id,
entity.tripUpdate.trip.routeId
);
}
}
// 6. Filter vehicles by route
const routeVehicles = vehicles.filter(v => {
const vehicleKey = v.identificationNumber || v.inventoryNumber.toString();
return vehicleRouteMap.get(vehicleKey) == targetRouteId;
});
console.log(`Found ${routeVehicles.length} vehicles on route ${targetRouteId}`);
```
### Polling for Updates
```javascript
// Recommended polling intervals
const VEHICLE_UPDATE_INTERVAL = 10000; // 10 seconds
const ARRIVAL_UPDATE_INTERVAL = 15000; // 15 seconds
const TRIP_UPDATE_INTERVAL = 10000; // 10 seconds
// Poll for vehicle positions
setInterval(async () => {
const vehicles = await fetch(vehiclesEndpoint).then(r => r.json());
updateMap(vehicles);
}, VEHICLE_UPDATE_INTERVAL);
// Poll for arrival predictions
setInterval(async () => {
const arrivals = await fetch(nearbyTimesEndpoint).then(r => r.json());
updateArrivalBoard(arrivals);
}, ARRIVAL_UPDATE_INTERVAL);
```
---
## GTFS Static Data
The API uses GTFS static data which should be downloaded separately. Static data includes:
- **stops.txt**: All stop locations and names
- **routes.txt**: All route definitions
- **trips.txt**: Trip schedules
- **stop_times.txt**: Stop time schedules
- **calendar.txt**: Service calendar
- **shapes.txt**: Route shapes (geographic paths)
**Note:** The static GTFS feed download URL should be obtained from ModeShift or the transit agency.
---
## Rate Limiting & Best Practices
1. **Polling Frequency**:
- Don't poll more frequently than every 10 seconds
- Cache responses when appropriate
- Use conditional requests when possible
2. **Radius Parameter**:
- Use smallest radius that meets your needs (typically 50-100m)
- Larger radius = more data returned = slower response
3. **Error Handling**:
- Handle network errors gracefully
- Implement exponential backoff for failures
- Parse protobuf data carefully (can be empty)
4. **Data Volume**:
- Vehicle positions endpoint returns ALL vehicles (100+)
- Filter client-side rather than making multiple requests
- Consider caching vehicle positions
---
## Common Issues & Solutions
### Empty GTFS-RT Feed
**Problem:** `tripupdates.pb` returns empty or zero-length buffer
**Solution:** This can happen during off-peak hours or system maintenance. Handle gracefully and retry later.
### Vehicle Matching
**Problem:** Vehicles in JSON don't match trip updates
**Solution:** Try matching by both `identificationNumber` and `inventoryNumber`. Some vehicles may not have active trips.
### Time Zone Issues
**Problem:** Arrival times are incorrect
**Solution:** The API uses Europe/Skopje timezone. Service day timestamps are at midnight local time.
### Delay Calculation
**Problem:** Delays seem wrong
**Solution:** Use `realtimeArrival - scheduledArrival` (already provided as `arrivalDelay` field).
---
## Related Documentation
- **GTFS Specification**: https://gtfs.org/
- **GTFS-Realtime**: https://gtfs.org/realtime/
- **Protocol Buffers**: https://protobuf.dev/
- **gtfs-realtime-bindings**: https://www.npmjs.com/package/gtfs-realtime-bindings
---
## Support
For issues with the ModeShift API:
- Contact ModeShift support
- Check ModeShift documentation at https://www.modeshift.app/
For issues with this tracker application:
- See [README.md](README.md)
- Check [API-DOCUMENTATION.md](API-DOCUMENTATION.md) for the local API
---
**Last Updated:** February 7, 2026
**API Provider:** ModeShift
**Coverage:** Skopje, North Macedonia