58 lines
8.4 KiB
Markdown
58 lines
8.4 KiB
Markdown
# HBOX RTK vs CalAmp on RTG ZPMC-20 — Wialon trip discrepancy: root cause, framework, fixes (2026-10-06)
|
||
|
||
**Context.** ICAVE compares Wialon trips from two devices on the same RTG crane (ZPMC-20): HBOX **H0013** (EinarBoard V5.5, fw 0.6.0, Wialon unit `RT-ZPM-20 (RTK)`) and the legacy CalAmp (`RT-ZPMC-20`). They derive fuel consumption from trips and asked why the two disagree. Data from Jacobo (Drive folder, 5 and 6 Oct 2026): Wialon NMEA message exports of both units, Wialon *Trips* .xlsx of both, and H0013's raw `historial` from htracker. Everything is now in `Drogheda/06-Testing-Validation/wialon-trips/` (raw data per day + `informe_viajes_2026-10-0{5,6}.html` + `analisis_*.json`).
|
||
|
||
## Bottom line
|
||
|
||
The complaint as phrased ("more trips from the RTK → more fuel") is the **opposite** of what the data shows. The CalAmp over-reports trips and mileage by **8–18×** because it reports movement while the crane is parked; the HBOX is clean but Wialon's trip-distance threshold discards most real RTG manoeuvres.
|
||
|
||
| | CalAmp (RT-ZPMC-20) | HBOX H0013 (RTK) | Real crane motion (from the 1 Hz RTK track) |
|
||
|---|---|---|---|
|
||
| 5-Oct 15:00–17:00 | 1 trip · 1 h 47 min · **3.30 km** | 1 trip · 6 min · 0.19 km | **0.41 km in 8 min** — 6 manoeuvres (12–166 m, 2–5 km/h) |
|
||
| 6-Oct 08:00–10:46 | 2 trips · 2 h 40 min · **6.17 km** | 1 trip · 2 min · 0.17 km | **0.35 km in 6 min** — 8 manoeuvres (11–170 m, 2–6 km/h) |
|
||
|
||
### Evidence
|
||
1. **Same Wialon trip-detector settings on both units.** A Wialon-equivalent detector (by GPS speed ≥ 1 km/h, parking ≥ 300 s, min trip distance ≈ 150 m, speed floored to integer km/h as Wialon stores it) reproduces **all four** Wialon Trips reports: RTK boundaries within ±3–5 s, CalAmp within 0–30 s (one 30-s sample). Parameter sweep over 525 combinations: only this family fits all four. So the difference is in the data, not the configuration.
|
||
2. **CalAmp false motion.** Per 30-s CalAmp sample, with the RTK as ground truth (0.2 m resolution; H0013 historial matches the Wialon export to 0.07 m, zero lag; Wialon speed = floor(km/h)):
|
||
- while the crane is stationary, CalAmp reports 1–5 km/h **33.5 % (5-Oct) / 43.8 % (6-Oct)** of samples;
|
||
- its position jumps 6.5 / 10.4 m median (p90 15 / 26 m, max 34 / 140 m) between consecutive messages while parked → **1.67 km / 3.64 km phantom mileage** per window;
|
||
- it detects real motion only 70 % / 47 % of the time;
|
||
- stationary drift it integrates: 785 / 1152 m/h. Because it rarely strings 300 s of zero speed, Wialon never closes the trip → one all-afternoon trip.
|
||
- Wialon's printed CalAmp mileage (3.30 / 5.81 km) is even higher than the path sum of its own exported positions (1.94 / 3.81 km) — Wialon's counter for that unit likely uses the CalAmp odometer parameter; either way it is noise.
|
||
3. **HBOX under-reporting is a Wialon threshold, not a device problem.** 93–97 % of samples at 0 km/h, stationary scatter p50 1.9 m / p95 4.9 m (DGPS fix quality 2, 12 sats, HDOP 0.5 — note: *not* RTK fixed/float; corrections issue is separate, see earlier notes). Real RTG moves last 20–120 s and cover 10–170 m; with min distance ≈ 150 m Wialon kept 1 of 6 (5-Oct) and 1 of 8 (6-Oct). The one it kept (15:16–15:22, 166 m) only crossed 150 m thanks to rest-jitter mileage during a 3.8-min pause inside the trip.
|
||
4. **HBOX stationary drift** at 1 Hz integrates ~0.33 km/h of path (does not affect Wialon trips, which ignore 0 km/h, but dirties tracks/counters) — fixed by the retranslator motion filter below.
|
||
|
||
### The map says it in one glance
|
||
RTK track = clean line along the container block; CalAmp track = scribble around a parked crane (see the reports).
|
||
|
||
## What was built — `htracker/trips/` (committed to the repo folder on Leo's Mac, not yet pushed)
|
||
|
||
Pure-stdlib Python package (openpyxl optional) + 40-line hook in `retranslator.py` + 26-line hook in `web_monitor.py`. Full details, env vars and deploy steps: `04-Source-Code/htracker/CHANGES-2026-10-06-viajes.md`.
|
||
|
||
- `nmea.py` readers: Wialon NMEA export, htracker `historial_unidades` ring files (unsorted → sorted/deduped), raw NMEA, `.gz`, Wialon Trips `.xlsx` (openpyxl must *not* be in read_only mode — Wialon's xlsx has a bad dimension).
|
||
- `detector.py`: `TripParams` + `detect_trips()` (Wialon-equivalent; defaults 1 km/h · 300 s · 150 m); `MotionFilter` (REPOSO/MOVIMIENTO hysteresis: ≥1 km/h × 2 samples or > 8 m from anchor → moving; < 1 km/h × 10 s → rest; at rest forward speed 0 and the anchor position). Hold radius 8 m because a parked F9P in DGPS wanders up to ~5 m over 20 min; 3 m caused 114 spurious transitions.
|
||
- `compare.py`: fine-vs-coarse alignment, confusion matrix, phantom mileage, true mileage, trip matching, chart series; `analyze()` builds the whole JSON.
|
||
- `report.py`: self-contained HTML (Chart.js + Leaflet CDN, monitor dark theme) — verdict cards, speed timeline with Wialon trips as bands, cumulative km (device path vs real vs Wialon-printed), map, noise histogram, Wialon-vs-recalculated tables, real-manoeuvre list, filter simulation, method. Also the `/trips` monitor page (same renderer, API-fed).
|
||
- `store.py`: `trips_positions.db` (SQLite WAL) fed by the retranslator at 1 Hz while moving, 1/10 s at rest (10 s, not 30: at 30 s the rest-jitter mileage disappears and the marginal 166-m Wialon trip would be lost; at 10 s all four Wialon reports still reproduce). ≈ 0.7 MB/unit/day, retention 14 d (~300 MB for 30 units). Writer thread, UDP loop never blocks.
|
||
- `gateway.py`: `TripsGateway.process()` hook (store raw fix; if the filter is enabled for the unit, send Wialon a copy with speed 0.00 and the held position). Never raises. Env: `GW_TRIPS_STORE_ENABLED=1` (default on), `GW_MOTION_FILTER_ENABLED=0` (default off), `GW_MOTION_FILTER_UNITS=H0013` for a staged rollout.
|
||
- `monitor_routes.py`: `GET /trips`, `GET /api/trips_1hz/units`, `POST /api/trips_1hz` (unit, window, params, optional reference NMEA file + Wialon .xlsx uploads, raw vs sent data). Same auth/unit gates as `/api/track`. Nav links added on dashboard and reports.
|
||
- Tests: 22 (`unittest`, real data fixtures 116 KB gzip): Wialon-report reproduction, readers, filter behaviour, store downsampling, gateway, FastAPI routes (skip without fastapi/openpyxl). Green here and on Leo's Mac (3.14; 7 skipped there for missing openpyxl/fastapi). End-to-end with a fake Wialon listener verified `#SD#` packets carry speed 0.00 + held position at rest and pass through unchanged when moving.
|
||
- CLI: `python3 -m trips.cli --device … --reference … --wialon-trips-device … --out informe.html --json …`.
|
||
|
||
Existing `/api/trips` (5-minute `gps_positions.db`) cannot see 20–120 s crane moves; it stays as is.
|
||
|
||
## Recommendations (in order)
|
||
1. **Wialon config for HBOX units:** min speed 1 km/h, parking 120–300 s, **min trip distance 10 m**, min time 0. Turns 1 → 6 and 1 → 8 trips (0.50 / 0.48 km ≈ real). Do *not* apply to the CalAmp (would explode its false trips).
|
||
2. **Enable the motion filter on H0013 only** (`GW_MOTION_FILTER_ENABLED=1`, `GW_MOTION_FILTER_UNITS=H0013`), validate 2–3 days with `/trips` ("raw" vs "sent"), then fleet-wide. It cleans tracks/counters; it does not change trip counts under the current 150 m threshold.
|
||
3. **Fuel = engine hours, not trip km** for an RTG. Forward ignition (`$PHEV IGN_ON/OFF`, fw 0.5.1+; PA5) as a Wialon IPS `#D#` param `ign` and use "ignition sensor" engine hours (and optionally ignition-based trips). Proposed, not implemented — needs a one-unit bench check that Wialon accepts the `D` packet with params from the current unit type.
|
||
4. Do not "match" the HBOX to the CalAmp.
|
||
|
||
Customer-facing note (es-MX, Leo's voice): `00-Project-Management/client-comms/2026-10-06_Drogheda_Viajes-Wialon-RTK-vs-CalAmp_Nota-ICAVE_v1.0.md`.
|
||
|
||
## Open points / caveats
|
||
- Wialon's detector parameters are inferred from the reports (they fit exactly), not read from the account — confirm in Wialon → unit → Trip detector before changing them.
|
||
- H0013 fix quality is 2 (DGPS), never 4/5; the RTK-corrections thread is unchanged by this work.
|
||
- The Wialon-printed CalAmp mileage source (odometer param vs GPS) is unverified; irrelevant to the conclusion.
|
||
- The `historial` ring covers only ~1 h and is unordered; the new store is the proper source for `/trips`.
|
||
- Git: Leo runs `git add trips/ retranslator.py web_monitor.py .gitignore CHANGES-2026-10-06-viajes.md README.md`, commit, `./push-deploy.sh`; then check `journalctl -u htracker` for "TripsStore iniciado".
|