Drogheda/06-Testing-Validation/wialon-trips/2026-10-06_Drogheda_Wialon-Trips_RTK-vs-CalAmp_Findings_v1.0.md

8.4 KiB
Raw Blame History

HBOX RTK vs CalAmp on RTG ZPMC-20 — Wialon trip discrepancy: root cause, framework, fixes (2026-10-06)

Context. ICAVE compares Wialon trips from two devices on the same RTG crane (ZPMC-20): HBOX H0013 (EinarBoard V5.5, fw 0.6.0, Wialon unit RT-ZPM-20 (RTK)) and the legacy CalAmp (RT-ZPMC-20). They derive fuel consumption from trips and asked why the two disagree. Data from Jacobo (Drive folder, 5 and 6 Oct 2026): Wialon NMEA message exports of both units, Wialon Trips .xlsx of both, and H0013's raw historial from htracker. Everything is now in Drogheda/06-Testing-Validation/wialon-trips/ (raw data per day + informe_viajes_2026-10-0{5,6}.html + analisis_*.json).

Bottom line

The complaint as phrased ("more trips from the RTK → more fuel") is the opposite of what the data shows. The CalAmp over-reports trips and mileage by 8–18× because it reports movement while the crane is parked; the HBOX is clean but Wialon's trip-distance threshold discards most real RTG manoeuvres.

CalAmp (RT-ZPMC-20) HBOX H0013 (RTK) Real crane motion (from the 1 Hz RTK track)
5-Oct 15:00–17:00 1 trip · 1 h 47 min · 3.30 km 1 trip · 6 min · 0.19 km 0.41 km in 8 min — 6 manoeuvres (12–166 m, 2–5 km/h)
6-Oct 08:00–10:46 2 trips · 2 h 40 min · 6.17 km 1 trip · 2 min · 0.17 km 0.35 km in 6 min — 8 manoeuvres (11–170 m, 2–6 km/h)

Evidence

  1. Same Wialon trip-detector settings on both units. A Wialon-equivalent detector (by GPS speed ≥ 1 km/h, parking ≥ 300 s, min trip distance ≈ 150 m, speed floored to integer km/h as Wialon stores it) reproduces all four Wialon Trips reports: RTK boundaries within ±3–5 s, CalAmp within 0–30 s (one 30-s sample). Parameter sweep over 525 combinations: only this family fits all four. So the difference is in the data, not the configuration.
  2. CalAmp false motion. Per 30-s CalAmp sample, with the RTK as ground truth (0.2 m resolution; H0013 historial matches the Wialon export to 0.07 m, zero lag; Wialon speed = floor(km/h)):
    • while the crane is stationary, CalAmp reports 1–5 km/h 33.5 % (5-Oct) / 43.8 % (6-Oct) of samples;
    • its position jumps 6.5 / 10.4 m median (p90 15 / 26 m, max 34 / 140 m) between consecutive messages while parked → 1.67 km / 3.64 km phantom mileage per window;
    • it detects real motion only 70 % / 47 % of the time;
    • stationary drift it integrates: 785 / 1152 m/h. Because it rarely strings 300 s of zero speed, Wialon never closes the trip → one all-afternoon trip.
    • Wialon's printed CalAmp mileage (3.30 / 5.81 km) is even higher than the path sum of its own exported positions (1.94 / 3.81 km) — Wialon's counter for that unit likely uses the CalAmp odometer parameter; either way it is noise.
  3. HBOX under-reporting is a Wialon threshold, not a device problem. 93–97 % of samples at 0 km/h, stationary scatter p50 1.9 m / p95 4.9 m (DGPS fix quality 2, 12 sats, HDOP 0.5 — note: not RTK fixed/float; corrections issue is separate, see earlier notes). Real RTG moves last 20–120 s and cover 10–170 m; with min distance ≈ 150 m Wialon kept 1 of 6 (5-Oct) and 1 of 8 (6-Oct). The one it kept (15:16–15:22, 166 m) only crossed 150 m thanks to rest-jitter mileage during a 3.8-min pause inside the trip.
  4. HBOX stationary drift at 1 Hz integrates ~0.33 km/h of path (does not affect Wialon trips, which ignore 0 km/h, but dirties tracks/counters) — fixed by the retranslator motion filter below.

The map says it in one glance

RTK track = clean line along the container block; CalAmp track = scribble around a parked crane (see the reports).

What was built — htracker/trips/ (committed to the repo folder on Leo's Mac, not yet pushed)

Pure-stdlib Python package (openpyxl optional) + 40-line hook in retranslator.py + 26-line hook in web_monitor.py. Full details, env vars and deploy steps: 04-Source-Code/htracker/CHANGES-2026-10-06-viajes.md.

  • nmea.py readers: Wialon NMEA export, htracker historial_unidades ring files (unsorted → sorted/deduped), raw NMEA, .gz, Wialon Trips .xlsx (openpyxl must not be in read_only mode — Wialon's xlsx has a bad dimension).
  • detector.py: TripParams + detect_trips() (Wialon-equivalent; defaults 1 km/h · 300 s · 150 m); MotionFilter (REPOSO/MOVIMIENTO hysteresis: ≥1 km/h × 2 samples or > 8 m from anchor → moving; < 1 km/h × 10 s → rest; at rest forward speed 0 and the anchor position). Hold radius 8 m because a parked F9P in DGPS wanders up to ~5 m over 20 min; 3 m caused 114 spurious transitions.
  • compare.py: fine-vs-coarse alignment, confusion matrix, phantom mileage, true mileage, trip matching, chart series; analyze() builds the whole JSON.
  • report.py: self-contained HTML (Chart.js + Leaflet CDN, monitor dark theme) — verdict cards, speed timeline with Wialon trips as bands, cumulative km (device path vs real vs Wialon-printed), map, noise histogram, Wialon-vs-recalculated tables, real-manoeuvre list, filter simulation, method. Also the /trips monitor page (same renderer, API-fed).
  • store.py: trips_positions.db (SQLite WAL) fed by the retranslator at 1 Hz while moving, 1/10 s at rest (10 s, not 30: at 30 s the rest-jitter mileage disappears and the marginal 166-m Wialon trip would be lost; at 10 s all four Wialon reports still reproduce). ≈ 0.7 MB/unit/day, retention 14 d (~300 MB for 30 units). Writer thread, UDP loop never blocks.
  • gateway.py: TripsGateway.process() hook (store raw fix; if the filter is enabled for the unit, send Wialon a copy with speed 0.00 and the held position). Never raises. Env: GW_TRIPS_STORE_ENABLED=1 (default on), GW_MOTION_FILTER_ENABLED=0 (default off), GW_MOTION_FILTER_UNITS=H0013 for a staged rollout.
  • monitor_routes.py: GET /trips, GET /api/trips_1hz/units, POST /api/trips_1hz (unit, window, params, optional reference NMEA file + Wialon .xlsx uploads, raw vs sent data). Same auth/unit gates as /api/track. Nav links added on dashboard and reports.
  • Tests: 22 (unittest, real data fixtures 116 KB gzip): Wialon-report reproduction, readers, filter behaviour, store downsampling, gateway, FastAPI routes (skip without fastapi/openpyxl). Green here and on Leo's Mac (3.14; 7 skipped there for missing openpyxl/fastapi). End-to-end with a fake Wialon listener verified #SD# packets carry speed 0.00 + held position at rest and pass through unchanged when moving.
  • CLI: python3 -m trips.cli --device … --reference … --wialon-trips-device … --out informe.html --json ….

Existing /api/trips (5-minute gps_positions.db) cannot see 20–120 s crane moves; it stays as is.

Recommendations (in order)

  1. Wialon config for HBOX units: min speed 1 km/h, parking 120–300 s, min trip distance 10 m, min time 0. Turns 1 → 6 and 1 → 8 trips (0.50 / 0.48 km ≈ real). Do not apply to the CalAmp (would explode its false trips).
  2. Enable the motion filter on H0013 only (GW_MOTION_FILTER_ENABLED=1, GW_MOTION_FILTER_UNITS=H0013), validate 2–3 days with /trips ("raw" vs "sent"), then fleet-wide. It cleans tracks/counters; it does not change trip counts under the current 150 m threshold.
  3. Fuel = engine hours, not trip km for an RTG. Forward ignition ($PHEV IGN_ON/OFF, fw 0.5.1+; PA5) as a Wialon IPS #D# param ign and use "ignition sensor" engine hours (and optionally ignition-based trips). Proposed, not implemented — needs a one-unit bench check that Wialon accepts the D packet with params from the current unit type.
  4. Do not "match" the HBOX to the CalAmp.

Customer-facing note (es-MX, Leo's voice): 00-Project-Management/client-comms/2026-10-06_Drogheda_Viajes-Wialon-RTK-vs-CalAmp_Nota-ICAVE_v1.0.md.

Open points / caveats

  • Wialon's detector parameters are inferred from the reports (they fit exactly), not read from the account — confirm in Wialon → unit → Trip detector before changing them.
  • H0013 fix quality is 2 (DGPS), never 4/5; the RTK-corrections thread is unchanged by this work.
  • The Wialon-printed CalAmp mileage source (odometer param vs GPS) is unverified; irrelevant to the conclusion.
  • The historial ring covers only ~1 h and is unordered; the new store is the proper source for /trips.
  • Git: Leo runs git add trips/ retranslator.py web_monitor.py .gitignore CHANGES-2026-10-06-viajes.md README.md, commit, ./push-deploy.sh; then check journalctl -u htracker for "TripsStore iniciado".