--- title: "Drogheda HBOX — Bench Log + Config Analysis (20 Jun 2026)" created: 2026-06-20 revised: 2026-06-20 (rev 2 — corrected swap labels + config files) owner: Leo Rodriguez (Helgen) status: draft-for-review type: test-evidence scope: 7 swap-matrix bench captures + NTRIP-caster & u-blox config files, v1 firmware source_logs: "Drive: 06-Testing-Validation/Logs Prueba 20JUN26 (*_GPS.ubx, *_NTRIP.log, 4G_NTRIP_HelgenSRV-NMEA.txt, configuration_GNSS.txt)" tooling: "06-Testing-Validation/log-analysis/analyze_logs.py + dashboard.html" tags: [drogheda, hbox, rtk, ntrip, gnss, bench-test, rca, thermal, config] --- # Drogheda HBOX — Bench Log + Config Analysis **One-line verdict (rev 3):** Of 7 bench captures, 4 pass the comms chain and **only 1 is a confirmed fault** — `RTK21NTRIP21` (master 21: modem won't register, 511 AT errors). The two "no-forward" captures are **indeterminate**, not failed: the NTRIP-master console can't prove forwarding either way (the healthy RTK14 baseline logs the same sparse console). The confirmed fault reproduces cold → not heat. The u-blox config is correct (GGA on UART2 @1 Hz); corrections come from the **free public rtk2go caster**, and no unit reached RTK fix (SPS only — bench sky + corrections rarely applied). > **Rev-3 correction (post-baseline, 2026-06-20).** A healthy RTK14 capture > (`RTK14NTRIP14_440`) — **proven forwarding 1 Hz to the server by tcpdump and live in > Wialon** — logs the *same* sparse NTRIP-master console (only `NET`; no `State main` / > `UDP Bytes sent`) as the captures I had called "no-forward." **Absence of those console > lines does not mean a unit failed.** So the earlier **"master-14 ingest fault" is > withdrawn** (master 14 demonstrably forwards — it *is* the baseline). Only > `RTK21NTRIP21` is a confirmed fault; `RTK14NTRIP14` and `RTK21NTRIP14` are > **indeterminate**. §§3–4 below predate this and are superseded on the no-forward point. > The structural fix: make the **Blackpill log fix + packets + state** so health is > unambiguous — see `04-Source-Code/firmware/docs/LOGGING-SPEC.md`. > **Baseline (RTK14NTRIP14_440), 3 views:** receiver `.ubx` = 341 epochs, 100% valid, > **12 sats, HDOP 0.75, SPS** (C/N0 ~20); server tcpdump = **984 pkts, steady 1 Hz, max > gap 0.86 s, zero dropouts** to `…:5010`; master console = registered only. Healthy > end-to-end **but SPS, not RTK** (corrections not applied). Use as the healthy-GNSS > baseline; raise the bar to a fix≥4 capture once corrections work. > **Rev-2 note:** two captures were re-labelled by their swap pair — > `RTK04NTRIP24 → RTK24NTRIP04` and `RTK14NTRIP21 → RTK21NTRIP14` (byte-identical, > md5-verified). This **moved the no-forward fault from "board 14" to "master 14"** > (the element common to both no-forward captures). Earlier drafts said board 14. --- ## 1. Executive summary Two log streams were captured per unit on the Querétaro bench (20 Jun 2026): `*_GPS.ubx` (the ZED-F9P over USB, via u-center) and `*_NTRIP.log` (the 4G NTRIP Master console). Filenames `RTKNTRIP` = RTK board `bb` + NTRIP master `mm`, forming a **swap matrix** that isolates board vs master faults. Two config files were also analysed: the 4G master config and the F9P config dump. **What the bench can / cannot show:** it validates the **integration/comms chain** (boot → GNSS position → cell registration → UDP forwarding). It does **not** validate RTK accuracy (needs open sky + live corrections) and **cannot** measure heat (v1 has no temperature channel; the bench runs cold). | Outcome | Captures | Why | |---|---|---| | **Healthy (bench)** | RTK17NTRIP17, RTK24NTRIP04, RTK24NTRIP24, RTK25NTRIP25 | boot → GNSS alive → TELCEL registered → UDP forwarding | | **Failed — no forwarding** | RTK14NTRIP14, RTK21NTRIP14 | GNSS valid on USB, modem registered, but **0 UDP** and no state machine — **both use master 14** | | **Failed — modem** | RTK21NTRIP21 | modem **never registers** (`0xFFFFF`, **511× AT-parser errors**), stuck in init | **Isolated faults:** 1. **NTRIP master 14 — GGA ingest.** With *two different* GPS boards (14 and 21), master 14 forwarded **nothing** while the F9P produced valid NMEA on USB. The common element is the master → **master 14's serial input / NMEA ingest**, not the board. (See config note: GGA *is* enabled on UART2, so it is not a message-rate error.) 2. **NTRIP master 21 — LTE modem.** Will not register on Telcel; continuous AT-parser errors. The GPS feeding it (board 21) is fine. Neither is the GNSS receiver; neither needs heat. --- ## 2. The heat question (headline for the deck) **These logs do not prove the field failures are thermal — and that is useful.** They are room-temperature bench captures with no thermal instrumentation, yet they reproduce hard faults **cold** (master-14 ingest, master-21 modem). A fault you can reproduce cold is **not** caused by heat → the fleet problem is **not a single thermal cause**, matching the RCA's "don't collapse everything into heat." This neither confirms nor refutes the **field** thermal hypothesis (18650 / boost / modem in a sealed cabinet in Veracruz sun, RCA H1/H7) — that needs a **thermocouple soak** of a representative cabinet and/or **v2 MCU-temp logging**. --- ## 3. Per-device findings Bench context is common: indoor/obstructed sky → **C/N0 ≈ 17–21 dB-Hz**, high HDOP, **zero RTK fixes** anywhere. Read the GNSS columns as "is the receiver alive and emitting valid NMEA," not as a positioning grade. | Capture | Board / Master | Status | GNSS (USB) | Cell | UDP out | dBm | AT err | Headline | |---|---|---|---|---|---|---|---|---| | **RTK24NTRIP04** | 24 / 04 | 🟢 Healthy | alive, SPS, 98% valid | TELCEL | **1555 dg** | −51 | 0 | full chain OK (master 04 good) | | **RTK24NTRIP24** | 24 / 24 | 🟢 Healthy | alive, SPS, 85% valid | TELCEL | **1831 dg** | −51 | 0 | full chain OK | | **RTK25NTRIP25** | 25 / 25 | 🟢 Healthy | alive, SPS, 100% valid | TELCEL | **2413 dg** | −51 | 0 | best forwarder | | **RTK17NTRIP17** | 17 / 17 | 🟢 Healthy | alive, 12% valid (poor sky this run) | TELCEL | **2267 dg** | −63 | 0 | comms healthiest; GNSS had no sky | | **RTK14NTRIP14** | 14 / 14 | 🔴 Failed | **alive, SPS, 100% valid** | TELCEL | **0** | −75 | 0 | registered but forwards nothing | | **RTK21NTRIP14** | 21 / 14 | 🔴 Failed | **alive, SPS, 100% valid, 10 sats** | TELCEL | **0** | −83 | 0 | same master-14 symptom, other board | | **RTK21NTRIP21** | 21 / 21 | 🔴 Failed | alive, SPS, 97% valid | **NOT reg** | **0** | −113 | **511** | modem never attaches | --- ## 4. Swap-matrix analysis (isolation) **By NTRIP master:** | Master | Captures | Result | |---|---|---| | 04, 17, 24, 25 | one each | **healthy** — forwards position | | **14** | RTK14NTRIP14 (board 14) **and** RTK21NTRIP14 (board 21) | **both fail to forward** despite valid USB NMEA | | **21** | RTK21NTRIP21 (board 21) | modem **not registered**, AT errors | → Master 14 fails with two different boards → **fault is master 14's NMEA ingest** (serial input), confirmed common factor. Master 21 is a modem-registration fault. Masters 04/17/24/25 are good. **By RTK board:** | Board | Captures | Read | |---|---|---| | 24 | NTRIP04 (✓), NTRIP24 (✓) | **healthy with two masters** → board 24 GNSS + UART2 output good | | 17, 25 | one each (✓) | healthy | | 21 | NTRIP14 (master-14 fault), NTRIP21 (master-21 fault) | GNSS alive on USB; its failures belong to the masters it was paired with | | 14 | NTRIP14 only (master-14 fault) | GNSS alive on USB; **board-14 UART2 output unproven** — only ever paired with the bad master 14 | **Open item:** test board 14 (and board 21) with a *known-good* master to clear/convict their UART2 output; test master 14 with a known-good board to confirm the ingest fault. --- ## 5. Configuration findings (from the two config files) ### 5.1 4G NTRIP Master (`4G_NTRIP_HelgenSRV-NMEA.txt`) | Item | Value | Note | |---|---|---| | NTRIP caster | **rtk2go.com : 2101**, mountpoint **HelgenTestStation** | **free public community caster — best-effort, no SLA** | | APN | internet.itelcel.com (Telcel) | correct | | UART baud (to F9P) | 115200 | matches F9P UART2 baud | | NTRIP restart timer | **300 s** | matches "NTRIP DELAY n/300" in the logs | | Telemetry out | UDP → Helgen server :5010, device **H0014** | position forwarding path | | PointPerfect | configured but disabled | not in use | > **Flag:** corrections from a **free public caster (rtk2go)** is a reliability risk > for a production fleet and a plausible reason the correction client rarely connected > and no unit reached RTK. Move production to a **dedicated/own caster (HelgenSRV)**. ### 5.2 ZED-F9P (`configuration_GNSS.txt`, UBX CFG-VALGET dump) | Item | Value | Note | |---|---|---| | Model / FW | ZED-F9P, HPG 1.32 | rover-grade | | Mode | **Rover** (TMODE=0) | correct (not a base) | | Update rate | **1 Hz** (1000 ms) | correct | | **GGA on UART2** | **ENABLED (1/epoch)** | the feed the master needs — **correctly configured** | | RMC on UART2 | ENABLED (1/epoch) | not required by NTRIP, harmless | | UART2 | enabled · NMEA-out + RTCM3-in · 115200 | textbook 4G-NTRIP rover wiring | | USB | full NMEA (GGA/GLL/GSA/GSV/RMC/VTG) | matches the `.ubx` captures | | **Dump layer** | **RAM only** | **flash persistence NOT verified** | > **Two implications:** > 1. **The message config is correct** — GGA *is* on UART2 @1 Hz. So master 14's dead > feed is **not** a missing-GGA setting → it's the master's ingest (or a per-unit > deviation), reinforcing §4. > 2. **The dump is RAM-layer only.** A RAM-only config is **lost on every power cycle** > (the classic "forgot Save Config to flash" trap). Confirm the **FLASH layer** on > each unit — a unit that boots without GGA-on-UART2 would feed the master nothing. --- ## 6. Recommendations / next steps 1. **Master 14 — bench the ingest.** Feed master 14 a known-good GGA stream (or pair with a healthy board) and watch for `State main` + `UDP Bytes sent`. If still zero, the master's serial-in / firmware is the fault → swap/repair the master. 2. **Master 21 — modem triage.** Re-test registration with a known-good SIM + antenna; 511 AT-parser errors + `0xFFFFF` point at the LTE modem / its AT link / power. Swap the modem to confirm. 3. **Persist & verify the F9P config to FLASH** on every unit (u-center → Save Config), then dump the **flash layer** to prove GGA-on-UART2 survives a power cycle. Check the simpleRTK2B Pro **XBEE↔UART2 switch** too (UART2 is off by default). 4. **Move corrections off rtk2go** to a dedicated caster (HelgenSRV) with uptime, then **re-run outdoors** with open sky to actually grade RTK (quality 4/5). 5. **Close the heat question separately** — thermocouple soak of a sealed cabinet in full sun and/or v2 MCU-temp logging. 6. **Retranslator log (separate effort)** — pair this unit-side analysis with the backend (HOL-stall / anti-jump) evidence for the full RCA picture. --- ## 7. Method & reproducibility - **Tool:** `log-analysis/analyze_logs.py` (Python stdlib). Parses NMEA (GGA/RMC/GSV) + UBX from `.ubx`, and the GPTXT diagnostic stream (`State main`, `NET`, `UDP Bytes sent`, `NTRIP DELAY`, `SM_REG`, AT errors) from `.log`. Emits per-device metrics, a swap-matrix with **common-factor isolation**, and per-device time-series. - **Dashboard:** `log-analysis/dashboard.html` — dark (GitHub Dark-Dimmed), self-contained: config summary, signal-flow diagram (marks the break + names the common master), zoomable time-series, status board, swap matrix. - **Config decode:** 4G master `$$$AS+` params; F9P UBX **CFG-VALGET** keys (CFG-MSGOUT-NMEA_ID_GGA_UART2, CFG-UART2*, CFG-RATE*, CFG-TMODE-MODE). - **Re-run:** `python3 analyze_logs.py --in raw_cur --out out && python3 build_dashboard.py` - **Validation:** classification spot-checked vs raw logs; the two relabels were md5-verified byte-identical to the originals. **Caveats:** ~10-min captures; indoor bench sky; v1 (no temperature); F9P config dump is RAM-layer only; `ntr/tcp/pp` enum values inferred from behavior. None affect the pass/fail conclusions, which rest on presence/absence of UDP forwarding, cellular registration, and the swap-matrix common factor. --- ## 8. Glossary - **GGA fix quality:** 0 no-fix · 1 SPS · 2 DGPS · **4 RTK-fixed** · **5 RTK-float**. - **C/N0 (dB-Hz):** satellite carrier-to-noise; ~35–50 open sky, ~20 indoors. - **NTRIP master state `main,ntr,tcp,pp,ntr_srv,udp`:** main=4 operational / 1 init; udp=1 forwarding position out; ntr = correction-client state (0 idle … 2 active). - **`NET,,,`:** TELCEL = registered; **`0xFFFFF` = not registered**. - **NTRIP DELAY n/300:** seconds since last correction vs the 300 s restart watchdog. - **GGA→UART2:** whether the F9P's GGA is reaching the 4G master over UART2 (XBee socket). Inferred from the master running its loop + forwarding UDP.