234 lines
13 KiB
Markdown
234 lines
13 KiB
Markdown
---
|
||
title: "Drogheda HBOX — Bench Log + Config Analysis (20 Jun 2026)"
|
||
created: 2026-06-20
|
||
revised: 2026-06-20 (rev 2 — corrected swap labels + config files)
|
||
owner: Leo Rodriguez (Helgen)
|
||
status: draft-for-review
|
||
type: test-evidence
|
||
scope: 7 swap-matrix bench captures + NTRIP-caster & u-blox config files, v1 firmware
|
||
source_logs: "Drive: 06-Testing-Validation/Logs Prueba 20JUN26 (*_GPS.ubx, *_NTRIP.log, 4G_NTRIP_HelgenSRV-NMEA.txt, configuration_GNSS.txt)"
|
||
tooling: "06-Testing-Validation/log-analysis/analyze_logs.py + dashboard.html"
|
||
tags: [drogheda, hbox, rtk, ntrip, gnss, bench-test, rca, thermal, config]
|
||
---
|
||
|
||
# Drogheda HBOX — Bench Log + Config Analysis
|
||
|
||
**One-line verdict (rev 3):** Of 7 bench captures, 4 pass the comms chain and
|
||
**only 1 is a confirmed fault** — `RTK21NTRIP21` (master 21: modem won't register,
|
||
511 AT errors). The two "no-forward" captures are **indeterminate**, not failed: the
|
||
NTRIP-master console can't prove forwarding either way (the healthy RTK14 baseline
|
||
logs the same sparse console). The confirmed fault reproduces cold → not heat. The
|
||
u-blox config is correct (GGA on UART2 @1 Hz); corrections come from the **free public
|
||
rtk2go caster**, and no unit reached RTK fix (SPS only — bench sky + corrections rarely
|
||
applied).
|
||
|
||
> **Rev-3 correction (post-baseline, 2026-06-20).** A healthy RTK14 capture
|
||
> (`RTK14NTRIP14_440`) — **proven forwarding 1 Hz to the server by tcpdump and live in
|
||
> Wialon** — logs the *same* sparse NTRIP-master console (only `NET`; no `State main` /
|
||
> `UDP Bytes sent`) as the captures I had called "no-forward." **Absence of those console
|
||
> lines does not mean a unit failed.** So the earlier **"master-14 ingest fault" is
|
||
> withdrawn** (master 14 demonstrably forwards — it *is* the baseline). Only
|
||
> `RTK21NTRIP21` is a confirmed fault; `RTK14NTRIP14` and `RTK21NTRIP14` are
|
||
> **indeterminate**. §§3–4 below predate this and are superseded on the no-forward point.
|
||
> The structural fix: make the **Blackpill log fix + packets + state** so health is
|
||
> unambiguous — see `04-Source-Code/firmware/docs/LOGGING-SPEC.md`.
|
||
|
||
> **Baseline (RTK14NTRIP14_440), 3 views:** receiver `.ubx` = 341 epochs, 100% valid,
|
||
> **12 sats, HDOP 0.75, SPS** (C/N0 ~20); server tcpdump = **984 pkts, steady 1 Hz, max
|
||
> gap 0.86 s, zero dropouts** to `…:5010`; master console = registered only. Healthy
|
||
> end-to-end **but SPS, not RTK** (corrections not applied). Use as the healthy-GNSS
|
||
> baseline; raise the bar to a fix≥4 capture once corrections work.
|
||
|
||
> **Rev-2 note:** two captures were re-labelled by their swap pair —
|
||
> `RTK04NTRIP24 → RTK24NTRIP04` and `RTK14NTRIP21 → RTK21NTRIP14` (byte-identical,
|
||
> md5-verified). This **moved the no-forward fault from "board 14" to "master 14"**
|
||
> (the element common to both no-forward captures). Earlier drafts said board 14.
|
||
|
||
---
|
||
|
||
## 1. Executive summary
|
||
|
||
Two log streams were captured per unit on the Querétaro bench (20 Jun 2026):
|
||
`*_GPS.ubx` (the ZED-F9P over USB, via u-center) and `*_NTRIP.log` (the 4G NTRIP
|
||
Master console). Filenames `RTK<bb>NTRIP<mm>` = RTK board `bb` + NTRIP master `mm`,
|
||
forming a **swap matrix** that isolates board vs master faults. Two config files
|
||
were also analysed: the 4G master config and the F9P config dump.
|
||
|
||
**What the bench can / cannot show:** it validates the **integration/comms chain**
|
||
(boot → GNSS position → cell registration → UDP forwarding). It does **not** validate
|
||
RTK accuracy (needs open sky + live corrections) and **cannot** measure heat (v1 has
|
||
no temperature channel; the bench runs cold).
|
||
|
||
| Outcome | Captures | Why |
|
||
|---|---|---|
|
||
| **Healthy (bench)** | RTK17NTRIP17, RTK24NTRIP04, RTK24NTRIP24, RTK25NTRIP25 | boot → GNSS alive → TELCEL registered → UDP forwarding |
|
||
| **Failed — no forwarding** | RTK14NTRIP14, RTK21NTRIP14 | GNSS valid on USB, modem registered, but **0 UDP** and no state machine — **both use master 14** |
|
||
| **Failed — modem** | RTK21NTRIP21 | modem **never registers** (`0xFFFFF`, **511× AT-parser errors**), stuck in init |
|
||
|
||
**Isolated faults:**
|
||
|
||
1. **NTRIP master 14 — GGA ingest.** With *two different* GPS boards (14 and 21),
|
||
master 14 forwarded **nothing** while the F9P produced valid NMEA on USB. The
|
||
common element is the master → **master 14's serial input / NMEA ingest**, not the
|
||
board. (See config note: GGA *is* enabled on UART2, so it is not a message-rate
|
||
error.)
|
||
2. **NTRIP master 21 — LTE modem.** Will not register on Telcel; continuous
|
||
AT-parser errors. The GPS feeding it (board 21) is fine.
|
||
|
||
Neither is the GNSS receiver; neither needs heat.
|
||
|
||
---
|
||
|
||
## 2. The heat question (headline for the deck)
|
||
|
||
**These logs do not prove the field failures are thermal — and that is useful.**
|
||
They are room-temperature bench captures with no thermal instrumentation, yet they
|
||
reproduce hard faults **cold** (master-14 ingest, master-21 modem). A fault you can
|
||
reproduce cold is **not** caused by heat → the fleet problem is **not a single
|
||
thermal cause**, matching the RCA's "don't collapse everything into heat."
|
||
|
||
This neither confirms nor refutes the **field** thermal hypothesis (18650 / boost / modem
|
||
in a sealed cabinet in Veracruz sun, RCA H1/H7) — that needs a **thermocouple soak**
|
||
of a representative cabinet and/or **v2 MCU-temp logging**.
|
||
|
||
---
|
||
|
||
## 3. Per-device findings
|
||
|
||
Bench context is common: indoor/obstructed sky → **C/N0 ≈ 17–21 dB-Hz**, high HDOP,
|
||
**zero RTK fixes** anywhere. Read the GNSS columns as "is the receiver alive and
|
||
emitting valid NMEA," not as a positioning grade.
|
||
|
||
| Capture | Board / Master | Status | GNSS (USB) | Cell | UDP out | dBm | AT err | Headline |
|
||
|---|---|---|---|---|---|---|---|---|
|
||
| **RTK24NTRIP04** | 24 / 04 | 🟢 Healthy | alive, SPS, 98% valid | TELCEL | **1555 dg** | −51 | 0 | full chain OK (master 04 good) |
|
||
| **RTK24NTRIP24** | 24 / 24 | 🟢 Healthy | alive, SPS, 85% valid | TELCEL | **1831 dg** | −51 | 0 | full chain OK |
|
||
| **RTK25NTRIP25** | 25 / 25 | 🟢 Healthy | alive, SPS, 100% valid | TELCEL | **2413 dg** | −51 | 0 | best forwarder |
|
||
| **RTK17NTRIP17** | 17 / 17 | 🟢 Healthy | alive, 12% valid (poor sky this run) | TELCEL | **2267 dg** | −63 | 0 | comms healthiest; GNSS had no sky |
|
||
| **RTK14NTRIP14** | 14 / 14 | 🔴 Failed | **alive, SPS, 100% valid** | TELCEL | **0** | −75 | 0 | registered but forwards nothing |
|
||
| **RTK21NTRIP14** | 21 / 14 | 🔴 Failed | **alive, SPS, 100% valid, 10 sats** | TELCEL | **0** | −83 | 0 | same master-14 symptom, other board |
|
||
| **RTK21NTRIP21** | 21 / 21 | 🔴 Failed | alive, SPS, 97% valid | **NOT reg** | **0** | −113 | **511** | modem never attaches |
|
||
|
||
---
|
||
|
||
## 4. Swap-matrix analysis (isolation)
|
||
|
||
**By NTRIP master:**
|
||
|
||
| Master | Captures | Result |
|
||
|---|---|---|
|
||
| 04, 17, 24, 25 | one each | **healthy** — forwards position |
|
||
| **14** | RTK14NTRIP14 (board 14) **and** RTK21NTRIP14 (board 21) | **both fail to forward** despite valid USB NMEA |
|
||
| **21** | RTK21NTRIP21 (board 21) | modem **not registered**, AT errors |
|
||
|
||
→ Master 14 fails with two different boards → **fault is master 14's NMEA ingest**
|
||
(serial input), confirmed common factor. Master 21 is a modem-registration fault.
|
||
Masters 04/17/24/25 are good.
|
||
|
||
**By RTK board:**
|
||
|
||
| Board | Captures | Read |
|
||
|---|---|---|
|
||
| 24 | NTRIP04 (✓), NTRIP24 (✓) | **healthy with two masters** → board 24 GNSS + UART2 output good |
|
||
| 17, 25 | one each (✓) | healthy |
|
||
| 21 | NTRIP14 (master-14 fault), NTRIP21 (master-21 fault) | GNSS alive on USB; its failures belong to the masters it was paired with |
|
||
| 14 | NTRIP14 only (master-14 fault) | GNSS alive on USB; **board-14 UART2 output unproven** — only ever paired with the bad master 14 |
|
||
|
||
**Open item:** test board 14 (and board 21) with a *known-good* master to clear/convict
|
||
their UART2 output; test master 14 with a known-good board to confirm the ingest fault.
|
||
|
||
---
|
||
|
||
## 5. Configuration findings (from the two config files)
|
||
|
||
### 5.1 4G NTRIP Master (`4G_NTRIP_HelgenSRV-NMEA.txt`)
|
||
|
||
| Item | Value | Note |
|
||
|---|---|---|
|
||
| NTRIP caster | **rtk2go.com : 2101**, mountpoint **HelgenTestStation** | **free public community caster — best-effort, no SLA** |
|
||
| APN | internet.itelcel.com (Telcel) | correct |
|
||
| UART baud (to F9P) | 115200 | matches F9P UART2 baud |
|
||
| NTRIP restart timer | **300 s** | matches "NTRIP DELAY n/300" in the logs |
|
||
| Telemetry out | UDP → Helgen server :5010, device **H0014** | position forwarding path |
|
||
| PointPerfect | configured but disabled | not in use |
|
||
|
||
> **Flag:** corrections from a **free public caster (rtk2go)** is a reliability risk
|
||
> for a production fleet and a plausible reason the correction client rarely connected
|
||
> and no unit reached RTK. Move production to a **dedicated/own caster (HelgenSRV)**.
|
||
|
||
### 5.2 ZED-F9P (`configuration_GNSS.txt`, UBX CFG-VALGET dump)
|
||
|
||
| Item | Value | Note |
|
||
|---|---|---|
|
||
| Model / FW | ZED-F9P, HPG 1.32 | rover-grade |
|
||
| Mode | **Rover** (TMODE=0) | correct (not a base) |
|
||
| Update rate | **1 Hz** (1000 ms) | correct |
|
||
| **GGA on UART2** | **ENABLED (1/epoch)** | the feed the master needs — **correctly configured** |
|
||
| RMC on UART2 | ENABLED (1/epoch) | not required by NTRIP, harmless |
|
||
| UART2 | enabled · NMEA-out + RTCM3-in · 115200 | textbook 4G-NTRIP rover wiring |
|
||
| USB | full NMEA (GGA/GLL/GSA/GSV/RMC/VTG) | matches the `.ubx` captures |
|
||
| **Dump layer** | **RAM only** | **flash persistence NOT verified** |
|
||
|
||
> **Two implications:**
|
||
> 1. **The message config is correct** — GGA *is* on UART2 @1 Hz. So master 14's dead
|
||
> feed is **not** a missing-GGA setting → it's the master's ingest (or a per-unit
|
||
> deviation), reinforcing §4.
|
||
> 2. **The dump is RAM-layer only.** A RAM-only config is **lost on every power cycle**
|
||
> (the classic "forgot Save Config to flash" trap). Confirm the **FLASH layer** on
|
||
> each unit — a unit that boots without GGA-on-UART2 would feed the master nothing.
|
||
|
||
---
|
||
|
||
## 6. Recommendations / next steps
|
||
|
||
1. **Master 14 — bench the ingest.** Feed master 14 a known-good GGA stream (or pair
|
||
with a healthy board) and watch for `State main` + `UDP Bytes sent`. If still zero,
|
||
the master's serial-in / firmware is the fault → swap/repair the master.
|
||
2. **Master 21 — modem triage.** Re-test registration with a known-good SIM + antenna;
|
||
511 AT-parser errors + `0xFFFFF` point at the LTE modem / its AT link / power. Swap
|
||
the modem to confirm.
|
||
3. **Persist & verify the F9P config to FLASH** on every unit (u-center → Save Config),
|
||
then dump the **flash layer** to prove GGA-on-UART2 survives a power cycle. Check the
|
||
simpleRTK2B Pro **XBEE↔UART2 switch** too (UART2 is off by default).
|
||
4. **Move corrections off rtk2go** to a dedicated caster (HelgenSRV) with uptime, then
|
||
**re-run outdoors** with open sky to actually grade RTK (quality 4/5).
|
||
5. **Close the heat question separately** — thermocouple soak of a sealed cabinet in
|
||
full sun and/or v2 MCU-temp logging.
|
||
6. **Retranslator log (separate effort)** — pair this unit-side analysis with the
|
||
backend (HOL-stall / anti-jump) evidence for the full RCA picture.
|
||
|
||
---
|
||
|
||
## 7. Method & reproducibility
|
||
|
||
- **Tool:** `log-analysis/analyze_logs.py` (Python stdlib). Parses NMEA (GGA/RMC/GSV) +
|
||
UBX from `.ubx`, and the GPTXT diagnostic stream (`State main`, `NET`, `UDP Bytes
|
||
sent`, `NTRIP DELAY`, `SM_REG`, AT errors) from `.log`. Emits per-device metrics,
|
||
a swap-matrix with **common-factor isolation**, and per-device time-series.
|
||
- **Dashboard:** `log-analysis/dashboard.html` — dark (GitHub Dark-Dimmed),
|
||
self-contained: config summary, signal-flow diagram (marks the break + names the
|
||
common master), zoomable time-series, status board, swap matrix.
|
||
- **Config decode:** 4G master `$$$AS+` params; F9P UBX **CFG-VALGET** keys
|
||
(CFG-MSGOUT-NMEA_ID_GGA_UART2, CFG-UART2*, CFG-RATE*, CFG-TMODE-MODE).
|
||
- **Re-run:** `python3 analyze_logs.py --in raw_cur --out out && python3 build_dashboard.py`
|
||
- **Validation:** classification spot-checked vs raw logs; the two relabels were
|
||
md5-verified byte-identical to the originals.
|
||
|
||
**Caveats:** ~10-min captures; indoor bench sky; v1 (no temperature); F9P config dump
|
||
is RAM-layer only; `ntr/tcp/pp` enum values inferred from behavior. None affect the
|
||
pass/fail conclusions, which rest on presence/absence of UDP forwarding, cellular
|
||
registration, and the swap-matrix common factor.
|
||
|
||
---
|
||
|
||
## 8. Glossary
|
||
|
||
- **GGA fix quality:** 0 no-fix · 1 SPS · 2 DGPS · **4 RTK-fixed** · **5 RTK-float**.
|
||
- **C/N0 (dB-Hz):** satellite carrier-to-noise; ~35–50 open sky, ~20 indoors.
|
||
- **NTRIP master state `main,ntr,tcp,pp,ntr_srv,udp`:** main=4 operational / 1 init;
|
||
udp=1 forwarding position out; ntr = correction-client state (0 idle … 2 active).
|
||
- **`NET,<op>,<dBm>,<MCCMNC>`:** TELCEL = registered; **`0xFFFFF` = not registered**.
|
||
- **NTRIP DELAY n/300:** seconds since last correction vs the 300 s restart watchdog.
|
||
- **GGA→UART2:** whether the F9P's GGA is reaching the 4G master over UART2 (XBee
|
||
socket). Inferred from the master running its loop + forwarding UDP.
|