---
tags: [project-drogheda, hardware, firmware, watchdog, telemetry, nmea]
project: Project Drogheda
author: Helgen CTO
date: 2026-06-20
status: Analysis — review & bench-test before any wiring/firmware change
---

# Watchdog Health Telemetry — Feasibility & Improvement Analysis

> **The question:** can the STM32F411 watchdog transmit its health (temp, reset
> cause, link state, recovery count…) over UART so it **lands in our server
> message**, e.g. as a custom `$GPTXT`? You've seen mixed information. This
> resolves it against the **as-built wiring** and proposes a clean design.

---

## TL;DR

- **As wired today: No.** The watchdog sits on the F9P's **UART1** (it reads NMEA
  and sends UBX rate commands). The data that reaches the server flows out the
  F9P's **UART2 → 4G NTRIP Master → server**. The watchdog is **not on that path**,
  so nothing it prints can reach the backend.
- The `$GPTXT,NET,<id>,TELCEL,<rssi>,<cell>` you see at the server is the **4G
  modem's** own network-status line (the `<id>` is the modem's station ID set via
  `AS+K`, which the backend treats as the `cell_id`). It is **not** produced by the
  F9P or the watchdog.
- **It is achievable** — but it requires putting the watchdog **in the modem's
  NMEA-uplink path** (a relay/injector) and emitting a **proprietary sentence**
  (`$PHLG…`, *not* `$GPTXT` — see the collision note). This is a **wiring change**,
  clean on the Phase-2 PCB, and it must be **bench-validated** against one unknown:
  *does the ArduSimple 4G NTRIP Master forward arbitrary/proprietary NMEA, or only a
  whitelist?* That single question is the source of the "mixed information."
- **Lower-risk interim (no rewiring):** the watchdog already has a full health
  record — ship the **v2.1 parseable log** (`docs/LOGGING-SPEC.md`) over its console.
  It's authoritative per-unit health, captured on pulled units; it just isn't *live
  to the server*. In-band-to-server telemetry is the Phase-2 upgrade.

---

## 1. The as-built data path (why the answer is "not yet")

```text
          ┌──────────────── simpleRTK2B Pro (ZED-F9P) ─────────────────┐
          │                                                            │
 UART1 ◄──┤ TX1 → NMEA (GGA…)  ──►  STM32 PA3 (RX)   [watchdog: READS] │
          │ RX1 ← UBX rate     ◄──  STM32 PA2 (TX)   [watchdog: WRITES]│
          │                                                            │
 UART2 ◄──┤ TX2 → NMEA/GGA @1Hz ──►  4G NTRIP Master (XBee) ──► SERVER │  ← server path
          │ RX2 ← RTCM3        ◄──  4G NTRIP Master (NTRIP caster)     │
          └────────────────────────────────────────────────────────────┘
                                         │
                       4G modem also emits its OWN  $GPTXT,NET,<AS+K id>,TELCEL,…
                       (network status + station id = the backend's cell_id)
```

Key facts (confirmed in firmware + the 20-Jun bench analysis):

- Firmware: `HardwareSerial GNSS(PA3, PA2)` → **F9P UART1**. `parseNMEA()` only
  *reads* GGA (for `fixQuality`/`satellites`/`rxCount`); `applyRate()` only *writes*
  UBX. On power-off it **floats** PA2/PA3. The watchdog never transmits toward the
  modem.
- The modem is on the Pro's **XBee socket = UART2** (the "XBEE↔UART2 switch"); GGA is
  configured out **UART2 @ 1 Hz**. The server stream and the NTRIP GGA-uplink both
  ride UART2.
- Therefore the watchdog (UART1) and the server path (UART2) are **electrically
  separate**. No firmware change alone can bridge them.

---

## 2. The three theoretical ways in (and which one to use)

1. **Make the F9P emit it.** ✗ Not possible. The ZED-F9P has no command to inject an
   arbitrary NMEA string into its UART2 output. It emits only its own sentences + UBX
   replies.
2. **Tell the 4G modem to carry it.** ✗ No runtime path. The modem's `$GPTXT` is
   network status; `AS+K` (station id) is *set-once* config, not a data channel the
   MCU can stuff health into per-cycle.
3. **Put the watchdog in the uplink NMEA path (relay/injector).** ✓ **The viable
   design.** Route the F9P→modem NMEA *through* the STM32:

   ```text
   F9P TX2 (NMEA) ──► STM32 RX  ──[relay verbatim]──► STM32 TX ──► modem RX
                                  └─[inject $PHLG health @ ~1/30s in the idle gap]
   modem ──► forwards EVERYTHING (GGA/RMC/GPTXT + $PHLG) ──► server
   RTCM from the modem can still go straight to the F9P RX2 (corrections path intact).
   ```

   The backend already maps **(ip,port) → cell_id** (from the modem's `$GPTXT`), so a
   `$PHLG` arriving on the same socket is **auto-associated to the right unit** — no
   need to put the id in the health sentence. (This reuses the CGNAT-safe identity work
   shipped in `retranslator.py`.)

> **Retrofit variant (no full relay):** diode-OR / open-drain the STM32 TX onto the
> modem's RX and inject only during the F9P's idle gaps (at 115200 baud, 1 Hz NMEA
> leaves the line idle >90 % of the time). Cheaper to bolt on, but collision-prone and
> harder to guarantee — **prefer the clean relay on the PCB.**

---

## 3. The proprietary sentence (don't reuse `$GPTXT`)

**Collision warning:** the backend parses `$GPTXT` by taking **field 2 as the
`cell_id`**. A health `$GPTXT` would be mis-read as a device id. Use a **proprietary
talker** instead — `$PHLG` (`P` = proprietary, `HLG` = Helgen):

```text
$PHLG,<fw>,<state>,<link>,<fix>,<sat>,<rx>,<vbat>,<v5v>,<tc>,<tmax>,<rec>,<limp>,<nmea_age_ms>,<reset>*CS
e.g.  $PHLG,2.1,RUN,LIMP,1,12,34192,4.01,5.02,63,71,2,1,420,IWDG*4F
```

Fields mirror the v2.1 `TLM` record (state, link, fix, sats, rx, battery, 5 V,
**MCU temp + session max**, recovery count, limp latch, NMEA age, **reset cause**).
`CS` = standard NMEA XOR checksum.

**Backend side** is a ~15-line addition to the hardened `retranslator.py`: a branch
that recognises `$PHLG`, verifies the checksum, and stores the fields per `cell_id`
(keyed by the existing `(ip,port)` map). Then the monitor can surface, **per unit and
live**: MCU temp / `tMax` (the thermal-hypothesis evidence, without pulling the unit),
reset cause (reboot-loop forensics), recovery count, and LIMP state — and it ships to
Loki for free.

---

## 4. What to bench-test before committing (resolves the "mixed info")

1. **Does the 4G NTRIP Master forward a proprietary `$P…` sentence?** Inject one on
   UART2-RX on the bench and watch the server `tcpdump`. *If it whitelists talkers:*
   fall back to a `$GPTXT,HLT,…` subtype and special-case it in the backend by **field
   1** (`HLT` vs `NET`) so it never collides with the cell-id parse. **This is the one
   experiment that settles the whole question.**
2. **GGA integrity for NTRIP.** The modem relays the F9P's GGA *up to the caster* for
   network RTK. The relay must be **byte-exact and low-latency** and only inject in
   idle gaps — never delay or mangle GGA, or corrections degrade.
3. **Rate / cost.** Health at **1 / 30–60 s** keeps cellular + backend load trivial.
4. **Checksum** must be correct or the backend (and possibly the modem) drops it.

---

## 5. Other watchdog improvements (independent of the telemetry question)

### 🔴 5a. The "no-RTK ≠ dead-modem" thrash risk — *act on this now*
v2 treats `fixQuality >= 4` (RTK) as the only healthy link, and after
`NTRIP_GRACE` (120 s) of `< 4` it **power-cycles the stack**. But the 20-Jun bench
reality is **every unit is SPS (quality 1), no RTK** (caster/FLASH/sky issue, *not* a
modem hang). With the watchdog connected as-is, a unit that is **GNSS-healthy and
forwarding fine at SPS** would be **power-cycled every ~2 min + backoff** — manufacturing
the very disconnects we're trying to kill. **Fix:** only power-cycle on evidence of a
*link/modem* hang (NMEA silence is already handled); treat *stable SPS with incrementing
`rx`* as a reportable **DEGRADED** state, **not** a recovery trigger — at least until the
corrections pipeline (move off rtk2go, save F9P config to FLASH, open-sky retest) is
fixed. This is the highest-value firmware change right now.

### 🟠 5b. Ship the v2.1 parseable log (`docs/LOGGING-SPEC.md`)
The ANSI dashboard overwrites itself and has no timestamps/reset-cause → useless as a
field record. The v2.1 spec (BOOT/COLS/TLM/EVT, reset cause, ~60 additive lines) makes
the watchdog the **authoritative per-unit health record**. It's also the exact data the
`$PHLG` sentence would carry — **do this first**, so every later change is observable.

### 🟠 5c. Independent watchdog (IWDG) — watch the watcher
Nothing currently recovers the Blackpill itself if *it* hangs. Enabling the STM32 IWDG
makes an MCU lockup self-reset and show `reset=IWDG` next boot — closing the last
"silent vanish" gap for a device whose whole job is fail-safe.

### 🟠 5d. External temp sensor + persisted counters
v2 logs **MCU** temp, but the heat casualties are the **18650 / 4G modem**, not the die.
Add a small external sensor near the cell/modem (PCB brief already flags this), and
**persist** `tMax` / `recoveryCount` / reset history to RTC backup registers or flash so
a pulled unit's forensics survive the power-cycle. Directly serves the thermal RCA.

---

## 6. Recommendation (phased)

| When | Action | Risk |
|---|---|---|
| **Now** (firmware only) | 5a (stop SPS-driven power-cycling) + 5b (v2.1 log) + 5c (IWDG). Bench-test, bump v2.1, then flash. | Low — additive / behavioural guard. |
| **Bench experiment** | Test #1: does the modem forward `$P…`? Decide `$PHLG` vs `$GPTXT,HLT`. | None — measurement. |
| **Phase 2 (PCB)** | Watchdog as NMEA relay/injector on the UART2 uplink + external temp sensor; backend `$PHLG` parser + monitor/Loki surfacing. | Medium — relay must preserve GGA/NTRIP; validate before fleet. |

**Bottom line for the deck:** *Yes, we can put unit health (incl. temperature) into the
live server feed — it needs the watchdog inserted into the modem's uplink path (a
Phase-2 PCB change) and one bench test of the modem's forwarding. Until then the same
health is captured on-unit via the v2.1 log. Separately, before we connect the watchdog
we must stop it from power-cycling healthy SPS units.*

## Related
- [[03-Architecture-Hardware/watchdog-schematic|watchdog-schematic]] ·
  [[03-Architecture-Hardware/firmware-review|firmware-review]] ·
  [[03-Architecture-Hardware/hardware-respec-and-pcb-brief|PCB brief]] ·
  `04-Source-Code/firmware/docs/LOGGING-SPEC.md` ·
  `04-Source-Code/firmware/blackpillrtkcode_v2/blackpillrtkcode_v2.ino`