auto-sync 2026-06-22 18:00

This commit is contained in:
Leopoldo Rodríguez 2026-06-22 18:00:16 -06:00
parent 0f74fde451
commit eef023e15a
16 changed files with 731 additions and 60 deletions

View File

@ -0,0 +1,109 @@
---
tags: [project-drogheda, backend, code-review, scale]
project: Project Drogheda
author: Helgen CTO
date: 2026-06-20
status: Review — for the "add more units" test campaign
---
# Code Review — retranslator + monitor (scale readiness)
> Review of the **hardened** `retranslator.py`, `web_monitor.py`, and the
> `gps_logger.py` they depend on, ahead of bringing more units online. The three
> 🔴 field-killers (HOL blocking, anti-jump black-hole, CGNAT identity) are already
> fixed — this pass is about **what else bites as the fleet grows**. Nothing here is
> on fire; the top items are the ones you'll actually hit while testing.
Severity: 🔴 blocks/badly misleads at scale · 🟠 should fix before the fleet grows ·
🟡 polish.
> **Status (2026-06-20):** **R1, R2, G1, G2 are implemented and verified** — smoke
> (10/10, no regression), an R1/R2 focused test (7/7), and a standalone index+prune
> test all pass. Remaining: **M1** (dashboard auth — an nginx change, server-side) and
> the 🟡 polish items. New env knob: `GPS_RETENTION_DAYS` (default 30, 0 = off).
## Summary
The hardened code is solid and the concurrency model is sound. The remaining risks
are mostly **operational at scale**: a new unit that isn't in `device_config.json`
silently won't forward (and the monitor won't tell you why); the SQLite history has
no index and no retention (it will scan slowly and grow until the small disk fills);
the dashboard has no auth and re-renders 500 rows every 2 s; and a couple of latent
parsing/ack assumptions. No SQL injection (queries are parameterized) and no obvious
XSS.
## 🔴 / 🟠 Issues to fix before adding units
| # | File · location | Issue | Sev |
|---|---|---|---|
| R1 | `retranslator.py` · `_process_device_telemetry` (`wialon_id = device_cell_id_to_wialon_id.get(cell_id)`) | **An unconfigured `cell_id` is silently never forwarded.** A new unit's GPTXT marks it *online* (so it appears active in the monitor) but with no `wialon_id` mapping it's never sent to Wialon — and it shows as `wialon_state="idle"`, not a clear "unconfigured". This is the #1 thing you'll hit adding units. | 🔴 |
| G2 | `gps_logger.py` · whole file | **SQLite history grows unbounded.** ~288 rows/unit/day → ~144k/day at 500 units, no pruning. On the space-limited server this **fills the disk** over weeks. | 🔴 |
| G1 | `gps_logger.py` · `_create_table` | **No index** on `gps_positions`. `/api/availability` does `WHERE received_at … GROUP BY day,device` → full scan; at millions of rows the report crawls. | 🟠 |
| M1 | `web_monitor.py` / nginx | **No auth on the dashboard or API.** `monitor.helgen.dev` exposes live fleet positions + `/api/status` + `/api/availability` to anyone with the URL. Client-sensitive. | 🟠 |
| R2 | `retranslator.py` · `main()` UDP loop (`raw = udp_data.decode(...).strip()`) | **One NMEA sentence per UDP datagram is assumed.** If a unit ever batches sentences in one packet (`$GPTXT…\r\n$GNGGA…`), only the first is parsed. Latent today (firmware sends one per datagram) but a trap for the planned `$PHLG` injection. | 🟠 |
| R3 | `retranslator.py` · `WialonForwarder._deliver` (`response = s.recv(1024)`) | **Wialon data-ack isn't validated.** Only the *login* ack (`#AL#1`) is checked; the per-packet ack is read and discarded, so a **rejected** packet still counts as `acked` → monitor shows the unit healthy when Wialon isn't accepting it. | 🟠 |
## 🟡 Worth doing
| # | File · location | Suggestion |
|---|---|---|
| R4 | `retranslator.py` · config load | New units need a **restart** (`device_config.json` read once). Add a `SIGHUP` handler to reload the map live — no fleet-wide blip every time you add a test unit. |
| M2 | `web_monitor.py` · `renderTable()` (`tbody.innerHTML = rows.map(...)`) | Re-renders **all 500 rows every 2 s**, churning the DOM and dropping scroll/selection. Diff rows, or only re-render on change. |
| R5 | `retranslator.py` · `_build_data_packet` (`datetime.utcnow()`) | Wialon timestamp = **server receive time**, not the GGA fix time. Sub-second for a slow fleet, but parse the GGA UTC if you want exact fix times. |
| R6 | `retranslator.py` · `WialonForwarder.forget` | A `forget()` on a full worker queue is silently dropped → a socket can leak until the next send error. Bounded; low impact. |
| R7 | `retranslator.py` · `_deliver` ack read | Residual per-worker HOL: one slow ack blocks that worker's ~8 devices for up to `TCP_RW_TIMEOUT` (15 s). Far better than global; consider lowering the read timeout. |
| G3 | `gps_logger.py` · `_insert` | `commit()` per row. Fine at the 300 s throttle; revisit if the throttle drops or inserts burst. |
| M3 | `web_monitor.py` · `sse_events` | Re-parses the whole status JSON every 2 s only to validate. Trivial now; grows with the device count. |
## What looks good ✅
- **Concurrency model** — UDP ingest fully decoupled from Wialon I/O; per-worker
queues with drop-oldest backpressure; stats behind a lock; `list()` snapshots in
the stats/cleanup threads (no "dict changed size").
- **Identity** — `(ip,port)` transport + `cell_id` state is the right CGNAT-safe shape.
- **Anti-jump** — time-aware + self-healing + cleared on disconnect; can't black-hole.
- **Safety** — atomic status-file write (`.tmp` + `rename`); fail-safe drops; SQL writer
is a bounded background thread that never blocks the loop.
- **Security hygiene** — availability SQL is **parameterized** (`?`), DB opened
read-only, free-text escaped (`esc()`), SSE degrades to "gateway down" cleanly.
## Before you test more units — do these first
1. **R1 — make unconfigured units obvious.** Two cheap parts: log a one-time
`WARNING` per unknown `cell_id`, and surface a distinct `wialon_state="unconfigured"`
in the monitor so a new unit that isn't forwarding is self-explanatory.
```python
# in _process_device_telemetry, where wialon_id is looked up:
wialon_id = device_cell_id_to_wialon_id.get(cell_id)
if not wialon_id:
if cell_id not in _warned_unconfigured: # module-level set()
_warned_unconfigured.add(cell_id)
log.warning(f"[CONFIG] cell_id {cell_id} sin wialon_id — "
f"agrégalo a device_config.json o no se reenvía.")
# still record telemetry/odometer; just don't forward
```
2. **G2 — cap the SQLite growth.** A nightly prune keeps the disk safe:
```sql
DELETE FROM gps_positions WHERE received_at < datetime('now','-30 days');
```
(run from the writer on a daily timer, or a cron `sqlite3` one-liner + `VACUUM`).
3. **G1 — add the index** (one line in `_create_table`, idempotent):
```sql
CREATE INDEX IF NOT EXISTS idx_gps_recv_dev
ON gps_positions(received_at, device_cell_id);
```
4. **M1 — gate the dashboard.** Fastest is nginx Basic Auth on `monitor.helgen.dev`
(one `auth_basic` + htpasswd), or an IP allowlist for the office/VPN.
5. **R2 — split multi-sentence datagrams** (`for line in raw.split('\n'): …`) so the
loop is robust before you wire any new sender (incl. `$PHLG`).
## Verdict
**Approve, with the R1/G1/G2 items recommended before the test campaign.** The core is
production-grade; the gaps are operational (config ergonomics, DB lifecycle, dashboard
auth) and quick to close. R1 and G2 are the two that will actually bite you while
adding units.
## Related
- [[02-Architecture-Software/backend-review|backend-review]] (the original 🔴 findings + resolution) ·
`04-Source-Code/htracker/retranslator.py` · `web_monitor.py` · `gps_logger.py`

Binary file not shown.

View File

@ -106,7 +106,10 @@ talker** instead — `$PHLG` (`P` = proprietary, `HLG` = Helgen):
```text
$PHLG,<fw>,<state>,<link>,<fix>,<sat>,<rx>,<vbat>,<v5v>,<tc>,<tmax>,<rec>,<limp>,<nmea_age_ms>,<reset>*CS
e.g. $PHLG,2.1,RUN,LIMP,1,12,34192,4.01,5.02,63,71,2,1,420,IWDG*4F
e.g. $PHLG,2.2,RUN,DEGR,1,12,34192,3.96,5.01,58,69,0,0,180,POR*11
(checksum is the standard NMEA XOR over the chars between $ and *, computed at
runtime by buildPHLG(); -127 in tc/tmax = "n/a". The earlier *4F here was a wrong
hand-written placeholder — the correct XOR for that body is *49.)
```
Fields mirror the v2.1 `TLM` record (state, link, fix, sats, rx, battery, 5 V,
@ -120,12 +123,55 @@ live**: MCU temp / `tMax` (the thermal-hypothesis evidence, without pulling the
reset cause (reboot-loop forensics), recovery count, and LIMP state — and it ships to
Loki for free.
### Backend `$PHLG` branch (concrete — review before deploy)
Drop-in for `retranslator.py`. `cell_id` is already resolved from `(ip,port)` by the
existing CGNAT-safe map; this only adds recognition + checksum + store. Tolerant of the
**extra trailing fields** the v3 board will append (air/RH/battery/fan).
```python
# --- $PHLG health branch (v2.2) — REVIEW BEFORE DEPLOY ------------------------
def _nmea_ok(s: str) -> bool:
if not s.startswith('$') or '*' not in s:
return False
body, _, cs = s[1:].partition('*')
x = 0
for ch in body:
x ^= ord(ch)
return f"{x:02X}" == cs.strip()[:2].upper()
PHLG_FIELDS = ("fw","state","link","fix","sat","rx","vbat","v5v",
"tc","tmax","rec","limp","nmea_age_ms","reset")
def handle_phlg(line: str, cell_id: str, store: dict) -> bool:
if not line.startswith('$PHLG,') or not _nmea_ok(line):
return False
body = line[1:].split('*', 1)[0]
parts = body.split(',')[1:] # drop the 'PHLG' talker
rec = dict(zip(PHLG_FIELDS, parts)) # extra v3 trailing fields are simply ignored
rec["ts"] = time.time()
store[cell_id] = rec # latest health per unit (also ship to Loki)
log.info("PHLG %s temp=%s tmax=%s reset=%s limp=%s rec=%s",
cell_id, rec.get("tc"), rec.get("tmax"), rec.get("reset"),
rec.get("limp"), rec.get("rec"))
return True
# In the inbound loop, BEFORE the GGA/position path:
# if handle_phlg(line, cell_id, PHLG_BY_CELL):
# continue # health record, NOT a position fix — don't forward to Wialon
# -----------------------------------------------------------------------------
```
---
## 4. What to bench-test before committing (resolves the "mixed info")
1. **Does the 4G NTRIP Master forward a proprietary `$P…` sentence?** Inject one on
UART2-RX on the bench and watch the server `tcpdump`. *If it whitelists talkers:*
1. **Does the 4G NTRIP Master forward a proprietary `$P…` sentence?**
✅ **RESOLVED 2026-06-22 — YES.** Enabling `$PUBX` (a u-blox proprietary `$P…` talker)
on the F9P UART2 → it arrives at the server `udp/5010` **intact**, alongside GGA/RMC. The
modem is a transparent forwarder; **no whitelist**, so the `$GPTXT,HLT` fallback is not
needed. (Detail: `03-Architecture-Hardware/watchdog-pcb-v3-design.md` §7a.) Original note:
inject one on UART2-RX on the bench and watch the server `tcpdump`. *If it whitelists talkers:*
fall back to a `$GPTXT,HLT,…` subtype and special-case it in the backend by **field
1** (`HLT` vs `NET`) so it never collides with the cell-id parse. **This is the one
experiment that settles the whole question.**

View File

@ -1,6 +1,6 @@
<svg viewBox="0 0 680 460" xmlns="http://www.w3.org/2000/svg" font-family="Helvetica, Arial, sans-serif">
<title>HBOX Watchdog Carrier PCB v3 — power and signal map</title>
<desc>24 V cabinet feed passes reverse-polarity protection, then splits: to a new dedicated fan buck feeding two 5 V fan drivers and the extraction fans; and to the external Mean Well 5 V supply that powers the STM32F411 Blackpill, the RTK-stack load switch (to the external simpleRTK2B + 4G stack), and sense circuits. New I2C SHT40 + TMP102 temperature sensors and the fan drivers connect to the MCU.</desc>
<desc>24 V cabinet feed passes reverse-polarity protection, then splits: to a new dedicated fan buck feeding a 5 V fan driver and the extraction fan; and to the external Mean Well 5 V supply that powers the STM32F411 Blackpill, the RTK-stack load switch (to the external simpleRTK2B + 4G stack), and sense circuits. New I2C SHT40 + TMP102 temperature sensors and the fan driver connect to the MCU.</desc>
<rect x="0" y="0" width="680" height="460" fill="#ffffff"/>
<defs>
<marker id="ap" markerWidth="9" markerHeight="9" refX="6.5" refY="3" orient="auto"><path d="M0,0 L6.5,3 L0,6 z" fill="#555555"/></marker>
@ -28,21 +28,16 @@
<text x="27" y="167" font-size="12.5" font-weight="600" fill="#1a2233">Fan buck U2</text>
<text x="27" y="181" font-size="10.5" fill="#555555">24→5 V dedicated rail · JP1</text>
<rect x="16" y="214" width="100" height="44" rx="7" fill="#FCF1E8" stroke="#E07A2F" stroke-width="1.8"/>
<text x="26" y="232" font-size="12" font-weight="600" fill="#1a2233">Fan drv 1</text>
<text x="26" y="232" font-size="12" font-weight="600" fill="#1a2233">Fan driver</text>
<text x="26" y="246" font-size="10.5" fill="#555555">Q2 · flyback</text>
<rect x="124" y="214" width="100" height="44" rx="7" fill="#FCF1E8" stroke="#E07A2F" stroke-width="1.8"/>
<text x="134" y="232" font-size="12" font-weight="600" fill="#1a2233">Fan drv 2</text>
<text x="134" y="246" font-size="10.5" fill="#555555">Q3 · flyback</text>
<rect x="40" y="300" width="160" height="40" rx="7" fill="#ffffff" stroke="#aaaaaa" stroke-width="1.3" stroke-dasharray="4 3"/>
<text x="51" y="317" font-size="12.5" font-weight="600" fill="#1a2233">2× 5 V fans</text>
<text x="51" y="317" font-size="12.5" font-weight="600" fill="#1a2233">1× 5 V fan</text>
<text x="51" y="331" font-size="10.5" fill="#555555">extraction · ext</text>
<path d="M205,92 L205,120 L120,120 L120,148" fill="none" stroke="#555555" stroke-width="1.6" marker-end="url(#ap)"/>
<text x="124" y="116" font-size="9.5" fill="#555555">24 V</text>
<line x1="66" y1="190" x2="66" y2="212" stroke="#555555" stroke-width="1.6" marker-end="url(#ap)"/>
<line x1="174" y1="190" x2="174" y2="212" stroke="#555555" stroke-width="1.6" marker-end="url(#ap)"/>
<text x="70" y="206" font-size="9.5" fill="#555555">5V_FAN</text>
<path d="M66,258 L66,300" fill="none" stroke="#555555" stroke-width="1.6" marker-end="url(#ap)"/>
<path d="M174,258 L174,300" fill="none" stroke="#555555" stroke-width="1.6" marker-end="url(#ap)"/>
<rect x="250" y="150" width="172" height="72" rx="7" fill="#ffffff" stroke="#1a2233" stroke-width="1.5"/>
<text x="262" y="180" font-size="13" font-weight="700" fill="#1a2233">STM32F411 Blackpill</text>
<text x="262" y="198" font-size="10.5" fill="#555555">watchdog · on headers</text>
@ -75,7 +70,7 @@
<path d="M414,222 L440,260 L440,316 L468,316" fill="none" stroke="#999999" stroke-width="1.3" stroke-dasharray="4 3" marker-end="url(#as)"/>
<text x="444" y="288" font-size="9.5" fill="#999999">PB0</text>
<path d="M250,196 L235,210 L228,228" fill="none" stroke="#999999" stroke-width="1.3" stroke-dasharray="4 3" marker-end="url(#as)"/>
<text x="150" y="200" font-size="9.5" fill="#999999">PWM + tach PA6/7 · PB10/12</text>
<text x="150" y="200" font-size="9.5" fill="#999999">PWM + tach PA6 · PB10</text>
<line x1="300" y1="300" x2="300" y2="224" stroke="#999999" stroke-width="1.3" stroke-dasharray="4 3" marker-end="url(#as)"/>
<text x="304" y="262" font-size="9.5" fill="#999999">BAT sense PA0</text>
<path d="M414,222 L436,250 L436,410 L424,410" fill="none" stroke="#999999" stroke-width="1.3" stroke-dasharray="4 3" marker-end="url(#as)"/>

Before

Width:  |  Height:  |  Size: 7.8 KiB

After

Width:  |  Height:  |  Size: 7.3 KiB

View File

@ -23,9 +23,10 @@ supersedes_section: hardware-respec-and-pcb-brief.md §4 (watchdog carrier detai
| # | Decision | Consequence |
|---|---|---|
| 1 | **Scope = watchdog + thermal *carrier*** | Blackpill plugs on via 2×20 headers (reuse v2.1 firmware, no MCU bring-up). Mean Well SD-15B-05 and the battery stay **external modules**. Lowest-risk first board. |
| 2 | **5 V fans** | Driven low-side (MOSFET PWM) + tach. ⚠ They load the same 5 V the modem bursts on — mitigated with a **dedicated fan rail** (see §0a). |
| 2 | **5 V fan ×1** | One extraction fan, low-side (MOSFET PWM) + tach. ⚠ Loads the same 5 V the modem bursts on — mitigated with a **dedicated fan rail** (§0a). Needs a **passive filtered inlet** to pull air through, and has **no redundancy** — a fan failure falls back to conduction + thermal derate (tach-fault flags it). |
| 3 | **I²C temp + humidity** | SHT40 (board ambient + RH) + TMP102 probe (battery/modem hot spot) on one I²C bus. |
| 4 | **JLCPCB with assembly** | Parts chosen from JLC/LCSC stock; we output Gerbers + BOM + CPL. Footprints biased to JLC-placeable + hand-reworkable. |
| 5 | **Live telemetry ($PHLG): bench-test first, NOT built into v3** (2026-06-22) | The 4G modem keeps its proven home on the F9P **XBee socket = UART2**; there is **no spare UART** to tap (see §2a). v3 ships health on the **USB STREAM log** (already works). PA9/PA10 header is **reserved/DNP** only. The forwarding question is settled by an isolated **bench experiment** (§7a), not by a board change. |
---
@ -33,15 +34,15 @@ supersedes_section: hardware-respec-and-pcb-brief.md §4 (watchdog carrier detai
### 0a. 5 V fans must NOT share the RTK stack rail
The Mean Well **SD-15B-05** gives **5 V @ 3 A**. The RTK+4G stack already peaks
**~2–2.5 A on LTE-TX bursts** — the brownout the whole redesign is fixing. Two
5 V fans add **~0.3–0.5 A plus inrush**. Coincident worst case ≈ 2.8–3.0 A sits
right on the supply limit, and **fan inrush + a modem burst at the same instant
is exactly the sag that drops the link.** So the board gives the fans their **own
**~2–2.5 A on LTE-TX bursts** — the brownout the whole redesign is fixing. One
5 V fan adds **~0.15–0.25 A plus inrush**. Coincident worst case ≈ 2.7 A still
nears the supply limit, and **fan inrush + a modem burst at the same instant
is exactly the sag that drops the link.** So the board gives the fan its **own
5 V** from a small **24→5 V buck** tapped off the cabinet 24 V (which is already
inside the HBOX). A **0 Ω jumper (JP1)** lets you instead bridge fans to the main
5 V if you later upsize the supply — one board, both strategies. **Do not power
fans from the *switched* (load-switch) stack 5 V** — fans must run when hot even
while the stack is power-cycling.
inside the HBOX). A **0 Ω jumper (JP1)** lets you instead bridge the fan to the main
5 V — with a single ~0.2 A fan that's more defensible than with two, but the
dedicated rail stays the safe default. **Do not power the fan from the *switched*
(load-switch) stack 5 V** — it must run when hot even while the stack is power-cycling.
### 0b. Fans + IP67 is a real tradeoff — own it
**Moving air in/out of the box breaks the IP67 seal.** A fan only helps if it
@ -85,7 +86,7 @@ fragile hardware**. Keep the brains, fix the body.
| LED + dividers on Dupont | mechanical | all on-board, 0603, conformal-coatable |
| `1.5 kΩ "????"` mystery R | undocumented | resolved on this schematic (it was the **LED series R**; dividers are 10k/10k — see §6) |
| MCU temp only | heat casualties are the **18650/modem**, not the die | **external** I²C sensors at the hot spot (§5) |
| No thermal *actuation* | firmware can derate but can't cool | **2× fan drivers + closed-loop control** (§4, §7) |
| No thermal *actuation* | firmware can derate but can't cool | **1 fan driver + closed-loop control** (§4, §7) |
| No SWD/header bring-out | can't debug a sealed unit | **SWD + USB + reserve-USART1** headers (§6) |
---
@ -103,11 +104,11 @@ fragile hardware**. Keep the brains, fix the body.
PA5 IGN_SENSE GPIO in ignition via DFR1015 (INPUT_PULLDOWN)
PB0 EXT_LED GPIO out external status LED (low-side MOSFET)
PC13 LED GPIO out onboard LED (active-LOW)
── NEW for v3 (fans + temperature) ───────────────────────────────────────
PA6 FAN1_PWM TIM3_CH1 25 kHz PWM → fan-1 low-side gate
PA7 FAN2_PWM TIM3_CH2 25 kHz PWM → fan-2 low-side gate
PB10 FAN1_TACH EXTI fan-1 tach (open-collector, 10k pull-up to 3V3)
PB12 FAN2_TACH EXTI fan-2 tach (open-collector, 10k pull-up to 3V3)
── NEW for v3 (fan + temperature) ────────────────────────────────────────
PA6 FAN_PWM TIM3_CH1 25 kHz PWM → fan low-side gate
PB10 FAN_TACH EXTI fan tach (open-collector, 10k pull-up to 3V3)
PA7 (spare) TIM3_CH2 reserved — 2nd fan PWM if ever needed
PB12 (spare) EXTI reserved — 2nd fan tach if ever needed
PB6 I2C1_SCL I2C1 SHT40 + TMP102 (4.7k pull-ups to 3V3)
PB7 I2C1_SDA I2C1 SHT40 + TMP102
── reserved (do not populate now; future in-band $PHLG telemetry relay) ───
@ -117,10 +118,29 @@ fragile hardware**. Keep the brains, fix the body.
PA11 USB_DM / PA12 USB_DP Blackpill micro-USB = STREAM log capture
```
No conflicts: TIM3_CH1/CH2 (PA6/PA7) do fan PWM; PB0 (TIM3_CH3) stays plain-GPIO
LED. The reserved USART1 pair future-proofs the board for the
No conflicts: TIM3_CH1 (PA6) does fan PWM; PB0 (TIM3_CH3) stays plain-GPIO
LED. The reserved USART1 pair (H3) **future-proofs** the board for the
[[03-Architecture-Hardware/watchdog-health-telemetry|`$PHLG` health-in-server]] upgrade
without a respin.
without a respin — but in v3 it is **left unpopulated; no relay is wired** (see §2a + the
decision row above). **PA10 caveat:** PA10 is USART1-RX *and* a known Blackpill DFU-entry
pin — a load/float here can block flash mode. Keep H3 unpopulated, or fit a **~22 kΩ
pull-up on PA10 to 3V3**, so this board never inherits the flaky-flashing gremlin.
### 2a. Why there is no spare UART (the telemetry reality)
The simpleRTK2B **Pro** breaks out **both** F9P UARTs, but only one is free:
| F9P port | Header pins | Used by | Free? |
|---|---|---|---|
| **UART1** | TX1 / RX1 | **watchdog** (NMEA read + UBX rate) — PA2/PA3 | taken |
| **UART2** | TX2 / RX2 **and the XBee socket** | **4G NTRIP Master** (server uplink) | **no — shared bus** |
`TX2` = F9P-UART2-TX = the **modem's DIN** (the wire the modem reads and forwards to the
server). The F9P already drives it, so the watchdog can't simply "connect a second wire" to
send `$PHLG` — that's two transmitters on one line. Getting health to the server therefore
needs a **relay** (interpose the MCU between F9P-UART2 and the modem), which is a deliberate
architecture change **deferred out of v3**. To *learn whether it's even possible*, run the
isolated bench experiment in **§7a** — it touches no PCB.
---
@ -139,10 +159,8 @@ without a respin.
18650 / PowerBoost (EXTERNAL) ──► P4 [BAT+ , BAT_SENSE] ─► PA0 divider
FAN rail: 24V ─► U2 buck (MP1584EN) ─► 5V_FAN (JP1: alt-bridge to MAIN 5V)
5V_FAN ─┬─► Fan1 (+) ───┐ ┌─── Fan1 tach ─10k─► PB10
│ Fan1 (−) ─ Q2 (AO3400) ◄PWM PA6 │ (D2 SS34 flyback ∥ fan)
└─► Fan2 (+) ───┘ └─── Fan2 tach ─10k─► PB12
Fan2 (−) ─ Q3 (AO3400) ◄PWM PA7
5V_FAN ───► Fan (+) ┌─── Fan tach ─10k─► PB10
Fan (−) ─ Q2 (AO3400) ◄PWM PA6 ┘ (D2 SS34 flyback ∥ fan)
I²C1 (PB6/7) ─┬─ SHT40 (U3, 0x44) board ambient temp + RH
└─ TMP102 (U4, 0x48) on P5 pigtail → battery/modem hot spot
@ -202,19 +220,19 @@ solid, sourceable SMD, plus a resettable fuse for the missing current limit:
eFuse (e.g., **AP2553W6-7** SOT-23-6, EN active-high, FLAGn). Lay the footprint
*and* the discrete; populate one. Verify LCSC stock at order; **default = discrete**.
### 4.4 Fan drivers ×2 (low-side PWM + tach) — Q2/Q3
Per fan (identical for fan-1 and fan-2):
### 4.4 Fan driver ×1 (low-side PWM + tach) — Q2
Single extraction fan:
```
5V_FAN ──► Fan(+) Fan tach ──[R_t 1k]──┬──► PBxx (EXTI)
Fan(−) ──┬──► Q2/Q3 Drain │
D2/D3 SS34 ∥ fan │ (AO3400A, N-MOS) [R 10k pull-up → 3V3]
5V_FAN ──► Fan(+) Fan tach ──[R_t 1k]──┬──► PB10 (EXTI)
Fan(−) ──┬──► Q2 Drain │
D2 SS34 ∥ fan │ (AO3400A, N-MOS) [R 10k pull-up → 3V3]
(cathode→5V_FAN) │ Source ── GND
PWM PAx ─[Rg 100Ω]─ Gate ──[Rgs 100k → GND]
```
- **Q2/Q3 = AO3400A** logic-level N-MOSFET (SOT-23): full-on from 3.3 V PWM, 5.7 A /
- **Q2 = AO3400A** logic-level N-MOSFET (SOT-23): full-on from 3.3 V PWM, 5.7 A /
30 V — vast margin over a <0.5 A fan.
- **D2/D3 = SS34** Schottky **flyback** across each fan (motor is inductive — protects
- **D2 = SS34** Schottky **flyback** across the fan (motor is inductive — protects
the FET at PWM turn-off). Cathode to 5V_FAN, anode to drain.
- **Rg 100 Ω** gate series (tames edges/EMI), **Rgs 100 kΩ** gate→GND pull-down =
**fans OFF until the MCU drives PWM** (fail-safe; consistent with the ethos — if the
@ -266,8 +284,7 @@ ADC stays ≤3.3 V with margin (verified in firmware-review). Firmware
| P3 | JST-XH 4-pos | 4 | SW_5V, GND, GNSS_TX(PA2), GNSS_RX(PA3) → simpleRTK2B |
| P4 | JST-XH 2-pos | 2 | BAT+ (to PA0 divider), GND (18650/PowerBoost) |
| P5 | JST-XH 4-pos | 4 | 3V3, GND, SDA, SCL → TMP102 probe pigtail |
| P6 | JST-XH 4-pos | 4 | 5V_FAN, Fan1−(PWM), GND, Tach1 → fan 1 |
| P7 | JST-XH 4-pos | 4 | 5V_FAN, Fan2−(PWM), GND, Tach2 → fan 2 |
| P6 | JST-XH 4-pos | 4 | 5V_FAN, Fan−(PWM), GND, Tach → fan |
| P8 | JST-XH 2-pos | 2 | EXT_LED+, EXT_LED− (panel LED) |
| H1 | 1×4 0.1″ | 4 | SWD: 3V3, SWDIO, SWCLK, GND |
| H2 | (Blackpill micro-USB) | — | STREAM log capture / flashing |
@ -284,9 +301,8 @@ ADC stays ≤3.3 V with margin (verified in firmware-review). Firmware
The v2.1 `.ino` changes are **small and additive** (the ethos: bench first; the
sandbox can't build STM32). New globals + functions, no state-machine surgery.
1. **Pins**: add `PIN_FAN1_PWM PA6`, `PIN_FAN2_PWM PA7`, `PIN_FAN1_TACH PB10`,
`PIN_FAN2_TACH PB12`; init TIM3 CH1/CH2 PWM at **25 kHz**; attach EXTI counters on
the tach pins.
1. **Pins**: add `PIN_FAN_PWM PA6`, `PIN_FAN_TACH PB10` (PA7/PB12 stay spare);
init TIM3 CH1 PWM at **25 kHz**; attach an EXTI counter on the tach pin.
2. **I²C temps**: `Wire.begin()` on PB6/PB7; read SHT40 (0x44, temp+RH) and TMP102
(0x48). Add `tAirC`, `rhPct`, `tBatC` to telemetry. Keep MCU-internal as cross-check.
3. **Closed-loop fan curve** (input = `max(tAirC, tBatC)`):
@ -298,15 +314,55 @@ sandbox can't build STM32). New globals + functions, no state-machine surgery.
4. **Tach fault**: once/30 s, command a fan to **100 % for ~300 ms**, count tach edges,
compute RPM. `0 RPM while commanded > 0` → latch `FAN_FAULT`, log `EVT`, raise it in
telemetry. A dead fan in a hot box must be **visible**, not silent.
5. **Telemetry**: extend the v2.1 `TLM`/`$PHLG` record with
`tair,rh,tbat,fan1_rpm,fan2_rpm,fan_pwm,fan_fault`. These are exactly the fields the
`$PHLG`-to-server upgrade will carry — capture them on-unit now via the STREAM log.
5. **Telemetry (on-unit only in v3)**: extend the v2.1 `TLM` record with
`tair,rh,tbat,fan_rpm,fan_pwm,fan_fault`, captured to the **USB STREAM log**.
These are the same fields a future `$PHLG`-to-server sentence would carry — but **v3 does
NOT push them live to the server** (no UART relay; see §2a + decision row 5). Live
telemetry is gated on the bench experiment below.
6. **IWDG-safe**: fan/tach/I²C work is non-blocking; keep petting the dog. I²C read uses
a short timeout so a stuck sensor can't hang the loop.
Pin map and control law live here; the implementation lands in
`blackpillrtkcode_v2.ino` **after** bench validation on the prototype board.
### 7a. The one bench experiment that decides live telemetry (no PCB change)
> **✅ RESULT — 2026-06-22: PASS (no extra hardware needed).** Enabled `$PUBX` (a u-blox
> **proprietary** `$P…` talker — same class + checksum format as `$PHLG`) on the F9P
> **UART2** and watched the server `udp/5010` capture: `$PUBX,00,…` arrives **end-to-end,
> intact**, alongside GGA/RMC. → **The 4G NTRIP Master forwards proprietary `$P…`
> sentences, so `$PHLG` health-in-server is confirmed viable.** No XBee-breakout test
> needed. Remaining work = the **UART2 relay** (interpose the watchdog to inject `$PHLG`)
> \+ backend parser — a deliberate later board, still out of v3 scope.
>
> *Incidental (flagged for GNSS follow-up):* the same capture caught two live units —
> bench **H0014 (Querétaro) at SPS (GGA quality 1), RSSI −93**, and field
> **H0017 (Veracruz/ICAVE) at DGPS (quality 2, RMC mode `D`, diff base id 0133), RSSI −51**.
> Per Leo the bench is in a **basement** (poor sky + weak cellular) — which alone explains
> the bench SPS, since **RTK fix needs open sky (carrier-phase) regardless of
> corrections/config**. But note H0017 has *good* sky+signal and still only reaches **DGPS,
> not RTK (4/5)** → the **corrections pipeline** (free rtk2go caster + RAM-only F9P config)
> is a *separate* ceiling from antenna placement. One **open-sky test with a proper caster +
> FLASH-saved config** separates the two. (Reinforces the v2.1 SPS-thrash guard: a healthy
> basement/field unit may legitimately sit at SPS/DGPS — do **not** power-cycle it.)
Settles "does the 4G NTRIP Master forward a proprietary `$P…` sentence?" by **isolating the
modem** — the F9P is not involved, so there's no bus collision:
1. Remove the **4G NTRIP Master** from the F9P XBee socket; mount it on a bench/XBee adapter
with **3.3 V power, the SIM, and the LTE antenna**.
2. Wire **Blackpill PA9 (USART1 TX) → modem DIN** (the `TX2`/F9P-UART2-TX net in the socket)
and **modem DOUT → Blackpill PA10 (RX)** *or* a USB-UART so a PC can watch it.
3. Match the modem's UART baud (ArduSimple default **115200**, 3.3 V). Let it register.
4. Emit `$PHLG,2.1,RUN,LIMP,1,12,34192,4.01,5.02,63,71,2,1,420,IWDG*CS\r\n` and watch the
**server `tcpdump`**.
- **Appears →** modem forwards proprietary sentences → `$PHLG` is viable → *then* plan a
relay-capable board.
- **Doesn't →** modem whitelists talkers → fall back to a `$GPTXT,HLT,…` subtype
(special-cased by **field 1** so it never collides with the cell-id parse).
Either way, **v3 ships unchanged** — this experiment only informs a *later* decision.
---
## 8. Layout, stackup & DFM (for a relative beginner — the rules that matter)
@ -342,8 +398,8 @@ Pin map and control law live here; the implementation lands in
blocks a flipped input, TVS not hot.
2. **Load switch**: toggle PA1 → SW_5V on/off; confirm OFF at reset; PPTC holds the
measured stack current, trips on a deliberate short.
3. **Fans**: ramp PA6/PA7 PWM 0→100 %, scope the gate + fan node, confirm flyback,
read **tach RPM** vs PWM; pull a fan → confirm `FAN_FAULT`.
3. **Fan**: ramp PA6 PWM 0→100 %, scope the gate + fan node, confirm flyback,
read **tach RPM** vs PWM; pull the fan → confirm `FAN_FAULT`.
4. **Sensors**: I²C scan finds 0x44 + 0x48; heat the TMP102 probe → fan ramps per curve.
5. **Power budget (the §0a check)**: with **JP1 open**, scope MAIN 5 V during an LTE
burst **and** a fan ramp simultaneously — confirm the stack rail no longer sags.
@ -359,7 +415,7 @@ Pin map and control law live here; the implementation lands in
- [ ] **IP rating target** with ICAVE: IP67 (no vents → conduction only) vs IP54/55
(filtered louvers + fans). Determines whether the fan section is populated.
- [ ] **Fan choice**: 5 V **3-wire** (tach) confirmed? Size (40/50/60 mm) + measured
- [ ] **Fan choice (×1)**: 5 V **3-wire** (tach), size (40/50/60 mm), + a **filtered passive inlet** so the single extraction fan can pull air through. Measured
static-pressure need through the chosen duct/filter. Source 2–3 candidates.
- [ ] **U2 buck vs upsize supply** (§4.2): lay the buck (default) or go 5 V/5 A external
+ JP1-closed. Recommend laying the buck for rail isolation.

View File

@ -96,11 +96,11 @@ For each, place the symbols, wire locally, and tie to power/net labels:
L2, C22, C23, the FB divider R20/R21, and **JP1** (solder-jumper symbol).
- **Load switch (§4.3):** Q1 AO3401A + Q5 MMBT3904 + R1/R2 + **F1 PPTC** + C3. Net the drain to
**SW_5V**. (Optional U1 eFuse footprint — leave it **DNP**.)
- **Fan drivers ×2 (§4.4):** Q2/Q3 AO3400A, D2/D3 SS34 flyback, Rg/Rgs gate parts, tach
pull-up + series into **PB10 / PB12**. PWM gates from **PA6 / PA7**.
- **Fan driver ×1 (§4.4):** Q2 AO3400A, D2 SS34 flyback, Rg/Rgs gate parts, tach
pull-up + series into **PB10**. PWM gate from **PA6** (PA7/PB12 left as spares).
- **Sensors (§5):** U3 SHT40 + U4 TMP102 on **PB6/PB7**, 4.7k pull-ups, 100nF each.
- **Sense + LED (§4.5–4.6):** the three 10k/10k dividers (PA0/PA4/PA5), D5 clamp, Q4 + D1 + R3.
- **Connectors (§6):** P1–P8, H1, H3 (DNP), and the two 2×20 sockets.
- **Connectors (§6):** P1–P6 + P8, H1, H3 (DNP), and the two 2×20 sockets.
### 4.4 Annotate + ERC
- **Tools → Annotate** (gives refs R1, U2, …).
@ -140,7 +140,7 @@ on **easyeda2kicad** / the JLC part page, grab the EasyEDA footprint and import.
**mounting holes** so the board can bolt to the cabinet **metal plate** (design-doc §8 —
that's your conduction path).
3. **Place by functional block** (keep each §4 sub-circuit together): power-in + buck in one
corner, load switch near P3, fan drivers near P6/P7, sensors near an air edge / P5, the
corner, load switch near P3, fan driver near P6, sensors near an air edge / P5, the
Blackpill sockets central, connectors on the edges.
4. **Stackup:** 2-layer is fine (cheaper); **4-layer** is better for RF/buck noise. Add a
**ground pour** (filled zone = GND) on the bottom (and an inner layer if 4-layer).
@ -204,7 +204,7 @@ pigtail when it arrives.
- **Rotations**: JLC's 0° reference often differs from KiCad's. Always eyeball the placement
preview; SOT-23 and electrolytics are the usual offenders.
- **Polarity**: double-check D10 TVS, C10 electrolytic, D2/D3 Schottky, the LED, and Q10/Q1
- **Polarity**: double-check D10 TVS, C10 electrolytic, D2 Schottky, the LED, and Q10/Q1
source/drain before ordering.
- **The buck (U2)** is the only "real" analog block — if you're nervous, **populate JP1
closed + upsize the 5 V supply** and leave U2 unpopulated for the first spin (design-doc

View File

@ -0,0 +1,123 @@
---
tags: [project-drogheda, hardware, watchdog, prototype, breadboard, bom, bring-up]
project: Project Drogheda
author: Helgen CTO
date: 2026-06-22
status: Active — shop list for the Blackpill bring-up
---
# Watchdog Bring‑up — Protoboard Shopping List (print me)
> Goal: breadboard the v2.2 watchdog on your **existing Blackpill**, prove the
> state machine + sensing + load switch + fans + temp loop, **before** any PCB.
> All parts are **through‑hole / breadboard‑friendly** and stocked at a normal MX
> electronics shop (Steren / AG Electrónica / local). Quantities include spares.
> Pin map: [[03-Architecture-Hardware/watchdog-pcb-v3-design#2|design §2]] ·
> firmware: `blackpillrtkcode_v2.ino` (v2.2).
## 0. Already have — do NOT buy
STM32F411 Blackpill · USB‑C/micro cable · 18650 + PowerBoost · Mean Well SD‑15B‑05
(or a bench PSU) · multimeter · the F9P + 4G master (only needed later, not for the
bring‑up). A **heat source** (hair dryer / lighter / hot water cup) to trigger the
thermal/fan test — household, no need to buy.
---
## 1. Core — bring up the "brains" (buy this even if the shop is sparse)
| # | Item | Qty | Value / part | ~MXN | For |
|---|---|---|---|---|---|
| 1 | Breadboard 830‑pt | 2 | half/full | 120 | build surface (2 = room for fans/sensors) |
| 2 | Jumper wires M‑M | 1 pk | 65 pcs | 60 | board wiring |
| 3 | Jumper wires M‑F | 1 pk | 40 pcs | 50 | to Blackpill / modules |
| 4 | Resistor assortment box | 1 | ¼ W (incl. 330Ω,1k,4.7k,10k,100k,220Ω) | 120 | dividers, gates, pull‑ups |
| 5 | LED (any) | 5 | 3/5 mm | 10 | status LED on PB0 |
| 6 | Ceramic cap 100 nF | 10 | 0.1 µF (104) | 20 | ADC filters + decoupling |
| 7 | Electrolytic 100 µF / ≥16 V | 4 | — | 20 | rail bulk / modem‑burst reservoir |
| 8 | **2N3904** (NPN, TO‑92) | 3 | load‑switch driver | 15 | PA1 → P‑FET gate |
| 9 | **IRF9540 / IRF9540N** (P‑MOSFET, TO‑220) | 2 | high‑side 5 V switch | 40 | switched 5 V to "stack" |
| 10 | **10 kΩ trim pot** | 2 | breadboard trimmer | 30 | sweep VBAT / ignition to test thresholds |
| 11 | Tactile button or slide switch | 2 | — | 20 | simulate ignition on/off |
| 12 | Male header strip | 1 | 1×40, 2.54 mm | 15 | solder onto any module that needs pins |
**Core subtotal ≈ 520 MXN.** With this you can flash v2.2, watch the `$PHLG`/TLM log,
and validate the **load switch + the three sense dividers + LED** — the whole watchdog
logic — without the thermal parts.
---
## 2. Thermal add‑ons — fans + temperature (the v3 new stuff)
| # | Item | Qty | Value / part | ~MXN | For / note |
|---|---|---|---|---|---|
| 13 | 5 V fan, 40–60 mm, **3‑wire** | 1 | 3‑wire = has tach | 60 | one extraction fan; 3‑wire so PB10 reads RPM. Grab a 2nd as a spare if cheap |
| 14 | **IRLZ44N** (logic‑level N‑MOSFET, TO‑220) | 2 | fan low‑side PWM | 30 | 1 + spare; **must be logic‑level**; see substitute below |
| 15 | Schottky **1N5819** | 3 | fan flyback | 10 | one across the fan (1N4007 ok if out) |
| 16 | I²C temp+RH breakout | 1 | **SHT31 / SHT40** ideal | 120 | matches firmware (0x44). RH catches condensation |
| 17 | Analog temp **LM35** (TO‑92) | 2 | sure‑thing fallback | 25 | every shop has it; heat it → watch fans ramp |
| 18 | **DS18B20** waterproof probe | 1 | 1‑wire | 45 | great as the battery/modem hot‑spot probe |
| 19 | LM2596 buck module (adj.) | 1 | 24/12 → 5 V | 35 | dedicated **fan 5 V** rail (caveat §0a) if no 2nd bench PSU |
**Thermal subtotal ≈ 260 MXN** (one fan + driver + one temp option). Grab #13–15 for the fan, and **one** temperature option
(#16 ideal, else #17/#18).
---
## 3. Substitutions (shop stock varies — these all work)
| If they're out of… | Buy instead | Catch |
|---|---|---|
| IRLZ44N (logic‑level FET) | **2N2222A** (TO‑92) or **TIP120** (Darlington) | BJT: add 1 kΩ base resistor; fine for a <0.5 A fan. **Plain IRF540/IRFZ44 won't fully turn on from 3.3 V — avoid.** |
| SHT31/SHT40 I²C breakout | **BME280/BMP280** (I²C, common) | address 0x76/0x77 → small firmware change |
| Any I²C sensor | **LM35** (analog) or **DS18B20** (1‑wire) | not I²C → firmware reads ADC / 1‑wire instead (I'll patch it) |
| IRF9540 P‑MOSFET | keep your original, or **2N2222 high‑side** is awkward — ask for any P‑channel ≥30 V | gate fully pulled to GND by the 2N3904, so a standard P‑FET is OK here |
| 1N5819 Schottky | **1N4007** | slightly slower; fine at fan PWM |
---
## 4. Bring‑up order (so the list maps to actions)
1. **Just USB** — flash v2.2, open serial @115200. Confirm `BOOT/COLS/TLM` lines **and a
`$PHLG,...*CS` every 30 s**. (No external parts — this proves the firmware.)
2. **LED on PB0** via 330 Ω → watch the blink codes change with state.
3. **Load switch** — 2N3904 + IRF9540 + 1 kΩ (base) + 10 kΩ (gate pull‑up) from a 5 V rail;
put an LED+330 Ω on the switched output. Toggle: confirm **OFF at reset**, ON when PA1
drives. (This is the heart of recovery.)
4. **Sense dividers** — two 10 kΩ each (÷2) into **PA0** (battery via 18650) and **PA4**
(5 V rail); a 10 kΩ pot into **PA5** (ignition). Sweep them and watch `veh`/`ign` flip in
the TLM with hysteresis/debounce.
5. **Fan** — PA6 → gate (220 Ω) of IRLZ44N → fan(−); flyback diode across the fan;
100 kΩ gate→GND. Watch it spin/ramp; tach wire → 10 kΩ pull‑up to 3V3 → PB10 →
RPM in the log; **unplug the fan → `FAN_FAULT`**.
6. **Temperature** — I²C breakout on PB6/PB7 (+4.7 kΩ pull‑ups), or LM35 on an ADC pin.
Heat it → fans ramp per the 45→70 °C curve; at 75 °C the existing thermal derate kicks in.
---
## 5. Wiring cheat‑sheet (Blackpill side)
```
PA0 ← BAT ÷2 (10k/10k) from 18650+ PA6 → 220Ω → IRLZ44N gate; fan(−); 1N5819 ∥ fan
PA4 ← 5V ÷2 (10k/10k) from 5V rail PB10 ← fan tach, 10k pull-up → 3V3 (1k series)
PA5 ← IGN ÷2 (10k/10k) from pot/switch PB6/PB7 → I²C SCL/SDA (+4.7k each → 3V3)
PA1 → 1k → 2N3904 base; collector → IRF9540 PB0 → 330Ω → LED → GND
gate; 10k gate→5V; drain → switched (PA7/PB12 left free — 2nd fan if ever needed)
5V → stack (use an LED load to test)
GND/5V/3V3 rails common; fan 5V from a SEPARATE 5V (LM2596 or 2nd PSU) — not the stack rail.
```
---
## 6. Note on the temp sensor + firmware
The v2.2 firmware reads **I²C SHT40 (0x44) + TMP102 (0x48)**. Whatever you bring back:
- **SHT31/SHT40** → works as‑is (SHT40) or a 1‑line address tweak (SHT31).
- **BME280** → change address + read call.
- **LM35 / DS18B20** → analog / 1‑wire; tell me which and I'll patch `buildPHLG()` +
the fan‑curve input to read it. For *tomorrow's* bring‑up the fan loop can also be
driven off the **MCU‑internal** temp (already in firmware) — heat the whole board and
the derate still triggers, so you can prove the fan path even before the sensor arrives.
## Related
- [[03-Architecture-Hardware/watchdog-pcb-v3-bom|v3 BOM (production + bench, xlsx)]] ·
[[03-Architecture-Hardware/watchdog-pcb-v3-design|v3 design (pin map, sub‑circuits)]] ·
`04-Source-Code/firmware/blackpillrtkcode_v2/blackpillrtkcode_v2.ino`

Binary file not shown.

Binary file not shown.

@ -1 +1 @@
Subproject commit f93a36d3ae8d64ead1036f422a32024eb3cb9cbf
Subproject commit ed51abe602ba401350347b0be135197af333e027

View File

@ -0,0 +1,195 @@
% Helgen Drogheda — Manual de Consulta de Logs (Grafana / Loki · backend htracker)
% Compilar: latexmk -xelatex 2026-06-22_Drogheda_Manual-Logs-Grafana_v1.0.tex (XeLaTeX)
\documentclass[10pt]{article}
\newcommand{\sistema}{Helgen Drogheda}
\newcommand{\sistemapdf}{Manual de Consulta de Logs — Grafana / Loki}
\newcommand{\doctipo}{Manual de Operación}
\input{estilo-helgen.tex}
\usepackage{amssymb}
\begin{document}
\helgenportada{Consulta de Logs en Grafana}{Observabilidad del backend \texttt{htracker} — Grafana Loki (capa gratuita)}{%
\begin{tabular}{@{}ll@{}}
Documento & Manual de Operación\\
Versión & 1.0\\
Fecha & 22 de junio de 2026\\
Autor & Dr.\ Leopoldo Rodríguez Salazar — Helgen Technologies\\
Sistema & \texttt{htracker} (retranslator + monitor), servidor \texttt{HelgenServer}\\
Fuente & \texttt{04-Source-Code/htracker/log-shipping/}\\
Clasificación & Confidencial — Uso interno\\
\end{tabular}}
\tableofcontents
\vspace{4pt}
\section{Propósito}
El servidor del backend tiene \textbf{muy poco disco}, por lo que el gateway \textbf{no}
guarda logs localmente. En su lugar, los registros se \textbf{envían fuera del servidor}
a \textbf{Grafana Loki} (capa gratuita) y se consultan desde \textbf{Grafana}. Este manual
explica cómo \emph{entrar} a Grafana y \emph{consultar} los logs de la flota — no requiere
acceso al servidor para leer.
\begin{helgennotice}[En una frase]
Los logs del backend viven en \textbf{Grafana Loki}. Para verlos: abre Grafana
$\rightarrow$ \textbf{Explore} $\rightarrow$ fuente de datos \textbf{Loki} $\rightarrow$
escribe \texttt{\{job="htracker"\}}.
\end{helgennotice}
\section{Arquitectura del envío de logs}
El gateway escribe a la salida estándar; \texttt{systemd} la captura en \texttt{journald}
(un búfer en anillo \textbf{limitado a 200 MB}, que nunca llena el disco); un agente ligero
(\textbf{Vector}, o \textbf{Grafana Alloy}) lee el journal y lo empuja por HTTPS a Loki.
\begin{center}
\begin{tikzpicture}[node distance=6mm and 8mm]
\node[caja] (rt) {retranslator\\\scriptsize(stdout)};
\node[caja, right=of rt] (jd) {journald\\\scriptsize(tope 200 MB)};
\node[caja, right=of jd] (vec) {Vector\\\scriptsize(o Alloy)};
\node[cajaoscura, right=of vec] (loki) {Grafana\\Loki};
\node[caja, right=of loki] (graf) {Grafana\\\scriptsize(UI)};
\draw[flecha] (rt)--(jd); \draw[flecha] (jd)--(vec);
\draw[flechaacento] (vec)-- node[etiqueta, above]{HTTPS} (loki); \draw[flecha] (loki)--(graf);
\end{tikzpicture}
\end{center}
\noindent\textbf{Por qué así:} el gateway nunca escribe un archivo que crece; \texttt{journald}
es un búfer pequeño y limitado (sobrevive cortes breves de Loki); el agente empuja con un
búfer acotado que \emph{descarta lo más nuevo} si Loki no responde — de modo que la ruta de
logs \textbf{jamás} bloquea ni tumba al gateway.
\begin{helgennotice}[¿Vector o Alloy? — verifícalo una vez]
El \texttt{setup-logging.sh} instala \textbf{Vector} por defecto (etiqueta
\texttt{job="htracker"}). Si en su lugar se usó el \emph{onboarding de Linux} de Grafana
Cloud, el agente es \textbf{Alloy} y los logs aparecen por unidad de systemd
(\texttt{\{unit="htracker.service"\}}). Si una consulta no devuelve nada, usa el
\textbf{Label browser} de Grafana para ver qué etiquetas existen.
\end{helgennotice}
\section{Acceso a Grafana}
\subsection{Grafana en la nube (nada que instalar)}
\begin{enumerate}[leftmargin=1.5em]
\item Entra a tu \textbf{stack de Grafana Cloud} (URL del tipo \texttt{https://<stack>.grafana.net}).
\item Menú lateral $\rightarrow$ \textbf{Explore}.
\item Arriba, selecciona la fuente de datos \textbf{Loki}.
\item Escribe la consulta (sección 4) y presiona \textbf{Run query}. Ajusta el rango de
tiempo arriba a la derecha.
\end{enumerate}
\subsection{Grafana local (si prefieres correr la interfaz tú mismo)}
Los datos siguen en Grafana Cloud; sólo el visor es local.
\begin{verbatim}
docker run -d --name grafana -p 3000:3000 grafana/grafana-oss
# http://localhost:3000 (admin/admin)
# Connections -> Data sources -> Loki:
# URL: <LOKI_URL> Auth: Basic user: <LOKI_USER> password: <LOKI_TOKEN>
\end{verbatim}
\begin{helgennotice}[Credenciales]
\texttt{LOKI\_URL}, \texttt{LOKI\_USER} y \texttt{LOKI\_TOKEN} están en
\texttt{/etc/vector/htracker.env} del servidor y en la consola de Grafana Cloud
(\emph{Connections} $\rightarrow$ \emph{Hosted logs}). \textbf{Son secretos} — viven en
\texttt{.env.vector} (ignorado por git); nunca se suben al repo.
\end{helgennotice}
\section{Cómo consultar los logs (LogQL)}
En \textbf{Explore} con la fuente \textbf{Loki}. Loki indexa \textbf{etiquetas}, no el texto;
todo lo demás se filtra con \texttt{|=} (contiene) en el momento de la consulta.
\begin{verbatim}
{job="htracker"} # todo el backend
{job="htracker", level="error"} # sólo errores
{job="htracker", unit="htracker.service"} # sólo el gateway (retranslator)
{job="htracker"} |= "Anti-Salto" # rechazos del filtro anti-salto
{job="htracker"} |= "victimas_backend" # el resumen 1/min (victimas de backend)
{job="htracker"} |= "ADDR-CHANGE" # cambios de direccion SIM/NAT
\end{verbatim}
\noindent Si el envío es por \textbf{Alloy} (no Vector), usa la unidad de systemd:
\begin{verbatim}
{unit="htracker.service"}
{unit="htracker.service"} |= "victimas_backend"
\end{verbatim}
\section{Qué se registra}
\begin{center}\footnotesize\renewcommand{\arraystretch}{1.3}
\begin{tabularx}{\linewidth}{@{}>{\bfseries\color{HelgenIndigo}}L{3.6cm} X@{}}
\toprule
Etiquetas (labels) & \texttt{job=htracker}, \texttt{host}, \texttt{unit}
(\texttt{htracker.service} / \texttt{-web} / \texttt{-notifier}), \texttt{level}
(\texttt{info}/\texttt{warn}/\texttt{error}/\texttt{debug})\\
Nivel por defecto & \texttt{INFO}: sólo \textbf{eventos} + una línea de \textbf{resumen cada
60 s} (\texttt{[SUMMARY]} / \texttt{victimas\_backend}). El detalle por paquete es
\texttt{DEBUG} y \textbf{no} se envía.\\
Eventos clave & anti-salto (\texttt{Anti-Salto}), cambios de dirección SIM/NAT
(\texttt{ADDR-CHANGE}), víctimas de backend (unidad en línea en htracker pero ausente en
Wialon).\\
\bottomrule
\end{tabularx}
\end{center}
\section{Retención, límites y seguridad de disco}
\begin{itemize}
\item \textbf{Loki (capa gratuita):} 50 GB/mes, \textbf{retención de 14 días}, Grafana
incluido. Tras el recorte de verbosidad se usa una fracción mínima.
\item \textbf{journald (servidor):} \texttt{SystemMaxUse=200M}, retención 3 días
(\texttt{/etc/systemd/journald.conf.d/htracker.conf}). Aunque Loki esté caído días,
el disco \textbf{no} se llena.
\item \textbf{Búfer de Vector:} en memoria, descarta lo más nuevo si Loki no responde.
\end{itemize}
\section{Verbosidad (nivel de log)}
\begin{itemize}
\item \texttt{GW\_LOG\_LEVEL} (por defecto \texttt{INFO}). Para depurar una unidad en vivo,
ponlo en \texttt{DEBUG} \emph{temporalmente}.
\item \texttt{GW\_LOG\_SUMMARY\_S} (por defecto 60) — cadencia del resumen.
\end{itemize}
\begin{helgennotice}[No dejes DEBUG encendido]
\texttt{DEBUG} registra cada paquete e inunda la tubería (y consume la cuota de Loki).
Úsalo unos minutos y regresa a \texttt{INFO}.
\end{helgennotice}
\section{Salud del envío y diagnóstico}
En el servidor (\texttt{ssh HelgenServer}):
\begin{verbatim}
systemctl status vector # esta activo el agente?
journalctl -u vector -f # esta enviando? errores de Loki?
\end{verbatim}
\noindent \textbf{No aparecen logs en Grafana:} (1) confirma \texttt{vector} activo; (2) revisa
credenciales/endpoint en \texttt{/etc/vector/htracker.env}; (3) verifica el rango de tiempo en
Grafana; (4) revisa el \textbf{Label browser} (¿\texttt{job} o \texttt{unit}?, Vector vs Alloy).
\section{Puesta en marcha / reinstalación}
Sólo si hay que reinstalar el envío (normalmente ya está activo). En el servidor, desde
\texttt{/home/itadmin/htracker/log-shipping}:
\begin{verbatim}
cp .env.vector.example .env.vector
nano .env.vector # pega LOKI_URL / LOKI_USER / LOKI_TOKEN
./setup-logging.sh # limita journald, --no-filelog, instala Vector, arranca
# ./setup-logging.sh --journald-only # si Grafana Alloy ya envia el journal
\end{verbatim}
\noindent \texttt{deploy.sh} ya sincroniza la carpeta \texttt{log-shipping/} al servidor.
Detalle completo: \texttt{04-Source-Code/htracker/log-shipping/README.md}.
\section{Referencia rápida de consultas}
\begin{center}\footnotesize\renewcommand{\arraystretch}{1.3}
\begin{tabularx}{\linewidth}{@{}>{\ttfamily\color{HelgenIndigo}}L{6.3cm} X@{}}
\toprule
\normalfont\bfseries Consulta & \normalfont\bfseries Qué muestra\\
\midrule
\{job="htracker"\} & Todo el backend\\
\{job="htracker", level="error"\} & Sólo errores\\
\{job="htracker", unit="htracker.service"\} & Sólo el gateway (retranslator)\\
\{job="htracker"\} |= "victimas\_backend" & Resumen 1/min — víctimas de backend\\
\{job="htracker"\} |= "Anti-Salto" & Rechazos del filtro anti-salto\\
\{job="htracker"\} |= "ADDR-CHANGE" & Cambios de dirección SIM/NAT\\
\bottomrule
\end{tabularx}
\end{center}
\vfill
\begin{center}\scriptsize\color{gray}
Helgen Technologies · Project Drogheda · Observabilidad del backend · Documento interno · v1.0 (22 jun 2026)
\end{center}
\end{document}

View File

@ -0,0 +1,117 @@
% ---------- Estilo de marca Helgen (compartido, Project Drogheda) ----------
% Reutiliza la identidad de Project Ennis (_estilo-helgen.tex). Cada documento define
% \sistema, \sistemapdf y \doctipo antes de \input{...}. Compilar con XeLaTeX.
% ---------- Geometría y base ----------
\usepackage[letterpaper, margin=2.4cm, headheight=14pt]{geometry}
\usepackage{polyglossia}
\setmainlanguage[variant=mexican]{spanish}
\usepackage{xcolor}
\usepackage{colortbl}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{array}
\usepackage{tabularx}
\usepackage{enumitem}
\usepackage{titlesec}
\usepackage{float}
\usepackage{fancyhdr}
\usepackage{tikz}
\usetikzlibrary{arrows.meta, positioning, shapes.misc, calc, fit, automata}
\usepackage{caption}
\captionsetup{font=small, labelfont={bf,color=HelgenIndigo}}
\usepackage{microtype}
\setlength{\emergencystretch}{3em}\tolerance=2000\hbadness=2000
\usepackage{hyperref}
\usepackage{textcomp}
\usepackage{ragged2e}
\usepackage{xltabular}
\usepackage[export]{adjustbox}
\usepackage[most]{tcolorbox}
\providecommand{\sistema}{Helgen Drogheda}
\providecommand{\sistemapdf}{Helgen Drogheda}
\providecommand{\doctipo}{Documento}
% ---------- Paleta oficial Helgen ----------
\definecolor{HelgenIndigo}{HTML}{2C3C5B}
\definecolor{HelgenCyan}{HTML}{51F3DA}
\definecolor{HelgenTeal}{HTML}{17A08C}
\definecolor{HelgenMist}{HTML}{EFF1F5}
\definecolor{HelgenLightCyan}{HTML}{E7FCF8}
\definecolor{HelgenLightIndigo}{HTML}{EDEFF4}
\newtcolorbox{helgennotice}[1][Aviso]{enhanced,breakable,colback=HelgenLightCyan,colframe=HelgenIndigo,boxrule=0.6pt,arc=2pt,left=9pt,right=9pt,top=7pt,bottom=7pt,coltitle=white,colbacktitle=HelgenIndigo,fonttitle=\helgentitle\bfseries\small,title={#1}}
\newcommand{\hwtab}[1]{\begin{center}\footnotesize\renewcommand{\arraystretch}{1.25}#1\end{center}}
\hypersetup{colorlinks=true, linkcolor=HelgenTeal, urlcolor=HelgenTeal, pdftitle={\sistemapdf{} - \doctipo}, pdfauthor={Dr. Leopoldo Rodríguez Salazar (Helgen Technologies)}}
% ---------- Tipografía de marca (XeLaTeX) ----------
\usepackage{iftex}
\ifXeTeX
\usepackage{fontspec}
\IfFontExistsTF{MADE Evolve Sans}{\setmainfont{MADE Evolve Sans}[AutoFakeSlant=0.18]}{\setmainfont{Latin Modern Sans}[AutoFakeSlant=0.18]}
\IfFontExistsTF{League Spartan}{\newfontfamily\helgentitle{League Spartan}[BoldFont={League Spartan SemiBold}]}{\newcommand{\helgentitle}{\sffamily\bfseries}}
\else
\usepackage[T1]{fontenc}
\renewcommand{\familydefault}{\sfdefault}
\newcommand{\helgentitle}{\sffamily\bfseries}
\fi
% ---------- Color de texto global ----------
\color{HelgenIndigo}
% ---------- Títulos de sección ----------
\titleformat{\section}{\helgentitle\Large\bfseries\color{HelgenIndigo}}{\thesection}{0.8em}{}[{\color{HelgenCyan}\titlerule[1.6pt]}]
\titleformat{\subsection}{\helgentitle\large\bfseries\color{HelgenIndigo}}{\thesubsection}{0.8em}{}
\titleformat{\subsubsection}{\helgentitle\normalsize\bfseries\color{HelgenIndigo}}{\thesubsubsection}{0.8em}{}
\titlespacing{\section}{0pt}{18pt}{10pt}
% ---------- Encabezado / pie ----------
\pagestyle{fancy}
\fancyhf{}
\fancyhead[L]{\helgentitle\small\color{HelgenIndigo}\textbf{\sistema} — \doctipo}
\fancyhead[R]{\helgentitle\small\color{HelgenIndigo}Hutchison Ports · ICAVE}
\fancyfoot[L]{\helgentitle\scriptsize\color{gray}Confidencial — Helgen Servicios Especializados S.A. de C.V.}
\fancyfoot[R]{\helgentitle\scriptsize\color{HelgenIndigo}\thepage}
\renewcommand{\headrulewidth}{0.4pt}
\renewcommand{\headrule}{\color{HelgenCyan}\hrule width\headwidth height 1.2pt}
% ---------- Estilos TikZ Helgen ----------
\tikzset{
caja/.style={draw=HelgenIndigo, fill=HelgenMist, rounded corners=2pt, text=HelgenIndigo,
align=center, font=\small, inner sep=5pt, minimum height=9mm},
cajaoscura/.style={draw=HelgenIndigo, fill=HelgenIndigo, rounded corners=2pt, text=white,
align=center, font=\small\bfseries, inner sep=5pt, minimum height=9mm},
zona/.style={draw=HelgenIndigo!50, dashed, rounded corners=4pt, inner sep=8pt},
etiqueta/.style={font=\scriptsize\bfseries, text=HelgenTeal},
flecha/.style={-{Stealth[length=2.6mm]}, thick, draw=HelgenIndigo},
flechaacento/.style={-{Stealth[length=2.6mm]}, very thick, draw=HelgenTeal},
}
\newcolumntype{L}[1]{>{\raggedright\arraybackslash}p{#1}}
\newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}}
% ---------- Listas con viñeta de marca ----------
\setlist[itemize]{label={\textcolor{HelgenTeal}{\rule[0.18ex]{0.5em}{0.5em}}}, leftmargin=1.4em, itemsep=2pt}
% ---------- Portada de marca (comando reutilizable) ----------
% Uso: \helgenportada{Título}{Subtítulo cian}{Bloque de metadatos (tabular)}
\newcommand{\helgenportada}[3]{%
\begin{titlepage}
\begin{tikzpicture}[remember picture, overlay]
\fill[HelgenIndigo] (current page.south west) rectangle (current page.north east);
\draw[HelgenCyan, line width=2.5pt] ([xshift=2.4cm, yshift=-8.1cm]current page.north west) -- ([xshift=-2.4cm, yshift=-8.1cm]current page.north east);
\node[anchor=north west, text=white] at ([xshift=2.4cm, yshift=-3.2cm]current page.north west)
{\helgentitle\fontsize{30}{34}\selectfont Helgen};
\node[anchor=north west, text=HelgenCyan] at ([xshift=2.4cm, yshift=-4.4cm]current page.north west)
{\helgentitle\small Ingeniería que produce, mantenimiento que protege};
\node[anchor=north west, text=white, text width=16.5cm] at ([xshift=2.4cm, yshift=-9.0cm]current page.north west)
{\helgentitle\fontsize{24}{30}\selectfont #1\\[6pt]
\helgentitle\fontsize{14}{19}\selectfont\color{HelgenCyan} #2};
\node[anchor=north west, text=white, text width=16.5cm] at ([xshift=2.4cm, yshift=-12.6cm]current page.north west)
{\helgentitle\fontsize{18}{22}\selectfont \sistema\\[5pt]
\normalsize Fleet RTK GPS — Hutchison Ports (ICAVE · TIMSA · EIT · LCT)};
\node[anchor=south west, text=white] at ([xshift=2.4cm, yshift=2.2cm]current page.south west)
{\footnotesize\helgentitle #3};
\end{tikzpicture}
\end{titlepage}}

View File

@ -41,6 +41,30 @@ tags: [drogheda, hbox, gnss, rtk, ntrip, findings, presentation]
- So **"healthy delivery" ≠ "RTK."** The baseline is *healthy SPS + reliable delivery*;
raise the bar to a fix-≥4 capture once corrections work.
## Update — 22 Jun 2026: live field capture + two ceilings on RTK
> A fresh live `udp/5010` server capture (not the 20-Jun bench) caught two units at once.
- **Telemetry forwarding CONFIRMED.** Enabling `$PUBX` (u-blox *proprietary* `$P…`, the
same class as our planned `$PHLG`) on the F9P **UART2** → it reaches the server **intact**,
next to GGA/RMC. The 4G NTRIP Master is a **transparent forwarder**, so **in-band unit
health (`$PHLG`) to the server is viable** — proven with zero extra hardware.
- **A natural A/B on RTK** (same capture):
- **H0014 (Querétaro bench)** — RSSI **−93**, **SPS (quality 1)**. It sits in a **basement**.
- **H0017 (Veracruz / ICAVE field)** — RSSI **−51**, **DGPS (quality 2, RMC mode `D`,
base id 0133)** — i.e. it **is** applying corrections.
- **Two independent ceilings on RTK** (refines §2's "RTK not achieved"):
1. **Sky / antenna placement** — RTK fix is carrier-phase; a **basement caps it** no matter
how good the corrections are. (Per Leo: bench antenna placement — not fixable there.)
2. **Corrections pipeline** — H0017 has good sky + signal yet only reaches **DGPS, not RTK
(4/5)**, pointing back at the **free rtk2go caster + RAM-only F9P config** as a *separate*
limit present everywhere.
- **Cleanest next test:** one **open-sky capture** (roof/outdoors) with a **proper caster +
FLASH-saved config** — that isolates the two ceilings and shows whether a fix ≥4 is reachable.
- **Watchdog implication:** a healthy unit can legitimately sit at SPS/DGPS (basement, or field
on a weak caster) → **never power-cycle on no-RTK alone** (already enforced by the v2.1
SPS-thrash guard; the v2.2 `$PHLG` will make this visible per-unit).
## 4 · The diagnostic trap (and a retraction)
- The **4G-NTRIP-Master console is an unreliable health signal.** The provably-healthy

View File

@ -130,9 +130,9 @@ these before presenting; if it breaks one, say so and propose the compliant fix.
| `00-Project-Management` | plan, ethos, hackathon, **financials**, **handover**, client comms, Drive tooling | [[00-Project-Management/PLAN\|PLAN]] · [[00-Project-Management/Engineering-Handover\|Engineering-Handover]] · `ethos.md` · `hackathon-plan.md` · `Financials.md` · `Drive-Upload/` · `client-comms/` |
| `01-Requirements` | what we build | `SRS.md` · `HBOX-product-spec.md` |
| `02-Architecture-Software` | backend/software analysis, ADRs | `backend-review.md` |
| `03-Architecture-Hardware` | HBOX/cabinet, wiring, BOM, **watchdog PCB v3** | `hardware-respec-and-pcb-brief.md` · `cabinet-electrical-bom.md` · `watchdog-schematic.md` · `watchdog-health-telemetry.md` · `firmware-review.md` · `BOM-master.md` · **`watchdog-pcb-v3-design.md`** (schematic spec: +2 fans, +I²C temp/RH) · `watchdog-pcb-v3-bom.xlsx` (proto+production, LCSC) · `watchdog-pcb-v3-kicad-guide.md` (KiCad→JLCPCB) · `watchdog-pcb-v3-blockdiagram.svg` |
| `03-Architecture-Hardware` | HBOX/cabinet, wiring, BOM, **watchdog PCB v3** | `hardware-respec-and-pcb-brief.md` · `cabinet-electrical-bom.md` · `watchdog-schematic.md` · `watchdog-health-telemetry.md` · `firmware-review.md` · `BOM-master.md` · **`watchdog-pcb-v3-design.md`** (schematic spec: +1 fan, +I²C temp/RH) · `watchdog-pcb-v3-bom.xlsx` (proto+production, LCSC) · `watchdog-pcb-v3-kicad-guide.md` (KiCad→JLCPCB) · `watchdog-pcb-v3-blockdiagram.svg` · `watchdog-protoboard-bringup.md` (shop list) · `watchdog-connections.xlsx` (pin/net connection list, PCB+proto) |
| `04-Source-Code` | firmware + backend repos | `firmware/` (**own git repo `hbox-firmware`**: `blackpillrtkcode_v2/` deploy · `legacy/` v1 · `docs/LOGGING-SPEC.md` v2.1) · `htracker/` · `notifications/` · `rtk-config/` |
| `05-Analytics-Reports` | SQL views, dashboards (VERONICA/PowerBI) | _(placeholder)_ |
| `05-Analytics-Reports` | SQL views, dashboards (VERONICA/PowerBI), **observability** | `2026-06-22_Drogheda_Manual-Logs-Grafana_v1.0.pdf` (Grafana/Loki log-consultation manual) |
| `06-Testing-Validation` | RCA, diagnostics, test evidence, **log analysis & tooling** | `RCA-2026-06-field-failures.md` · `remote-diagnostic-protocol.md` · `2026-06-20_Drogheda_Bench-Log-Analysis_v1.0.md` (rev-3) · `2026-06-20_Drogheda_GNSS-Findings-Summary.md` · `log-analysis/` (repo **`hbox-log-tools`**: `log-explorer.html` · `analyze_logs.py` · `build_dashboard.py` · `docs/manual.pdf`) |
| `Marketing` | product-facing collateral (LaTeX→PDF) | `HBOX-Datasheet/` (industrial spec sheet) · `HBOX-Brochure-RTK/` (flyer) |
| `99-Archive` | superseded raw inputs (read-only) | `situation_ORIGINAL.md` |
@ -185,6 +185,12 @@ Montacargas 12 V. **Target:** 30 units by August.
`Project-Drogheda/` once synced.
- **Backend repo:** `ssh://git.helgen.dev:6611/htracker` (retranslator + monitor),
cloned at `04-Source-Code/htracker` (branch `feature/test500unidades`).
- **Observability (logs):** backend logs ship off-box to **Grafana Loki** (free tier, 50 GB/mo,
14-day) via **Vector** (default) or **Grafana Alloy** reading journald (capped 200 MB) —
`04-Source-Code/htracker/log-shipping/` (`README.md` runbook · `setup-logging.sh` · `vector.toml`).
Read in Grafana → Explore → Loki: `{job="htracker"}` (Vector) or `{unit="htracker.service"}`
(Alloy). Consultation manual (PDF): `05-Analytics-Reports/2026-06-22_Drogheda_Manual-Logs-Grafana_v1.0.pdf`.
Creds in `/etc/vector/htracker.env` (server) — never in the repo.
- **Firmware repo:** `ssh://git.helgen.dev:6611/hbox-firmware` (STM32F411 watchdog),
lives at `04-Source-Code/firmware` (branch `main`) — its own git repo with README,
`docs/PINOUT.md`, redacted RTK-config template, CHANGELOG. Push instructions in its README.