# Remote Diagnostic Protocol — Pulled HBOX Units

**Date:** 2026-06-19 · **Owner:** Leo Rodriguez
**Situation:** all HBOX units were **removed from the machines**. We have **4 units
in hand: 3 not working, 1 working-ish.** They are **physically in Veracruz**; the
team is in **Querétaro** → debugging is **remote** until they're shipped or someone
on-site acts as our hands.
**Firmware on these units = v1** (no MCU thermal log) → thermal evidence must come
from **physical inspection + a bench power test**, not from the firmware.

> The single working-ish unit is the **control**. Every measurement below should be
> taken on it too, so "bad" readings on the dead units have a baseline to compare
> against.

---

## 0. Decision gate (make this call first)

**Recommendation: ship all 4 units to Querétaro.** They are small, and remote-
debugging *dead* electronics is slow and rarely conclusive — a 1–2 day courier buys
full bench access (PSU with current limit, scope, u-center, reflash, thermal soak,
teardown). Bring the working-ish unit too; it's the reference.

Choose remote-hands **only if** shipping is blocked (client chain-of-custody, the
units must stay on-site, or courier delay is unacceptable). Confirm with ICAVE who
**owns** the units and whether they may leave the terminal.

| Path | When | Cost |
|---|---|---|
| **Ship to QRO** *(recommended)* | units may leave Veracruz | courier + 1–2 days |
| **Remote-hands in Veracruz** | units must stay; cooperative on-site tech exists | needs the protocol below + their time |

Either way, **start §2 (backend analysis) now from Querétaro** — it needs no units.

---

## 1. What we can / cannot learn remotely

| Question | Remote-hands? | How |
|---|---|---|
| Is the LiPo swollen / burnt / leaking? | ✅ | photo checklist (§3 Tier 0) |
| Connectors/SMA loose, solder cracked? | ✅ | photos + wiggle test |
| Does it draw current / boot on power? | ✅ | bench PSU **current reading** (§3) |
| LED state on power-up? | ✅ | film the LED, decode (§4) |
| Battery V, fix quality, state, packets | ✅ (Tier 1) | USB dashboard capture |
| Does the 4G modem attach / get a fix? | ✅ (Tier 1–2) | dashboard / u-center / remote desktop |
| Does it reach our gateway when powered? | ✅ | watch htracker for that unit's `cell_id` |
| Exact failed component / thermal soak | ❌ (needs bench) | **ship to QRO** |

So ~80% of triage is remotely achievable; only component-level confirmation and
thermal reproduction strictly require the units in QRO.

---

## 2. Do this NOW from Querétaro (no units needed)

The backend is on our own server — fully debuggable from QRO, and per
`02-Architecture-Software/backend-review.md` it can take healthy units off Wialon by itself.

1. **Historical online-vs-offline split.** For the days before removal, compare each
   unit's **htracker** record (online on UDP receipt) against **Wialon** (online on
   positions received). A unit that was **online in htracker but offline in Wialon**
   was a **backend victim** (anti-jump lock-out / HOL stall), *not* dead hardware —
   that reframes which "dead" units are actually hardware failures.
2. **Gateway logs around the death window:** look for the anti-jump warning
   (`¡Filtro Anti-Salto Activado!`) spamming one `cell_id` (→ lock-out), and for
   `netstat -su` UDP receive-buffer drops (→ HOL stall).
3. **Bench-reproduce with the working-ish unit** once it's powered (in VER via
   remote-hands, or in QRO): point it at the gateway and watch whether the
   retranslator forwards it cleanly or trips the filter.
4. Caster (HelgenSRV) uptime + **per-SIM data balance/suspension** for the window.

This is the highest-leverage work we can do today and it needs nobody in Veracruz.

---

## 3. Remote-hands protocol (tiered by what the on-site person has)

Give the on-site tech the matching tier. A ready-to-send Spanish version is in §6.

### Tier 0 — power + a phone camera (minimum)
For **each** unit, before powering:
1. **Photograph**: the whole inside, then close-ups of the **LiPo** (any swelling,
   bulging, brown/black marks, leaks?), the **PowerBoost**, every **connector/Dupont**,
   the **SMA antenna** joints, and any **discoloured/burnt** components.
2. **Wiggle test** (gently): do any Dupont/terminal/SMA connectors move or fall off?
3. **Bench power-on with a current-limited PSU** at the unit's normal voltage
   (12 V or 24 V), current limit ~2 A:
   - Read **inrush** then **steady current**. Record the number.
   - **0 mA / trips at 0:** open circuit / dead input stage / blown fuse.
   - **Hits the current limit / very hot:** short → **power off immediately**.
   - **~50–150 mA steady, nothing else:** MCU only, stack not powering (load switch
     / boost / modem).
   - **Normal-ish draw + LED activity:** boots — go to Tier 1.
   - Compare every reading to the **working-ish unit**.
4. **Film the LED** for ~60 s from power-up (decode in §4).

### Tier 1 — Tier 0 + USB cable + a laptop
5. Plug the Blackpill **USB** into the laptop; open **PuTTY/TeraTerm at 115200**.
   The v1 watchdog prints a live dashboard. **Capture 3–5 min** (log to file) and
   send it. Read off: **BATTERY V**, **SYSTEM STATE**, **RTK FIX**, **PACKETS TX'd**,
   PWR/IGN inputs. (No MCU-temp line — these are v1.)
6. Note whether **PACKETS TX'd increments** (GNSS+serial alive) and whether **RTK
   FIX** ever reaches 4/5 (corrections arriving = 4G/NTRIP working).

### Tier 2 — Tier 1 + remote-desktop (AnyDesk/TeamViewer) into the laptop
7. We drive from QRO: read the live dashboard, run **u-center** on the ZED-F9P
   (fix, sat levels, config), issue **AT commands** to the 4G modem (attach, signal),
   and if useful **reflash v2** to gain thermal logging + better recovery.

---

## 4. LED quick-decode (v1)

| LED | State | Read as |
|---|---|---|
| Fast blink | BOOTING | powered, waiting for 4G/GNSS |
| Solid ON | RUNNING, vehicle power, RTK fixed | healthy |
| Heartbeat (1 s) | RUNNING on battery | on LiPo, 30 s rate |
| Slow blink | searching / no corrections | 4G/NTRIP not delivering |
| Off (with power applied) | SLEEP / dead | battery-critical logic **or** not booting |

"Off while powered from a good bench supply" on a unit that should boot = a strong
**no-boot / power-stage** signal — correlate with the current reading.

---

## 5. Per-unit triage decision tree

```
Power on bench PSU (current-limited)
├─ 0 mA ............................. dead input / blown fuse / open  → ship to QRO
├─ hits limit / hot ................ SHORT (power off!) ............... → ship to QRO
└─ draws current
   ├─ no LED, no USB dashboard ...... MCU/boot dead .................... → ship to QRO
   └─ boots (LED + dashboard)
      ├─ PACKETS not incrementing ... GNSS/serial dead ................ Tier 2 / ship
      ├─ FIX stuck 0/1 (never 4/5) .. 4G/NTRIP not delivering ......... check modem/SIM/antenna
      └─ FIX 4/5 + packets OK ........ HARDWARE IS FINE ............... → backend victim (§2)
```

The last branch is the important one: a pulled unit that **boots, fixes, and
transmits on the bench is not the failure** — its disappearance from Wialon is then
backend (§2), and re-installing it unchanged would just repeat the problem.

---

## 6. Safety — heat-damaged LiPo (tell the on-site tech)

A swollen/puffy or hot LiPo is a **fire risk**. Do **not** charge it, puncture it, or
keep powering a unit whose battery is visibly swollen. Disconnect it, place it in a
**LiPo-safe / fireproof bag or a metal container**, keep it away from anything
flammable, and ventilate. Photograph first, then isolate. If it's hot, vented, or
smells: stop and isolate immediately.

---

## 6b. Spanish — instrucciones para el técnico en sitio (Tier 0/1)

> Para **cada** unidad: (1) **Fotos** del interior y de cerca de la **batería LiPo**
> (¿inflada, manchas cafés/negras, fuga?), del **PowerBoost**, de **cada conector
> Dupont**, de las **antenas SMA**, y de cualquier componente **quemado/decolorado**.
> (2) Prueba de **movimiento** suave: ¿algún conector se mueve o se cae? (3) Enciéndela
> con una **fuente de banco** a su voltaje normal (12 V/24 V), límite de corriente ~2 A,
> y anota la **corriente** (arranque y estable). Si llega al límite o se calienta,
> **apágala de inmediato** (corto). (4) **Graba el LED** ~60 s desde el encendido.
> (5) Si hay laptop: conecta el **USB** del Blackpill, abre **PuTTY/TeraTerm a 115200**
> y **graba 3–5 min** del tablero (BATERÍA V, ESTADO, RTK FIX, PAQUETES). **Seguridad:**
> una batería **inflada o caliente** es riesgo de incendio — no la cargues ni la
> perfores; aíslala en bolsa/recipiente seguro y ventila.

---

## 7. What to send back to Querétaro (capture checklist)

Per unit, labeled with its `cell_id`/machine: the **photos**, the **current reading**
(inrush + steady), the **LED video**, the **3–5 min USB dashboard capture**, and a
one-line note (boots? fix? packets?). With that, we can classify all 4 units against
the tree in §5 and the backend evidence in §2 — and decide which truly need teardown
in QRO.