6a703c314d
Incomplete: step 1 (fixture markers + track_truth.json), step 6 (frontend wiring), step 8 (tests) not done; suite not yet run. Salvaged from interrupted prior session. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
172 lines
9.2 KiB
Python
172 lines
9.2 KiB
Python
"""Marker detection (spec M19, lane H) — STUB with the frozen contracts baked in.
|
|
|
|
foundation3 ships this file with:
|
|
|
|
- the **concrete, foundation-owned protocol constants + pure encoders** every side of the
|
|
system must agree on (the blink OOK protocol #4, the color-mode marker-key convention #1).
|
|
``synthetic.py`` renders the fixture markers with these, ``scripts/badge_selftest.py``
|
|
(lane K) imports this decoder path, and ``hardware/badge/protocol.h`` (lane K) duplicates
|
|
the constants with a documented copy-check. **One source of truth lives here.**
|
|
- a **stub** :func:`run_detect` that raises ``NotImplementedError``. Lane H fills the body
|
|
(and adds ``backend/tests/test_tracker_detect.py``); it must not change the constants or
|
|
the emitted ``marker_key`` conventions, only implement the detection logic.
|
|
|
|
Nothing in this module raises at import time — ``synthetic.py`` imports the constants and the
|
|
``encode_word`` / ``led_on`` helpers, so the import must stay side-effect-free and light
|
|
(no OpenCV at module scope; import ``cv2`` inside ``run_detect``).
|
|
|
|
================================================================================
|
|
FROZEN CONTRACTS (plan/30-phase6.md — copy, do not reinterpret)
|
|
================================================================================
|
|
|
|
Contract #1 — Marker hues (color mode)
|
|
--------------------------------------
|
|
Magenta ``#ff00ff`` and cyan ``#00ffff`` are the two supported fixture hues; real shoots may
|
|
configure others via ``FESTIVAL4D_MARKER_HUES`` (comma-separated hex). Detection converts to
|
|
HSV and gates on **saturation AND value before hue** (pitfall #2 — stage lighting lies about
|
|
color). A color blob's ``marker_key`` is ``"hue:<H>"`` where ``<H>`` is the **OpenCV hue**
|
|
(0..179) of the configured marker color, i.e. ``round(matplotlib_hue_degrees / 2)``:
|
|
magenta → ``hue:150``, cyan → ``hue:90``. Use :func:`color_marker_key` — never format the
|
|
string by hand.
|
|
|
|
Contract #2 — Schema (what the solver, not the detector, writes; here for context)
|
|
----------------------------------------------------------------------------------
|
|
``tracks(id INTEGER PK, marker_key TEXT NOT NULL, label TEXT, color TEXT, created_at TEXT)``
|
|
``track_points(id INTEGER PK, track_id FK, t_global_s FLOAT, x FLOAT, y FLOAT, z FLOAT,
|
|
quality FLOAT, views INTEGER)`` — points ordered by ``t_global_s``;
|
|
``views`` = cameras used (1 ⇒ ground-plane fallback); coordinates in **Three.js scene space**.
|
|
|
|
Contract #3 — detections.json (this module's OUTPUT — write via `db`? No: a JSON file)
|
|
--------------------------------------------------------------------------------------
|
|
Write ``config.DETECTIONS_JSON`` (``data/work/detections.json``) with EXACTLY this shape::
|
|
|
|
{"videos": {"<video_id>": [{"t_video_s": float, "marker_key": str,
|
|
"cx": float, "cy": float, "area": float,
|
|
"conf": float 0..1}]},
|
|
"generated_by": "festival4d.tracker_detect"}
|
|
|
|
``cx`` / ``cy`` are **normalized 0..1** (the bbox-annotation convention: ``cx = px / width``,
|
|
``cy = py / height``). ``area`` is the blob area in pixels. Detections at ``t_video`` (the
|
|
frame's local time) — the solver converts to ``t_global`` with the FROZEN
|
|
``config.t_global_from_video`` helper (pitfall #1). No markers / dark / markerless video →
|
|
a valid-empty ``{"videos": {}, "generated_by": ...}``, never a crash.
|
|
|
|
Contract #4 — Blink protocol (OOK, camera-decodable)
|
|
----------------------------------------------------
|
|
Bit period **133 ms** (≈4 frames @ 30 fps). One **word** = preamble ``11100`` + 6-bit ID
|
|
(MSB first) + even parity = **12 bits**, repeated continuously (~1.6 s/word). ID **63** is
|
|
reserved for the sync beacon. Decoding needs **≥24 fps** video and **≥1 full word** of
|
|
visibility. Decode by **integrating brightness per bit window using the video's real fps**
|
|
(``meta.fps``) — never by counting frames (pitfall #3). Emit ``marker_key = "code:<id>"``
|
|
via :func:`code_marker_key`. The constants below are the byte-identical source for
|
|
``hardware/badge/protocol.h`` (lane K).
|
|
|
|
Contract #5 — Ground plane (solver context)
|
|
-------------------------------------------
|
|
Synthetic ground is ``y = 0`` (scene space). Real projects estimate it as the
|
|
5th-percentile-y horizontal plane of the point cloud. Single-view fallback intersects the
|
|
pixel ray with this plane; such points get ``views = 1`` and ``quality ≤ 0.5``.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import os
|
|
|
|
log = logging.getLogger("festival4d.tracker_detect")
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Contract #4 — blink OOK protocol. FROZEN. Byte-identical to hardware/badge/protocol.h.
|
|
# ---------------------------------------------------------------------------
|
|
BLINK_BIT_PERIOD_S: float = 0.133 # 133 ms bit period (≈4 frames @ 30 fps)
|
|
BLINK_PREAMBLE: tuple[int, ...] = (1, 1, 1, 0, 0) # start-of-word marker
|
|
BLINK_ID_BITS: int = 6 # payload width (0..63)
|
|
BLINK_WORD_BITS: int = len(BLINK_PREAMBLE) + BLINK_ID_BITS + 1 # +1 parity = 12
|
|
BLINK_BEACON_ID: int = 63 # reserved: the sync/calibration beacon
|
|
BLINK_MIN_FPS: float = 24.0 # decoder needs at least this frame rate
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Contract #1 — color mode marker hues. OpenCV hue is 0..179 (degrees / 2).
|
|
# ---------------------------------------------------------------------------
|
|
MAGENTA_OPENCV_HUE: int = 150 # #ff00ff
|
|
CYAN_OPENCV_HUE: int = 90 # #00ffff
|
|
DEFAULT_MARKER_HUES_HEX: tuple[str, ...] = ("#ff00ff", "#00ffff")
|
|
|
|
|
|
def encode_word(marker_id: int) -> list[int]:
|
|
"""The 12-bit OOK word for ``marker_id`` (contract #4). Pure; no I/O.
|
|
|
|
``preamble(11100) + 6-bit ID MSB-first + even-parity bit``. The parity bit is chosen so
|
|
the **whole word has an even number of 1-bits** (parity over preamble+ID). This is the
|
|
exact bit sequence a badge emits, ``synthetic.py`` renders, and lane H's decoder inverts.
|
|
|
|
>>> encode_word(5)
|
|
[1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1]
|
|
"""
|
|
mid = int(marker_id)
|
|
if not 0 <= mid < (1 << BLINK_ID_BITS):
|
|
raise ValueError(f"marker_id {marker_id} out of range 0..{(1 << BLINK_ID_BITS) - 1}")
|
|
id_bits = [(mid >> (BLINK_ID_BITS - 1 - i)) & 1 for i in range(BLINK_ID_BITS)]
|
|
body = list(BLINK_PREAMBLE) + id_bits
|
|
parity = sum(body) % 2 # even parity: word weight (incl. parity) is even
|
|
return body + [parity]
|
|
|
|
|
|
def led_on(t_seconds: float, marker_id: int, epoch: float = 0.0) -> bool:
|
|
"""Whether the badge LED for ``marker_id`` is lit at wall-clock time ``t_seconds``.
|
|
|
|
The word repeats continuously from ``epoch``; a single physical badge blinks in real
|
|
(global) time, so every camera that captures a frame at the same ``t_global`` sees the
|
|
same LED state. Used by the synthetic fixture to render marker B faithfully.
|
|
"""
|
|
bit_index = int((t_seconds - epoch) // BLINK_BIT_PERIOD_S)
|
|
word = encode_word(marker_id)
|
|
return bool(word[bit_index % BLINK_WORD_BITS])
|
|
|
|
|
|
def color_marker_key(opencv_hue: int) -> str:
|
|
"""The ``marker_key`` for a color blob of the given OpenCV hue (contract #1)."""
|
|
return f"hue:{int(opencv_hue)}"
|
|
|
|
|
|
def code_marker_key(marker_id: int) -> str:
|
|
"""The ``marker_key`` for a decoded blink ID (contract #4)."""
|
|
return f"code:{int(marker_id)}"
|
|
|
|
|
|
def marker_hues_hex() -> list[str]:
|
|
"""Configured color-mode hues as hex strings (``FESTIVAL4D_MARKER_HUES`` or the default)."""
|
|
env = os.environ.get("FESTIVAL4D_MARKER_HUES")
|
|
if env:
|
|
hues = [h.strip() for h in env.split(",") if h.strip()]
|
|
if hues:
|
|
return hues
|
|
return list(DEFAULT_MARKER_HUES_HEX)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Lane H fills these. Signatures FROZEN.
|
|
# ---------------------------------------------------------------------------
|
|
def run_detect(sample_fps: float = 8.0) -> dict:
|
|
"""Detect markers in every video and write ``detections.json`` (spec M19, contract #3).
|
|
|
|
Lane H implements:
|
|
1. Sample frames per video with the ``frames.py`` machinery at ~``sample_fps`` fps
|
|
(document the exact rate; ~8 fps comfortably oversamples the 133 ms bit period).
|
|
2. **Color mode:** BGR→HSV; gate on saturation AND value *before* hue (pitfall #2);
|
|
connected components per configured hue (:func:`marker_hues_hex`); blob centroid →
|
|
normalized ``cx``/``cy``; ``marker_key = color_marker_key(<opencv_hue>)``.
|
|
3. **Blink mode:** track bright blobs across the sampled window; integrate brightness
|
|
per :data:`BLINK_BIT_PERIOD_S` window using the video's **real fps** (``meta.fps``,
|
|
pitfall #3); correlate against :data:`BLINK_PREAMBLE`; check even parity; emit
|
|
``marker_key = code_marker_key(<id>)``.
|
|
4. Write ``config.DETECTIONS_JSON`` in the contract-#3 shape. No markers / dark /
|
|
markerless → valid-empty ``{"videos": {}, "generated_by": "festival4d.tracker_detect"}``.
|
|
|
|
Returns a summary dict (per-video detection counts, decoded blink IDs). Never crash on a
|
|
markerless or unreadable video — log and continue.
|
|
"""
|
|
raise NotImplementedError(
|
|
"marker detection not implemented yet (lane H / M19) — fill tracker_detect.run_detect"
|
|
)
|