Source: docs/backend.md
Backend
A FastAPI app (src/herdtag/server/) with SQLite storage. It has four jobs:
- Take uplinks from The Things Stack (TTS) by webhook.
- Store tag state, position history and events.
- Manage fences and push them to tags as downlinks.
- Serve the map UI, the dashboards and the analytics.
Running it
cd ~/herdtag
uv run herdtag-server # http://127.0.0.1:8000, DB file ./herdtag.db
HERDTAG_DB=/path/farm.db PORT=8080 HOST=0.0.0.0 uv run herdtag-server
| Env var | Default | Meaning |
|---|---|---|
HERDTAG_DB |
herdtag.db (current directory) |
SQLite file path. Created if missing |
HOST |
127.0.0.1 |
Bind address. Keep it local unless something in front of it does auth |
PORT |
8000 |
Port |
HERDTAG_WEBHOOK_SECRET |
unset | If set, /ingest/ttn requires an X-Webhook-Secret header with this value |
TTS_BASE |
unset | e.g. https://nam1.cloud.thethings.network |
TTS_APP_ID |
unset | TTS application id |
TTS_API_KEY |
unset | TTS API key that can write downlinks |
With all three TTS_* variables set, fence assignments are pushed to TTS. Without them, the server works in local mode: downlinks go into a queue that only the virtual tags read, and the UI shows a banner saying so.
Endpoints
All bodies are JSON unless noted. Interactive OpenAPI docs are at /docs.
Ingest
POST /ingest/ttn: the TTS v3 uplink webhook.
- Body: the standard TTS uplink message. The fields used are
end_device_ids.device_id,end_device_ids.dev_eui,received_at,uplink_message.f_port,uplink_message.frm_payload(base64), anduplink_message.rx_metadata[].rssi/snr(the strongest gateway is kept). - Responses:
{"ok":true,"state":"warning","events":[...],"cue":true}on success{"ignored":"not an uplink"}for joins, acks and other webhook types{"ignored":"unknown payload"}when the port isn't 1 or the payload isn't 11 bytes; this also logs anunknown_payloadevent401if the webhook secret is wrong
P=$(printf '16eb3159c673a873572101' | xxd -r -p | base64)
curl -s -X POST localhost:8000/ingest/ttn -H 'content-type: application/json' -d "{
\"end_device_ids\": {\"device_id\": \"tag-01\", \"dev_eui\": \"70B3D57ED0000001\"},
\"received_at\": \"2026-11-07T15:04:05.123456789Z\",
\"uplink_message\": {\"f_port\": 1, \"frm_payload\": \"$P\", \"rx_metadata\": [{\"rssi\": -101, \"snr\": 4.5}]}}"
# {"ok":true,"state":"warning","events":[],"cue":true}
Events the backend derives from uplinks:
| Event | Trigger |
|---|---|
breach |
State changed to breach |
returned |
State changed away from breach |
cue_ignored |
FLAG_IGNORED set |
gps_degraded |
FLAG_DEGRADED set and wasn't set on the previous uplink |
fence_synced |
FLAG_FENCE_ACK set, the reported fence matches the assigned one, and it differs from what the tag was enforcing before |
battery_low |
Battery ≤ 20 %, at most once per tag per 24 h |
fence_assigned |
Logged by the assign endpoint, not by ingest |
Tags and events
| Method and path | Returns |
|---|---|
GET /api/tags |
Every tag: dev_id, dev_eui, lat, lon, state, battery, flags, enforcing_fence, assigned_fence, rssi, snr, last_seen, plus computed synced (assigned is none, or equals enforcing) and age_s |
GET /api/tags/{dev_id}/track?hours=6 |
[{t, lat, lon, state}] in time order |
GET /api/events?limit=100 |
Newest first, at most 1000. [{id, dev_id, t, kind, detail}] |
GET /api/status |
{"tts_configured": bool, "max_vertices": 32} |
curl -s localhost:8000/api/tags
curl -s 'localhost:8000/api/tags/tag-01/track?hours=24'
curl -s 'localhost:8000/api/events?limit=20'
Fences
GET /api/fences lists the active (non-archived) fences: [{id, name, points:[[lat,lon],...], created_at, archived}].
POST /api/fences creates a fence. Responds 201 with the fence.
- Body:
{"name": "North", "points": [[lat, lon], ...]}. A closing point equal to the first is dropped. - Validation (422): 3–32 corners, valid lat/lon, no edges crossing each other, within ±32 km of its centre.
- Ids: fences get the lowest unused id from 1–255. The id is what tags report back.
curl -s -X POST localhost:8000/api/fences -H 'content-type: application/json' \
-d '{"name":"North","points":[[38.4500,-96.5520],[38.4500,-96.5480],[38.4530,-96.5480],[38.4530,-96.5520]]}'
# {"id":1,"name":"North","points":[[38.45,-96.552],...],"created_at":...,"archived":0}
POST /api/fences/{id}/assign sends a fence to tags.
- Body:
{"dev_ids": ["tag-01", "tag-02"]}, or{"dev_ids": null}for every known tag. - For each tag it:
- sets
assigned_fence - marks older still-queued fence downlinks as
superseded - encodes the fence into fragments of at most 51 bytes
- pushes them to TTS if configured, otherwise queues them locally
- logs a
fence_assignedevent - Response:
[{dev_id, frames, status, detail}], wherestatusisqueued,sent_to_ttsorerror. A TTS failure for one tag doesn't stop the others.
curl -s -X POST localhost:8000/api/fences/1/assign -H 'content-type: application/json' -d '{"dev_ids":["tag-01"]}'
# [{"dev_id":"tag-01","frames":1,"status":"queued","detail":""}]
DELETE /api/fences/{id} archives a fence. It returns 409 if any tag is still assigned to it. Archived fences keep their id, so ids aren't reused.
Dashboards and analytics
| Method and path | What |
|---|---|
GET / |
Operator map: live tags, draw fences, send them to tags, events |
GET /farmer, /ops, /trial, /explore |
Farmer view, command centre, trial/research view, simulator explorer |
GET /api/farmer |
Plain-language herd summary: headline, level (ok/attention/urgent/info), cows, alerts (out, not checking in for over 1 h, battery ≤ 20 %, fence not picked up, cue_ignored in the last 2 h), fences, counts |
GET /api/ops?hours=24&bins=24 |
Fleet and herd metrics: per-tag battery series, staleness, uplinks, estimated missed uplinks / loss %, RSSI/SNR, cue uplinks, breaches, time share by state; uplinks per bin; state share over time; breaches per day; RSSI coverage cells; fences |
GET /api/trial?hours=744 |
Per-animal cue episodes (grouped from consecutive cue-flagged uplinks), rolling success, habituation delta, a latency upper bound, cue hours; pooled learning curve in blocks of five episodes; the controller's caps; list of data gaps |
GET /api/export/positions.csv?hours=744 |
CSV: dev_id, iso_time, t, lat, lon, state, battery, cue_since_last, cue_ignored, flags |
GET /api/export/events.csv?hours=744 |
CSV: dev_id, iso_time, t, kind, detail |
The analytics only use what tags actually send. Anything they can't know (cue bursts per uplink, turn timestamps, accelerometer activity, gateway id, sound level) comes back in a gaps object, rather than being estimated.
Development only
POST /dev/downlinks/{dev_id}/take returns [{f_port, frm_payload}] for that tag's queued downlinks and marks them delivered_local. The virtual tags in herdtag demo call it right after each uplink, to mimic Class A. It has no auth, so don't expose it publicly.
Database schema
SQLite, created on start-up (server/db.py).
| Table | Columns | Notes |
|---|---|---|
tags |
dev_id PK, dev_eui, name, lat, lon, state, battery, flags, enforcing_fence, assigned_fence, rssi, snr, last_seen |
One row per tag, holding its latest values. enforcing_fence is what the tag reports; assigned_fence is what we told it |
positions |
id, dev_id, t, lat, lon, state, battery, flags |
One row per uplink. Indexed on (dev_id, t) |
events |
id, dev_id, t, kind, detail |
|
fences |
id (1–255), name, points (JSON [[lat,lon],...]), created_at, archived |
|
rf |
id, dev_id, t, lat, lon, rssi, snr |
Radio history for coverage maps. Filled by a trigger when tags.last_seen changes |
downlinks |
id, dev_id, fence_id, f_port, payload (blob), status, created_at, detail |
status: queued, sent_to_tts, error, superseded, delivered_local |
Times are Unix seconds (float, UTC). Uplink times come from TTS's received_at, not the server clock.
Connecting The Things Stack
The gateway and device registration steps are in hardware/BUILD.md §7. On the backend side:
- Make the server reachable over HTTPS, for example with a Cloudflare Tunnel or a small VPS behind a reverse proxy.
- In the TTS console, go to Application → Integrations → Webhooks → add a custom webhook:
- Format JSON, base URL
https://<host>, and enable Uplink message with path/ingest/ttn. - Add a headerX-Webhook-Secret: <random string>, and start the server with the sameHERDTAG_WEBHOOK_SECRET. - Create an API key in the application with permission to write downlink application traffic. Start the server with
TTS_BASE,TTS_APP_IDandTTS_API_KEY.
Downlink push calls POST {TTS_BASE}/api/v3/as/applications/{app}/devices/{dev}/down/push. It sends one confirmed f_port: 2 entry per fragment, with a Bearer token. This path hasn't been tested against a live TTS yet. Check it against your TTS version's API reference on the first real run.
Security caveats
- No user authentication. Anyone who can reach the server can read positions, create and assign fences, and drain the local downlink queue. Keep it on
127.0.0.1, or put it behind an authenticating proxy (e.g. Cloudflare Access). - The webhook secret protects only
/ingest/ttn, and only whenHERDTAG_WEBHOOK_SECRETis set. /dev/downlinks/*exists for the demo. It should be disabled or protected before any internet-facing deployment.- Positions of animals on a named farm are sensitive. Treat the DB file and the CSV exports accordingly.