# MQTT to Kafka bridge lab

This loopback-only lab demonstrates a verifiable MQTT-to-Kafka ingestion boundary. It uses:

- Eclipse Mosquitto 2.0.22 for MQTT;
- Redpanda 25.1.9 as a Kafka-compatible event log and HTTP proxy;
- MQTT.js 5.15.2 plus Node.js `fetch` in the bridge.

No real credentials or production endpoints are required.

## Run the lab

From the RunMQTT repository root:

```bash
docker compose -f public/examples/mqtt-kafka-bridge/compose.yaml up -d
pnpm mqtt:kafka-bridge:self-test
pnpm mqtt:kafka-bridge
```

Wait for the bridge to print `bridge_ready`. In a second shell, publish a valid synthetic event:

```bash
docker compose -f public/examples/mqtt-kafka-bridge/compose.yaml exec mosquitto \
  mosquitto_pub -h 127.0.0.1 -t lab/device-042/telemetry -q 1 \
  -m '{"eventId":"evt-001","recordedAt":"2026-01-01T00:00:00Z","value":21.4}'
```

The bridge validates the payload, derives `deviceId` from the MQTT Topic, adds `schemaVersion` and `ingestedAt`, and sends the record to `iot.telemetry` with `device-042` as its Kafka key. Inspect one record:

```bash
docker compose -f public/examples/mqtt-kafka-bridge/compose.yaml exec redpanda \
  rpk topic consume iot.telemetry --num 1 --brokers redpanda:9092
```

Publish `{}` to the same MQTT Topic and consume `iot.telemetry.dlq` to verify schema rejection. The DLQ envelope records the reason, byte length, and SHA-256 digest, but does not copy the rejected payload.

Stop the bridge with Ctrl+C, then remove the disposable containers:

```bash
docker compose -f public/examples/mqtt-kafka-bridge/compose.yaml down -v
```

## What the example proves

- an MQTT Topic can be normalized into a smaller Kafka Topic taxonomy;
- the authorized Topic path, rather than an untrusted payload field, supplies the partition key;
- valid events and validation failures take observable, separate paths;
- transient HTTP proxy failures use four bounded attempts with exponential backoff;
- same-device events use the same Kafka record key and therefore the same partition.

## Production limits

This fixture is deliberately not a production connector:

- Mosquitto permits anonymous access because it binds only to `127.0.0.1`; production must authenticate every publisher and enforce device-scoped Topic ACLs.
- MQTT delivery is acknowledged independently of the asynchronous Kafka write. A bridge crash between those events can lose a record. Use a supported connector or durable local spool/outbox when loss at this boundary is unacceptable.
- The bridge is a single process without leader election, admission limits, schema-registry integration, durable retry storage, or secret management.
- If Kafka itself is unavailable, the example cannot write to its Kafka DLQ. Production needs an independent durable failure path and an operator alert.
- A DLQ is not a retry loop. Restrict access, set retention, attach ownership, and replay only after correcting the cause.
- The HTTP endpoints and anonymous test broker are loopback-only. Do not expose this Compose project to a shared network.

Use the [MQTT to Kafka architecture guide](/mqtt-vs-kafka) for the full identity, Topic, schema, partition, retry, DLQ, observability, and command-return boundaries.