MQTT
A compact publish-subscribe protocol built for intermittent networks, constrained devices, hierarchical topics, and broker-pushed delivery.
Best for device-to-cloud and command-and-control messaging.
Protocol comparison
MQTT vs Kafka: device messaging or durable event streaming?
Use MQTT alone for live device connectivity and commands without a replayable backend log. Use Kafka alone for trusted backend producers and durable event processing. Use both when constrained devices need MQTT sessions at the edge and multiple backend consumers need retained, replayable events behind a controlled bridge.
Compare the operating model, not just throughput
The protocols solve different boundaries. MQTT optimizes the device connection; Kafka optimizes the durable processing backbone.
Apache Kafka
A distributed event log built for durable retention, partitioned throughput, consumer-controlled offsets, replay, and stream processing.
Best for service-to-service event pipelines, analytics, and replay.
| Capability | MQTT | Kafka |
|---|---|---|
| Device connection lifecycle | Native fit: lightweight clients, keepalive, sessions, reconnect, and Last Will | Backend clients assume stable connectivity and Kafka protocol support |
| Core model | Brokered publish-subscribe with hierarchical topics | Partitioned append-only log with consumer offsets |
| Typical clients | Sensors, embedded devices, gateways, mobile clients | Backend services, connectors, stream processors |
| Delivery | Broker pushes matching messages; QoS 0, 1, or 2 | Consumers pull and commit offsets per consumer group |
| Command downlink | Broker-pushed command topics with per-device authorization, expiry, and acknowledgments | Requires an application gateway to authorize and translate events into device commands |
| Retention and replay | Retained latest value and session queues; not a general event archive | Time- or size-based durable retention with offset replay |
| Ordering and partitioning | Order is scoped to a client connection and Topic delivery; no consumer partition model | A record key selects a partition; order is guaranteed only within that partition |
| Independent processing consumers | Subscriptions fan out live delivery; shared subscriptions divide work without durable offsets | Consumer groups keep independent offsets and scale processing across partitions |
| Routing | Hierarchical topic filters with + and # wildcards | Flat topics and partitions; routing belongs in producers or processors |
| Network assumptions | Designed for low bandwidth, high latency, and reconnecting clients | Designed for stable data-center or cloud service connectivity |
Choose by where the data is crossing
A single system can use both without duplicating their responsibilities.
Choose MQTT
When constrained or intermittently connected clients need low-overhead telemetry, commands, session behavior, and topic-level fan-out.
Choose Kafka
When backend teams need durable event history, multiple independent consumers, replay, partition ordering, and stream processing.
Use both
When MQTT terminates device sessions and selected events flow into Kafka for durable analytics, enrichment, and downstream products.
A practical MQTT-to-Kafka boundary
Keep protocol responsibilities explicit so replay, backpressure, and authorization remain understandable.
1. Authenticate before the bridge
The MQTT broker authenticates each device and enforces its publish Topic. The bridge trusts broker-authorized routing, not a deviceId supplied inside the payload.
2. Map Topics deliberately
Subscribe only to approved telemetry filters. Map them to a small Kafka Topic set by event domain instead of copying every MQTT Topic into Kafka.
3. Validate and envelope
Validate versioned payloads, derive tenant and device context from the authorized Topic, and add ingestion time and a stable event ID before producing.
4. Choose the ordering key
Key records by the smallest business unit that needs order—often deviceId or assetId. Kafka does not provide total order across partitions.
5. Bound retries and dead letters
Retry transient produce failures with backoff. Send invalid or exhausted records to a restricted DLQ with the reason, then alert; never loop them silently.
6. Measure both sides
Correlate MQTT client and Topic with Kafka Topic, partition, offset, event ID, retry count, DLQ rate, queue age, and end-to-end latency without logging secrets.
7. Keep commands on a gated return path
A separate service must authorize Kafka-derived actions, add operation ID and expiry, publish through a command-specific MQTT identity, and reconcile device acknowledgments.
Reference flow and ownership handoff
- 01
Device fleet
Unique identity; versioned telemetry
- 02
MQTT broker
TLS, sessions, Topic ACLs, command delivery
- 03
Ingestion bridge
Schema, envelope, retry, DLQ
- 04
Kafka topics
Partitioned retention and replay
- 05
Consumer groups
Independent processing and offsets
Runnable local bridge, with explicit limits
The repository example starts loopback-only Mosquitto and Redpanda containers, validates a synthetic telemetry schema, keys Kafka records by deviceId, retries bounded failures, and routes invalid records to a DLQ. It is a learning fixture—not a production connector: the anonymous local broker has no device identity, and the bridge has no durable spool between MQTT acknowledgment and Kafka acknowledgment.
Run from the repository root
docker compose -f public/examples/mqtt-kafka-bridge/compose.yaml up -d
pnpm mqtt:kafka-bridge
# In a second shell, publish a synthetic event
docker compose -f public/examples/mqtt-kafka-bridge/compose.yaml exec mosquitto \
mosquitto_pub -h 127.0.0.1 -t lab/device-042/telemetry -q 1 \
-m '{"eventId":"evt-001","recordedAt":"2026-01-01T00:00:00Z","value":21.4}'Protocol and implementation sources
MQTT vs Kafka FAQ
Can Kafka replace an MQTT broker?
Kafka is not a drop-in device broker. It does not provide MQTT sessions, QoS flows, retained messages, hierarchical topic filters, or the same constrained-client footprint.
Can MQTT replace Kafka for event history?
MQTT can queue session messages and retain a latest value, but it is not designed as a long-lived replayable event log for many independent backend consumers.
Where should an MQTT-to-Kafka bridge run?
Run it at a controlled ingestion boundary where credentials, schemas, retry behavior, partition keys, and dead-letter handling can be operated and observed.
Related architecture guides
Continue from architecture boundaries into MQTT implementation, security, operating model, fleet scaling, and plan selection.
Browser MQTT over WSS
Understand browser transport, MQTT responsibility boundaries, and reconnect behavior.
MQTT tutorial
Follow a message from client connection through Topic delivery.
MQTT security
Protect device identity, transport, and topic permissions.
Managed vs self-hosted MQTT
Assign ownership for broker upgrades, availability, and security operations.
Scaling MQTT fleets
Plan connection, subscription, reconnect, and regional capacity before adding downstream pipelines.
RunMQTT pricing
Compare managed MQTT broker plans for the device connection layer.