Waterstream 1.1.0 introduces the Bridge mode that allows connecting Waterstream to another MQTT broker and exchanging messages between them, appearing as a single distributed broker.
To showcase this new capability, we connected Waterstream to the Lufthansa Open API notification service. This public API allows monitoring changes of Lufthansa flights (and others) using the MQTT protocol. Communication flows only from the Lufthansa endpoint toward Waterstream that behaves like a simple MQTT client consuming available MQTT topics.
Below there's an example of a message coming from the Lufthansa endpoint.
{
"Update":{
"ScheduledFlightDate":"2020-10-05",
"Message":"New Estimated Arrival",
"ScheduledFlightTime":"0110",
"FlightNumber":"LX967",
"Timestamp":"2020-10-05T13:45:03"
}
}
Waterstream uses Apache Kafka as the persistence layer. Every message ingested from the source MQTT broker by the bridge is written directly into Kafka. It's straightforward to configure a Kafka Connector to import that data into Elasticsearch and build some Kibana dashboards.
The final result consists of a seamless integration that allows you to get insights of the ingested data. You can select one airline to filter data, or you can apply a different time window to get notifications for a specific day or hour.
The source code of this demo is available on Github
Visit Waterstream product site at waterstream.io and learn how to combine MQTT with Kafka, the standard de-facto streaming platform.