🚀
Introducing Versions: Develop data products using Git. Join the waitlist

The power of real-time streaming analytics with Confluent and Tinybird

Today, Tinybird is excited to announce the launch of our new Confluent Connector, offering Confluent users the ability to stream their data into Tinybird in real time. With today’s announcement, Confluent users have a new way to query their streaming data and build fast APIs, faster.
Alejandro Martín
Product Manager
Apr 18, 2023
 ・ 
  min read

Modern applications are built on the most accurate, up-to-the-second data. Whether it’s stock inventory management, website analytics, in-product analytics, or more, making real-time decisions can make all the difference. Some of the biggest applications on the planet leverage streaming data at scale, and Confluent is the leading platform provider for streaming applications. Confluent is a cloud-native, complete, and fully managed service that goes above and beyond Apache Kafka.

Many of Tinybird’s customers also use Confluent:

  • A top 5 retailer uses Confluent and Tinybird together for real-time smart inventory management to reconcile stock across fulfillment centers, retail locations, and online shopping.
  • Fanduel uses Confluent and Tinybird to power real-time personalization across all their sports betting solutions to improve time-to-first-bet and reduce the risk of fraud.
  • Vercel uses Confluent and Tinybird to provide user-facing real-time analytics tools so their customers can get accurate visibility into the performance of their sites

Often, developers need to combine and enrich their streaming data with data from elsewhere. Streaming data on its often is often not valuable, or only partially valuable. To truly get the most out of your streams, you often need to enrich them with dimensions from other sources.

To get the most out of your streaming data sources, you need to enrich them with corresponding dimensions.

For example, if you have a stream of real-time sales events from an eCommerce website, you can work out your up-to-the-second sales figures, which is nice to have. 

But, if you also join this sales event stream with a table of inventory data, you can also begin to make automated, real-time decisions to purchase stock so that you can continue to fulfill orders. For modern retailers, this is a must-have.

Announcing the Tinybird Confluent Connector

Today, Tinybird is excited to launch the Tinybird Confluent Connector, which enables our users to ingest their data from Confluent streams, query and combine it with other data sources using SQL, and publish the results as high-concurrency, low-latency APIs. These APIs can power numerous user-facing scenarios, including real-time business intelligence, in-app personalization, smart inventory management, and many more.

The Tinybird Confluent Connector makes it easy for Confluent users to build low-latency APIs over Confluent data at scale.
A gif showing the Tinybird Confluent Connector workflow in the Tinybird UI
The Tinybird Confluent Connector makes it easy to ingest data from a Confluent topic into a Tinybird Data Source.

Already, several Tinybird customers are using the Confluent Connector, with hundreds of topics being ingested daily.

Read on for more information about how customers use the Confluent Connector today and what it’s like to use it.

In the meantime, if you’re not a Tinybird customer, you can learn more and sign up for free (no credit card required). You can also check out our documentation, or feel free to join the Tinybird Community on Slack and ask us any questions or request additional features.

Setting up the Confluent Connector in Tinybird

You can start using the Tinybird Confluent Connector today in the Tinybird UI. Add a new Data Source as you typically would, and then select the Confluent option in the dialog box.

A screenshot showing the Tinybird Confluent Connector being selected in the Tinybird UI
You can easily connect send Confluent data to Tinybird with the Confluent Connector.

Then, navigate to Confluent to get your Bootstrap Server, API Key, and Secret to set up the connection in Tinybird.

A screenshot showing the Tinybird server details being configured in the Tinybird UI
Configuring your Confluent Connection is simple in the Tinybird UI.

Once you’re connected, select your topic and give the consumer group a unique name.

A screenshot showing a Confluent topic being selected in the Tinybird UI
You can select your Confluent topics to ingest into Tinybird Data Sources.

Finally, choose which data you want to ingest and define the table schema of your resulting Data Source.

A screenshot showing a Tinybird Data Source being created from a Confluent topic in the Tinybird UI
You can create a Tinybird Data Source from your Confluent Topics by selecting your data an defining your table schema.

Of course, if you prefer to use the Tinybird CLI, you can use the familiar ``tb connection create kafka`` command and follow the prompts using your Confluent details.

A screenshot showing a Confluent connection being created in the Tinybird CLI
You can use the standard Kafka Connection workflow to connect to Tinybird using the CLI.

Get started today

Still want to learn more? Check out the Confluent Connector Screencast:

If you’re not yet a Tinybird customer, you can learn more and sign up for free (no credit card required) and get started today. The Tinybird Build plan is free forever, with no time limit, but if you need more, use the code ``CONFLUENT`` for $300 off a Pro subscription.

Also, feel free to join the Tinybird Community on Slack and ask us any questions or request any additional features.

And, if you’re keen to learn more about the Confluent Connector, join our Launch Week Release Round-up on Friday, where we’ll demonstrate all the amazing features we’ve launched this week. Sign up to get a calendar invite and be notified when the Release Round-up begins.

Tinybird Launch Week #2 Release Roundup - Click to register
Click here to get a calendar invite to our release round-up.