🚀
Introducing Versions: Develop data products using Git. Join the waitlist

Use AWS SNS to send data to Tinybird

SNS is a popular pub/sub messaging system for AWS users. Here's how to use SNS to send data to Tinybird.
David Manzanares
Software Engineer
Jul 12, 2022
 ・ 
  min read

If you operate on AWS infrastructure, AWS’s Simple Notification Service (SNS) provides a simple managed service for pub/sub messaging to various AWS and external services.

Given that a lot of Tinybird customers operate on AWS, SNS tends to be one of our most requested data origins. It provides an easy-to-use path to distribute messages to multiple destinations, allows for message filtering, and has built in data-loss prevention mechanisms. Because it’s a tool our customers regularly use and like, we’ve prioritized making it possible to connect SNS to Tinybird.

And now, we are happy to say that we can support data ingestion from SNS! We’ve conditionally rolled it out for all customers and are opening up the feature to feedback. If you have any, please share it with us in our community Slack.

How to use SNS to send data to Tinybird

Here’s how to use AWS SNS to send data to Tinybird:

Step 1: Use an existing SNS topic or create a new one

From your AWS console, navigate to an existing SNS topic or create a new one.

A webpage showing how to create an SNS topic to send data to Tinybird
Create an SNS topic to send data to Tinybird.

Step 2: Create a new subscription for the topic

Define a new subscription using the Tinybird {%code-line%}/events{%code-line-end%} API endpoint as the subscription Endpoint using the following syntax:

{%code-line%}https://api.tinybird.co/v0/events?name=YOUR_DATA_SOURCE_NAME&token=YOUR_TOKEN{%code-line-end%}

Make sure to select HTTPS as the protocol and check “Enable Raw Message Delivery”, then click “Create Subscription.”

A webpage showing how to create a subscription from AWS SNS to the Tinybird /events endpoint
Creating a subscription to an SNS topic using the Tinybird /events endpoint.

You should see a confirmation screen like the one below.

AWS SNS confirmation screen
Success!

Step 3. Start sending messages to your SNS topic

You can manually publish messages to your new SNS topic from the topic page by clicking “Publish Message”

A webpage showing how to publish a message to an SNS topic.
Publish a message to your SNS topic to test the connection.

Just define the JSON payload to be sent in the Message Body section

A webpage to publish a message to an SNS topic showing the JSON message structure
Use JSON structure to send data to /events.

You should see data appear in your Tinybird data source.


A Tinybird data source
Boom. Data in Tinybird!

Here’s a bit of JavaScript showing how to publish a new message to the SNS topic. Just replace your AWS region and Topic ARN in the code.

As always, if you have any questions or run into issues, you can join our Slack community.