Amazon S3
From files in Amazon S3 to
low-latency APIs
Use the data you have in S3 to build real-time data products. Automate ingestion and publish low-latency API endpoints with Tinybird.
Trusted in production by engineers at...
No hassle
You have your CSV files in S3 buckets. You can easily Build APIs on top now.
Fast & Smart
We detect new files in the S3 buckets and ingest automatically, up to millions of rows per second.
SQL based
Once ingested, run serverless fast transformations using our Data Pipes.
Secure
Use Auth tokens to control access to API endpoints. Implement access policies as you need. Support for row-level security.
Build in minutes, not weeks
Ingest, query and build APIs for your data at scale in a matter of minutes. Forget about ETLs, performance and complex security rules.
$ tb create connection s3 --key $KEY --secret $SECRET my-s3
** Creating connection...
** my-s3.connection created successfully!
$ cat s3_datasource.datasource
IMPORT_SERVICE s3
IMPORT_CONNECTION_NAME my-s3
IMPORT_BUCKET_URI s3://my_bucket/*.csv
IMPORT_STRATEGY append
IMPORT_SCHEDULE @auto
$ tb push s3_datasource.datasource
** s3_datasource created successfully!
NODE avg_triptime_endpoint
SQL >
SELECT
toDayOfMonth(pickup_datetime) as day,
avg (dateDiff('minute', pickup_datetime, dropoff_datetime)) as avg_trip_time_minutes
FROM s3_datasource
{% if defined(start_date) and defined(end_date) %}
WHERE pickup_dt BETWEEN {{Date(start_date)}} AND {{Date(end_date)}}
{% end %}
GROUP BY day
$ tb push endpoints/avg_triptime.pipe
** Processing avg_triptime.pipe
** Building dependencies
** Creating avg_triptime
** Token read API token not found, creating one
=> Test endpoint with:
$ curl https://api.tinybird.co/v0/pipes/avg_triptime.json?token=<TOKEN>&start_date=2021-01-01&end_date=2021-03-01
** 'avg_triptime' created
Automate ingestion of files from your S3 bucket with our UI, CLI, or REST API.
Filter, shape, or enrich your data using Pipes, an easy way of chaining SQL queries designed to ease development and maintenance.
Share secure access to your data in a click and get full OpenAPI and Postman documentation for your APIs.
We accelerate your data, no matter where it is.
Connect data from Relational Databases, Data Warehouses and Data Streams.
Amazon Redshift
Amazon S3
Google BigQuery
Apache Kafka
PostgreSQL
MySQL
Snowflake
FAQ
Build fast data products, faster.
