daily.dev uses Tinybird to serve your feed in a flash
daily.dev feeds you the news you want to read right now. Keep reading to learn what happens under the hood every time you open a new tab.
How daily.dev creates your feed
.png)
“Whenever you open a new tab or use our feed, just know that we use Tinybird to fetch all the articles.”
What data is used to create your feed?
A few sources of data inform the contents of your feed. There’s some dimensional data (posts and user preferences) and events streams (the interactivity events that get generated when daily.dev users engage with their feed every moment).
daily.dev streams events using Tinybird’s high-frequency ingestion Events API. The endpoint gives them the flexibility to send new streaming data to Tinybird with a simple HTTP POST request.
Likewise, Tinybird ingests data from the daily.dev application server to keep track of the historical inventory of posts and user preferences.
Opening a new tab with the daily.dev extension triggers a request to the Tinybird Feed API Endpoint using your unique user ID as a query parameter.
The Feed API Endpoint is the final node in a series of chained, composable SQL nodes in Tinybird, published as a REST API. When it’s called, it runs a query that joins the current content rankings snapshot to your specific feed configuration: your favorite tags, reading history, blocked sources, and content preferences. It also injects new posts at a certain ratio. The query then returns a personalized ranked list of post IDs that gets packaged as JSON in the request response.
Finally, the Feed Service parses the response and hydrates the list of post IDs with metadata, including titles, authors, sources, and social preview images to display in your browser.
Speed, speed, and more speed
Your personalized feed should appear instantly with each new tab. That means the Feed API created in Tinybird should response in 50 ms or less.
This is a non-trivial problem, especially because the query must join streaming events to dimensions with every new request.
To make that happen, daily.dev uses Materialized Views.
Tinybird’s Materialized Views allow daily.dev to keep rolling rankings of sources, tags, and the performance of individual posts stored in pre-aggregated tables that update incrementally as soon as new data is ingested. Whenever new posts or events hit the analytics service, Tinybird automatically updates these aggregations so the rolling rankings are always fresh.
Every few minutes, Tinybird uses the materializations to produce a new global content inventory, ranked by engagement and age.
Since these Materialized Views do the heavy lifting of filtering and aggregating during ingestion time - and not query time - the Feed API responds with exceptionally low-latency thanks to a vastly reduced scan size.
Serverless for simplicity and scale
The critical measure for the daily.dev team is how quickly they can serve a highly relevant feed. Tinybird is serverless, so it scales even as daily.dev adds new users and content sources every day. The team at daily.dev don’t need to worry about managing infrastructure, instead focusing on building a smarter feed algorithm and a better user experience.
“My favorite feature of Tinybird is that you can materialize data as soon as it gets into the database.”
Building something with Tinybird
