Feeds and timelines

feeds gives your app a follow graph and a home timeline that stays fast at any follower count, plus trending counts per region. The feed is chronological by default and has an end.

People follow, block and mute each other, post and read, whether an author has ten followers or ten million. Reach for it for a social app, a community or an activity stream.

The feed is chronological by default. You opt in to ranking, and every ranked entry carries a reason and a way back to chronological. The feed has an end: no endless scroll, and no new items inserted while a person reads.

feeds:
  driver: db
  items: posts
  fanout: { threshold: 10000, batch: 500 }
  timeline: { order: chronological }
  page: { end: true, pause_after: 200 }
  trending: { window: 1h, k: 20, min-count: 25 }

import { feeds } from "mainsdk";

await feeds.publish(user, { id: post.id, visibility: "public" });
const home = await feeds.timeline(user, { limit: 20 });   // ends with a caught-up or pause entry

feeds:
  driver: db

Quick start

feeds:
  driver: db
  items: posts

import { feeds } from "mainsdk";

declare const user: { id: string };
declare const post: { id: string };

await feeds.publish(user, { id: post.id, visibility: "public" });
const home = await feeds.timeline(user, { limit: 20 });   // ends with a caught-up or pause entry

feeds stores references to your records, not copies. items names the db model whose records are the feed items.

Configuration

feeds:
  driver: db                       # db (default) | local for tests
  items: posts                     # the db model whose records are feed items
  fanout:
    threshold: 10000               # authors with more followers are merged at read time
    batch: 500                     # timeline writes per job
  max-following: 5000
  timeline:
    order: chronological           # chronological (default) | ranked
    max: 800                       # entries kept per timeline
    keep: 30d
  page:
    limit: 20                      # default page size; at most 100
    end: true                      # a caught-up marker where the last session ended
    pause_after: 200               # items per session before a pause marker; 50 for children
  ranked:
    candidates: 500                # entries ranked per request
    budget: 50ms                   # ranker time per page; over it, chronological is served
  trending:
    window: 1h
    k: 20
    min-count: 25                  # keys counted for fewer people are hidden; may only be raised
  realtime: true                   # tell viewers how many new items wait

What you get

A follow graph with blocks and mutes, home timelines that fan out on write (merged at read time for large authors), profiles and trending keys. Every call below takes the acting user as its first argument.

Calls

Follow graph

feeds.follow(follower, target)                  // { state: "following" | "requested" }; private accounts get a request
feeds.unfollow(follower, target)
feeds.followers(user, o?)                       // a page of accounts, newest first
feeds.following(user, o?)
feeds.counts(user)                              // { followers, following }
feeds.requests(user, o?)                        // pending follow requests
feeds.accept(user, from)
feeds.decline(user, from)
feeds.relation(a, b)                            // { following, followedBy, requested, blocking, blockedBy, muting }

Following yourself, or following twice, returns the current state and changes nothing. Following more than max-following fails with MAIN-FEED-0005.

Block and mute

feeds.block(user, target)
feeds.unblock(user, target)
feeds.mute(user, target, { until? })            // hides the target from the muter's timeline only
feeds.unmute(user, target)

A block deletes follows and requests both ways, refuses new ones with MAIN-FEED-0001, and hides each side's items from the other. A mute leaves follows alone. Neither tells the other person.

Items and timelines

feeds.publish(author, { id, visibility })       // visibility: "public" | "followers"
feeds.edit(author, id)                          // bumps the version after the item record changed
feeds.remove(by, id, { reason? })               // the author, or a subject that access allows
feeds.timeline(user, { order?, limit?, cursor?, continue? })
feeds.profile(viewer, author, o?)               // one author's items, as the viewer may see them
feeds.ranker(fn)                                // register the ranking function

feeds.publish resolves once the record reference and the fan-out job are accepted, and writes no follower's timeline in the request. An author sees their own new item at the top of their chronological timeline on the next read.

A timeline entry is one of three kinds:

order: "chronological" is always accepted, so a person can leave ranking at any time. For a children's audience the timeline is always chronological and pauses after at most 50 items.

Trending

feeds.count(key, { by })                        // one occurrence of a key such as "tag:launch"
feeds.trending({ window?, k? })                 // { region, window, keys: [{ key, count }] }

feeds.count is synchronous and in memory, and a job flushes it to db each minute. Each person counts once per key per window, using a salted hash that rotates every window. No person id is stored. Keys counted for fewer than min-count people are hidden, and regions are never merged.

Examples

Rank with a reason

import { feeds } from "mainsdk";

declare function affinity(user: { id: string }, item: { id: string }): number;

feeds.ranker((candidates, { user }) =>
  candidates.map((c) => ({ id: c.id, score: affinity(user, c), reason: "Because you follow this author" })),
);

A ranker that throws, omits a reason or passes ranked.budget makes feeds.timeline serve the chronological timeline and log MAIN-FEED-0007.

Show new-item counts

With realtime: true, each fan-out batch publishes the event feeds.new with { count } on user:<id>. It never carries an item. Show "3 new posts" and let the person ask for them.

Errors and checks

Two codes are runtime only: MAIN-FEED-0006 (a failed fan-out batch, served by merging at read time) and MAIN-FEED-0008 (a pushed value that loosens a limit is refused). See check codes.

mainsdk check has a feeds gate for MAIN-FEED-0010 to 0013, each reported with its file and line.

Swapping drivers

feeds.driver is db (the default, on your app's db stores) or local, an in-process SQLite store for tests. Your calls do not change when you change it.

Testing

driver: local keeps the graph, items and timelines in an in-process SQLite store, so tests run offline on the fake clock and the local job queue. See Test your app.

Pushed config

A push may only tighten (ADR-0017): feeds.page.end is never turned off, feeds.page.pause_after and feeds.max-following only go lower, feeds.trending.min-count only goes higher. A looser value is refused with MAIN-FEED-0008, the last good value stays, and the refusal is written to the audit log.

What is connected, and what is not

app.feeds (and import { feeds } from "mainsdk") is built on first use from the feeds section over the app's db SQL stores, tenancy, jobs, access and the log. These modules are not built yet, so set app.socialPorts.feeds = { ... } before the first use if you have your own:

Security and privacy

• A timeline entry holds only the item id, author id and time, and lives in the follower's region. Item records stay in their author's region.
• Follow, block and mute changes are audited without content and visible only to the person who made them.
• privacy.erase removes a person's follows, blocks, mutes, item references, timeline entries and counts.
• A link to an item goes through links, not through feeds.

FAQ

Are likes, replies and reposts built in? No. Model them as records and count them for your ranker.

Can I push people more items while they read? No. A timeline never inserts items while a person reads; use the feeds.new count.

Next

• Build a feed with an end: the steps, end to end.
• Realtime updates
• Jobs and queues
• Moderation