Halving Kaguya’s frontend CPU allocation with edge caching

Kaguya’s Next.js frontend and Phoenix API ran as separate apps on Fly.io. Every now and then, Fly’s concurrency graph for the frontend would spike. Before I added rate limiting, the worst bursts reached roughly 700–1,000 concurrent requests.

That didn’t match with what Plausible was showing. Fly counted everything reaching the application, while Plausible mostly reflected browser activity, so the numbers weren’t going to match exactly. But there were nowhere near enough users active on the site to explain those spikes.

Kaguya had hundreds of thousands of public catalog pages for visual novels, characters, developers, and series. A request for any of them went through the full stack:

Cloudflare
  -> Next.js
  -> Supabase session lookup
  -> Phoenix / Absinthe
  -> PostgreSQL

Most of the page was the same for every visitor, but every request still triggered a Next.js render, a Supabase session lookup, a GraphQL request, and PostgreSQL queries.

Adding rate limits

I started by blocking the obvious offenders.

One ASN had made around 4,500 requests when I first noticed it after a slow-response spike. When I checked again, it was closer to 15,000. A Cloudflare managed challenge blocked 845 requests from that source.

That reduced the immediate pressure, but it wasn’t robust. Another IP or ASN would eventually take its place.

I added an in-memory rate limiter to the Next.js middleware. Normal routes allowed 120 requests per IP per minute, while auth routes allowed 40. The limiter ran before the Supabase session lookup, so requests over the limit were rejected before we spent time on session handling or rendering.

I added another limiter at the start of the Phoenix API pipeline, before authentication or GraphQL execution. This covered clients that called the API directly instead of going through the Next.js app.

While testing it, I found that GraphQL requests made by Next.js during server rendering all arrived under the Fly machine’s IP. Page renders for different users could therefore end up sharing the same rate-limit bucket. I moved the Next.js-to-Phoenix traffic onto Fly’s private network, gave it a separate higher limit, and forwarded the original client IP alongside it. Browser GraphQL requests continued to use the public endpoint.

After deploying the limiters, the 700–1,000 concurrency spikes stopped, but during crawler bursts, the graph would still climb to whatever Fly concurrency ceiling we had configured and stay there for several minutes.

The spikes were under control, but the work per request had not changed. With enough IPs, crawlers could still make us render the same catalog pages over and over.

I had previously spent some time looking at how Letterboxd loaded its pages. Most of the page would come from cache, while likes, ratings, and other user data were fetched separately in the browser.

Cloudflare was already caching Kaguya’s images and Next.js assets, so I started looking at whether we could use the same split for the catalog pages.

Separating catalog and account data

Kaguya had catalog pages for visual novels, characters, developers, and series. Most of those pages showed the same content whether someone was signed in or not.

Visual novel pages had the most user-specific data mixed in. Along with the title, description, images, characters, developers, relations, and ratings, the page also needed the signed-in user’s rating, reading status, shelves, review, likes, and votes.

Before this change, the visual-novel query fetched both together. That made a mostly public page depend on the current session and made the response difficult to reuse at the edge.

I split the query into two parts:

VnPublicBySlug
  -> title, description, images, relationships,
     developers, series, aggregate ratings, and other catalog data

MyVnData
  -> my rating, reading status, review, shelves,
     votes, and the aggregate fields affected by my actions

The bulk of the page could now follow the cached path:

request -> Cloudflare -- HIT --> cached HTML/RSC
                       \
                        MISS -> Next.js -> Phoenix/GraphQL -> PostgreSQL

signed-in browser -> Supabase session -> Apollo -> viewer-specific data

Guests and crawlers needed only the cached page. Signed-in users received the same page, then Apollo loaded their account-specific data in the browser. We used cache-and-network, so state already in Apollo’s cache could appear immediately while a fresh response loaded in the background.

The user query also returned the aggregate values affected by user actions where needed. When someone rated a visual novel, the optimistic update could change both their rating and the displayed aggregate without waiting for the cached page to refresh.

I later used the same approach for reviews. The server rendered the public review data, while a smaller query returned the review IDs liked by the current user. That kept the like interaction optimistic without making the entire review page depend on user data.

Caching the catalog routes

Visual-novel pages changed more often than the other catalog pages because they included ratings, reviews, and other community data. I gave those pages a shorter edge TTL, while character, developer, and series pages could stay cached longer.

Cloudflare cached eligible successful responses. Error responses were excluded, and 429 responses were marked no-store. On a hit, the request ended at Cloudflare instead of going through Next.js, Supabase, Phoenix, and PostgreSQL again.

For changes that shouldn’t have to wait for the cache to expire, I later added targeted invalidation from Phoenix for selected updates, including reviews, tag votes, covers, featured screenshots, and deletions.

Handling cached pages across deployments

Caching pages for days introduced another problem: a cached page could outlive the build that created it.

After a deployment, Cloudflare could still return HTML that referenced JavaScript chunks from the previous build. If the new container no longer had those files, the HTML would load but the page could fail while fetching its scripts.

We already had a simple reload for chunk errors, but I did not want that to be the main recovery path. I kept the previous build’s static chunks in the production image and updated the deploy script to purge cached catalog routes once the new version was live.

I also updated the browser recovery to retry the navigation once with a cache-busting URL. A session flag prevented a persistent error from causing a reload loop.

Keeping the previous chunks covered the gap between a new deployment going live and the old HTML disappearing from Cloudflare.

Back to two CPUs

During the worst crawler spikes, we had doubled the frontend’s CPU allocation from two shared CPUs to four.

The rate limiters removed the 700–1,000-request bursts, but bot traffic could still hold Fly near its concurrency ceiling for several minutes at a time. After the catalog cache went live, the graph stopped flattening at that ceiling and moved much closer to what Plausible showed.

I monitored it for a few days, then reduced the frontend from four shared CPUs back to two.

I no longer have access to the old Cloudflare dashboard, so I cannot give a reliable cache-hit ratio or origin-request reduction. What I can verify is that Fly’s concurrency stabilized and we were able to halve the frontend’s CPU allocation within days of the rollout.