blog post

A Post Gateway World: Transitioning Users to Direct Retrieval with IPFS

Summary

The public-good IPFS gateways at ipfs.io and dweb.link have played a foundational role in IPFS adoption—serving over 614 million requests and 45TB of data to 10 million users daily. However, this centralized infrastructure has become a major obstacle to sustainability and decentralization.

Our traffic analysis identifies three main user categories:

  • A) Browser-based website visitors
  • B) Hot-linking users (web and app-based embeds)
  • C) Backend services and automated clients

We propose a targeted transition plan to help each group move toward decentralized, verifiable content retrieval. This shift is made possible by recent Shipyard innovations:

  • Verified Fetch – a library for apps and hot-linked content
  • Service Worker Gateway – enabling trustless in-browser access
  • Self-hosting tools – for backend services and infrastructure operators

Our analysis shows:

  • 67% of requests come from backend clients — ideal candidates to run their own IPFS infrastructure
  • 23% are hot-links or app embeds — can migrate to @helia/verified-fetch
  • 9% are direct browser users — can transition to inbrowser.link for secure, offline-ready access

This strategy aligns with IPFS’s core vision of resiliency and verifiability through decentralization and offers practical solutions for sustainable content delivery at scale.


Introduction

IPFS public gateways were instrumental in driving early adoption by bridging Web2 and Web3. However, their centralized architecture now presents serious challenges to scalability, security, and protocol evolution.

This article outlines a strategic transition plan grounded in real gateway traffic data. We identify how different user types interact with IPFS content and describe drop-in tools to shift from centralized gateways to decentralized, verifiable alternatives.


The Role of Public Gateways (Today)

The IPFS gateways at ipfs.io and dweb.link allow standard web clients to retrieve IPFS content via HTTP without verifying content-addressed data on their own . While this accessibility has driven adoption, it has also created technical and economic burdens:

  • Scalability limits and rate limiting
  • Single points of failure
  • Delegated trust and lack of verifiability
  • Centralized bottleneck for protocol upgrades

To make IPFS sustainable and scalable, we must move beyond public-good gateways.


Understanding the Traffic

From May 12–14, 2025, we analysed gateway traffic, filtering out known abuse via Cloudflare and Badbits. Although imperfect, this data gives a representative view of usage patterns across three user categories:

A) Browser Visitors

Direct visitors loading IPFS-hosted static sites or dApps via ipfs.io or dweb.link. These appear as modern browser user agents without cross-origin headers.

B) Hot-linking / App Embeds

Cross-origin media, JSON, or resource fetches embedded in websites or mobile/desktop apps. These use IPFS gateways as a CDN for partial content rather than full pages.

C) Automated Clients / Services

Backend scripts, crawlers, proxy services, and other headless HTTP clients. These should ideally run their own IPFS nodes or gateways.


Traffic Breakdown by Category

Bandwidth Share

Gateway Traffic Categories by Bandwidth

  • A: Browser Visitors — 9.2%
  • B: Hot-linking / Apps — 21.3%
  • C: Automated Clients — 69.5%

Request Share

Gateway Traffic Categories by Requests

  • A: Browser Visitors — 9.4%
  • B: Hot-linking / Apps — 23.1%
  • C: Automated Clients — 67.4%

The takeaway: the vast majority of traffic comes from clients that could—and should—be using native tooling to fetch directly from the network. Only about one-third is web browser–based. Still, it was important to develop tools to support browser users, so we had a clear migration path for that segment before moving into the next phase of the transition roadmap. Without a viable upgrade strategy for browser users, it would be difficult to justify limiting automated clients that can easily masquerade as browsers. By establishing a migration path for real browser users, we’ve effectively closed off this loophole as a way to disguise bogus traffic going forward.


Content & Client Patterns

  • Images (PNG, JPEG, GIF) account for 63% of bandwidth
  • JSON dominates 57% of all requests — mostly metadata lookups
  • Top user agents include:
    • Go-http-client (33.3% bandwidth / 27.2% requests)
    • go-resty, node-fetch, Lavf, etc.
  • Browser UAs represent ~25% bandwidth but only ~15% of requests

Key Insight

Most IPFS gateway usage today stems from backend services treating ipfs.io as a free CDN—undermining decentralization and harming performance for legitimate browser users. And some of the largest share of user agents we see are from some of our best supported languages, Go and NodeJS, and as such should be easy candidates for migration to native retrieval.


Migration Strategy

We propose a phased transition, offering targeted tools for each user type:

Redirect organic users to the Service Worker Gateway at https://inbrowser.link. This maintains the same user experience while enabling local verification, offline access, and better security.

Encourage apps that hotlink images or JSON to switch to Drop-In Service Worker . Or for those looking for more verification and resiliency, there’s Verified Fetch which offers trustless content retrieval and automatic fallback to multiple providers—drop-in compatible with fetch().

C) Backend Services → Self-Hosted Gateways

For backend and automated clients, we recommend running dedicated IPFS nodes or gateways using tools like Rainbow and Someguy . These services shouldn’t rely on the public-good gateway at all.


Tooling Overview

✅ Verified Fetch Library

  • Drop-in replacement for the fetch() Web API
  • Verifies content hashes locally
  • Supports ipfs://, ipns://, and https://
  • Auto-fallback to trustless gateways
  • Drop-in for dweb.link subdomain gateway
  • Runs in-browser with strict origin isolation
  • Supports directory views, range requests, and full UnixFS features

✅ Delegated Routing

  • Vendor-agnostic HTTP API Helps browsers find content providers
  • Filters for WSS/HTTPS compatibility
  • Backed by delegated-ipfs.dev (or self-hosted)

✅ Self-Hosting Tools

  • Run your own IPFS HTTP gateway (Rainbow)
  • Self-host delegated routing (Someguy)
  • Automate deploys via ipfs-deploy-action

Transition Roadmap

Phase 1: Infrastructure (Complete)

Tools like Verified Fetch, Service Worker Gateway, and delegated routing are live and ready.

Phase 2: Outreach (Active)

We’re sharing migration guides, offering support, and reaching out to top gateway users across all categories.

Phase 3: Migration & Rate-Limiting

  • Browser traffic → redirect to inbrowser.link
  • Hot-linked and backend traffic → begin rate-limiting on ipfs.io and dweb.link
  • 429 responses will link to relevant migration docs
  • More to come with details and timelines

Looking Ahead

This transition is about more than infrastructure—it’s about returning IPFS to its decentralized roots.

By decentralizing retrieval, we improve:

  • 🔐 Security through trustless verification
  • 🌍 Resilience via redundant, independent infrastructure
  • 📈 Sustainability by reducing reliance on centralised gateways

We’re building a more robust IPFS, one that scales with its users instead of against them.


Need Help?

We’re actively monitoring issues and providing support across our tool chain:

You can also find migration guides and community support at discuss.ipfs.tech .

If you’re confused about how to approach this migration, or how this will affect you, please get in touch with us. Either via the IPFS discussion forums, GitHub, Slack, or other mediums, as outlined on https://ipfs.tech/community/ .

Related Articles

Keep Web3 Online

Our free IPFS tools and integrations have over 75 million monthly active users around the globe.

Help Fund Shipyard