IPFS for pioneers - Enable building interoperable IPFS systems using HTTP protocols
🐎

IPFS for pioneers - Enable building interoperable IPFS systems using HTTP protocols

What is this?

There is a very minimal HTTP-based set of protocols and requirements for building IPFS implementations that can be interoperable across most common ecosystem implementations.

As a north star, the amount of effort to build compatible IPFS tooling in something like Python, .NET, etc. (that doesn’t have much IPFS or libp2p tooling today) should be extremely minimal.

This means not just demonstrating that this can be done as an example, but working with the ecosystem on making support widespread enough that this can become something they can rely on for compatibility (e.g., it seems simpler for a developer looking to build tooling that works with other IPFS tools today to build via block requests over HTTP vs block requests over Bitswap since they might not yet have even a minimal libp2p implementation available).

Why is this a good idea?

  • Makes it easier for people to build new implementations in new environments.
  • By making the base interop layer small (and gradually expanding), it’s much easier to build something compatible so that developers can get onto building out the more unique capabilities of their implementation.
  • Investing in some of our existing HTTP-based protocols (e.g., Trustless Gateway HTTP API) gives us some leverage in that by building in support we can end up being able to solve multiple problems at once
    • Combined with a CAR format supporting large blocks, we automatically get a data transfer protocol for moving around DAGs with large blocks. (See
      👋
      Welcome pre-existing content addressed data via supporting large blocks
       for large blocks plans.)
    • Combined with work in Helia, we enable innovation around how to express non-UnixFS data in IPFS URIs. HTTP-based requests make it easy to piggyback on tooling like the browser fetch API to iterate on proposals for non-UnixFS URIs in ways that can be deployed permissionlessly in a browser. This means progress can be made here even without agreement from maintainers of important infrastructure like ipfs.io or heavily used applications like Kubo.
  • This has the added benefit of becoming a practical hands-on teaching example about IPFS implementations. With this core minimal nugget of an implementation defined (likely covered in a “we wrote an IPFS block server in Python in X days” blog post), we can branch to more advanced “lessons” (e.g., incremental verifiability, p2p connectivity, content routing).

What we need to do

  • (This assumes
    🔁
    Specifications for end-to-end HTTP retrieval and deserialization of popular content-addressed data
     .)
  • First class HTTP data transfer support in some common implementations (e.g., Kubo and Helia).
    • Should be able to serve data using Trustless Gateway via HTTP over libp2p in at least Kubo
    • Should be able to retrieve data using the Trustless Gateway via HTTP or HTTP over libp2p in Kubo + Helia
      • Similar to when we did this in Rhea, we’ll start with block requests and then move towards DAG requests
  • Deploying a public HTTP /routing/v1 endpoint (e.g., routing.delegate.ipfs.io which could be backed by cid.contact) that people can use to substitute for how the combination of the IPFS Public DHT and IPNI are used today.

Fun: why call this “IPFS for pioneers”?

  • Pioneers head into new frontiers, and we believe this core minimal nugget of an implementation expands the horizons for people.
  • Pioneers are often seeking more degrees of freedom or a fresh start.
  • Pioneers pack light (“one wagon load to cross the Rockies”). These HTTP-based implementations have the essentials for going the distance, but nothing more.
  • Pioneers create paths. They might start small but the initial trail blazes become paths, get grooves, expand to railways, etc.