Merge pull request #476 from ipfs/feat/add-get-closest-peers-to-http-routing

IPIP-0476: Delegated Routing DHT Closest Peers API

Author: lidel

src/ipips/ipip-0476.md

@@ -0,0 +1,151 @@
+---
+title: "IPIP-0476: Delegated Routing DHT Closest Peers API"
+date: 2025-11-20
+ipip: ratified
+editors:
+  - name: Alex Potsides
+    github: achingbrain
+    affiliation:
+      name: Shipyard
+      url: https://ipshipyard.com
+  - name: Marcin Rataj
+    github: lidel
+    affiliation:
+      name: Shipyard
+      url: https://ipshipyard.com
+relatedIssues:
+  - https://github.com/ipfs/specs/pull/476
+  - https://github.com/ipfs/specs/pull/497
+order: 476
+tags: ['ipips']
+---
+
+## Summary
+
+Add a new HTTP endpoint to the Delegated Routing API that allows clients to find the closest peers to a given peer ID without being full DHT clients.
+
+## Motivation
+
+Browser nodes and other resource-constrained clients need to perform peer discovery operations without the overhead of being full DHT clients. The primary use case is for browser nodes performing random walks to find peers that they can make circuit relay reservations on.
+
+Currently, to find peers close to a particular key in the DHT keyspace, a node must:
+1. Be a full DHT client with all the associated overhead
+2. Maintain connections to many peers
+3. Handle the complexity of the DHT protocol
+
+This is particularly problematic for:
+- Browser nodes that need to find circuit relay servers
+- Light clients that want to populate their routing tables
+- Applications that need to find peers to host provider records
+
+## Detailed design
+
+This IPIP introduces a new "DHT Routing API" section to the Delegated Routing V1 HTTP API specification with the following endpoint:
+
+### `GET /routing/v1/dht/closest/peers/{key}`
+
+#### Path Parameters
+
+- `key` is a [CID](https://github.com/multiformats/cid) or Peer ID to find the closest peers to.
+  - CID SHOULD be a CIDv1 in any encoding.
+  - Peer ID can be represented as a Multihash in Base58btc, or a CIDv1 with `libp2p-key` (`0x72`) codec in Base36 or Base32.
+  - Arbitrary multihash lookups can be performed by wrapping the multihash in a CIDv1 with `raw` (`0x55`) codec.
+
+#### Response
+
+The response follows the same format as the existing peer routing endpoints, returning a JSON object with a `Peers` array containing peer records conforming to the Peer Schema.
+
+### Specification Changes
+
+The following changes are made to `src/routing/http-routing-v1.md`:
+
+1. Add a new "## DHT Routing API" section after the "Peer Routing API" section
+2. Document the `/routing/v1/dht/closest/peers/{key}` endpoint with its parameters and response format
+3. Update the document date to reflect the modification
+
+## Design rationale
+
+The design follows several key principles:
+
+### Simple MVP Approach
+
+The endpoint provides the minimum viable functionality needed for the primary use cases. It can be expanded later if more complex routing scenarios emerge.
+
+### Future-Proofed Path Structure
+
+The path `/routing/v1/dht/closest/peers/{key}` is intentionally structured to allow future expansion:
+- The `/dht/` segment clearly indicates DHT-specific operations
+- The `/closest/peers/` structure allows for potential future endpoints like `/closest/keys/` for keyspace queries
+- This organization keeps the API logical and extensible
+
+### Routing-Agnostic Implementation
+
+While the endpoint is in the DHT namespace, implementations have flexibility in how they determine "closest" peers. This allows for optimization based on the specific routing system being used.
+
+### Consistency with Existing API
+
+The endpoint follows the established patterns of the Delegated Routing API:
+- Uses the same response format as other peer routing endpoints
+- Follows the same parameter conventions
+- Maintains consistency in error handling and status codes
+
+### User benefit
+
+This enhancement provides significant benefits to end users:
+
+1. **Browser Compatibility**: Browser nodes can discover circuit relay servers without implementing the full DHT protocol
+2. **Reduced Resource Usage**: Light clients save bandwidth and processing power by delegating peer discovery
+3. **Faster Peer Discovery**: Delegated routing servers can provide cached results more quickly than performing DHT walks
+4. **Simplified Implementation**: Application developers can implement peer discovery with simple HTTP requests instead of complex DHT logic
+
+### Compatibility
+
+This change is fully backward compatible:
+- It adds a new endpoint without modifying existing ones
+- Existing clients continue to work unchanged
+- Servers that don't implement the endpoint return 501 (Not Implemented) as per the specification
+
+### Security
+
+The new endpoint introduces no additional security considerations beyond those already present in the Delegated Routing API:
+
+- Standard rate limiting should be applied to prevent abuse
+- The endpoint reveals no more information than a DHT query would
+- Access controls can be implemented at the HTTP layer if needed
+- Response caching helps mitigate potential DoS attacks
+
+### Alternatives
+
+Several alternatives were considered:
+
+1. **Full DHT Client Implementation**: Rejected due to excessive resource requirements for browser and mobile environments
+
+2. **Custom libp2p Protocol**: Would require all nodes to implement a new protocol, creating adoption barriers
+
+3. **Extension of Existing Peer Routing**: The `/routing/v1/peers/` endpoint serves a different purpose (finding specific peers rather than closest peers)
+
+4. **Amino-Specific Endpoint**: Initially considered `/routing/v1/amino/` namespace but rejected in favor of the more generic `/dht/` approach
+
+5. **Query Parameters for Filtering and Limiting**: During design discussions, `count` and `closer-than` query parameters were considered:
+   - `count`: limit the number of results returned
+   - `closer-than`: filter peers to only return those closer than a reference peer
+
+   These were omitted from the initial implementation (boxo v0.35.2, someguy v0.11.0) for simplicity. The endpoint defaults to returning up to the DHT bucket size (20 for Amino DHT). Future backwards-compatible updates MAY add these parameters if use cases emerge that require more fine-grained control.
+
+### Peer Ordering
+
+Implementations SHOULD return peers sorted by closeness to the key. For Kademlia-based DHT implementations (such as Amino DHT), this means sorting by XOR distance with the closest peers first.
+
+## Test fixtures
+
+This IPIP does not deal with content-addressed data, so specific test CIDs are not applicable. However, implementations should test:
+
+1. Valid CID and Peer ID inputs return appropriate closest peers
+2. Results are limited to the DHT bucket size
+3. Peers are sorted by closeness (XOR distance for Kademlia DHTs)
+4. Invalid keys return appropriate error responses
+5. Both streaming (NDJSON) and non-streaming (JSON) response formats work correctly
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

src/routing/http-routing-v1.md

@@ -4,9 +4,28 @@ description: >
   Delegated routing is a mechanism for IPFS implementations to use for offloading
   content routing, peer routing and naming to another process/server. This specification describes
   an HTTP API for delegated routing of content, peers, and IPNS.
-date: 2024-10-29
+date: 2025-11-20
 maturity: reliable
 editors:
+  - name: Marcin Rataj
+    github: lidel
+    url: https://lidel.org/
+    affiliation:
+      name: Shipyard
+      url: https://ipshipyard.com
+former_editors:
+  - name: Henrique Dias
+    url: https://hacdias.com/
+    github: hacdias
+    affiliation:
+      name: Shipyard
+      url: https://ipshipyard.com
+  - name: Daniel Norman
+    github: 2color
+    affiliation:
+      name: Shipyard
+      url: https://ipshipyard.com
+thanks:
   - name: Gus Eggert
     github: guseggert
     affiliation:
@@ -17,20 +36,15 @@ editors:
     affiliation:
       name: Protocol Labs
       url: https://protocol.ai/
-  - name: Henrique Dias
-    url: https://hacdias.com/
-    github: hacdias
+  - name: Alex Potsides
+    github: achingbrain
     affiliation:
       name: Shipyard
       url: https://ipshipyard.com
-  - name: Marcin Rataj
-    github: lidel
-    url: https://lidel.org/
-    affiliation:
-      name: Shipyard
-      url: https://ipshipyard.com
-  - name: Daniel Norman
-    github: 2color
+  - name: Will Scott
+    github: willscott
+  - name: Hector Sanjuan
+    github: hsanjuan
     affiliation:
       name: Shipyard
       url: https://ipshipyard.com
@@ -245,6 +259,65 @@ The content body must be a [`application/vnd.ipfs.ipns-record`][application/vnd.
 - `400` (Bad Request): the provided :ref[IPNS Record] or :ref[IPNS Name] are not valid.
 - `406` (Not Acceptable): submitted content type is not supported. Error message returned in body should inform the user to retry with `Content-Type: application/vnd.ipfs.ipns-record`.
 
+## DHT Routing API
+
+The DHT Routing API is OPTIONAL. Implementations that do not support DHT operations MAY return `404` (Not Found) or `501` (Not Implemented) as specified in [Error Codes](#error-codes).
+
+### `GET /routing/v1/dht/closest/peers/{key}`
+
+This optional endpoint allows light clients to lower the cost of DHT walks in browser contexts.
+
+#### Path Parameters
+
+- `key` is a [CID] or [Peer ID][peer-id-representation] to find the closest peers to.
+  - [CID] SHOULD be a CIDv1 in any encoding.
+  - [Peer ID][peer-id-representation] can be represented as a Multihash in Base58btc, or a CIDv1 with `libp2p-key` (`0x72`) codec in Base36 or Base32.
+  - Arbitrary multihash lookups can be performed by wrapping the multihash in a CIDv1 with `raw` (`0x55`) codec.
+  - Implementations SHOULD support both CID and Peer ID formats for maximum interoperability.
+
+#### Response Status Codes
+
+- `200` (OK): the response body contains peer records.
+- `404` (Not Found): must be returned if no matching records are found.
+- `422` (Unprocessable Entity): request does not conform to schema or semantic constraints.
+- `501` (Not Implemented): may be returned if DHT operations are not supported.
+
+#### Response Headers
+
+- `Content-Type`: the content type of this response, which MUST be `application/json` or `application/x-ndjson` (see [streaming](#streaming)).
+- `Last-Modified`: an HTTP-date timestamp ([RFC9110, Section 5.6.7](https://www.rfc-editor.org/rfc/rfc9110#section-5.6.7)) of the resolution, allowing HTTP proxies and CDNs to support inexpensive update checks via `If-Modified-Since`
+- `Cache-Control: public, max-age={ttl}, public, stale-while-revalidate={max-ttl}, stale-if-error={max-ttl}`: meaningful cache TTL returned with the response.
+  - When present, `ttl` SHOULD be shorter for responses whose resolution ended in no results (e.g. 15 seconds),
+    and longer for responses that have results (e.g. 5 minutes).
+  - Implementations SHOULD include `max-ttl`, set to the maximum cache window of the underlying routing system.
+    For example, if Amino DHT results are returned, `stale-while-revalidate` SHOULD be set to `172800` (48h, which at the time of writing this specification, is the provider record expiration window).
+- `Vary: Accept`: allows intermediate caches to play nicely with the different possible content types.
+
+#### Response Body
+
+```json
+{
+  "Peers": [
+    {
+      "Schema": "<schema>",
+      "Protocols": ["<protocol-a>", "<protocol-b>", ...],
+      "ID": "bafz...",
+      "Addrs": ["/ip4/..."],
+      ...
+    },
+    ...
+  ]
+}
+```
+
+The number of peer records in the response SHOULD be limited to the DHT bucket size (20 for Amino DHT).
+
+Peers SHOULD be returned sorted by closeness to the key. For Kademlia-based DHT implementations (such as Amino DHT), this means sorting by XOR distance with the closest peers first.
+
+The client SHOULD be able to make a request with `Accept: application/x-ndjson` and get a [stream](#streaming) with results. Note that due to the XOR sorting requirement, the streamed response may be blocked until the DHT lookup completes and peers can be sorted before transmission.
+
+Each object in the `Peers` list is a record conforming to the [Peer Schema](#peer-schema).
+
 ## Pagination
 
 This API does not support pagination, but optional pagination can be added in a backwards-compatible spec update.