Aggregation has to happen on the migrator's own machine today. Content is downloaded, packed into an aggregate CAR, and uploaded, so every migrator pays local disk plus bandwidth in both directions.
The paths that avoid local disk all require a public HTTPS origin the provider can pull from: redirect-serve behind a tunnel, or --source-relay. Most consumers cannot provide one. CGNAT, corporate networks, and no-VPS users are simply excluded, and for the ipfs2filecoin campaign that is the largest single drop-off point.
PoDSI in Curio would remove the need for client-side aggregation, but we decided not to block on it and to aggregate client-side for now.
Worth investigating: a service that accepts a CID list, fetches from public gateways, builds aggregate CARs, and serves them at /piece/{pieceCidV2} for a provider to pull. That removes the local-disk requirement and the public-ingress requirement together.
Open questions:
- Who operates and funds it. The existing
packages/relay Worker runs on a personal free-tier Cloudflare account and will not carry campaign traffic.
- Whether the trust model is acceptable. Bytes flow through a third party, though the provider still verifies CommP against the PieceCID the client committed, so a tampering relay fails the add rather than corrupting data.
- Rate limiting and abuse, given there is no registration step.
Aggregation has to happen on the migrator's own machine today. Content is downloaded, packed into an aggregate CAR, and uploaded, so every migrator pays local disk plus bandwidth in both directions.
The paths that avoid local disk all require a public HTTPS origin the provider can pull from:
redirect-servebehind a tunnel, or--source-relay. Most consumers cannot provide one. CGNAT, corporate networks, and no-VPS users are simply excluded, and for the ipfs2filecoin campaign that is the largest single drop-off point.PoDSI in Curio would remove the need for client-side aggregation, but we decided not to block on it and to aggregate client-side for now.
Worth investigating: a service that accepts a CID list, fetches from public gateways, builds aggregate CARs, and serves them at
/piece/{pieceCidV2}for a provider to pull. That removes the local-disk requirement and the public-ingress requirement together.Open questions:
packages/relayWorker runs on a personal free-tier Cloudflare account and will not carry campaign traffic.