Module iroh_blobs::downloader

Available on crate feature downloader only.

Handle downloading blobs and collections concurrently and from nodes.

The Downloader interacts with four main components to this end.

• Dialer: Used to queue opening connections to nodes we need to perform downloads.
• ProviderMap: Where the downloader obtains information about nodes that could be used to perform a download.
• Store: Where data is stored.

Once a download request is received, the logic is as follows:

1. The ProviderMap is queried for nodes. From these nodes some are selected prioritizing connected nodes with lower number of active requests. If no useful node is connected, or useful connected nodes have no capacity to perform the request, a connection attempt is started using the Dialer.
2. The download is queued for processing at a later time. Downloads are not performed right away. Instead, they are initially delayed to allow the node to obtain the data itself, and to wait for the new connection to be established if necessary.
3. Once a request is ready to be sent after a delay (initial or for a retry), the preferred node is used if available. The request is now considered active.

Concurrency is limited in different ways:

• Total number of active request: This is a way to prevent a self DoS by overwhelming our own bandwidth capacity. This is a best effort heuristic since it doesn’t take into account how much data we are actually requesting or receiving.
• Total number of connected nodes: Peer connections are kept for a longer time than they are strictly needed since it’s likely they will be useful soon again.
• Requests per node: to avoid overwhelming nodes with requests, the number of concurrent requests to a single node is also limited.

Structs

• ConcurrencyLimits
  Concurrency limits for the Downloader.
• DownloadHandle
  Handle to interact with a download request.
• DownloadKind
  The kind of resource to download.
• DownloadRequest
  A download request.
• Downloader
  Handle for the download services.
• IntentId
  Identifier for a download intent.
• RetryConfig
  Configuration for retry behavior of the Downloader.

Enums

• DownloadError
  Error returned when a download could not be completed.
• FailureAction
  Signals what should be done with the request when it fails.
• GetOutput
  Output returned from Getter::get.

Traits

• Dialer
  Trait modeling a dialer. This allows for IO-less testing.
• Getter
  Trait modelling performing a single request over a connection. This allows for IO-less testing.
• NeedsConn
  Trait modelling the intermediary state when a connection is needed to proceed.