By default the Boost daemon repository is located at ~/.boost
It contains the following files:
api The local multi-address of Boost's libp2p API
boost.db The sqlite database with all deal metadata
boost.logs.db The sqlite database with the logs for deals
config.toml The config file with all Boost's settings
repo.lock A lock file created when Boost is running
storage.json Deprecated (needed by legacy markets)
token The token used when calling Boost's JSON RPC endpoints
It has the following directories:
dagstore Contains indexes of CAR files stored with Boost
datastore Contains metadata about deals for legacy markets
deal-staging The directory used by legacy markets for incoming data transfers
incoming The directory used by Boost for incoming data transfers
journal Contains journal events (used by legacy markets)
keystore Contains the secret keys used by libp2p (eg the peer ID)
kvlog Used by legacy markets datastore
Boost supports the same libp2p protocols as legacy markets, and adds new versions of the protocols used to propose a storage deal and to check the deal's status.
The client makes a deal proposal over v1.2.0 or v1.2.1 of the Propose Storage Deal Protocol: /fil/storage/mk/1.2.0 or /fil/storage/mk/1.2.1
It is a request / response protocol, where the request and response are CBOR-marshalled.
There are two new fields in the Request of v1.2.1 of the protocol, described in the table below.
| Field | Type | Description |
|---|---|---|
The client requests the status of a deal over v1.2.0 of the Storage Deal Status Protocol: /fil/storage/status/1.2.0
It is a request / response protocol, where the request and response are CBOR-marshalled.
| Field | Type | Description |
|---|---|---|
| Field | Type | Description |
|---|---|---|
| Field | Type | Description |
|---|---|---|
DealUUID
uuid
A uuid for the deal specified by the client
IsOffline
boolean
Indicates whether the deal is online or offline
ClientDealProposal
ClientDealProposal
Same as <v1 proposal>.DealProposal
DealDataRoot
cid
The root cid of the CAR file. Same as <v1 proposal>.Piece.Root
Transfer.Type
string
eg "http"
Transfer.ClientID
string
Any id the client wants (useful for matching logs between client and server)
Transfer.Params
byte array
Interpreted according to Type. eg for "http" Transfer.Params contains the http headers as JSON
Transfer.Size
integer
The size of the data that is sent across the network
SkipIPNIAnnounce (v1.2.1)
boolean
Whether the provider should announce the deal to IPNI or not (default: false)
RemoveUnsealedCopy (v1.2.1)
boolean
Whether the provider should keep an unsealed copy of the deal (default: false)
Accepted
boolean
Indicates whether the deal proposal was accepted
Message
string
A message about why the deal proposal was rejected
DealUUID
uuid
The uuid of the deal
Signature
A signature over the uuid with the client's wallet
DealUUID
uuid
The uuid of the deal
Error
string
Non-empty if there's an error getting the deal status
IsOffline
boolean
Indicates whether the deal is online or offline
TransferSize
integer
The total size of the transfer in bytes
NBytesReceived
integer
The number of bytes that have been downloaded
DealStatus.Error
string
Non-empty if the deal has failed
DealStatus.Status
string
The checkpoint that the deal has reached
DealStatus.Proposal
DealProposal
SignedProposalCid
cid
cid of the client deal proposal + signature
PublishCid
cid
The cid of the publish message, if the deal has been published
ChainDealID
integer
The ID of the deal on chain, if it's been published
Boost stores metadata about deals in a sqlite database in the root directory of the Boost repo.
To open the database use a sqlite client:
sqlite3 boost.db
The database tables are
Deals metadata about Boost storage deals (eg deal proposal) and their current state (eg checkpoint)
FundsLogs log of each change in funds reserved for a deal
FundsTagged how much FIL is tagged for deal collateral and publish message for a deal
StorageLogs log of each change in storage reserved for a deal
StorageTagged how much storage is tagged for a deal
Boost keeps a separate database just for deal logs, so as to make it easier to manage log data separately from deal metadata. The logs database is named boost.logs.db and it has a single table DealLogs that stores logs for each deal, indexed by uuid.
Boost uses goose (https://pressly.github.io/goose/) tool and library for handling sqlite3 migrations.
goose can be installed following the instructions at https://pressly.github.io/goose/installation/
Migrations in Boost are stored in the /db/migrations directory.
Boost handles database migrations on start-up. If a user is running an older version of Boost, migrations up to the latest version are automatically applied on start-up.
Developers can use goose to inspect and apply migrations using the CLI:
The boostd executable runs as a daemon alongside a lotus node and lotus miner. This daemon replaces the current markets subsystem in the lotus miner. The boost daemon exposes a libp2p interface for storage and retrieval deals. It performs on-chain operations by making API calls to the lotus node. The daemon hands off downloaded data to the lotus miner for sealing via API calls to the lotus miner.
boostd has a web interface for fund management and deal monitoring. The web interface is a react app that consumes a graphql interface exposed by the daemon.
The typical flow for a Storage Deal is:
The Client puts funds in escrow with the Storage Market Actor on chain.
The Client uploads a CAR file to a web server.
The Client sends a storage deal proposal to Boost with the URL of the CAR file.
Boost checks that the client has enough funds in escrow to pay for storing the file.
Boost accepts the storage deal proposal.
Boost downloads the CAR file from the web server.
Boost publishes the deal on chain.
The client checks that the deal was successfully published on chain.
Boost exposes a libp2p interface to listen for storage deal proposals from clients. This is similar to the libp2p interface exposed by the lotus market subsystem.
Boost communicates with the lotus node over its JSON-RPC API for on-chain operations like checking client funds and publishing the deal.
Once the deal has been published, Boost hands off the downloaded file to lotus-miner for sealing.
The DAG store manages a copy of unsealed deal data stored as CAR files. It maintains indexes over the CAR files to facilitate efficient querying of multihashes.
By default, the dagstore root will be:
$BOOST_PATH/dagstore
The directory structure is as follows:
index: holds the shard indices.
transients: holds temporary shard data (unsealed pieces) while they're being indexed.
datastore: records shard state and metadata so it can survive restarts.
.shard-registration-complete: marker file that signals that initial migration for legacy markets deals is complete.
.boost-shard-registration-complete: marker file that signals that initial migration for boost deals is complete.
When you first start your boost process without a dagstore repo, a migration process will register all shards for both legacy and Boost deals in lazy initialization mode. As deals come in, shards are fetched and initialized just in time to serve the retrieval.
For legacy deals, you can monitor the progress of the migration in your log output, by grepping for the keyword migrator. Here's example output. Notice the first line, which specifies how many deals will be evaluated (this number includes failed deals that never went on chain, and therefore will not be migrated), and the last lines (which communicate that migration completed successfully):
For Boost deals, you can do the same by grepping for the keyword boost-migrator.
Forcing bulk initialization will become important in the near future, when miners begin publishing indices to the network to advertise content they have, and new retrieval features become available (e.g. automatic shard routing).
Initialization places IO workload on your storage system. You can stop/start this command at your wish/convenience as proving deadlines approach and elapse, to avoid IOPS starvation or competition with window PoSt.
To stop a bulk initialization(see the next paragraph), press Control-C. Shards being initialized at that time will continue in the background, but no more initializations will be performed. The next time you run the command, it will resume from where it left off.
You can force bulk initialization using the boostd dagstore initialize-all command. This command will force initialization of every shard that is still in ShardStateNew state for both legacy and Boost deals. To control the operation:
You must set a concurrency level through the --concurrency=N flag.
A value of 0 will disable throttling and all shards will be initialized at once. ⚠️ Use with caution!
By default, only unsealed pieces will be indexed to avoid forcing unsealing jobs. To index also sealed pieces, use the --include-sealed flag.
In our test environments, we found the migration to proceed at a rate of 400-500 shards/deals per second, on the following hardware specs: AMD Ryzen Threadripper 3970X, 256GB DDR4 3200 RAM, Samsung 970 EVO 2TB SSD, RTX3080 10GB GPU.
The DAG store can be configured through the config.toml file of the node that runs the boost subsystem. Refer to the [DAGStore] section. Boost ships with sane defaults:
Shards can error for various reasons, e.g. if the storage system cannot serve the unsealed CAR for a deal/shard, if the shard index is accidentally deleted, etc.
Boost will automatically try to recover failed shards by triggering a recovery once.
You can view failed shards by using the boostd dagstore list-shards command, and optionally grepping for ShardStateErrored.
The boostd executable contains a dagstore command with several useful subcommands:
boostd dagstore list-shards
boostd dagstore initialize-shard <key>
boostd dagstore initialize-all --concurrency=10
boostd dagstore gc
Refer to the --help texts for more information.
