New Boost Setup

How setup a new Boost instance for a new lotus-miner

If you are already running a Boost node then please follow the migration tutorial and do not attempt the below steps. This can result in permanent data loss.

Make sure you have read the Components page before proceeding. Boost v2 introduces a new service called boostd-data which requires a database to be installed - YugabyteDB or LevelDB.

Introduction

Boost v2 introduces the Local Index Directory as a replacement for the DAG store. It scales horizontally and provides a more intuitive experience for users, by surfacing problems in the UI and providing repair functionality.

Prerequisites

Install YugabyteDB

The Local Index Directory (LID) stores retrieval indices in YugabyteDB. Retrieval indices store the size and location of each block in the deal data.

It is recommend to run YugabyteDB on a dedicated host with SSD drives. Depending on how many blocks there are in the user data, the retrieval indices may require up to 2% of the size of the unsealed data. e.g. 1 TiB of unsealed user data may require a 20 GiB index.

YugabyteDB requires about the same amount of space as your DAG store requires today.

You can find more information about YugabyteDB in the Components section:

pageYugabyteDB

Instructions

Follow these instructions in order to setup a new Boost instance with boostd-data service:

1. Make sure you have a Lotus node and miner running

2. Create and send funds to two new wallets on the lotus node to be used for Boost. Boost currently uses two wallets for storage deals:

• The publish storage deals wallet - This wallet pays the gas cost when Boost sends the PublishStorageDeals message.

• The deal collateral wallet - When the Storage Provider accepts a deal, they must put collateral for the deal into escrow. Boost moves funds from this wallet into escrow with the StorageMarketActor.

export PUBLISH_STORAGE_DEALS_WALLET=`lotus wallet new bls`
export COLLAT_WALLET=`lotus wallet new bls`
lotus send --from mywallet $PUBLISH_STORAGE_DEALS_WALLET 10
lotus send --from mywallet $COLLAT_WALLET 10

3. Set the publish storage deals wallet as a control wallet

lotus-miner actor control set --really-do-it $PUBLISH_STORAGE_DEALS_WALLET

4. Build the latest Boost V2 binaries

5. Run the boostd-data service

boostd-data is a data proxy service which abstracts the access to LID through an established API. It makes it easier to secure the underlying database and not expose it. boostd-data listens to a websocket interface, which is the entrypoint which should be exposed to boostd, andbooster-http

Start the boostd-data service with parameters to connect to YugabyteDB on its Cassandra and PostgreSQL interfaces:

./boostd-data run yugabyte \
  --hosts <yugabytedb-hosts> \
  --connect-string="postgresql://<username>:<password>@<yugabytedb>:5433" \
  --addr 0.0.0.0:8044

The boostd-data service can run a separate machine from Boost as long as the service is reachable via the network from the boost node

6. Create and initialize the Boost repository

Boost keeps all data in a directory called the repository. By default the repository is at ~/.boost. To use a different location pass the --boost-repo parameter (must precede any particular command verb, e.g. boostd --boost-repo=<PATH> init).

Export the environment variables needed for boostd init to connect to the lotus daemon and lotus miner.

export $(lotus auth api-info --perm=admin)
export $(lotus-miner auth api-info --perm=admin)

Export environment variables that point to the API endpoints for the sealing and mining processes. They will be used by the boost node to make JSON-RPC calls to the mining/sealing/proving node.

export APISEALER=`echo $MINER_API_INFO`
export APISECTORINDEX=`echo $MINER_API_INFO`

Run boostd init to create and initialize the repository:

boostd --vv init \
       --api-sealer=$APISEALER \
       --api-sector-index=$APISECTORINDEX \
       --wallet-publish-storage-deals=$PUBLISH_STORAGE_DEALS_WALLET \
       --wallet-deal-collateral=$COLLAT_WALLET \
       --max-staging-deals-bytes=50000000000

• --api-sealer is the API info for the lotus-miner instance that does sealing

• --api-sector-index is the API info for the lotus-miner instance that provides storage

• --max-staging-deals-bytes is the maximum amount of storage to be used for downloaded files (once the limit is reached Boost will reject subsequent incoming deals)

7. Update ulimit file descriptor limit if necessary. Boost deals will fail if the file descriptor limit for the process is not set high enough. This limit can be raised temporarily before starting the Boost process by running the command ulimit -n 1048576. We recommend setting it permanently by following the Permanently Setting Your ULIMIT System Value guide.

8. Before you start your boostd, it is important that it is reachable from any peer in the Filecoin network. For this, you will need a stable public IP and edit your <boostd repo>/config.toml as follows:

   [Libp2p]
     ListenAddresses = ["/ip4/0.0.0.0/tcp/24001"] # choose a fixed port
     AnnounceAddresses = ["/ip4/<YOUR_PUBLIC_IP_ADDRESS>/tcp/24001"] # important!

9. Add boostd-data details to the boostd config Configure boostd repository config (located at <boostd repo>/config.toml) to point to the exposed boostd-data service endpoint. Note that the connection must be configured to go over a websocket. For example:

     [LocalIndexDirectory]
     ServiceApiInfo = "ws://<boostd-data>:8044"

10. Start the boostd process

    boostd --vv run

11. Make sure that the correct peer id and multiaddr for your SP is set on chain, given that boost init generates a new identity. Use the following commands to update the values on chain:

lotus-miner actor set-addrs <MULTIADDR>
lotus-miner actor set-peer-id <PEER_ID>

<MULTIADDR> should be the same as the AnnounceAddresses you set in the Libp2p section of the config.toml of Boost <PEER_ID> can be found in the output of boostd net id command

Conclusion

At this stage you should have the latest version of Boost running with the Local Index Directory. Go to the Local Index Directory page and review the number sections:

Pieces

Pieces section shows counters for total pieces of user data that your SP is storing as well as whether you are keeping unsealed and indexed copies of them.

Flagged pieces

Flagged pieces are pieces that either lack an unsealed copy or are missing an index. For the sealed-only user data, you should make sure that you unseal individual sectors if you want this data to be retrievable.

Sealed only copies of data are not retrievable and are only being proven on-chain within the corresponding deadline / window. Typically sealed only data is considered as archival as it is not immediately retrievable. If the client requests it, the SP sealing pipeline must first unseal it, which typically takes 1-2 hours, and only then the data becomes available.

Flagged (unsealed) pieces is user data that your SP is hosting, which is not indexed.

We recommend that you trigger re-indexing for these pieces, so that data becomes retrievable. Check the tutorial on re-indexing flagged unsealed pieces for more information.

Deal Sector Copies

Deal Sector Copies section displays counters of your sectors state - whether you keep unsealed copies for all sectors or not. Ideally the SP should keep unsealed copies for all data that should be immediately retrievable.

Sector Proving State

Sector Proving State section displays counters of your active and inactive sectors - active sectors are those that are actively proven on-chain, inactive sectors are those that you might have failed to publish a WindowPoSt for, or are expired or removed.