Development

Indexer

Here you can find information related to the indexer used by the Blobscan explorer.

Check out the repository here.

How it works?

The indexer traverses the Ethereum blockchain, retrieving data from both the Execution and Consensus layer clients. It examines each slot's execution payload in the Consensus Layer to identify blocks with blob versioned hashes. Afterward, it retrieves the blob data from the Consensus Layer, the remaining block and tx data from the Execution Layer client, and forwards it to the Blobscan API. The data is then stored in a PostgreSQL database, while the blobs are persisted in alternative storage solutions.

How to run locally?

  1. Install dependencies
sudo apt install libssl-dev
  1. Git clone this repository.
git clone https://github.com/Blobscan/blobscan-indexer.rs.git
cd blobscan-indexer.rs
  1. Set the environment variables.

The indexer interacts with other services (such as the execution and consensus clients). They can be configured by creating a .env file. You can use the .env.example file as a reference.

echo "SECRET_KEY=$(openssl rand -base64 32)" > .env
  1. Run the indexer.
cargo run
  1. Build a release
cargo build -r

Environment variables

Below you can find a list of supported variables:

NameRequiredDescriptionDefault value
SECRET_KEYYesShared secret key Blobscan API JWT authentication.
BLOBSCAN_API_ENDPOINTNoEndpoint for the Blobscan API.http://localhost:3001
BEACON_NODE_ENDPOINTNoA consensus node REST endpoint.http://localhost:3500
EXECUTION_NODE_ENDPOINTNoAn execution node RPC endpoint.http://localhost:8545
SENTRY_DSNNoSentry client key.

Docker images

For convenience, we also provide docker images for blobscan-indexer.

Running with defaults

docker run --rm blossomlabs/blobscan-indexer:master

Using environment variables

docker run --rm \
  -e BLOBSCAN_API_ENDPOINT=http://blobscan-api:3001 \
  -e BEACON_NODE_ENDPOINT=http://beacon:3500 \
  -e EXECUTION_NODE_ENDPOINT=http://execution:8545 \
  blossomlabs/blobscan-indexer:master

Or directly using the .env file

docker run --env-file=.env --rm blossomlabs/blobscan-indexer:master

For more information, check out Docker Hub.

Command-Line Arguments

The indexer supports the following command-line arguments for configuring the indexing process:

  • -f, --from-slot SLOT: Start indexing from slot SLOT. If the argument is not provided, the indexer will start from the latest slot in the database.

  • -n, --num-threads NUM_THREADS: Use NUM_THREADS threads to index the blockchain. It allows you to specify the number of threads that will be utilized to parallelize the indexing process. Default: the number of CPU cores.

  • -s, --slots-per-save SLOTS_PER_SAVE: Save the latest slot in the database every SLOTS_PER_SAVE slots. If the argument is not provided, the default value is 1000.

Example usage

cargo run -- -f 1000 -n 10

A note on tracing

The indexer uses the EnvFilter and Bunyan tracing layers to provide more customizable and legible events by using the bunyan format.

To display the formatted logs you'll need to have the bunyan CLI installed and pipe the indexer's output to the bunyan cli as shown below:

cargo run -q | bunyan

To filter spans and events through the EnvFilter layer you can use the default env variable RUST_LOG to define the directives to be used.

For example:

RUST_LOG=blob_indexer[span{field=value}]=level cargo run
Previous
Running locally
Next
Clis