@attestate/groupie

A follower to keep track of EVM logs (also known as: “Solidity events” to make them available for local querying in LMDB.

Context

Solidity contracts are emitting events as state transition markers that off-chain applications can use to stay synchronized.

contract Token {
  event Transfer(
    address indexed from,
    address indexed to,
    uint256 indexed tokenId
  );
  // ...
  function transfer(/* ... */) external {
    emit Transfer(from, to, tokenId);
  }
}

Synchronizing the event history of a large Ethereum smart contract can be technically challenging. Downloading block event logs can fail, event parsing can be ambiguous and follow-up downloads can complicate the process.

@attestate/groupie is an embeddable JavaScript process built on top of @attestate/crawler that keeps an ordered record of specified Ethereum events in a thread-safe LMDB.

Developers can spin up groupie and synchronize to Ethereum’s latest state while thread-safely reading entries from their disk using LMDB.

Installation

Install via NPM:

npm i @attestate/groupie

Usage

There are two options for running the process. One is using function loop(config) which will result in taking-over the entire process (blocking). An alternative is function launch(config) which will spin up a node.js worker thread to keep the main thread unclogged. Both functions may be used interchangably.

import { env } from "process";

import { loop, /* launch */ } from "@attestate/groupie";

const config = {
  database: {
    // NOTE: Environment variable set by @attestate/crawler
    path: `${env.DATA_DIR}/events/`,
    index: {
      prefix: "events",
    },
  },
  blocks: {
    start: 14566826,
    stepSize: 10000, // default: 1
    interval: 5000,
  },
  contract: {
    address: "0x0bC2A24ce568DAd89691116d5B34DEB6C203F342",
  },
  topics: [
    "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
    "0x0000000000000000000000000000000000000000000000000000000000000000",
  ],
  crawler: {
    queue: {
      options: {
        // NOTE: Environment variable set by @attestate/crawler
        concurrent: parseInt(env.EXTRACTION_WORKER_CONCURRENCY, 10),
      },
    },
  },
};

(async () => {
  /*
   * Alternatively, you can use `await launch(config)` to spawn a
   * worker_thread instead of having `loop(config)` take over the entire
   * process.
   *
   */
  await loop(config);
});

There are two files needing configuration. @attestate/groupie needs a valid @attestate/crawler environment variables file and it also needs to configuration that is passed to function loop or function launch.

Environment Variables

A .env file must be present in accordance to what is outlined in attestate/crawler’s documentation.

Configuration

In addition to the Environment variables file, a JSON configuration must be passed to the start function of groupie. Its schema specification can be found in the code base.