Crawlori SDK

Let's generate a configuration for our Crawlori blockchain indexer. Crawlori requires a bit more setup than the other quick start tutorials, but it's well worth the (small) hassle! It allows to crawl Tezos blockchains in a very fast and modular way.

Import KT1 from the blockchain

Start by generating an SDK for the contract in the current directory, with the extra --crawlori option. If no option is given for the language, the Typescript SDK is generated:

factori import kt1 . KT1Fjiwsmo49yM7jch6KG4ETLFkatF3Qj8of \
    --crawlori \
    --name my_contract \
    --network ghostnet \
    --force

Before we go further, you need to do some setup.

Setting things up

Postgresql setup

In the following, <USER> is the user of your shell (typically you can obtain it through the $USER variable.

If this is your first time using pgocaml, you might need to install postgresql and configure it to give the required rights to your user.

sudo -i -u postgres -- psql -c 'create user <USER> with createdb;'

Opam setup

Warning: if you run into problems, check that you have disabled sandboxing as indicated at the end of this section.

You want to install the necessary OCaml dependencies. You need to be inside an opam switch, or you can create one with

make _opam

Then install needed dependencies:

opam update
make deps

Note: even if you already did this previously in the OCaml quickstart, you still need to re-do it because the dependencies for Crawlori are slightly bigger.

Now cleanly format the generated code:

make format-ocaml

The pgocaml package doesn't play well with the sandbox mode of OPAM. So you might need to disable it.

On a fresh install of opam you can proceed like this:

opam init --reinit --disable-sandboxing --bare
opam reinstall pgocaml pgocaml_ppx ez_pgocaml

Otherwise, you can edit the opam config file (default location is ~/.opam/config). To disable the sandboxing you can comment out the last three lines:

wrap-build-commands:
  ["%{hooks}%/sandbox.sh" "build"] {os = "linux" | os = "macos"}
wrap-install-commands:
  ["%{hooks}%/sandbox.sh" "install"] {os = "linux" | os = "macos"}
wrap-remove-commands:
  ["%{hooks}%/sandbox.sh" "remove"] {os = "linux" | os = "macos"}

If you edited ~/.opam/config as said above, you will need to reinstall pgocaml:

opam reinstall pgocaml

Building the crawler

make crawlori

Hello world example

Let's generate a Crawlori plugin for KT1Fjiwsmo49yM7jch6KG4ETLFkatF3Qj8of. See section SDK generation for the details about this contract.

Look at the code

First, you may want to have a look at the content of src/typescript_sdk/src/ directory.

ls src/ocaml_crawlori/

Edit the configuration file

Edit the file src/ocaml_crawlori/config.json:

{ "nodes": [
    "https://api.ghostnet.tzkt.io/"
  ],
  "sleep": 1,
  "start": 2061886,
  "confirmations_needed": 2,
  "step_forward": 30,
  "register_kinds": [],
  "originator_address" : "tz1XcgTCbVj8nbJb3DhrsYjNtjVdouZHMmYb"
 }

Then to start crawlori, you can run the following command:

_build/default/src/ocaml_crawlori/crawler.exe src/ocaml_crawlori/config.json