keeping books in sync

[elyase]

Is there any logic in cryptocrawler to keep books in sync with the exchange?

It seems that there is some support for that but no sure if there is some additional logic like for example here.

Update: Related logic explanation for gateio

[soulmachine]

The crypto-crawler library never modifies data received from exchanges, as a crawler, its job is to receive data only.

The crypto-msg-parser will parse the raw data from the crawler, if it sees fields related to seq_id and prev_seq_id, it will fill these fields. For now only a few exchanges have related fields.

[elyase]

ah ok, do I understand from your message that having the whole book in sync (or reconstructing the book at all) is out of scope for the current libraries?

[soulmachine]

@elyase correct, crypto-crawler just receives data as it is. If you're building a trading bot which needs to maintain a correct local orderbook, you can restart the crawler if the seq_id has a gap.

[elyase]

Got it. Would be great to have an example of that as it seems there are a couple details that need to be taken into account like getting the initial snapshot before running the crawler, caching and applying the updates fromcrypto-crawler, which symbols need to be refreshed when etc, and it is not clear how to combine the pieces from this great library correctly to make it work.
May be there is a need for a higher level library ala ccxt. They just stream the whole snapshot and take care of resents in background so that the user doesn't need to deal with those complexities.
I opened a separate issue.