Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

review the need for entry_seqno_index #237

Open
achamayou opened this issue Nov 21, 2024 · 0 comments
Open

review the need for entry_seqno_index #237

achamayou opened this issue Nov 21, 2024 · 0 comments

Comments

@achamayou
Copy link
Member

The current code maintains a entry_seqno_index with a ccf::indexing::strategies::SeqnosForValue_Bucketed<EntryTable> that's essentially a list of the seqnos at which there is a write to the entry table (i.e. CTS business transactions as opposed to CCF internal/governance).

This is exposed by the /entries/txIds endpoint, presumably to facilitate scanning through all the CTS ledger entries using the API, as opposed to the ledger files.

This may be a premature and ultimately harmful optimisation, because it trades off an ever-growing in-memory index for being able to skip deserialising what should amount to a fairly small amount of transactions normally (governance is rarely the major part of a ledger). Aside from the memory cost, this will also greatly increase first-response latency on new nodes, which currently won't respond usefully before they can build the index up to that point. An index-less historical query will come back much faster in that situation.

My sense is that it would be best to:

  1. Remove the index
  2. Convert /entries/txIds to a regular historical query, if it's needed at all
  3. Establish one or more benchmarks that we think /entries/txIds should meet, and decide what, if any optimisation is needed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant