Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve historical query cache #83

Open
letmaik opened this issue Jan 19, 2023 · 4 comments
Open

Improve historical query cache #83

letmaik opened this issue Jan 19, 2023 · 4 comments
Labels
enhancement New feature or request
Milestone

Comments

@letmaik
Copy link
Member

letmaik commented Jan 19, 2023

We currently use a simple LRU cache in the historical query adapter to keep track of when to drop cached transactions to avoid going out of memory. This is based on the transaction count and is hard-coded to 100. Given that transactions have different sizes it would be more economical to use the transaction size to determine the cache size and then set a limit like 100 MiB, based on the available enclave memory set in oe_sign.conf. With small claim sizes this would effectively raise the cache limit by one or more magnitudes of transactions.

@letmaik letmaik added the enhancement New feature or request label Jan 19, 2023
@achamayou
Copy link
Member

Related to microsoft/CCF#3234

@ivarprudnikov ivarprudnikov added this to the 0.6.0 milestone Sep 12, 2023
@ivarprudnikov ivarprudnikov changed the title Use transaction size instead of count for historical query cache Improve historical query cache Jan 12, 2024
@ivarprudnikov ivarprudnikov modified the milestones: Next version, Backlog Jan 12, 2024
@ivarprudnikov
Copy link
Member

@achamayou
Copy link
Member

A set_soft_cache_limit() API was added to CCF in microsoft/CCF#6282, to allow applications to easily set a limit.

@achamayou
Copy link
Member

Some of the code in historical_queries_adapter.h can be unwound in favour of setting a soft cache limit on the historical state, as is done in https://github.com/maxtropets/CCF/blob/8db27adc7b318cab2bed3257d0ef710eb2998cc0/samples/apps/logging/logging.cpp#L467

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants