Data retention for analytics and aggregation functions

[BlancoMY]

Hello Everyone,

In the context of analytics functions like "latest" and "lag" in ekuiper, I would like to inquire about the duration for which data history of variables is retained.

Also, i'm wondering what is the maximum duration for windows when using aggregation functions and how does that affects RAM consumption of eKuiper if we collect a lot of data in the source.

Thank you

[ngjaying]

It seems like you have two questions.

1. analytic function state retention, it will retain in the whole rule lifecycle. Mostly, you'll need one message retained for any time. For example, latest(temperature) will only return one temperature value during the rule execution and it is usually unawarable for memory.
2. We didn't limit the window duration and leave it to the user to determine the limitation based on the data volume and resource. The window will affect RAM comsumptions a lot. If you have 2 minutes' window, the two minutes' data of the source will be cached for any given time.