Add statistical type aggregate functions, including autocorrelation, skewness, and linear regression#17292
Open
Cool6689 wants to merge 7 commits intoapache:masterfrom
Open
Conversation
…ity by removing unnecessary comments and whitespace
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support for several advanced statistical aggregation functions to the IoTDB query engine, including correlation, covariance, regression, skewness, and kurtosis. It introduces new accumulator classes to implement these functions and updates the relevant factory and enum classes to register and handle them appropriately.
New statistical aggregation functions:
CORR,COVAR_POP,COVAR_SAMP,REGR_SLOPE,REGR_INTERCEPT,SKEWNESS, andKURTOSIStoBuiltinAggregationFunctionEnumfor recognition in the system.AccumulatorFactoryto recognize these new functions as multi-input or single-input aggregations and to instantiate the appropriate new accumulator classes when requested. [1] [2] [3]Implementation of new accumulator classes:
CorrelationAccumulatorfor correlation and covariance calculations, supporting both population and sample variants.RegressionAccumulatorfor regression slope and intercept calculations.CentralMomentAccumulatorfor skewness and kurtosis calculations.Integration with sliding window aggregators:
SlidingWindowAggregatorFactoryto support the new statistical functions within sliding window queries.This pull request introduces support for several advanced statistical aggregation functions in the IoTDB query engine, including correlation, covariance, regression, skewness, and kurtosis. It adds new accumulator implementations for these functions and integrates them into the aggregation and sliding window frameworks.The most important changes are:
New statistical accumulator implementations:
CentralMomentAccumulatorfor computing skewness and kurtosis. (iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/aggregation/CentralMomentAccumulator.java)CorrelationAccumulatorfor correlation and covariance calculations. (iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/aggregation/CorrelationAccumulator.java)RegressionAccumulatorfor regression slope and intercept calculations. (iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/aggregation/RegressionAccumulator.java)Integration with aggregation framework:
AccumulatorFactoryto support the new aggregation functions, including logic for multi-input and single-input accumulators, and to instantiate the new accumulator classes as appropriate. (iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/aggregation/AccumulatorFactory.java) [1] [2] [3]Sliding window support:
SlidingWindowAggregatorFactoryto support the new statistical functions within sliding window aggregations. (iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/aggregation/slidingwindow/SlidingWindowAggregatorFactory.java)Enumeration and function registration:
BuiltinAggregationFunctionEnumto include the new statistical aggregation function names. (integration-test/src/main/java/org/apache/iotdb/itbase/constant/BuiltinAggregationFunctionEnum.java)## DescriptionContent1 ...
Content2 ...
Content3 ...
This PR has:
for an unfamiliar reader.
for code coverage.
Key changed/added classes (or packages if there are too many classes) in this PR