Implement PromQL functions sum_over_time(), avg_over_time() and count_over_time() - #112353

…er_time()

Add a new AggregateFunctionTimeseriesSumCountAvg aggregate function that
computes a shared (sum, count) accumulator over the sliding grid window
and projects it to sum, avg or count depending on a compile-time Kind
selector, mirroring the existing bool-template idiom used by
AggregateFunctionTimeseriesExtrapolatedValue. The (sum, count) Summary is
a trivially invertible monoid, so it reuses
AggregateFunctionTimeseriesSlidingSum's fast running-sum path instead of
the two-stacks/recompute paths needed by non-invertible aggregates.

Register timeSeriesSumToGrid, timeSeriesAvgToGrid and timeSeriesCountToGrid
in AggregateFunctionTimeseriesHelpers.cpp, and wire sum_over_time,
avg_over_time and count_over_time into applyFunctionOverRange.cpp's
impl_map with drop_metric_name=true (these are value-transforming
aggregations, like rate/increase/delta, not last_over_time's passthrough).

Add SQL-level test 04628_timeseries_sum_avg_count_over_time and PromQL
regression tests in test_evaluation.py covering all three functions plus
the empty-window/no-samples case.

…input value type

timeSeriesCountToGrid projected its result as the input value column's
ValueType, so a Float32 `value` column made count_over_time return
Array(Nullable(Float32)): counts above 16777216 rounded, and even small
counts were constrained to Float32 precision. This contradicted the
function's documented Array(Nullable(Float64)) result.

AggregateFunctionTimeseriesBase now lets Traits optionally declare a
ResultType distinct from ValueType (defaulting to ValueType for every
other timeseries aggregate, so their behavior is unchanged), and the
Count specialization of AggregateFunctionTimeseriesSumCountAvgTraits
uses it to always project to Float64. Sum and Avg keep projecting to
ValueType as before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…on and fix Float32 date-function precision

Two Float32 TimeSeries-table precision gaps remained after the previous
commit, both flagged by review on this PR:

- count_over_time() was only fixed at the aggregate-function layer.
  applyFunctionOverRange() still handed its result to a VECTOR_GRID, and
  finalizeSQL()/getResultColumns() unconditionally cast/declared every
  VECTOR_GRID's value column as context.scalar_data_type (the table's
  sample type), rounding the new exact Float64 count back down to Float32
  before it reached the client. SQLQueryPiece gains an optional
  value_data_type override (nullptr by default, so every other function is
  unaffected); applyFunctionOverRange()'s ImplInfo sets it to Float64 only
  for count_over_time(), and finalizeSQL()'s two VECTOR_GRID branches now
  consult it. getResultColumns() had no SQLQueryPiece to consult at all, so
  Converter::getResultColumns() now builds one via the same tree walk
  getSQL() uses, but only trusts the override when the top-level piece's
  own value column is actually produced with StoreMethod::VECTOR_GRID -
  matching exactly the condition finalizeSQL() now checks, so the declared
  and actual result types can never disagree for any composition.

- Date/time functions (minute(), hour(), day_of_week(), etc.) still
  computed their result from an evaluation-time argument that flowed
  through context.scalar_data_type, which can be Float32 and only has
  ~128 seconds of precision at today's epoch magnitude - enough to flip
  the returned calendar component near a boundary (e.g. minute(vector(time()))
  at 1770582700, which rounds to 1770582656). This is the same bug already
  fixed on the unrelated promql/date-funcs-0arg branch, but that fix relied
  on a 0-argument date-function feature and a makeTimeQueryPiece()/Native()
  split this branch doesn't have. Ported the same idea instead:
  fromFunctionTimeNative() is a sibling of fromFunctionTime() that keeps
  the evaluation time typed context.timestamp_data_type; applyDateTimeFunction()
  detects the vector(time()) argument shape at the AST level and swaps in
  the native-precision piece before delegating to applySimpleFunction().
  time() itself is untouched and still legitimately returns a scalar/float
  value.

Added Float32-table PromQL regressions for both: an end-to-end
count_over_time() check (16777217 raw samples, verified via both the SQL
table-function path and the standalone dynamic-table HTTP API path) and a
minute(vector(time())) check at a non-128-aligned timestamp, both via
instant and range queries.

Verified against the built binary: count_over_time's Float64 precision and
the date-function fix work end-to-end; sum_over_time/avg_over_time/rate/
increase/delta/last_over_time remain unaffected (spot-checked and via the
unchanged 04628_timeseries_sum_avg_count_over_time.sql reference); the
standalone compliance harness still reports the unchanged baseline of
402 pass / 11 fail / 124 unsupported / 2 ref_mismatch (539 total).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…ppers

The previous fix only preserved count_over_time()'s exact Float64 result
through the direct top-level VECTOR_GRID finalization path. Once the result
flowed through a wrapper, the override was silently dropped and the value
was cast back down to the table's sample type:

- applySimpleFunction() (abs(), sin(), date/time functions, etc.) built its
  result as a fresh SQLQueryPiece and never copied value_data_type from its
  VECTOR_GRID argument, so abs(count_over_time(...)) lost the override
  entirely.
- finalizeSQL()'s SINGLE_SCALAR/SCALAR_GRID branches (used by scalar() and
  by vector()-wrapped/subqueried scalars) unconditionally cast to
  context.scalar_data_type, ignoring value_data_type even when set.
- Converter::getResultColumns() only honored value_data_type for
  INSTANT_VECTOR/RANGE_VECTOR results produced via VECTOR_GRID, so a
  top-level scalar(count_over_time(...)) query would still advertise the
  table's sample type as its column type.

Fix all three: applySimpleFunction() now copies value_data_type from its
(at most one) VECTOR_GRID argument; every SINGLE_SCALAR/SCALAR_GRID
finalization branch now applies the same value_data_type-or-fallback
pattern already used by the VECTOR_GRID branches; and the getResultColumns()
override condition now matches store method (VECTOR_GRID/SINGLE_SCALAR/
SCALAR_GRID) instead of gating on ResultType, keeping it in sync with
finalizeSQL().

Verified end-to-end on a Float32 TimeSeries table that scalar(), abs(), and
vector(scalar(...)) wrapping (including through a subquery) all now return
the exact Float64 count instead of rounding it back to Float32, while
sum_over_time()/avg_over_time() and unrelated applySimpleFunction() users
(rate(), bool-modifier comparisons) are unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Three more places silently dropped count_over_time()'s Float64 override
back to the table's sample type:

- applyFunctionOverRange() gated the override on `has_group`, but `has_group`
  only reflects whether the *argument* was grouped, not the type of the
  aggregate function actually being built. count_over_time() over a
  scalar/subquery-sourced range vector (e.g. count_over_time(vector(1)[...]))
  produced a SCALAR_GRID/ungrouped piece and lost the override entirely.

- applySimpleFunction() only copied value_data_type in its VECTOR_GRID
  branch. A scalar()-wrapped count_over_time() result reaching this function
  as a SINGLE_SCALAR/SCALAR_GRID/CONST_SCALAR argument (e.g. as an operand of
  a math/comparison operator) lost the override again.

- The binary-operator files that build a fresh VECTOR_GRID SQLQueryPiece
  (applySimpleBinaryOperator, applyBinaryOperatorOr/And/Unless) never copied
  value_data_type from either operand, so combining two count_over_time()
  results (or mixing one with an ordinary metric) via +, or, and, or unless
  lost the override too.

Fixed each based on which operand(s) actually determine the output value:
arithmetic operators and `or` may draw from either operand, so they merge
value_data_type via a new mergeValueDataType() helper (falling back to the
default type on a genuine conflict, which can't happen today since Float64
is the only override in use); `and`/`unless` only ever forward the left
operand's own values, so they take left's value_data_type unconditionally.

Comparison operators without the bool modifier needed special handling:
they act as a filter that keeps exactly one operand's value unmodified
(whichever side is/contains the instant vector), so applyComparisonOperator()
now captures that side's value_data_type before calling
applySimpleBinaryOperator() and applies it to the result afterwards -
otherwise a plain metric compared against a count_over_time() result would
wrongly inherit Float64 from the *discarded* operand.

Audited the remaining files that reference these store methods
(applyAggregationOperatorQuantile, applyHistogramQuantile,
applyLabelManipulationFunction, applyLimitAggregationOperator, applyOffset,
applyOneArgumentAggregationOperator, applyUnaryOperator, dropMetricName,
fromFunctionTime, toVectorGrid): all but applyHistogramQuantile and
fromFunctionTime already preserve value_data_type correctly because they
mutate their argument/result in place or copy it wholesale, and doing so is
the right call - e.g. sum/min/max/quantile/topk of count_over_time() results
should stay exact Float64. applyHistogramQuantile() computes a genuinely new
interpolated estimate (not a copy of its input), so it correctly keeps
defaulting to the table's sample type; fromFunctionTime() takes no arguments
and can never receive a count_over_time()-derived value at all.

Verified end-to-end on a Float32 TimeSeries table: all newly-fixed cases now
return exact Float64, all four previously-fixed cases (bare count_over_time,
scalar(), abs(), vector(scalar())) still work, ordinary (non-count_over_time)
uses of every touched file are unaffected (regular +, comparisons with and
without bool, or/and/unless, unary minus, topk, sum/quantile by(...),
label_replace, offset/@), and quantile/sum aggregation of count_over_time()
correctly stays exact Float64 through aggregation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…e docs

Two issues found by review, both confirmed against current code:

1. test_date_time_functions_float32_precision()'s instant- and range-query
   assertions used execute_query_in_prometheus()/execute_range_query_in_prometheus(),
   which hit the real Prometheus reader/receiver services. Per
   configs/prometheus.xml, those services' /api/v1/query and /api/v1/query_range
   handlers (my_rule_5/my_rule_6) are hardcoded to `default.prometheus`, never
   `prometheus_f32` - so neither assertion ever touched the Float32 table this
   test exists to cover. Worse, since the query is vector(time())-based (no
   series selector), real Prometheus can evaluate it entirely client-side
   without even issuing a remote-read, so these assertions may not have
   exercised ClickHouse's PromQL engine at all. The test would have stayed
   green even if the Float32 date-function fix were completely reverted.

   Fixed both assertions using the same dynamic-table HTTP API pattern
   test_count_over_time_float32_precision() already uses: hit ClickHouse's
   own /dynamic_table/api/v1/query handler (my_rule_10) directly, with the
   table supplied via the `table` query parameter so it can actually target
   `prometheus_f32`. No equivalent range-query handler existed, so added
   my_rule_12 (/dynamic_table/api/v1/query_range) mirroring my_rule_10. This
   is a low-risk, purely additive config change: PrometheusRequestHandler.cpp
   dispatches Instant vs. Range by the trailing path segment of the actual
   request URL (uri_path.ends_with("/query_range") vs "/query"), not by
   anything baked into the handler's configured <url>, so the new rule can't
   affect any existing handler's behavior. Chose this over dropping the
   range-query HTTP-API assertion as redundant with the SQL-level check below
   it, since covering both the HTTP API and SQL surfaces (like the
   count_over_time precision test already does) is more thorough for the
   same low cost.

   Audited every other execute_query_in_prometheus()/execute_range_query_in_prometheus()
   call in the file for the same table-mismatch bug: only this test function
   had it. test_count_over_time_float32_precision() already used the correct
   dynamic-table pattern throughout.

2. AggregateFunctionTimeseriesHelpers.cpp's ReturnedValue doc strings for
   timeSeriesSumToGrid/timeSeriesAvgToGrid claimed a fixed
   `Array(Nullable(Float64))` result. That's only true for
   timeSeriesCountToGrid - per AggregateFunctionTimeseriesSumCountAvg.h's
   ResultType, Sum/Avg deliberately keep projecting to the input `value`
   column's own type (confirmed correct and already covered by
   04628_timeseries_sum_avg_count_over_time.sql, which explicitly checks
   Float32 input yielding Array(Nullable(Float32)) for both). Fixed the docs,
   not the behavior: reworded both descriptions to "of the same type as
   `value`" (matching the existing "of the same type as `x`" convention used
   in FunctionsRound.cpp/changeDate.cpp) and changed their `types` list to
   `Array(Nullable(Float*))` (matching the `Float*` wildcard convention
   AggregateFunctionQuantileExactWeighted.cpp etc. use for the same
   type-preserving pattern).

Verification:
- Built standalone via ninja; `SELECT returned_value FROM system.functions`
  confirms the new Sum/Avg doc text renders correctly at runtime, and
  timeSeriesCountToGrid's doc is untouched.
- Ran 04628_timeseries_sum_avg_count_over_time.sql directly and diffed
  against its .reference file: zero differences, confirming the doc fix is
  behavior-neutral.
- Docker isn't available in this sandbox (as in prior rounds on this
  branch), so replicated the fixed integration-test assertions by hand
  against a standalone clickhouse-server using the exact same
  prometheus_test_utils helpers the test calls: both the instant- and
  range-query assertions pass against a live `prometheus_f32` table
  (minute=31 via both endpoints, matching the date-function fix), and
  pointing the same query at a nonexistent table returns an error rather
  than silently succeeding - confirming the assertions are genuinely keyed
  off the `table` parameter and therefore do exercise `prometheus_f32`,
  unlike the assertions they replace.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…pers around time()

The Float32-precision fast path for date/time functions (minute(), hour(), etc.)
previously only fired for the exact vector(time()) AST shape. But applyFunctionScalar()
and applyFunctionVector() are value-preserving no-ops for their argument's underlying
value, and applyUnaryOperator()'s '+' case is too, so wrapper forms like
scalar(vector(time())) or vector(scalar(vector(time()))) bypassed the fast path and
fell back to the lossy context.scalar_data_type cast, which can be Float32 on a
TimeSeries table with Float32 samples.

Generalize asVectorOfTime() into findTimeCallThroughScalarVectorWrappers(), which
recursively peels off any nesting of scalar(...), vector(...), and unary '+' around a
bare time() call before falling back to the generic (possibly lossy) conversion path.
Unary '-' is intentionally not peeled, since it actually negates the value rather than
passing it through unchanged.

…ppers

findTimeCallThroughScalarVectorWrappers() already peeled scalar()/vector()/
unary '+' wrappers around time() before falling back to the lossy
context.scalar_data_type cast. Extend it to also peel an Offset node
(the "@ <timestamp>" / "offset <duration>" AST wrapper): applyOffset.cpp's
offsetEvaluationTime()/setEvaluationTime() are value-preserving no-ops for
CONST_SCALAR/SINGLE_SCALAR/SCALAR_GRID pieces (only start_time/end_time/step
bookkeeping changes, never scalar_value/select_query), and
NodeEvaluationRangeGetter pre-computes each node's evaluation range in a
separate upfront pass, already folding the @/offset adjustment into the
range it assigns to the wrapped inner expression - so looking up the range
for the innermost time() node directly still yields the correctly shifted
start_time/end_time.

Verified empirically that, per the current PromQL grammar/parser, an Offset
node is only ever constructed directly around an InstantSelector,
RangeSelector, or Subquery - never directly around a Function or
UnaryOperator - so this branch is presently unreachable via any valid
PromQL syntax (queries like `vector(time()) @ 123` fail to parse). It's
kept as a defensive, provably non-regressing generalization consistent
with the existing scalar()/vector()/unary-'+' peeling, documented inline.
No new regression test was added for this specific path since no valid
query exercises it; all pre-existing regression tests (Float32 date/time
precision, count_over_time/sum_over_time/avg_over_time, value_data_type
propagation) and SQL-level PromQL parser tests were re-verified to still
pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Resolves a conflict in getResultColumns.cpp where master renamed the
PQT alias to PrometheusQueryTree while this branch added the
value_data_type_override parameter for sum_over_time/avg_over_time/
count_over_time result typing; kept both changes.

Also fixes two spots left over by git's automatic (non-conflicting)
merge in fromFunctionTime.{h,cpp} and applyDateTimeFunction.cpp that
still referenced the now-removed PQT alias after master dropped it,
replacing them with PrometheusQueryTree to match the rest of the
codebase.

These functions return Array(Nullable(Float64)), one value per grid point, so
the added assertions must not apply tupleElement to an element: the tuples in
this test's other expectations come from a query that zips the timestamps in
itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC

Review feedback from vitlibar: keep only what sum_over_time / avg_over_time /
count_over_time need.

Dropped the evaluation-time carrier feature entirely - applyDateTimeFunction's
carrier walk, fromFunctionTime, the round_evaluation_time_down context flag and
the Converter refactor that exported convertNode for it, along with its tests
and the query_range handler they used. Those four source files are back to their
master contents.

Kept the value type override plumbing: it exists because count_over_time returns
Float64 on a table whose samples are Float32, so it has to survive every
operator, the empty-result short circuits and the result-column schema. That is
this PR's own contract, not the removed feature's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC

Review feedback from vitlibar: split the shared implementation and its traits
into three, one per file, and use Kahan summation for the average.

Each function now owns its summary, which is what the three actually differ in:

- count keeps only a UInt64, exact under subtraction, so it alone stays
  invertible and its window is an O(1) running sum.
- sum keeps a plain Float64 and has no unmerge, so its window is recomputed.
- avg sums with Kahan-Babuska-Neumaier compensation. Each addition's rounding
  error is carried in a second accumulator and folded back in at the end, so
  samples far smaller than the running sum survive: beside 1e16, whose ulp is 2,
  a plain sum drops every following 1.

The kind enum, the conditional ResultType, the constrained unmerge and the
if-constexpr projection in getResult all disappear with the split.

Test: a window holding 1e16 followed by four 1s must average (1e16 + 4)/5,
which a plain Float64 sum cannot produce.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC

…) revert

tryEvaluateOneArgumentMathFunction() had its only caller in
applyDateTimeFunction.cpp's constant-folding carrier walk, which the previous
commit removed entirely. Reverts applyOneArgumentMathFunction.cpp/.h to their
master contents; isOneArgumentMathFunction() still dispatches these functions
in applyFunction.cpp and is unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Conflict in AggregateFunctionTimeseriesSamples.h: master extracted the
duplicate-timestamp rule into timeseriesMaxValueForDuplicateTimestamp(),
this branch made the collapse conditional on `keep_duplicates`. Kept both -
the policy switch now calls the shared helper.

Master also reworked the two-stacks API the sum/avg traits use, so the three
new headers are adapted to it: FORMAT_VERSION moves into the traits (the base
now reads Traits::FORMAT_VERSION), createAggregator() takes the stack size the
base computes via getStackSizeForTwoStacks() instead of redoing the density
math, and the redundant DateTime64Supported declarations are dropped along
with the siblings'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The Kahan-Babuska-Neumaier summary's merge is not commutative: which of the
two sums is larger decides where the rounding error lands, and the added
compensation folds into a different running total. The two-stacks queue
combines bucket summaries in queue order, so on dense windows the reported
average depended on the queue state. Drop the two-stacks thresholds from the
avg traits so getStackSizeForTwoStacks always returns 0 and the window is
recomputed in time order; timeSeriesSumToGrid keeps the shortcut, its merge
is commutative.

The regression pins the last grid point of a 25-bucket-per-window grid (the
old forcing threshold) against the single-point grid over the same window,
which combines its samples sequentially: values whose compensation is
order-sensitive make the two orders differ by an ulp, and the dense grid now
equals the sequential reference exactly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC

Both operators pass the left operand's values through, but neither copied
its value_data_type: the empty fast paths and the VECTOR_GRID results
dropped it, so a Float64 override reverted to the table's Float32 and
exact counts above 2^24 rounded again. The override now follows the
values, independent of whether the result contains rows.  already
carried it (it moves the surviving side), as does the arithmetic
operator's empty path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC

The non-empty or path built its result without copying either operand's
value_data_type, so a Float64 grid narrowed back to the table type before
reaching the client. My previous reply on this thread said or carried the
override everywhere; that was only true of its empty fast paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC

The three new registrations copied an older doc block that advertises
only UInt32/DateTime timestamps and integer step/staleness, while the
factory accepts the DateTime64 / fractional / decimal / string surface
that timeSeriesRateToGrid in the same file already documents; 04628
exercises these functions with DateTime64(3) timestamps. Mirror that
wording, and state that these three keep duplicate timestamps rather
than collapsing them to the greatest value like their neighbours.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcH1uX3QLcLes3XfidQqsC