Do not consult the query context when building a CAST - #114769
Conversation
CastOverloadResolverImpl inherited WithContext, which holds only a weak_ptr<const Context>, and buildImpl passed getContext() to createFunctionBaseCast. That is safe only during analysis, while a caller still holds a strong ContextPtr. PR ClickHouse#109946 made FunctionCastOrDefault store that resolver as a member and call build() at execute time. The ActionsDAG keeps the resolver alive but nothing keeps its context alive, and both affected paths build the DAG from a function-local Context::createCopy that dies on return (inplaceBlockConversions.cpp for materialized-column defaults, MutationsInterpreter.cpp for mutations). Evaluating such a stored expression later throws LOGICAL_ERROR "Context has expired": an exception in release builds, a server abort under debug and sanitizer builds. The AST fuzzer hit it on two unrelated PRs within five minutes of that merge, and the signature has no prior occurrence in 90 days of CI history. Capture the conversion settings in the resolver's constructor, while the context is alive, and have buildImpl consume that snapshot. FunctionCast used its ContextPtr for nothing but building the same FunctionConvertSettings, so this moves an existing snapshot one step earlier in the same call chain rather than introducing a new one. The resolver now holds no context at all, so the WithContext exception for this file in various_checks.sh is removed. The two build paths keep their distinct DateTimeOverflowBehavior: Saturate for createInternalCast, Ignore for buildImpl, the sentinel that defers to date_time_overflow_behavior. Sharing one snapshot between them would silently change date-time conversion, so they are built separately. The snapshot is threaded as a shared_ptr so that CastOverloadResolver.cpp can keep forward-declaring the factory instead of including the whole of FunctionsConversion.h. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both mutation arms asserted 16/120, a value that already held before their mutation executed: the MATERIALIZE COLUMN arm read a column whose stored bytes and current default expression agreed, and the UPDATE arm recomputed the values the INSERT had already written. Neither could tell a mutation that applied from one that was skipped, so they measured nothing about the mutation reach path they exist to cover. Give each arm a discriminating before value. The MATERIALIZE COLUMN arm now redefines the default with a metadata-only MODIFY COLUMN, so the stored bytes (sum 120) differ from what the current expression produces, and only a mutation that actually runs can reach 16120. The UPDATE arm inserts a -1 sentinel (sum -16) that the UPDATE must overwrite to reach 120. Each arm gets its own table so they cannot mask each other. Verified by mutation rather than by assertion: deleting either statement makes the test fail with a value diff (16 120 where 16 16120 is expected, and 16 -16 where 16 120 is expected).
Both reproduction arms of the test only reach the defect with the analyzer enabled, so on an old-analyzer CI flavour the test passed even with the fix reverted. The expiring function-local context exists only on the analyzer branch of each path. inplaceBlockConversions.cpp:280-283 (and the same gate in evaluateMissingDefaults at :311-315) calls createExpressionsAnalyzer, which takes a Context::createCopy that dies at return; the other branch, createExpressions, forwards the caller's longer-lived context. MutationsInterpreter.cpp:1758-1762 likewise creates the expiring copy only under use_analyzer, while the old path at :2066-2072 hands ExpressionAnalyzer the stored context. tests/config/install.sh:297 symlinks users.d/analyzer.xml when USE_OLD_ANALYZER=1, and that file sets allow_experimental_analyzer to 0 in the default profile. Such jobs exist, and one of the recorded sightings is on AST fuzzer (amd_debug, targeted, old_compatibility). A session SET is enough for the mutation half as well, because use_analyzer_for_mutations is not set anywhere under tests/config or ci, so shouldUseAnalyzerForMutations falls through to the session setting. Verified against the pre-fix binary with the default profile forced to the old analyzer: with this line the test aborts with 'Context has expired' at CastOverloadResolver.cpp:195, without it the test passes. enable_analyzer already defaults to 1, so the reference is unchanged.
Internal second-model review (click to expand)Three review rounds on this change: an independent cold read of the code plus an
Session id: cron:clickhouse-review-slot-8:20260814-063002 |
Pre-PR validation gate (click to expand)
Session id: cron:clickhouse-review-slot-10:20260814-023002 |
|
Workflow [PR], commit [0b50647] Summary: ❌
AI ReviewSummaryThis PR snapshots Final Verdict
LLVM Coverage Report
Changed lines: Changed C/C++ lines covered: 34/34 (100.00%) · Uncovered code |
|
cc @Ergus @Algunenano, could you review this? |
Build profile diff (arm_release)Comparing Binary sizes
Only the stripped binary is compared: the official master build keeps debug symbols while PR builds strip them, so the other binaries differ by construction. Object file sizes
|
| Object file | Master | PR | Δ |
|---|---|---|---|
src/CMakeFiles/dbms.dir/Interpreters/Aggregator.cpp.o |
11.18 MiB | 11.53 MiB | +366.26 KiB (+3.20%) |
src/CMakeFiles/dbms.dir/Processors/Transforms/LimitByTransform.cpp.o |
1.07 MiB | 1.10 MiB | +32.88 KiB (+3.00%) |
src/CMakeFiles/dbms.dir/Processors/Transforms/NegativeLimitByTransform.cpp.o |
784.98 KiB | 802.46 KiB | +17.48 KiB (+2.23%) |
737 more object files are built by the master warmup baseline only (it builds every object-file target, a pull request build only clickhouse-bundle) and not compared.
Compile time of recompiled translation units
48 translation units recompiled, 318 s compile time in total, 48 of them have a recent master baseline.
CI finish ledger - da4e7e6CI is fully finished on this head and there are no failures to own. Gates: no check queued/in_progress/pending; Coverage verified from the praktika report rather than the skip ratio: The only CIDB FAIL rows at this SHA are
Session id: cron:our-pr-ci-monitor:20260815-070000 |
|
📊 Cloud Performance Report ✅ AI verdict: This PR is a correctness fix: it snapshots the CAST FunctionConvertSettings once while the constructing context is alive (so casts in stored defaults and mutation expressions no longer read an expired context), and drops WithContext from the resolver. It moves only where those settings are built at plan/build time and changes nothing in the per-row execution path. Because no query hot path is touched, the broad 6-26% ClickBench improvements and the large TPC-H Q7 (×8.6 faster) and Q8 (×2.2 faster) readings cannot be attributed to this change, so all flagged improvements are downgraded to not-sure and read as run-to-run variance (note the small source sample and the very noisy source measurements on ClickBench Q23). clickbenchFlagged queries (11 of 43)
Change = percent below ×2; the ratio of medians (×N faster/slower) beyond, where percent understates the scale. q-value = BH-FDR adjusted p; smaller is stronger evidence. MIRAI flags a query when q < fdr_q (default 0.10) — the value the verdict is based on. tpch_adapted_1_officialFlagged queries (2 of 22)
Change = percent below ×2; the ratio of medians (×N faster/slower) beyond, where percent understates the scale. q-value = BH-FDR adjusted p; smaller is stronger evidence. MIRAI flags a query when q < fdr_q (default 0.10) — the value the verdict is based on. Debug info
|
|
@groeneai merge master into this branch, build and check if the test is passed, because in master this now returns differnet result (and that's why sync in private is failed - test failes): |
The `-- MATERIALIZED default whose cast overflows` arm took its expected value from `toDateTimeOrDefault`, so it asserted that a `to<Type>OrDefault` function honours `date_time_overflow_behavior`. That only holds while ClickHouse#109946 is in the tree. It is reverted on master by ClickHouse#114911, where `FunctionCastOrDefault` goes back to `castColumnAccurateOrNull` with a null context and therefore always saturates. The arm passes on this pull request's base, which predates the revert, and fails against master: 2149-06-06 2013-04-29 17:31:44 expected, wrap-around 2149-06-06 2106-02-06 00:00:00 actual, saturating That is why the private sync reported this test as the only new failure, in every stateless job it ran in, while the public checks stayed green. Spelling the conversion as an explicit `CAST` pins the arm to `CastOverloadResolverImpl`, which is what this change is about, and makes it read the same before and after the revert. Verified against three binaries - master `d59f2be34ed3` with the revert in effect, this pull request's head, and the private sync build - all of which reproduce the reference byte for byte. The reference file is unchanged. Related: ClickHouse#109946 Related: ClickHouse#114911 Related: ClickHouse#114912 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Confirmed, and it is master's behaviour rather than anything this PR does. I merged master ( The route changed under the arm. A stored I see @alexey-milovidov has already pushed One thing worth flagging, since it changes the merge order. #114912 re-lands #109946 and does not carry this PR's fix: at its head |
CI finish ledger - 0b50647Every failure below has an owner: a fixing PR (ours or external), or a full-effort fix task
CI is otherwise clean on this head: 179 check-runs, 0 incomplete, 160 success, 18 skipped, Context, since the gate result is expected rather than surprising: The merge-blocking decision sits in Session id: cron:our-pr-ci-monitor:20260818-013000 |
|
The blocking One measurement that I think decides the category question, because it is not visible from master: the defect is live on the 26.7 branch. #114689 backported #109946 there on 08-13 and no revert followed, so the branch still carries both the retaining caller and The INSERT is rejected outright. The last tagged release, So I have not pushed anything. The master merge remains your call. |
91e2361
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Fixed a
LOGICAL_ERROR: Context has expiredexception when a stored expression containingaccurateCastOrDefaultor ato<Type>OrDefaultfunction is evaluated after the query that created it has finished, for example aMATERIALIZEDcolumn default onINSERTor a mutation expression.Description
Related: #109946
CastOverloadResolverImplinheritedWithContext, which holds only aweak_ptr<const Context>, and itsbuildImplpassedgetContext()tocreateFunctionBaseCast. That is safe only during analysis, while a caller still holds a strongContextPtr.#109946 made
FunctionCastOrDefaultstore that resolver as a member and callbuild()at execute time. TheActionsDAGkeeps the resolver alive but nothing keeps its context alive, and both affected paths build the DAG from a function-localContext::createCopythat dies on return (inplaceBlockConversions.cpp:230for materialized-column defaults,MutationsInterpreter.cpp:1762for mutations). Evaluating such an expression later throwsContext has expired: an exception in release builds, a server abort under debug and sanitizers.This captures the conversion settings in the resolver's constructor, while the context is provably alive, and has
buildImplconsume that snapshot.FunctionCastused itsContextPtrfor nothing butsettings(context, ...), so the snapshot moves one step earlier in the same call chain rather than being introduced. The resolver now holds no context at all, so theWithContextexception for this file invarious_checks.shis removed.The two build paths keep their distinct
DateTimeOverflowBehavior:SaturateforcreateInternalCast,IgnoreforbuildImpl. Sharing one snapshot between them would silently change date-time conversion.Since a snapshot replaces a live context, every context-derived conversion setting was compared before and after and is byte-identical, including all three
date_time_overflow_behaviormodes,cast_keep_nullable,precise_float_parsing, the IPv4/IPv6 error settings and timezone substitution.04510_accurateCastOrDefault_settingsfrom #109946 passes unchanged. The new test aborts a master server atCastOverloadResolver.cpp:195and passes with this change, 50/50 green under randomized settings.The defect rides along with #109946, so it is present wherever that landed: 26.7 (#114689, merged); the 25.8, 26.3 and 26.5 backports are still open.
CI provenance
Logical error: Context has expired, STID1805-460a, first seen 2026-08-13 22:45:16 UTC, about four hours after #109946 merged. The frame pairCastOverloadResolverImpl::buildImpl+Context has expiredhas no earlier occurrence in the 90 days CIDB retains.17d2798b9626The master row is bucketed as STID
1805-4810because an STID is a stack hash, but it carries both discriminating frames and17d2798b9626contains #109946.Workflow [PR]
Sync PR [sync-upstream/pr/114769]
Version info
26.8.1.1625(included in26.8and later)