[Bug] SUM() over DECIMAL silently returns a wrong value on overflow, while check_overflow_for_decimal errors on the same overflow in binary arithmetic

[hanke580]

Search before asking

• I had searched in the issues and found no similar issues.

Version

Built from master at commit c4dee4bd5e82c6f0f380e0243cbaf265999e6a6a
(2026-09-17). ./build.sh --fe --be, single FE + single BE, default fe.conf
and be.conf.

Session defaults, unchanged: check_overflow_for_decimal = 1,
enable_decimal256 = 0.

What's Wrong?

SUM() over a DECIMAL column returns a silently wrong value once the
running total leaves the representable range. Doris already detects the very same
overflow in binary arithmetic and refuses the query; the aggregate does not check
at all.

On two rows of DECIMAL(38,0) holding 10^38 - 1:

expression	result
a + a	errors — [E-255]Arithmetic overflow: 99999999999999999999999999999999999999 add 99999...
a * 2	errors — [E-255]Arithmetic overflow: ... multiply 2 = 1999999...
SUM(a)	-14028236692093846346337460743176821145 — no error

The same overflow, on the same two values, in the same query: refused in one
expression and answered with a wrong number in the other.

There are two distinct silent regimes, and the first one is the dangerous one

With an ordinary 34-digit per-row value of 10^33, summing n rows:

n	exact total	returned
99 999	99999000000000000000000000000000000000	same	ok
100 000	100000000000000000000000000000000000000	10000000000000000000000000000000000000	exactly one tenth — positive and plausible
170 141	170141000000000000000000000000000000000	17014100000000000000000000000000000000	one tenth
170 142	170142000000000000000000000000000000000	-17014036692093846346337460743176821145	wrapped negative
199 998	199998000000000000000000000000000000000	-14028436692093846346337460743176821145	wrapped negative

The two boundaries are exactly the two representable limits involved:

• n = 100 000 is where the total first exceeds DECIMAL(38,0)'s maximum
  (10^38 - 1). From here the answer is the true total divided by ten —
  positive, 38 digits, entirely plausible, and wrong by an order of magnitude.
• n = 170 142 is where the total first exceeds int128 max
  (170141183460469231731687303715884105727). From here it wraps negative.

A negative sum of positive values is at least obviously broken. A result that is
one tenth of the truth is not — nothing in the result, the type, or the log says
anything is wrong.

Every accumulation path is affected, including one that persists to disk

Two rows of DECIMAL(38,0) holding 10^38 - 1; exact total
199999999999999999999999999999999999998:

path	returned
SUM(a)	-14028236692093846346337460743176821145	wrong
SUM(a) ... GROUP BY g	-14028236692093846346337460743176821145	wrong
SUM(a) OVER ()	-14028236692093846346337460743176821145	wrong
AVG(a)	8026480282232511287371606174911062.6544	wrong
AGGREGATE KEY table, a DECIMAL(38,0) SUM	-14028236692093846346337460743176821145	wrong, and stored on disk
SUM(a) + 0	errors	refuses
SUM(DISTINCT a)	99999999999999999999999999999999999999	correct

Three things worth separating out:

AVG breaks an invariant that needs no arithmetic to check. The average of
two identical values is that value. Both inputs are 10^38 - 1; the answer comes
back as about 8.03 x 10^33. AVG is dividing the already-wrapped total.

The AGGREGATE KEY case is not a query-time artifact. That table
pre-aggregates on write, so the wrapped value is what the storage engine
persists. Two inserts of 10^38 - 1 leave the table holding a negative number
and the inputs are gone.

SUM(a) + 0 errors, and the error quotes the corrupted value:

[E-255]Arithmetic overflow: -14028236692093846346337460743176821145 add 0 = DECIMAL128I ...

So the aggregate has already produced the wrapped number and handed it
downstream; the only reason anything complains is that the binary operator is
checked. This localises the gap precisely to the aggregate's own accumulation.

enable_decimal256 = true fixes every case

enable_decimal256	99 999 rows	100 000 rows	199 998 rows
false (default)	ok	one tenth	negative
true	ok	ok	ok

What You Expected?

Either the correct total, or an error. check_overflow_for_decimal is 1 by
default and is documented as the switch that makes decimal overflow an error;
it should cover the aggregate the same way it covers a + a and a * 2.
Returning a positive, plausible number that is one tenth of the answer is the
one outcome that should not happen.

How to Reproduce?

CREATE DATABASE IF NOT EXISTS ovf;

-- 1. the contract, on two rows
CREATE TABLE ovf.two (id INT, a DECIMAL(38,0))
  DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES('replication_num'='1');
INSERT INTO ovf.two VALUES (1, 99999999999999999999999999999999999999),
                           (2, 99999999999999999999999999999999999999);

SELECT a + a  FROM ovf.two LIMIT 1;   -- E-255 Arithmetic overflow  (correct)
SELECT a * 2  FROM ovf.two LIMIT 1;   -- E-255 Arithmetic overflow  (correct)
SELECT SUM(a) FROM ovf.two;           -- -14028236692093846346337460743176821145
                                      --    <-- no error

-- 2. it is persisted by an AGGREGATE KEY table
CREATE TABLE ovf.agg (k INT, a DECIMAL(38,0) SUM)
  AGGREGATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES('replication_num'='1');
INSERT INTO ovf.agg VALUES (1, 99999999999999999999999999999999999999);
INSERT INTO ovf.agg VALUES (1, 99999999999999999999999999999999999999);
SELECT a FROM ovf.agg;                -- -14028236692093846346337460743176821145

-- 3. the two regimes, with an ordinary 34-digit value
CREATE TABLE ovf.deep (id BIGINT, v DECIMAL(38,0))
  DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES('replication_num'='1');
-- load 320000 rows of 10^33 (the script below does this by doubling)

SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT  99999) x;  -- correct
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 100000) x;  -- one tenth
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 170142) x;  -- negative

SET enable_decimal256 = true;
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 100000) x;  -- correct

A self-contained script that builds the data and prints the tables above:

python3 repro.py --port 9030        # parts 1 and 2 of the report
python3 verify-main.py --port 9030  # regime boundaries, all aggregate paths, AGGREGATE KEY

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Doris-Breakwater]

Initial assessment

Verdict: confirmed high-impact decimal correctness bug on c4dee4bd5e82c6f0f380e0243cbaf265999e6a6a. The report is sufficiently detailed to start work; no query profile or additional logs are needed for the query-time cases. I confirmed the mechanisms by inspecting the exact commit and reproducing the arithmetic/string behavior locally, although I did not rebuild and run a Doris cluster. The issue currently has no labels, assignee, milestone, or prior comments.

There are two cooperating defects, not one:

1. Decimal SUM/AVG aggregation does not observe check_overflow_for_decimal, so an out-of-range logical DECIMAL(38,s) value is allowed into the result column and later native int128 overflow is also unchecked.
2. Decimal-to-string formatting assumes the native value has at most the declared physical precision. Once aggregation violates that invariant, a 39-digit positive int128 (or sign plus 39 digits) is written past the logical size of a string preallocated for 38 digits, and the last digit is lost when the string is resized. This explains the apparently plausible “one tenth” result.

Verified code path and exact values

• With enable_decimal256 = false, FE assigns decimal SUM a DECIMAL(38, input_scale) result; when enabled it assigns precision 76 (ComputePrecisionForSum). BE then uses that planned result type as the aggregate state type.
• The decimal sum state performs plain sum += value and sum += rhs.sum, with no logical-precision or native-overflow check (sum state). Row add, partial-state merge, and final insertion simply call those operations and push the raw state into the result column (aggregate implementation). Serialized-state range merge and window incremental paths contain the same unchecked additions elsewhere in this class.
• check_overflow_for_decimal is copied into scalar FunctionContext (context setup), which is why binary arithmetic checks it. Aggregate construction has no corresponding policy field in AggregateFunctionAttr (aggregate attributes). Therefore the session variable cannot currently affect SUM, AVG, window aggregation, or storage reader/load sum.
• At 100,000 rows of 10^33, the aggregate's native state is still the mathematically correct 10^38; it fits in signed int128 but exceeds DECIMAL(38,0). The formatter allocates 38 characters based on MAX_DECIMAL128_PRECISION, writes all 39 digits, then resizes from the old logical size (formatter). With the same libstdc++ behavior, a focused probe produces 38 visible digits followed by NUL, exactly matching the reported value divided by ten. This formatter behavior is unsafe and should be fixed independently, but the primary invariant violation originates in aggregation.
• For two inputs of 10^38 - 1, the native two's-complement result after overflowing signed 128-bit storage is actually -140282366920938463463374607431768211458. The reported -14028236692093846346337460743176821145 is that value with its final digit lost by the same formatter. Consequently, the text in the SUM(a) + 0 error is not the exact native aggregate state; the binary operator does receive an already-wrapped negative value, but its diagnostic formatting drops the final digit too.
• AVG has unchecked decimal accumulation and merge, then also computes sum * multiplier in the same result type before division (AVG result calculation, AVG accumulation). Applying 128-bit wrap to the reported inputs, then the scale-4 multiplication and division by two, gives exactly 8026480282232511287371606174911062.6544.
• Storage aggregation reuses the same unchecked sum implementation for both _reader and _load (registration); aggregate-key memtables select the _load function (memtable path). Thus storage materialization is a credible and code-confirmed risk, not only a SQL result-sink problem.
• SUM(DISTINCT a) is not a counterexample: the two equal inputs reduce to one value, so that case never overflows.

One clarification needed for the on-disk claim

The shown aggregate-key reproduction uses two separate INSERT statements. Those can create separate rowsets and overflow later in the _reader merge; that SQL alone does not prove the negative native value was already persisted in a segment. The _load path is nevertheless unchecked and can materialize it when duplicate keys are combined in one memtable, and a later compaction can do likewise.

Please strengthen that part of the reproduction with either:

• one INSERT ... VALUES (1, max), (1, max) so both duplicate keys aggregate in one load memtable, followed by a segment/rowset inspection; or
• the existing two inserts followed by a forced compaction, then evidence that the compacted segment contains the overflowed value.

The referenced repro.py and verify-main.py contents are not attached to the issue. The SQL already suffices for the query bug, but attaching them would make the boundary and storage regression cases reusable.

Recommended fix scope

1. Propagate the decimal-overflow policy into aggregate construction/execution, or use a wider internal decimal state and enforce the planned result precision at finalization. Cover every arithmetic site: row add/subtract, partial-state merge, serialized-state range/vector merge, window incremental updates, and AVG's scale multiplication. Checking only insert_result_into is too late after native int128 wrap.
2. Apply equivalent protection to storage _load/_reader aggregation. Storage operations do not naturally have a query session policy, so their failure contract must be explicit; silently storing a wrapped DECIMAL(38,s) is not acceptable.
3. Fix decimal_to_string so capacity is based on the actual formatted value (or use a bounded/dynamically growing formatter). It must never write outside the string's logical size or silently emit a truncated value, even if an invalid internal decimal reaches serialization.
4. If the aggregate state is widened from 128 to 256 bits, account for serialized aggregate-state size, spill/exchange compatibility, mixed-BE-version execution, and storage reader/load compatibility rather than changing the state layout unversioned.
5. Add focused regression coverage for:
   • logical precision overflow below int128 max (10^38) with check_overflow_for_decimal = 1;
   • native int128 overflow in both row accumulation and partial-state merge;
   • grouped, global, window, and multi-stage/distributed SUM;
   • AVG, including the scale-multiplication overflow;
   • aggregate-key same-load aggregation, read merge, and post-compaction data;
   • decimal string conversion of 39-digit positive and negative int128 values, ensuring no dropped byte or embedded trailing NUL;
   • enable_decimal256 = true as the non-overflow control case.

For ordinary query SUM/AVG, enable_decimal256 = true is a valid temporary mitigation for these magnitudes because it changes the planned state/result to Decimal256. It should not be presented as protection for an existing DECIMAL(38,0) SUM aggregate-key column: storage _load/_reader aggregation follows the schema's Decimal128 type and does not consume that session setting.

Maintainers should apply the project-standard high-severity/data-correctness, BE aggregate/expression, decimal, and storage labels and assign owners from both aggregate execution and storage aggregation.

Breakwater-GitHub-Analysis-Slot: slot_47c5ef12290a