Fix formatting, numeric, unicode, codec, complex and marshal mismatches found by differential testing - #8600
Merged
Conversation
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (4)
📒 Files selected for processing (16)
📝 WalkthroughWalkthroughChangesFormatting and numeric behavior
String and Unicode data handling
Marshal formats and read errors
Complex arithmetic and parsing
Object exposure and interning
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant PythonOperation
participant ComplexRealBinop
participant ComplexArithmetic
participant ErrorMapping
PythonOperation->>ComplexRealBinop: dispatch multiply or true_divide
ComplexRealBinop->>ComplexArithmetic: call prod, quot, or rc_quot
ComplexArithmetic-->>ComplexRealBinop: recovered result or zero-divisor result
ComplexRealBinop->>ErrorMapping: map zero-divisor result to ZeroDivisionError
Suggested reviewers: ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [x] lib: cpython/Lib/codecs.py dependencies:
dependent tests: (161 tests)
[x] test: cpython/Lib/test/test_complex.py (TODO: 2) dependencies: dependent tests: (no tests depend on complex) [x] lib: cpython/Lib/re dependencies:
dependent tests: (82 tests)
Legend:
|
ShaharNaveh
approved these changes
Aug 28, 2026
youknowone
force-pushed
the
compiler-cpython-parity
branch
from
August 28, 2026 09:08
2ad40f3 to
7758500
Compare
`format_bytes` subtracted the byte length from the field width without saturating, so `b'%1s' % (b'abcde',)` aborted with a capacity overflow. A precision written as a bare dot now truncates to nothing, and `%r`/`%a` on bytes now honour the precision and width like the other conversions. `%c` with a bytes, bytearray or str argument that is not one unit long now names the argument by its length instead of by its type. Assisted-by: Claude Code:claude-opus-5
A grouping option next to an unrecognised presentation type now reports "Cannot specify ',' with 'a'." instead of falling through to the type check. `format(True, spec)` now validates the spec the way an int does. The 'c' type now pads and aligns its character, and an integer outside the C long range raises OverflowError rather than reporting a code point out of range. A complex formatted with the 'n'/'g' types or with a precision and no type now goes through the general format without forcing a fraction. Assisted-by: Claude Code:claude-opus-5
Dividing by a power of ten rounded twice: once when the quotient was stored and again at the tie, so e.g. `round(2.5e2, -2)` came out as 300.0. The digits now come from the exact decimal expansion of the value. Assisted-by: Claude Code:claude-opus-5
An invalid literal parsed with base 0 named the base that the prefix turned out to mean rather than 0. A string of nothing but a sign was taken for the old octal form and returned 0 instead of raising. Assisted-by: Claude Code:claude-opus-5
`str.split`, `rsplit`, `strip`, `lstrip` and `rstrip` with no separator used Rust's whitespace predicate, which differs from the 29 code points `Py_UNICODE_ISSPACE` names: U+001C-U+001F and U+0085 were skipped and U+200B was treated as a separator. `expandtabs` went through `str`, so it dropped the lone surrogates of a WTF-8 string; it now works on `Wtf8` throughout. Indexing a str out of range now says "string index out of range". Assisted-by: Claude Code:claude-opus-5
`DerivedNumericValues.txt` field 1 carries a rounded decimal, so
`unicodedata.numeric('⅐')` and its neighbours came back short of the
value CPython reports. The generator now reads the rational in field 3,
and takes the 3.2 values from field 8 of `UnicodeData-3.2.0.txt`.
Assisted-by: Claude Code:claude-opus-5
`pattern_groupindex` wraps the group index in a dict proxy, and hands back a plain dict only when the pattern has no named group. Assisted-by: Claude Code:claude-opus-5
The utf-32 decoder dropped a code point above U+10FFFF instead of calling the error handler, and both the utf-16 and utf-32 handlers were told the encoding was "utf-16"/"utf-32" whichever byte order the data was in. The encoders now take a bytes replacement from the handler as it stands, rejecting one that does not fill whole code units, rather than iterating over it as text; a str replacement from the utf-16 encoder now splits an astral code point into a surrogate pair. The utf-7 decoder names its encoding "utf7", as its codec does. Assisted-by: Claude Code:claude-opus-5
Division now divides by whichever part of the divisor is larger, the way `_Py_c_quot` does, and both it and multiplication recover the infinities that the plain formulas turn into nan, following C11 Annex G. An operand that is a real number is now multiplied and divided component-wise, which keeps the sign of a zero and the other part of an infinity; a real numerator has its own quotient so that its zero keeps a sign too. An integer exponent no greater than 100 in magnitude is now reached by repeated multiplication rather than through the polar form, so e.g. `(1+2j) ** 2` is exact. A zero exponent gives 1, and an exponent with no imaginary part no longer reaches for a logarithm that turns an infinite magnitude into a nan. A result that came out infinite raises OverflowError. `complex() arg is a malformed string` is replaced by `could not convert string to complex: ...` when the string carries a misplaced underscore, `complex division by zero` by `division by zero`, and the message for zero raised to a negative or complex power no longer names the base. Assisted-by: Claude Code:claude-opus-5
A code object's string constants were interned by length, so a short string that is not a name was interned and a long name was not. They are now picked out the way `all_name_chars` does. The one-character latin-1 strings are interned as well, so that every route to one of them lands on the same object. Assisted-by: Claude Code:claude-opus-5
Versions below 2 now write a float or complex as its digits under TYPE_FLOAT/TYPE_COMPLEX, versions from 4 on write a tuple of fewer than 256 items under TYPE_SMALL_TUPLE and a long ascii string under TYPE_ASCII, and a string is marked interned only when it is. `marshal.loads` now tells apart the three ways its data can run out: "EOF read where object expected" where an object was to start, "EOF read where not expected" where a single byte was, and "marshal data too short" where a run of bytes was cut short. Assisted-by: Claude Code:claude-opus-5
youknowone
force-pushed
the
compiler-cpython-parity
branch
from
August 29, 2026 11:52
7758500 to
a5fbe53
Compare
youknowone
marked this pull request as ready for review
August 29, 2026 13:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Differential testing against CPython 3.14 over generated corpora (numeric literals, format specs,
%-formatting,strmethods,re, codecs,marshal, complex arithmetic) surfaced the following mismatches. Each commit fixes one area.crates/commonccee8074d—%-formatting of bytes precision, width and%cerrors. A bare.precision panicked instead of truncating to zero; width padding underflowed;%r/%aon bytes did not go throughascii();%cerror messages did not name the argument.5e156f22f— format spec handling of grouping, bool,'c'and complex.Unknownformat types skipped the grouping check;format(True, ...)did not validate the spec againstDecimal;'c'ignored fill/align and did not raiseOverflowErrorfor out-of-range ints; complex'g'handling was wrong.6ac591664—round()at negativendigits. Divide-then-round loses the exact decimal; now rounds half-to-even from the exact decimal digits at the requested power of ten.7dc90f4f3—int()literal errors. The error message reported the resolved base rather than the requested one for base 0, and a bare'-'/'+'took the old-octal path.a65fbbd2c— whitespace set.split/rsplit/stripdefaults andexpandtabsnow use thePy_UNICODE_ISSPACEset;str[i]gained the "string index out of range" message.crates/unicode6dc80a1ba— numeric values as exact rationals.UnicodeDatafield 8 carriesnumerator/denominator; these were being read as integers, sounicodedata.numeric()was wrong for every fractional code point. Also picks up the Unicode 3.2 fraction table.crates/vmfadd4cdc2—Pattern.groupindexreturns amappingproxy(and a plaindictwhen there are no named groups).d3491485d— complex multiplication, division and power. Implements the C11 Annex G.5.1/G.5.2 nan/infinity recovery for*and/, Smith's algorithm for the quotient, the dedicated real-numerator quotient, repeated squaring for integral exponents in[-100, 100],OverflowError("complex exponentiation"), and the real-operand fast paths.01758138e— interning of code string constants. Was gated on length ≤ 20; now onall_name_chars. Single-character latin-1 strings are interned singletons.2ad40f3f7—marshalversion handling. Versions < 2 write'f'/'x'with 17 significant digits;FLAG_REF/TYPE_INTERNEDonly from version 3; the short-ascii, ascii and small-tuple forms only from version 4. The three EOF conditions now produce their distinct messages ("EOF read where object expected" / "EOF read where not expected" / "marshal data too short").Lib/7a685bc19— utf-16/32/7 codec error handling. The utf-32 decoder did not call the error handler for code points above0x10FFFF;encodingnames passed to handlers dropped the byte order; abytesreplacement from a handler was mishandled; the utf-16 encoder did not split astral code points into surrogate pairs; the utf-7 handler name was wrong.Tests
Five
@unittest.expectedFailuremarkers removed (test_complex,test_re,test_codeccallbacks) — they now pass.Verification
clippywith CI flags: cleanclippy: cleancargo test: passResidual differences that remain in the sweeps were traced to the CPython build itself and are not fixed here: FMA contraction in its complex division,
hash(nan)being id-based, an errno leak in_Py_c_abs, refcount-dependentFLAG_REFinmarshal, and the UCD 16 ↔ 17 version gap.— drafted by Claude
Summary by CodeRabbit
Bug Fixes
Compatibility
groupindexbehavior to provide read-only mappings for named groups.