buffer: fix 32-bit truncation in SlowCopy for buffers larger than 4 GiB#62500
buffer: fix 32-bit truncation in SlowCopy for buffers larger than 4 GiB#62500crawfordxx wants to merge 1 commit intonodejs:mainfrom
Conversation
src/node_buffer.cc
Outdated
| const size_t target_start = | ||
| static_cast<size_t>(args[2]->IntegerValue(ctx).ToChecked()); | ||
| const size_t source_start = | ||
| static_cast<size_t>(args[3]->IntegerValue(ctx).ToChecked()); | ||
| const size_t to_copy = | ||
| static_cast<size_t>(args[4]->IntegerValue(ctx).ToChecked()); | ||
|
|
||
| CopyImpl(source_obj, target_obj, target_start, source_start, to_copy); |
There was a problem hiding this comment.
Don't use checked operations here.
| const size_t target_start = | |
| static_cast<size_t>(args[2]->IntegerValue(ctx).ToChecked()); | |
| const size_t source_start = | |
| static_cast<size_t>(args[3]->IntegerValue(ctx).ToChecked()); | |
| const size_t to_copy = | |
| static_cast<size_t>(args[4]->IntegerValue(ctx).ToChecked()); | |
| CopyImpl(source_obj, target_obj, target_start, source_start, to_copy); | |
| int64_t target_start, source_start, to_copy; | |
| if (!args[2]->IntegerValue(env->context()).To(&target_start) || | |
| !args[3]->IntegerValue(env->context()).To(&source_start) || | |
| !args[4]->IntegerValue(env->context()).To(&to_copy)) { | |
| return; | |
| } | |
| CopyImpl(source_obj, | |
| target_obj, | |
| static_cast<size_t>(target_start), | |
| static_cast<size_t>(source_start), | |
| static_cast<size_t>(to_copy)); |
src/node_buffer.cc
Outdated
There was a problem hiding this comment.
Please also change the FastCopy argument and return types from uint32_t to uint64_t.
| // Copy the entire large buffer. The native _copy binding receives | ||
| // to_copy = FOUR_GB + 5. Before the fix, Uint32 truncation reduced this to | ||
| // 5, leaving all but the first 5 bytes of dst unchanged (still 0xaa). | ||
| src.copy(dst); |
There was a problem hiding this comment.
Copying gigabytes of buffer memory is a massive operation. Can this just test copying a small chunk of memory with source/target offsets of >2^32?
SlowCopy extracted targetStart, sourceStart, and to_copy using `.As<Uint32>()->Value()`, which silently truncates values that exceed 2^32-1 (4 GiB). As a consequence, Buffer.prototype.copy() produced incorrect results when operating on buffers larger than 4 GiB: only the lower 32 bits of each offset/count were used, causing writes to the wrong location or copying far fewer bytes than requested. Fix by extracting those arguments with `IntegerValue()` (which yields int64_t) and casting to size_t, matching the behaviour of the previous `ParseArrayIndex`-based implementation that existed before PR nodejs#54087. CopyImpl's parameter types are widened to size_t accordingly. FastCopy retains uint32_t parameters because V8's Fast API contract guarantees it is only invoked when all arguments fit in uint32_t; for larger values V8 falls back to SlowCopy automatically. Add test/parallel/test-buffer-copy-large.js to cover the regression. The test allocates a buffer slightly larger than 4 GiB and verifies that Buffer.prototype.copy() copies all bytes correctly; it skips gracefully on systems with insufficient memory. Fixes: nodejs#55422
92f4933 to
4ce7086
Compare
|
Addressed all three review comments:
|
Summary
SlowCopyinsrc/node_buffer.ccextractedtargetStart,sourceStart,and
to_copyvia.As<Uint32>()->Value(), silently truncating values thatexceed
2^32 - 1(4 GiB). This causedBuffer.prototype.copy()to copythe wrong number of bytes or write to the wrong position when dealing with
buffers larger than 4 GiB.
Root cause was introduced in #54087 ("buffer: use native copy impl"), which
replaced the original
ParseArrayIndex-based extraction (usingsize_t) withUint32Value(32-bit).Changes
src/node_buffer.cc: UseIntegerValue()(returnsint64_t) inSlowCopyinstead of
.As<Uint32>()->Value(), and widenCopyImpl's parameter typesfrom
uint32_ttosize_t.FastCopyis left unchanged β V8's Fast API guarantees it is only invokedwhen all arguments fit in
uint32_t; for larger values V8 falls back toSlowCopyautomatically.test/parallel/test-buffer-copy-large.js: New test that allocates a bufferslightly larger than 4 GiB and verifies
Buffer.prototype.copy()copies allbytes correctly. The test skips gracefully on systems with insufficient memory.
Test
Fixes #55422