Fix cross-platform math determinism (Mac/Linux)

AGENTS.md

@@ -28,15 +28,19 @@ GeneralsX is a cross-platform port of Command & Conquer: Generals Zero Hour for
 1. **Single codebase** – Linux and macOS build from same source
 2. **SDL3 everywhere** – No native platform calls in game code
 3. **DXVK everywhere** – DX8 → Vulkan translation on all platforms
-4. **OpenAL default (MiniAudio WIP)** – Cross-platform audio stack. Audio fixes must prioritize OpenAL but be backported to MiniAudio.
+4. **OpenAL / MiniAudio Parity** – Cross-platform audio stack. Implementations and bug fixes in one MUST be replicated in the other to maintain strict feature parity.
 5. **64-bit native** – x86_64 only (32-bit via VC6 upstream)
 6. **Retail compatibility** – Original replays and mods must work
 7. **Determinism** – Rendering/audio changes must not affect gameplay logic
 8. **No band-aids** – Fix underlying issues, not symptoms
 9. **Update worklog** – Update `docs/WORKLOG/YYYY-MM-DIARY.md` before committing (see [.github/instructions/docs.instructions.md](.github/instructions/docs.instructions.md) for details)
 10. **Reference repos** – Study patterns, don't copy-paste
 11. **Backport to Generals** – Bugfixes and improvements must be backported to the Generals base game.
-12. **Deterministic Math** – The native math functions were replaced by deterministic multi-platform WWMath equivalents to maintain determinism. This pattern must be preserved. When syncing upstream, replace these functions:
+
+## Cross-Platform Determinism (Mac vs Linux)
+To guarantee cross-play between macOS ARM64 and Linux x86_64 without SyncCrash desyncs, you must obey the following rules:
+
+1. **Deterministic Math** – The native math functions were replaced by deterministic multi-platform WWMath equivalents to maintain determinism. This pattern must be preserved. When syncing upstream, replace these functions:
     - `sqrt` / `sqrtf` -> `WWMath::SqrtOrigin` / `WWMath::SqrtfOrigin` or `WWMath::Sqrt`
     - `sin` / `sinf` -> `WWMath::SinTrig` or `WWMath::Sin`
     - `cos` / `cosf` -> `WWMath::CosTrig` or `WWMath::Cos`
@@ -48,6 +52,11 @@ GeneralsX is a cross-platform port of Command & Conquer: Generals Zero Hour for
     - `ceil` / `ceilf` -> `WWMath::Ceil`
     - `floor` / `floorf` -> `WWMath::Floor`
     - `pow` -> `WWMath::PowOrigin`
+2. **NaN/Inf Integer Casting** – ARM64 casts `(Int)NaN` to `0`, while x86 casts it to `INT_MIN`. Any division by potentially zero (e.g. `val / maxHealth`) whose result might be cast to an integer or affect simulation flow MUST be guarded (`if (maxHealth > 0)`) or use `WWMath::Div_FixNaN` to avoid immediate CRC desyncs.
+3. **Double-Precision Isolation** – Do not allow implicit `double` promotion in transcendental/power math evaluations (e.g. `gm_sqrt`, `gm_pow`, `gm_atan`). x87 (24-bit mantissa) and NEON (53-bit mantissa) diverge by 1-ULP on double-precision. Ensure all `WWMath` wrappers explicitly accept/return `float` or internally downcast `double` to `float` prior to `gm_*f` execution to guarantee cross-platform bit-identical outputs.
+4. **FPU Environment State Leaks** – Audio drivers (OpenAL/MiniAudio) and OS callbacks aggressively alter hardware FPU registers (e.g., Flush-To-Zero, Rounding mode) for DSP performance. If this state leaks into the main thread, the simulation math diverges. **Always inject `ScopedFPUGuard`** at the boundaries of `GameLogic::update()`, audio thread callbacks, and mouse-picking logic (e.g., `View::pickDrawable()`).
+5. **FMA Contraction** – Fused Multiply-Add combines instructions with infinite intermediate precision, yielding different results on ARM64 vs x86_64. We strictly enforce `-ffp-contract=off` globally (and `/fp:precise` for MSVC). Never bypass this with `-ffast-math` or `/fp:fast`.
+6. **Deep CRC Memory Buffer Logging** – When a sync crash occurs and the root cause isn't obvious, the game automatically dumps a `Debug/deep_crc_YYYY-MM-DD-HH-MM-SS.bin` file containing a binary snapshot of the last 64 frames of state transfers. Use `scripts/qa/parse_deep_crc.py` to inspect these dumps and identify the exact object ID and state data that first diverged between players. Note: This requires the `RTS_BUILD_OPTION_DEEP_CRC=ON` CMake flag (which is enabled by default).
 
 ## Reference Repositories
 - **fighter19-dxvk-port** – Primary graphics/platform reference (DXVK + SDL3 on Linux)
@@ -161,8 +170,6 @@ mkdir -p logs && gdb -batch -ex "run -win" -ex "bt full" -ex "thread apply all b
   ./build/linux64-deploy/GeneralsMD/GeneralsXZH 2>&1 | tee logs/gdb.log
 ```
 
-
-
 ## Branching & Sync
 ### TheSuperHackers upstream sync
 ```bash
@@ -197,6 +204,12 @@ git merge thesuperhackers/main
 - Use `--body-file` with real Markdown file instead of `--body`
 - Avoid literal `\n` sequences; prefer actual newlines in multi-line strings
 
+## Git Commit Standards
+- **Conventional Commits**: Format must be `<type>(optional scope): <description>`. (e.g. `fix(audio): restart sound groups in reset`)
+- **DO NOT use `@`**: The commit message (both title and body) must never contain an `@` symbol. The `// GeneralsX @keyword` format is strictly for inline code annotations in C++ files. Do not append these signatures to commit messages.
+- **Imperative mood**: Use "add", "fix", "change" (not "added" or "fixes").
+- **Read the Docs**: For full details, valid types, and PR standards, you **MUST** read `.github/instructions/git-commit.instructions.md`.
+
 ## VS Code Tasks
 - Prefer task-first execution for build/test/debug
 - Logs captured to `logs/` directory

Core/GameEngine/Include/Common/GameDefines.h

@@ -139,6 +139,10 @@
 #define USE_BUFFERED_IO (1)
 #endif
 
+#if (!defined(DEEP_CRC_TO_MEMORY) && !defined(DEBUG_CRC))
+#define DEEP_CRC_TO_MEMORY 0
+#endif
+
 // Enable cache for local file existence. Reduces amount of disk accesses for better performance,
 // but decreases file existence correctness and runtime stability, if a cached file is deleted on runtime.
 #ifndef ENABLE_FILESYSTEM_EXISTENCE_CACHE

Core/GameEngine/Include/Common/Xfer.h

@@ -178,6 +178,11 @@ class Xfer
 	virtual void xferMatrix3D( Matrix3D* mtx );
 	virtual void xferMapName( AsciiString *mapNameData );
 
+#if DEEP_CRC_TO_MEMORY
+	virtual void xferLogString(const AsciiString& str) {}
+#endif
+
+
 protected:
 
 	// this is the actual xfer implementation that each derived class should implement

Core/GameEngine/Include/Common/XferDeepCRC.h

@@ -33,6 +33,10 @@
 #include "Common/Xfer.h"
 #include "Common/XferCRC.h"
 
+#if DEEP_CRC_TO_MEMORY
+#include <vector>
+#endif
+
 // FORWARD REFERENCES /////////////////////////////////////////////////////////////////////////////
 class Snapshot;
 
@@ -55,9 +59,18 @@ class XferDeepCRC : public XferCRC
 	virtual void xferAsciiString( AsciiString *asciiStringData ) override;  ///< xfer ascii string (need our own)
 	virtual void xferUnicodeString( UnicodeString *unicodeStringData ) override;	///< xfer unicode string (need our own);
 
+#if DEEP_CRC_TO_MEMORY
+	virtual void xferLogString(const AsciiString& str) override;
+	void changeXferMode(XferMode xferMode);
+#endif
+
 protected:
 
 	virtual void xferImplementation( void *data, Int dataSize ) override;
 
+#if DEEP_CRC_TO_MEMORY
+	std::vector<UnsignedByte>* m_buffer;									///< pointer to buffer
+	size_t m_bufferIndex;																	///< current index in buffer
+#endif
 	FILE * m_fileFP;																			///< pointer to file
 };

Core/GameEngine/Include/GameNetwork/NetworkInterface.h

@@ -104,7 +104,11 @@ class NetworkInterface : public SubsystemInterface
 	virtual void attachTransport(Transport *transport) = 0;
 	virtual void initTransport() = 0;
 	virtual Bool sawCRCMismatch() = 0;
+#if DEEP_CRC_TO_MEMORY
+	virtual void setSawCRCMismatch(const UnicodeString& strMismatchDetails) = 0;
+#else
 	virtual void setSawCRCMismatch() = 0;
+#endif
 
 	virtual Bool isPlayerConnected(Int playerID) = 0;
 

Core/GameEngine/Source/Common/System/XferCRC.cpp

@@ -36,6 +36,10 @@
 #include "Common/Snapshot.h"
 #include "Utility/endian_compat.h"
 
+#if DEEP_CRC_TO_MEMORY
+#include "GameLogic/GameLogic.h"
+#endif
+
 //-------------------------------------------------------------------------------------------------
 //-------------------------------------------------------------------------------------------------
 XferCRC::XferCRC()
@@ -175,7 +179,14 @@ UnsignedInt XferCRC::getCRC()
 XferDeepCRC::XferDeepCRC()
 {
 
+#if DEEP_CRC_TO_MEMORY
+	m_xferMode = XFER_CRC;
+	m_buffer = nullptr;
+	m_bufferIndex = 0;
+#else
 	m_xferMode = XFER_SAVE;
+#endif
+
 	m_fileFP = nullptr;
 
 }
@@ -202,8 +213,13 @@ XferDeepCRC::~XferDeepCRC()
 void XferDeepCRC::open( AsciiString identifier )
 {
 
+#if DEEP_CRC_TO_MEMORY
+	m_xferMode = XFER_CRC;
+#else
 	m_xferMode = XFER_SAVE;
+#endif
 
+#if !DEEP_CRC_TO_MEMORY
 	// sanity, check to see if we're already open
 	if( m_fileFP != nullptr )
 	{
@@ -213,10 +229,27 @@ void XferDeepCRC::open( AsciiString identifier )
 		throw XFER_FILE_ALREADY_OPEN;
 
 	}
+#endif
 
 	// call base class
 	Xfer::open( identifier );
 
+#if DEEP_CRC_TO_MEMORY
+	m_buffer = &TheGameLogic->getCRCBuffer();
+	m_bufferIndex = 0;
+
+	AsciiString str;
+	str.format("[ START OF DEEP CRC FRAME %d ]", TheGameLogic->getFrame());
+	const UnsignedInt length = str.getLength();
+
+	while (m_bufferIndex + length >= m_buffer->size())
+	{
+		m_buffer->resize(m_buffer->size() * 2);
+	}
+
+	memcpy(&(*m_buffer)[m_bufferIndex], str.str(), length);
+	m_bufferIndex += length;
+#else
 	// open the file
 	m_fileFP = fopen( identifier.str(), "w+b" );
 	if( m_fileFP == nullptr )
@@ -226,6 +259,7 @@ void XferDeepCRC::open( AsciiString identifier )
 		throw XFER_FILE_NOT_FOUND;
 
 	}
+#endif
 
 	// initialize CRC to brand new one at zero
 	m_crc = 0;
@@ -238,6 +272,21 @@ void XferDeepCRC::open( AsciiString identifier )
 void XferDeepCRC::close()
 {
 
+#if DEEP_CRC_TO_MEMORY
+	AsciiString str;
+	str.format("[ END OF DEEP CRC FRAME %d ]", TheGameLogic->getFrame());
+	const UnsignedInt length = str.getLength();
+
+	while (m_bufferIndex + length >= m_buffer->size())
+	{
+		m_buffer->resize(m_buffer->size() * 2);
+	}
+
+	memcpy(&(*m_buffer)[m_bufferIndex], str.str(), length);
+	m_bufferIndex += length;
+
+	TheGameLogic->storeCRCBuffer(m_bufferIndex);
+#else
 	// sanity, if we don't have an open file we can do nothing
 	if( m_fileFP == nullptr )
 	{
@@ -250,6 +299,11 @@ void XferDeepCRC::close()
 	// close the file
 	fclose( m_fileFP );
 	m_fileFP = nullptr;
+#endif
+
+#if DEEP_CRC_TO_MEMORY
+	m_buffer = nullptr;
+#endif
 
 	// erase the filename
 	m_identifier.clear();
@@ -267,6 +321,15 @@ void XferDeepCRC::xferImplementation( void *data, Int dataSize )
 		return;
 	}
 
+#if DEEP_CRC_TO_MEMORY
+	while (m_bufferIndex + dataSize >= m_buffer->size())
+	{
+		m_buffer->resize(m_buffer->size() * 2);
+	}
+
+	memcpy(&(*m_buffer)[m_bufferIndex], data, dataSize);
+	m_bufferIndex += dataSize;
+#else
 	// sanity
 	DEBUG_ASSERTCRASH( m_fileFP != nullptr, ("XferSave - file pointer for '%s' is null",
 										 m_identifier.str()) );
@@ -279,6 +342,7 @@ void XferDeepCRC::xferImplementation( void *data, Int dataSize )
 		throw XFER_WRITE_ERROR;
 
 	}
+#endif
 
 	XferCRC::xferImplementation( data, dataSize );
 
@@ -341,3 +405,18 @@ void XferDeepCRC::xferUnicodeString( UnicodeString *unicodeStringData )
 		xferUser( (void *)unicodeStringData->str(), sizeof( WideChar ) * len );
 
 }
+
+#if DEEP_CRC_TO_MEMORY
+void XferDeepCRC::changeXferMode(XferMode xferMode)
+{
+	m_xferMode = xferMode;
+}
+
+void XferDeepCRC::xferLogString(const AsciiString& str)
+{
+	if (m_buffer)
+	{
+		xferUser(const_cast<char*>(str.str()), str.getLength());
+	}
+}
+#endif

Core/GameEngine/Source/GameClient/MessageStream/PlaceEventTranslator.cpp

@@ -339,7 +339,7 @@ GameMessageDisposition PlaceEventTranslator::translateGameMessage(const GameMess
 				Int x, y;
 				x = mouse.x - start.x;
 				y = mouse.y - start.y;
-				if( WWMath::SqrtOrigin( (x * x) + (y * y) ) >= PLACEMENT_DRAG_THRESHOLD_DIST )
+				if( WWMath::SqrtOrigin( (float)((x * x) + (y * y)) ) >= PLACEMENT_DRAG_THRESHOLD_DIST )
 				{
 
 					TheInGameUI->setPlacementEnd(&mouse);

Core/GameEngine/Source/GameLogic/AI/AIPathfind.cpp

@@ -929,7 +929,7 @@ void Path::computePointOnPath(
 		// projected position on the path.  If we are very far off the path, we will move
 		// directly towards the nearest point on the path, and not the next path node.
 		const Real maxPathError = 3.0f * PATHFIND_CELL_SIZE_F;
-		const Real maxPathErrorInv = 1.0 / maxPathError;
+		const Real maxPathErrorInv = 1.0f / maxPathError;
 		Real k = offsetDist * maxPathErrorInv;
 		if (k > 1.0f)
 			k = 1.0f;
@@ -2071,7 +2071,7 @@ UnsignedInt PathfindCell::costToGoal( PathfindCell *goal )
 	Int dy = m_info->m_pos.y - goal->getYIndex();
 #define NO_REAL_DIST
 #ifdef REAL_DIST
-	Int cost = COST_ORTHOGONAL*WWMath::SqrtOrigin(dx*dx + dy*dy);
+	Int cost = COST_ORTHOGONAL*WWMath::SqrtOrigin((float)(dx*dx + dy*dy));
 #else
 	if (dx<0) dx = -dx;
 	if (dy<0) dy = -dy;
@@ -2097,7 +2097,7 @@ UnsignedInt PathfindCell::costToHierGoal( PathfindCell *goal )
 	}
 	Int dx = m_info->m_pos.x - goal->getXIndex();
 	Int dy = m_info->m_pos.y - goal->getYIndex();
-	Int cost = REAL_TO_INT_FLOOR(COST_ORTHOGONAL*WWMath::SqrtOrigin(dx*dx + dy*dy) + 0.5f);
+	Int cost = REAL_TO_INT_FLOOR(COST_ORTHOGONAL*WWMath::SqrtOrigin((float)(dx*dx + dy*dy)) + 0.5f);
 	return cost;
 }
 
@@ -6445,7 +6445,7 @@ Int Pathfinder::examineNeighboringCells(PathfindCell *parentCell, PathfindCell *
 				}	else {
 					dx = newCellCoord.x - goalCell->getXIndex();
 					dy = newCellCoord.y - goalCell->getYIndex();
-					costRemaining = COST_ORTHOGONAL*WWMath::SqrtOrigin(dx*dx + dy*dy);
+					costRemaining = COST_ORTHOGONAL*WWMath::SqrtOrigin((float)(dx*dx + dy*dy));
 					costRemaining -= attackDistance/2;
 					if (costRemaining<0)
 						costRemaining=0;
@@ -6769,7 +6769,7 @@ Path *Pathfinder::internalFindPath( Object *obj, const LocomotorSet& locomotorSe
 		dx = from->x - to->x;
 		dy = from->y - to->y;
 
-		Int count = WWMath::SqrtOrigin(dx*dx+dy*dy)/(PATHFIND_CELL_SIZE_F/2);
+		Int count = WWMath::SqrtOrigin((float)(dx*dx+dy*dy))/(PATHFIND_CELL_SIZE_F/2);
 		if (count<2) count = 2;
 		Int i;
 		color.green = 0;
@@ -7460,7 +7460,7 @@ Path *Pathfinder::findGroundPath( const Coord3D *from,
 		dx = from->x - to->x;
 		dy = from->y - to->y;
 
-		Int count = WWMath::SqrtOrigin(dx*dx+dy*dy)/(PATHFIND_CELL_SIZE_F/2);
+		Int count = WWMath::SqrtOrigin((float)(dx*dx+dy*dy))/(PATHFIND_CELL_SIZE_F/2);
 		if (count<2) count = 2;
 		Int i;
 		color.green = 0;
@@ -8164,7 +8164,7 @@ Path *Pathfinder::internal_findHierarchicalPath( Bool isHuman, const LocomotorSu
 		dx = from->x - to->x;
 		dy = from->y - to->y;
 
-		Int count = WWMath::SqrtOrigin(dx*dx+dy*dy)/(PATHFIND_CELL_SIZE_F/2);
+		Int count = WWMath::SqrtOrigin((float)(dx*dx+dy*dy))/(PATHFIND_CELL_SIZE_F/2);
 		if (count<2) count = 2;
 		Int i;
 		color.green = 0;

Core/GameEngine/Source/GameLogic/System/GameLogicDispatch.cpp

@@ -2302,6 +2302,14 @@ bool GameLogic::onSelfDestruct(MAYBE_UNUSED GameMessage *msg)
 		msgPlayer->killPlayer();
 	}
 
+	// ignore CRC messages when a player is defeated
+	// because there's a mismatch risk at low NET_CRC_INTERVAL values
+	// example: runahead decreases, clients are expected to send more frames
+	// defeated player sends some but not the required number of frames (and CRC messages);
+	// the game mismatches because it only retains the most recent CRC value
+	// and the CRC value from the defeated player is not up-to-date
+	m_shouldValidateCRCs = -1;
+
 	// There is no reason to do any notification here, it now takes place in the victory conditions.
 	// bonehead.
 
@@ -2406,7 +2414,8 @@ bool GameLogic::onLogicCrc(MAYBE_UNUSED GameMessage *msg)
 			if (!TheDebugIgnoreSyncErrors)
 			{
 #endif
-				m_shouldValidateCRCs = TRUE;
+				if (m_shouldValidateCRCs != -1)
+					m_shouldValidateCRCs = 1;
 #if defined(RTS_DEBUG)
 			}
 #endif

Core/GameEngine/Source/GameNetwork/Network.cpp

@@ -164,7 +164,11 @@ class Network : public NetworkInterface
 	virtual void attachTransport(Transport *transport) override;
 	virtual void initTransport() override;
 
+#if DEEP_CRC_TO_MEMORY
+	virtual void setSawCRCMismatch(const UnicodeString& strMismatchDetails) override;
+#else
 	virtual void setSawCRCMismatch() override;
+#endif
 	virtual Bool sawCRCMismatch() override { return m_sawCRCMismatch; }
 	virtual Bool isPlayerConnected( Int playerID ) override;
 
@@ -368,7 +372,11 @@ void Network::init()
 #endif
 }
 
+#if DEEP_CRC_TO_MEMORY
+void Network::setSawCRCMismatch(const UnicodeString& strMismatchDetails)
+#else
 void Network::setSawCRCMismatch()
+#endif
 {
 	m_sawCRCMismatch = TRUE;
 	// GeneralsX @build GitHubCopilot 12/04/2026 Surface mismatch UI activation in manual Linux/macOS captures.