[VL] Fix GlutenRemoveRedundantSortsSuite in Spark 4.x

[zml1206]

No description provided.

[Copilot]

Pull request overview

This PR updates the Spark 4.x (Spark 4.0 and 4.1) Gluten unit tests for redundant sort removal so they validate Gluten’s columnar sort operator (SortExecTransformer) and re-enables the suite in Velox backend test settings.

Changes:

• Rewrote GlutenRemoveRedundantSortsSuite (Spark 4.0/4.1) to assert redundant-sort behavior by counting SortExecTransformer nodes.
• Added SparkConf override to disable forced SHJ for the suite to stabilize join-related sort expectations.
• Enabled GlutenRemoveRedundantSortsSuite in Spark 4.0/4.1 VeloxTestSettings, including all Gluten-prefixed tests.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenRemoveRedundantSortsSuite.scala	Replaces Spark SortExec-based assumptions with SortExecTransformer-based assertions for Spark 4.1.
gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala	Re-enables the redundant-sorts suite for Spark 4.1 Velox UTs and includes Gluten tests.
gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenRemoveRedundantSortsSuite.scala	Same test rewrite as Spark 4.1, adapted for Spark 4.0 module.
gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala	Re-enables the redundant-sorts suite for Spark 4.0 Velox UTs and includes Gluten tests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zml1206]

@zhouyuan @philo-he Should we disable tests that only verify optimization rules and are unrelated to Gluten? Examples include GlutenPlannerSuite, GlutenProjectedOrderingAndPartitioningSuite, GlutenRemoveRedundantProjectsSuite, GlutenRandomDataGeneratorSuite, GlutenSetCommandSuite, GlutenWholeStageCodegenSparkSubmitSuite, GlutenWholeStageCodegenSuite, GlutenExplainSuite, GlutenPythonDataSourceSuite, GlutenPythonUDFSuite, GlutenPythonUDTFSuite, and GlutenRowQueueSuite.