Comet 1.0.0 Release

[andygrove]

What is the problem the feature request solves?

I would like to start planning ahead for a 1.0.0 release. Let's use this issue to discuss what that means.

Current planned items:

• Demonstrated cost savings for TPC-H and TPC-DS at SF1000 (1TB)
• Consistent logging
• Thorough documentation for compatibility status
• Review all configuration options and consider renaming some for consistency
• Documented policy for how long we will support Spark versions
• Documented process for preventing major performance regressions
• Documented process for versioning (semver) moving forward. What does that mean for Comet?

Nice to have:

• Implement more aggregate expressions
  • feat: support approx_count_distinct aggregate expression #4819
  • feat: support grouping() and grouping_id() indicator functions #4815
  • feat: support listagg / string_agg aggregate (Spark 4.0+) #4816
  • feat: support max_by and min_by aggregate expressions #4817
  • feat: support Spark HyperLogLog sketch functions (hll_sketch_agg, hll_union_agg, hll_sketch_estimate, hll_union) #4802
  • feat: support approx_percentile / percentile_approx aggregate #4801
  • feat: support native aggregate function mode #4782
  • feat: support PivotFirst aggregate for the optimized PIVOT fast path #4798
  • feat: support kurtosis aggregate #4818
  • feat: implement regr_slope, regr_intercept, regr_r2, regr_sxx, regr_syy, regr_sxy aggregates #4775
• Implement more window functions

Describe the potential solution

No response

Additional context

No response

[parthchandra]

I'd like to see iceberg metadata columns support make it #4082 (depends on a bunch of PRs in iceberg-rust though).