[PERF]: Epic for binding overhead improvements #1645
Description
This issue is tracking performance improvements and investigations to Python-to-C binding overhead, mostly driven by the benchmark of cuTensorMapEncodeTiled devised in #659. That is a useful benchmark because it is a function with an unusually high number of arguments (and therefore unusually high Python-to-C overhead).
Comparison to a more limited Cython binding
As an interesting experimental datapoint, a colleague provided a vibe-coded Cython binding for cuTensorMapEncodeTiled that runs about 4x faster than cuda-bindings official one. It is useful to see where some overheads may be reduced, but care should be taken looking at its raw performance: this wrapper accepts far fewer things as inputs than the CUDA bindings, and doesn't include developer niceties, like enums.
Merged or in-progress fixes
Timings below are per-iteration of the benchmark in #659. This includes /both/ binding overhead and some fixed amount of time in the actual CUDA call.
- 4.80us Baseline time
- 3.63us Use faster method to get enum values #1543
- 2.73us Statically-allocate small arrays of numbers #1545
- 2.70us Faster enum implementation (Pure Python version) #1581
- 2.59us [PERF]: Faster void * conversion #1616
- 2.38us [PERF]: Improve speed of returning the result pair #1638
Under investigation
Issues in this category are theoretical findings to reduce the operations required for type conversion, but haven't necessarily yet been confirmed to have a measurable effect.
- [PERF]: Remove redundant type checks #1639
- [PERF]: Handle generic sequences more efficiently #1640
- [PERF]: Converting arguments of generic int or fixed-size int (e.g. cuuint32_t) are suboptimal #1642