May 27, 2023
I think in the case where optimizations significantly affect the run time, but have some variability, that’s really a case where you need to be analysing the results, in which case neither a simple min or mean will do.
I’d be very wary of using the max without analysing the data first (in a lossless way).