Extend a single iteration of each workload?

[VStepany]

Is there a way to extend a single iteration of each workload?

The duration of each iteration is very short for performance profiling (JFR, Async, perf). I'm aware of the "--duration" knob of the Renaissance harness, however, this option simply stitches few iteration together to get the desired execution time.

What I need though is a longer execution time for each iteration in order to profile the workload.

thank you

[ceresek]

At this time, a generic way to scale all workloads is not available - this is something we are looking into (@lbulej will have more details). Eventually, we want to provide scaling along multiple axes (more parallelism, larger computations, etc.) but the individual options are not equally suitable for all workloads.

What kind of scaling exactly are you interested in ? If it is just doing the same work for longer, doing more repetitions (either with --repetitions or --run-seconds) should be almost exactly that (except for the forced GC between repetitions, which can be disabled with --no-forced-gc, and collecting the results, which should constitute a small share of the time).

Some workloads offer individual scaling options, for example with the Reactors workload you can do --override scaling_factor=x to get roughly x times longer repetitions, but this is workload-specific, so you would have to go through each workload individually, and some workloads do not have any scaling options (look for @Configuration in the sources).

[VStepany]

Hi @ceresek, thank you for the reply.
I need a longer iteration, not just few iterations stitched together.
Where do I look up information on each workload in order to see which of them support --override scaling_factor=x or similar? Quick search shows that only Reactors allows this particular option.
Looking through the src code of each workload?

Thank you.

[lbulej]

Many benchmarks have workload-specific parameters that have an impact on the duration of a repetition, but the duration might not respond linearly to simple proportional changes in the parameters. We are experimenting with scaling, but so far we were mostly focusing on scaling the workloads with available processors (and validate the results properly) and currently there is no generic tuning knob (with a linear scale) for the repetition duration.

I would not completely disregard the option of stitching the repetitions together (with disabled forced GC), even though I understand that you may be concerned about profile pollution from the setup/teardown code before/after each repetition. Unlike benchmark-level setup/teardown, this code is usually a tiny fraction of the measured operation. This leaves the validation code, which can be thought of as something that uses the results.

There is also the option of cleaning up the profile after collection — the harness provides information about the start time of each repetition, or controlling the profiler at runtime from a plugin that receives start/stop notifications from the harness, but this does not work with JFR or perf and I can understand that you want to avoid having to mess with the profiles. In that case, a way forward is either stitching more repetitions together or tweaking the benchmark parameters.

All benchmarks have a test configuration which usually sets benchmark-specific parameters to make the repetitions shorter (it is used during CI). Searching the sources in the benchmarks directory for @Configuration annotation will give you an idea which parameters to override to make the repetitions longer, but you will have to play with the parameter values to get the desired duration. However, you should expect that changing the parameters will probably break validation (unless you also provide the expected results) and there is currently no option to ignore validation failures.