Quickstart

To illustrate neurtu usage, will will benchmark array sorting in numpy. First, we will generator of cases,

import numpy as np
import neurtu

def cases()
    rng = np.random.RandomState(42)

    for N in [1000, 10000, 100000]:
        X = rng.rand(N)
        tags = {'N' : N}
        yield neurtu.delayed(X, tags=tags).sort()

that yields a sequence of delayed calculations, each tagged with the parameters defining individual runs.

We can evaluate the run time with,

>>> df = neurtu.timeit(cases())
>>> print(df)
        wall_time
N
1000     0.000014
10000    0.000134
100000   0.001474

which will internally use timeit module with a sufficient number of evaluation to work around the timer precision limitations (similarly to IPython’s %timeit). It will also display a progress bar for long running benchmarks, and return the results as a pandas.DataFrame (if pandas is installed).

By default, all evaluations are run with repeat=1. If more statistical confidence is required, this value can be increased,

>>> neurtu.timeit(cases(), repeat=3)
       wall_time
            mean       max       std
N
1000    0.000012  0.000014  0.000002
10000   0.000116  0.000149  0.000029
100000  0.001323  0.001714  0.000339

In this case we will get a frame with a pandas.MultiIndex for columns, where the first level represents the metric name (wall_time) and the second – the aggregation method. By default neurtu.timeit is called with aggregate=['mean', 'max', 'std'] methods, as supported by the pandas aggregation API. To disable, aggregation and obtains timings for individual runs, use aggregate=False. See neurtu.timeit documentation for more details.

To evaluate the peak memory usage, one can use the neurtu.memit function with the same API,

>>> neurtu.memit(cases(), repeat=3)
        peak_memory
               mean  max  std
N
10000           0.0  0.0  0.0
100000          0.0  0.0  0.0
1000000         0.0  0.0  0.0

More generally neurtu.Benchmark supports a wide number of evaluation metrics,

>>> bench = neurtu.Benchmark(wall_time=True, cpu_time=True, peak_memory=True)
>>> bench(cases())
         cpu_time  peak_memory  wall_time
N
10000    0.000100          0.0   0.000142
100000   0.001149          0.0   0.001680
1000000  0.013677          0.0   0.018347

including [psutil process metrics](https://psutil.readthedocs.io/en/latest/#psutil.Process).

For more information see the Examples.