Benchmarking

To run a benchmark, one can use the command line interface PyVRP provides. After installation, it is available as pyvrp from the command line. To find out about available options, run

pyvrp --help

To benchmark pyvrp, we mostly use instances from CRVPLIB instances. Particularly:

• For the CVRP, we use the X-instances of Uchoa. Each instance has \(n\) clients; the runtimes are computed as \(2.4 n\) seconds on a reference CPU with PassMark score 2183. Thus, an instance with 100 clients is ran for 240 seconds, assuming a CPU with PassMark score 2183.

• For the VRPTW, we use the Gehring and Homberger instances, particularly those with 1000 customers. These instances are each run for two hours on a reference CPU with PassMark score 2000.

• For the PC-VRPTW, we use the modified Gehring and Homberger instances with 1000 customers. These instances are each run for two hours on a reference CPU with PassMark score 2000.

The time limit should be scaled by the PassMark score of your CPU. Each instance is run ten times with different seeds. Each run is performed on a single core. For each instance, we take the average objectives/gaps of these ten runs with different seeds.

To get the benchmark instance sets quickly, the PyVRP/Instances repository may be initialised as a submodule:

git submodule init instances

After running this command, the instances will be available in instances/.