● Benchmarks
Measured against OR-Tools and Timefold.
Same instances, same time budget, every solution re-scored by the problem’s own rules. Where another solver wins or ties, the table says so.
26 / 32
best or tied
of the head-to-head instances
3
solvers
optimaru, Google OR-Tools, Timefold
30 s
per run
wall-clock budget for every solver
- optimaru
- OR-Tools
- Timefold
Vehicle routing — CVRPLIB Set X
Gap to the best-known solution, one median instance per size bucket (100–1000 customers). OR-Tools runs its Routing solver with guided local search; Timefold its open-source edition, which has no nearby selection. · lower is better
Planning problems — against Timefold’s own models
Twelve problems from the Timefold quickstarts. Timefold solves its own published model; optimaru and OR-Tools CP-SAT solve the same problem, and every solution is scored by Timefold’s constraint definitions. Values are the penalty on the score level that decides the row. · lower is better
| Problem | Instance | optimaru | OR-Tools CP-SAT | Timefold |
|---|---|---|---|---|
| Bed allocation soft | demo 77 stays / 13 beds / 10 rooms | 6,954 | 6,640 best | 6,990 |
| Bed allocation soft | medium 183 stays / 30 beds / 24 rooms | 13,686 best | 21,316 | 15,250 |
| Bed allocation soft | large 551 stays / 61 beds / 48 rooms | 34,556 best | no solution | 61,970 |
| Conference scheduling soft | demo Java quickstart demo conference | 54,420 tie | 54,420 tie | 61,665 |
| Facility location soft | demo 60 cons / 30 fac | 1,343,284 tie | 1,343,284 tie | 1,490,564 |
| Facility location soft | medium 200 cons / 50 fac | 3,144,802 | 3,099,950 best | 3,472,230 |
| Facility location soft | large 600 cons / 80 fac | 6,498,876 | 6,381,914 best | 8,050,744 |
| Flight crew scheduling soft | demo 62 seats / 50 crew / 14 flights | 23,000 tie | 23,000 tie | 38,000 |
| Flight crew scheduling soft | medium 120 seats / 50 crew / 28 flights | 25,000 best | 48,000 | 46,000 |
| Flight crew scheduling soft | large 236 seats / 50 crew / 56 flights | 35,000 best | 7,000 | 49,000 |
| Food packaging soft | demo Java quickstart demo plant | 95,221,284 best | no solution | 99,403,198 |
| Maintenance scheduling soft | small 14 jobs / 3 crews / 40 dates | 71 | 61 best | 63 |
| Maintenance scheduling soft | large 48 jobs / 5 crews / 80 dates | 3,472 | no solution | 3,467 best |
| Meeting scheduling soft | demo Java quickstart demo (24 meetings) | 3,040 best | no solution | 3,081 |
| Order picking soft | small 22 picks / 2 trolleys | 8,244 tie | 8,244 tie | 8,246 |
| Order picking soft | medium 56 picks / 4 trolleys | 20,478 best | 20,490 | 20,604 |
| Project job scheduling soft | demo 24 jobs / 2 projects / 7 res | 7 tie | 7 tie | 7 tie |
| Project job scheduling soft | medium 72 jobs / 4 projects / 13 res | 20 tie | 20 tie | 24 |
| Project job scheduling soft | large 156 jobs / 6 projects / 19 res | 35 | 28 best | 38 |
| Sports league scheduling soft | demo Java quickstart demo league | 111,868 best | 202,557 | 161,698 |
| Task assigning soft 3 | demo Java quickstart demo | 11,485 tie | 11,485 tie | 11,485 tie |
| Tournament scheduling medium | demo 72 slots / 7 teams / 18 days | 0 tie | 0 tie | 1.2 |
| Tournament scheduling soft | large 180 slots / 10 teams / 36 days | 0 best | 2 | 2.45 |
| Tournament scheduling soft | xlarge 800 slots / 20 teams / 100 days | 6.07 best | 109.64 | 9.74 |
School timetabling
Full school timetabling model, total weighted penalty with every hard rule met. · lower is better
Public instances
Classic sets from the literature, against OR-Tools CP-SAT.
| Problem | Measure | optimaru | OR-Tools CP-SAT |
|---|---|---|---|
| Pickup & delivery (synthetic Li & Lim, 200 requests) | vehicles · distance | 43 · 11,300 | no solution |
| Pickup & delivery (Li & Lim lc101, 10 requests) | distance | 156.0 tie | 156.0 |
| Project scheduling (Patterson Pat1) | makespan | 65 | 62 |
| Bin packing (t60_00, u120_04) | bins | 21 · 50 tie | 21 · 50 |
| Assembly line balancing (3 × 100 tasks) | stations | 26 · 25 · 29 | 23 · 22 · 25 |
How we measure
- Every solver gets the same instance and the same wall-clock budget. optimaru and Timefold run on one thread; OR-Tools CP-SAT gets 8 workers and its time includes building the model.
- Solutions are never taken at their word: each one is re-scored by an independent evaluator, and for the Timefold problems also by Timefold’s own constraint definitions. For every problem the two evaluators agree on optimaru’s solutions, so all three solvers optimise the same objective.
- Timefold Solver 2.6.0 (open source, default configuration) with the models of timefold-quickstarts v2.6.0; OR-Tools 9.15.
- A single run per instance and seed on one machine — a snapshot, not a statistical study. The harnesses are open source; re-run them on your own hardware.
Measured 2026-10-29 on one Apple Silicon Mac (16 cores), 30 s per solver per instance. Benchmark code on GitHub →