Benchmark Results - Problem C3 - Initialization Strategies
Tested Initialization strategies:
name |
class |
params |
Constraint-aware |
|---|---|---|---|
fast |
InitFast | False | |
ros(u) |
InitRandomOneShot | uniform=True ignore_constraints=False |
True |
ros(nu) |
InitRandomOneShot | uniform=False ignore_constraints=False |
True |
ros(u,uncon) |
InitRandomOneShot | uniform=True ignore_constraints=True |
False |
ros(nu,uncon) |
InitRandomOneShot | uniform=False ignore_constraints=True |
False |
rb(2) |
InitRandomBatched | b=2 ignore_constraints=False |
True |
rb(4) |
InitRandomBatched | b=4 ignore_constraints=False |
True |
rb(8) |
InitRandomBatched | b=8 ignore_constraints=False |
True |
rb(16) |
InitRandomBatched | b=16 ignore_constraints=False |
True |
e(2) |
InitEager | nc=2 ignore_constraints=False |
True |
e(4) |
InitEager | nc=4 ignore_constraints=False |
True |
e(8) |
InitEager | nc=8 ignore_constraints=False |
True |
e(16) |
InitEager | nc=16 ignore_constraints=False |
True |
Time Duration
d |
n |
k |
m |
fast |
ros(u) |
ros(nu) |
ros(u,uncon) |
ros(nu,uncon) |
rb(2) |
rb(4) |
rb(8) |
rb(16) |
e(2) |
e(4) |
e(8) |
e(16) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 150 | 10 | 2 | 34.21 μsec ± 4.3% | 43.83 μsec ± 4.5% | 42.44 μsec ± 3.6% | 35.27 μsec ± 3.5% | 35.65 μsec ± 1.9% | 66.40 μsec ± 4.4% | 92.02 μsec ± 4.2% | 141.4 μsec ± 2.2% | 164.8 μsec ± 1.9% | 414.5 μsec ± 2.2% | 649.7 μsec ± 1.8% | 1.114 msec ± 0.6% | 2.080 msec ± 3.2% |
| 2 | 300 | 20 | 4 | 66.23 μsec ± 3.0% | 92.04 μsec ± 4.3% | 89.23 μsec ± 2.0% | 66.96 μsec ± 5.3% | 68.17 μsec ± 5.6% | 124.9 μsec ± 4.8% | 141.5 μsec ± 2.5% | 199.4 μsec ± 2.2% | 312.1 μsec ± 4.4% | 898.4 μsec ± 2.1% | 1.424 msec ± 1.8% | 2.506 msec ± 2.6% | 4.552 msec ± 1.8% |
| 3 | 450 | 30 | 6 | 109.3 μsec ± 4.9% | 159.4 μsec ± 2.6% | 158.6 μsec ± 1.8% | 104.4 μsec ± 3.5% | 105.5 μsec ± 3.5% | 189.8 μsec ± 2.2% | 214.4 μsec ± 4.0% | 274.4 μsec ± 1.2% | 399.0 μsec ± 4.1% | 1.447 msec ± 1.9% | 2.348 msec ± 1.7% | 4.136 msec ± 1.9% | 7.709 msec ± 0.8% |
| 4 | 600 | 40 | 8 | 153.0 μsec ± 1.4% | 254.4 μsec ± 6.4% | 259.5 μsec ± 9.8% | 159.4 μsec ± 19.1% | 151.5 μsec ± 10.2% | 320.2 μsec ± 6.8% | 324.0 μsec ± 5.9% | 383.3 μsec ± 4.1% | 537.5 μsec ± 7.2% | 2.138 msec ± 2.2% | 3.539 msec ± 2.9% | 6.275 msec ± 1.7% | 11.83 msec ± 1.7% |
| 6 | 900 | 60 | 12 | 231.4 μsec ± 1.9% | 525.4 μsec ± 3.8% | 524.2 μsec ± 4.8% | 279.0 μsec ± 16.7% | 258.9 μsec ± 12.6% | 591.6 μsec ± 3.2% | 643.7 μsec ± 6.5% | 696.5 μsec ± 6.0% | 867.4 μsec ± 4.4% | 3.896 msec ± 2.7% | 6.421 msec ± 1.5% | 11.38 msec ± 2.2% | 21.74 msec ± 1.6% |
| 8 | 1200 | 80 | 16 | 328.6 μsec ± 2.3% | 958.9 μsec ± 3.0% | 1.044 msec ± 7.2% | 374.6 μsec ± 8.5% | 410.8 μsec ± 10.4% | 1.090 msec ± 3.4% | 1.108 msec ± 5.9% | 1.245 msec ± 5.2% | 1.440 msec ± 4.8% | 6.111 msec ± 4.1% | 10.30 msec ± 2.6% | 18.65 msec ± 1.9% | 34.98 msec ± 1.7% |
| 12 | 1800 | 120 | 24 | 575.3 μsec ± 0.8% | 2.501 msec ± 2.8% | 2.511 msec ± 3.4% | 751.9 μsec ± 9.0% | 834.5 μsec ± 10.8% | 2.717 msec ± 3.8% | 2.795 msec ± 2.5% | 2.925 msec ± 4.0% | 3.047 msec ± 2.3% | 12.06 msec ± 2.1% | 21.29 msec ± 2.0% | 38.39 msec ± 2.5% | 73.94 msec ± 2.0% |
| 16 | 2400 | 160 | 32 | 844.8 μsec ± 0.9% | 5.108 msec ± 2.3% | 5.199 msec ± 1.5% | 1.408 msec ± 9.4% | 1.389 msec ± 6.9% | 5.336 msec ± 2.8% | 5.348 msec ± 1.0% | 5.683 msec ± 3.3% | 5.811 msec ± 1.4% | 20.75 msec ± 0.4% | 36.69 msec ± 0.9% | 68.68 msec ± 1.5% | 130.9 msec ± 1.3% |
| 24 | 3600 | 240 | 48 | 1.494 msec ± 1.6% | 14.87 msec ± 0.7% | 14.93 msec ± 0.6% | 2.657 msec ± 2.8% | 2.750 msec ± 0.4% | 15.17 msec ± 1.0% | 15.23 msec ± 0.7% | 15.48 msec ± 0.6% | 15.74 msec ± 0.9% | 48.72 msec ± 0.8% | 88.53 msec ± 0.4% | 169.2 msec ± 0.3% | 331.0 msec ± 0.5% |
| 32 | 4800 | 320 | 64 | 2.367 msec ± 0.2% | 33.77 msec ± 0.9% | 33.88 msec ± 0.5% | 5.138 msec ± 1.4% | 5.046 msec ± 1.0% | 34.89 msec ± 1.0% | 34.81 msec ± 0.7% | 35.02 msec ± 0.3% | 35.61 msec ± 0.1% | 99.40 msec ± 0.9% | 180.5 msec ± 0.5% | 350.9 msec ± 0.7% | 680.7 msec ± 0.5% |
| 48 | 7200 | 480 | 96 | 4.770 msec ± 0.2% | 106.6 msec ± 0.3% | 107.4 msec ± 0.3% | 13.76 msec ± 0.7% | 14.03 msec ± 2.7% | 111.4 msec ± 0.7% | 111.0 msec ± 0.4% | 112.0 msec ± 1.0% | 113.1 msec ± 0.4% | 291.6 msec ± 0.3% | 543.8 msec ± 0.6% | 1.054 sec ± 0.2% | 2.072 sec ± 0.2% |
| 64 | 9600 | 640 | 128 | 8.089 msec ± 0.7% | 246.2 msec ± 0.3% | 246.5 msec ± 0.2% | 25.00 msec ± 1.5% | 23.50 msec ± 3.2% | 266.9 msec ± 1.8% | 267.9 msec ± 0.1% | 264.4 msec ± 2.0% | 265.2 msec ± 1.7% | 672.5 msec ± 1.3% | 1.186 sec ± 0.9% | 2.783 sec ± 3.9% | 5.515 sec ± 4.1% |
| Geomean: | 489.9 μsec ± 1.8% | 2.214 msec ± 2.6% | 2.227 msec ± 2.9% | 723.0 μsec ± 6.7% | 727.3 μsec ± 5.7% | 2.546 msec ± 3.0% | 2.703 msec ± 2.8% | 3.064 msec ± 2.7% | 3.557 msec ± 2.8% | 11.69 msec ± 1.8% | 20.00 msec ± 1.5% | 37.22 msec ± 1.7% | 70.99 msec ± 1.6% |
Diversity Score
d |
n |
k |
m |
fast |
ros(u) |
ros(nu) |
ros(u,uncon) |
ros(nu,uncon) |
rb(2) |
rb(4) |
rb(8) |
rb(16) |
e(2) |
e(4) |
e(8) |
e(16) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 150 | 10 | 2 | 0.004 ± 0.0% | 0.146 ± 31.4% | 0.259 ± 28.8% | 0.104 ± 43.6% | 0.251 ± 28.0% | 0.235 ± 15.3% | 0.327 ± 24.2% | 0.402 ± 17.1% | 0.427 ± 10.6% | 0.496 ± 2.0% | 0.514 ± 2.2% | 0.506 ± 1.9% | 0.511 ± 1.1% |
| 2 | 300 | 20 | 4 | 0.081 ± 0.0% | 0.386 ± 7.9% | 0.529 ± 11.2% | 0.375 ± 10.8% | 0.536 ± 15.2% | 0.568 ± 13.0% | 0.746 ± 11.1% | 0.836 ± 10.8% | 0.870 ± 9.2% | 1.027 ± 4.5% | 1.080 ± 2.7% | 1.095 ± 2.3% | 1.075 ± 1.9% |
| 3 | 450 | 30 | 6 | 0.247 ± 0.0% | 0.668 ± 9.7% | 0.829 ± 8.6% | 0.642 ± 7.1% | 0.814 ± 5.1% | 0.927 ± 5.8% | 1.136 ± 7.2% | 1.201 ± 3.4% | 1.203 ± 6.1% | 1.426 ± 1.7% | 1.463 ± 1.8% | 1.482 ± 1.0% | 1.516 ± 1.6% |
| 4 | 600 | 40 | 8 | 0.324 ± 0.0% | 0.954 ± 5.1% | 1.129 ± 6.4% | 0.966 ± 5.1% | 1.054 ± 6.8% | 1.240 ± 3.5% | 1.395 ± 2.7% | 1.497 ± 2.8% | 1.554 ± 2.1% | 1.796 ± 2.2% | 1.868 ± 1.5% | 1.901 ± 1.4% | 1.909 ± 1.4% |
| 6 | 900 | 60 | 12 | 0.712 ± 0.0% | 1.481 ± 2.6% | 1.577 ± 2.3% | 1.441 ± 2.3% | 1.560 ± 2.2% | 1.829 ± 2.4% | 1.960 ± 2.4% | 2.019 ± 2.2% | 2.097 ± 2.3% | 2.336 ± 1.8% | 2.461 ± 1.1% | 2.531 ± 0.8% | 2.567 ± 0.9% |
| 8 | 1200 | 80 | 16 | 1.109 ± 0.0% | 1.971 ± 2.3% | 2.070 ± 2.4% | 1.881 ± 3.0% | 2.006 ± 3.1% | 2.288 ± 1.9% | 2.409 ± 2.5% | 2.530 ± 1.4% | 2.573 ± 1.9% | 2.848 ± 1.3% | 2.972 ± 1.0% | 3.067 ± 1.2% | 3.116 ± 1.0% |
| 12 | 1800 | 120 | 24 | 1.788 ± 0.0% | 2.713 ± 1.4% | 2.778 ± 1.4% | 2.687 ± 1.5% | 2.738 ± 0.8% | 3.045 ± 1.2% | 3.180 ± 1.2% | 3.253 ± 1.1% | 3.267 ± 0.7% | 3.656 ± 0.7% | 3.826 ± 1.0% | 3.902 ± 1.0% | 3.973 ± 0.7% |
| 16 | 2400 | 160 | 32 | 2.459 ± 0.0% | 3.459 ± 0.9% | 3.511 ± 1.0% | 3.383 ± 1.1% | 3.462 ± 0.9% | 3.739 ± 1.1% | 3.841 ± 1.2% | 3.928 ± 1.0% | 3.980 ± 0.9% | 4.282 ± 1.2% | 4.520 ± 0.6% | 4.630 ± 0.4% | 4.699 ± 0.5% |
| 24 | 3600 | 240 | 48 | 3.438 ± 0.0% | 4.622 ± 0.6% | 4.641 ± 0.7% | 4.510 ± 1.0% | 4.587 ± 0.7% | 4.829 ± 0.3% | 4.927 ± 0.4% | 4.915 ± 1.1% | 4.954 ± 0.6% | 5.363 ± 0.5% | 5.563 ± 0.1% | 5.676 ± 0.6% | 5.767 ± 0.5% |
| 32 | 4800 | 320 | 64 | 4.441 ± 0.0% | 5.641 ± 0.4% | 5.681 ± 0.3% | 5.475 ± 0.2% | 5.587 ± 0.8% | 5.843 ± 0.4% | 5.916 ± 0.3% | 5.956 ± 0.4% | 5.976 ± 0.5% | 6.233 ± 0.2% | 6.366 ± 0.5% | 6.574 ± 0.3% | 6.643 ± 0.4% |
| 48 | 7200 | 480 | 96 | 6.055 ± 0.0% | 7.305 ± 0.2% | 7.341 ± 0.1% | 7.168 ± 0.2% | 7.219 ± 0.1% | 7.496 ± 0.3% | 7.567 ± 0.3% | 7.594 ± 0.2% | 7.602 ± 0.2% | 7.857 ± 0.3% | 7.997 ± 0.2% | 8.077 ± 0.2% | 8.133 ± 0.1% |
| 64 | 9600 | 640 | 128 | 7.497 ± 0.0% | 8.797 ± 0.1% | 8.823 ± 0.1% | 8.624 ± 0.1% | 8.683 ± 0.1% | 9.006 ± 0.1% | 9.063 ± 0.1% | 9.101 ± 0.1% | 9.160 ± 0.1% | 9.433 ± 0.2% | 9.551 ± 0.1% | 9.679 ± 0.1% | 9.728 ± 0.0% |
| Geomean: | 0.801 ± 0.0% | 1.855 ± 5.5% | 2.091 ± 5.5% | 1.764 ± 5.8% | 2.050 ± 5.7% | 2.214 ± 3.7% | 2.440 ± 4.7% | 2.561 ± 3.5% | 2.611 ± 3.0% | 2.888 ± 1.4% | 2.994 ± 1.1% | 3.046 ± 0.9% | 3.077 ± 0.8% |
Constraint Score
d |
n |
k |
m |
fast |
ros(u) |
ros(nu) |
ros(u,uncon) |
ros(nu,uncon) |
rb(2) |
rb(4) |
rb(8) |
rb(16) |
e(2) |
e(4) |
e(8) |
e(16) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 150 | 10 | 2 | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 0.889 ± 14.1% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% |
| 2 | 300 | 20 | 4 | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 0.879 ± 3.4% | 0.924 ± 4.9% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% |
| 3 | 450 | 30 | 6 | 0.986 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 0.890 ± 4.0% | 0.890 ± 4.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% |
| 4 | 600 | 40 | 8 | 0.984 ± 0.0% | 1.000 ± 0.4% | 1.000 ± 0.4% | 0.880 ± 3.3% | 0.891 ± 3.2% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% |
| 6 | 900 | 60 | 12 | 0.976 ± 0.0% | 0.997 ± 0.3% | 0.997 ± 0.3% | 0.882 ± 1.3% | 0.891 ± 1.7% | 1.000 ± 0.2% | 1.000 ± 0.2% | 1.000 ± 0.2% | 1.000 ± 0.2% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% |
| 8 | 1200 | 80 | 16 | 0.971 ± 0.0% | 0.996 ± 0.1% | 0.996 ± 0.2% | 0.891 ± 0.9% | 0.905 ± 1.5% | 0.995 ± 0.2% | 0.996 ± 0.2% | 0.996 ± 0.1% | 0.996 ± 0.2% | 1.000 ± 0.1% | 1.000 ± 0.0% | 1.000 ± 0.0% | 1.000 ± 0.0% |
| 12 | 1800 | 120 | 24 | 0.960 ± 0.0% | 0.997 ± 0.1% | 0.995 ± 0.1% | 0.889 ± 0.9% | 0.889 ± 1.4% | 0.996 ± 0.1% | 0.996 ± 0.1% | 0.996 ± 0.2% | 0.996 ± 0.1% | 0.997 ± 0.1% | 0.999 ± 0.1% | 0.999 ± 0.0% | 1.000 ± 0.0% |
| 16 | 2400 | 160 | 32 | 0.938 ± 0.0% | 0.995 ± 0.1% | 0.996 ± 0.1% | 0.889 ± 0.5% | 0.895 ± 1.1% | 0.996 ± 0.1% | 0.997 ± 0.1% | 0.996 ± 0.1% | 0.996 ± 0.1% | 0.997 ± 0.1% | 0.999 ± 0.0% | 0.999 ± 0.0% | 1.000 ± 0.0% |
| 24 | 3600 | 240 | 48 | 0.939 ± 0.0% | 0.991 ± 0.2% | 0.991 ± 0.1% | 0.889 ± 0.5% | 0.892 ± 0.3% | 0.990 ± 0.1% | 0.993 ± 0.2% | 0.991 ± 0.1% | 0.991 ± 0.1% | 0.994 ± 0.1% | 0.996 ± 0.1% | 0.997 ± 0.0% | 0.998 ± 0.1% |
| 32 | 4800 | 320 | 64 | 0.930 ± 0.0% | 0.979 ± 0.2% | 0.980 ± 0.1% | 0.883 ± 0.2% | 0.887 ± 0.3% | 0.984 ± 0.2% | 0.983 ± 0.0% | 0.983 ± 0.2% | 0.984 ± 0.1% | 0.989 ± 0.1% | 0.994 ± 0.1% | 0.995 ± 0.0% | 0.996 ± 0.0% |
| 48 | 7200 | 480 | 96 | 0.921 ± 0.0% | 0.968 ± 0.2% | 0.968 ± 0.1% | 0.886 ± 0.2% | 0.887 ± 0.1% | 0.969 ± 0.0% | 0.972 ± 0.0% | 0.974 ± 0.1% | 0.973 ± 0.1% | 0.978 ± 0.1% | 0.984 ± 0.1% | 0.986 ± 0.0% | 0.987 ± 0.0% |
| 64 | 9600 | 640 | 128 | 0.919 ± 0.0% | 0.956 ± 0.0% | 0.958 ± 0.1% | 0.890 ± 0.2% | 0.890 ± 0.1% | 0.960 ± 0.1% | 0.962 ± 0.0% | 0.960 ± 0.0% | 0.963 ± 0.0% | 0.966 ± 0.0% | 0.970 ± 0.0% | 0.974 ± 0.1% | 0.974 ± 0.1% |
| Mean: | 0.960 ± 0.0% | 0.990 ± 0.1% | 0.990 ± 0.1% | 0.886 ± 2.5% | 0.904 ± 1.5% | 0.991 ± 0.1% | 0.991 ± 0.1% | 0.991 ± 0.1% | 0.992 ± 0.1% | 0.993 ± 0.1% | 0.995 ± 0.0% | 0.996 ± 0.0% | 0.996 ± 0.0% |