EXPERIMENT 01 / SELECTIVITY
Selective JAK2 / LCK
Find molecules that bind strongly to JAK2 while avoiding LCK, using a limited number of docking measurements.
- Library
- 259,931 molecules
- Paid budget
- 5,000 measurements / run
- Targets
- JAK2 + LCK
- Search seeds
- 0 / 1 / 2
THE EXPERIMENT
What are we looking for?
A good candidate needs both a favorable JAK2 score and a weak LCK interaction. We rank only molecules with both measurements; QED adds a drug-likeness penalty.
Maximize F = −JAK2 + min(LCK, −8.1) − 10 × (1 − QED)
DOCKSTRING provides recorded docking scores. Both methods are evaluated on the same canonical molecules and the same QED values.
FINAL QUALITY
Results at the full budget
Means give equal weight to seeds 0 / 1 / 2. Sample SD describes variation across the 3 recorded runs of each method. Bold marks the higher mean.
| Metric | MolScout | Mobley EI Separate | Difference | Ratio |
|---|---|---|---|---|
| Recall@100 | 88.00%± 1.00 pp SD | 70.67%± 1.53 pp SD | +17.33 pp | 1.25× |
| Recall@250 | 88.27%± 0.46 pp SD | 66.27%± 0.92 pp SD | +22.00 pp | 1.33× |
| Recall@500 | 83.47%± 0.81 pp SD | 60.73%± 0.64 pp SD | +22.73 pp | 1.37× |
| Recall@1000 | 76.63%± 0.35 pp SD | 52.33%± 0.40 pp SD | +24.30 pp | 1.46× |
| AUC@100 | 0.595± 0.004 SD | 0.250± 0.013 SD | +0.345 | 2.38× |
| AUC@250 | 0.581± 0.009 SD | 0.228± 0.008 SD | +0.353 | 2.55× |
| AUC@500 | 0.512± 0.010 SD | 0.194± 0.007 SD | +0.318 | 2.64× |
| AUC@1000 | 0.420± 0.005 SD | 0.154± 0.004 SD | +0.266 | 2.73× |
Mean discovery curves
Compare the methods for the top 100, 250, 500, or 1,000 molecules.
Recall
Percentage of the fixed top K molecules discovered with all required measurements. Higher is better.
Top 100
Top 250
Top 500
Top 1,000
AUC
Average fractional Recall@K over the measurements made so far, from 0 to 1. Higher values mean earlier discovery.
Top 100
Top 250
Top 500
Top 1,000
INDIVIDUAL RESULTS
Individual seed results
Each run uses 5,000 charged measurements. Bold marks the higher value; higher is better for both metrics.
| Top K | MolScout | Mobley EI Separate |
|---|---|---|
| 100 | 87.00% | 69.00% |
| 250 | 88.00% | 65.20% |
| 500 | 84.20% | 60.00% |
| 1000 | 76.60% | 51.90% |
| Top K | MolScout | Mobley EI Separate |
|---|---|---|
| 100 | 89.00% | 71.00% |
| 250 | 88.80% | 66.80% |
| 500 | 83.60% | 61.20% |
| 1000 | 77.00% | 52.40% |
| Top K | MolScout | Mobley EI Separate |
|---|---|---|
| 100 | 88.00% | 72.00% |
| 250 | 88.00% | 66.80% |
| 500 | 82.60% | 61.00% |
| 1000 | 76.30% | 52.70% |
| Top K | MolScout | Mobley EI Separate |
|---|---|---|
| 100 | 0.593 | 0.235 |
| 250 | 0.571 | 0.219 |
| 500 | 0.510 | 0.187 |
| 1000 | 0.414 | 0.150 |
| Top K | MolScout | Mobley EI Separate |
|---|---|---|
| 100 | 0.594 | 0.261 |
| 250 | 0.589 | 0.235 |
| 500 | 0.523 | 0.194 |
| 1000 | 0.425 | 0.152 |
| Top K | MolScout | Mobley EI Separate |
|---|---|---|
| 100 | 0.599 | 0.254 |
| 250 | 0.582 | 0.229 |
| 500 | 0.503 | 0.200 |
| 1000 | 0.421 | 0.159 |
MEASUREMENT EFFICIENCY
Budget to match the reference
Each target level is the reference method’s final mean. The recorded crossing is the first budget that reaches that level. Budget reduction compares MolScout’s crossing with the full measurement budget.
| Metric | Reference final level | MolScout crossing | Reference first crossing | Full budget | Budget reduction |
|---|---|---|---|---|---|
| Recall@100 | 70.67% | 2,803 | 4,894 | 5,000 | 43.9% |
| Recall@250 | 66.27% | 2,502 | 4,946 | 5,000 | 50.0% |
| Recall@500 | 60.73% | 2,561 | 4,946 | 5,000 | 48.8% |
| Recall@1000 | 52.33% | 2,837 | 4,997 | 5,000 | 43.3% |
| AUC@100 | 0.250 | 1,520 | 5,000 | 5,000 | 69.6% |
| AUC@250 | 0.228 | 1,662 | 5,000 | 5,000 | 66.8% |
| AUC@500 | 0.194 | 1,781 | 5,000 | 5,000 | 64.4% |
| AUC@1000 | 0.154 | 1,956 | 5,000 | 5,000 | 60.9% |
HOW WE COUNT
Measurements, recall, and AUC
- One measurement
- One charged molecule–target score. Initialization, repeated calls, and unavailable evaluations keep the charges recorded by the experiment.
- Recall@K
- The fraction of the fixed top-K molecules discovered so far. A molecule must have every score required by the objective. Recall is displayed as a percentage.
- Running AUC@K
- Average fractional recall over the paid budget, using left-step integration. Values range from 0 to 1; a higher value means earlier discovery.
- Mean and SD
- Means give equal weight to seeds 0 / 1 / 2. Sample SD describes variation across the 3 recorded runs of each method.
- Budget percentage
- 100 × charged measurements / canonical library molecules. DUDE-Z uses the per-target budget.
- Budget reduction
- 100 × (1 − MolScout crossing budget / full budget). An unreached threshold is reported as an outcome, without extrapolation.
AUC@K(B) = sum of fractional Recall@K(b), for b = 0 … B−1, divided by B
REFERENCE