Repository navigation
run_model_on_task: make avoid_duplicate_runs=False the default #1143
Description
Description
run_model_on_task has an option to avoid running experiments that already exist on OpenML, called avoid_duplicate_runs. This, however, requires an API key. It is currently the default, meaning that people can't try out this function without setting their API key.
This creates an unnecessary obstacle, especially for beginners who don't know that the avoid_duplicate_runs option can be switched off.
Steps/Code to Reproduce
from sklearn import ensemble
from openml import tasks, runs
clf = ensemble.RandomForestClassifier()
task = tasks.get_task(3954)
run = runs.run_model_on_task(clf, task)
Expected Results
The model should just run. The user may have no intention to upload the run to OpenML later.
Actual Results
An API key error is thrown
Versions
Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
Python 3.7.13 (default, Apr 24 2022, 01:04:09)
[GCC 7.5.0]
NumPy 1.21.6
SciPy 1.4.1
Scikit-Learn 1.0.2
OpenML 0.12.2
Activity
PGijsbers commentedon Jun 27, 2022
I had a closer look, and the problem is actually a
postrequest which should have been agetrequest:openml-python/openml/flows/functions.py
Line 256 in 99a62f6
With that fixed, you can run above code without an API key configured while still having the
avoid_duplicate_runsfunctionality (at the cost of some additional server roundtrips to check if the run exists).So now the question becomes, should we still prefer to have it turned off by default regardless?
PGijsbers commentedon Jun 27, 2022
Also @mfeurer
mfeurer commentedon Jun 27, 2022
That's an interesting question - maybe we can move this flag to the upload/publish function instead? It will serve the same purpose, but slightly improve the user experience as users can still run things without having to worry about duplicate runs.
PGijsbers commentedon Jun 28, 2022
The idea of having it here is that the user may avoid unnecessary computation since they can identify there's a duplicate before running the experiment and download the results instead (not so much avoiding duplicates on the server, or so I thought).
mfeurer commentedon Jul 5, 2022
Interesting, I thought it's to avoid duplicate stuff on the server. @joaquinvanschoren would you still like to remove that flag now that @PGijsbers has found a workaround?
Change default avoid_duplicate_runs to False
SubhadityaMukherjee commentedon Jun 17, 2025
Looks like this was solved.