sc.external.pp.magic: what is n_jobs's default _really_?

[alexlenail]

scanpy docs:

n_jobs : Optional[int] (default: None)
Number of threads to use in training. All cores are used by default.

magic docs:

n_jobs (integer, optional, default: 1) – The number of jobs to use for the computation. If -1 all CPUs are used. If 1 is given, no parallel computing code is used at all, which is useful for debugging.

scanpy code:

scanpy/scanpy/external/pp/_magic.py

Line 164 in 536ed15

n_jobs = settings.n_jobs if n_jobs is None else n_jobs

scanpy/scanpy/_settings.py

Line 82 in 536ed15

n_jobs=1,

I'm guessing the scanpy docs are wrong when they say "All cores are used by default." ?

[jpmcga]

^^I've empirically confirmed this; been seeing multi-fold speedups when I set n_jobs=-1 corresponding to the number of cores, versus running with the default. From what I can tell, running without specifying n_jobs results in 1 core being used.

[flying-sheep]

Yeah, we definitely need to improve this. #2390 is also relevant here