generate: autodetect number of jobs

[pjonsson]

Detect the number of cores in the machine,
but cap the default value to 16.

Cap only applies to the default value, so
if a user specifies --jobs 48, they will
get 48 workers.

---

📚 Documentation preview 📚: https://datacube-explorer--844.org.readthedocs.build/en/844/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.36%. Comparing base (5737151) to head (cf6246b).
⚠️ Report is 16 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #844   +/-   ##
========================================
  Coverage    84.36%   84.36%           
========================================
  Files           35       35           
  Lines         4253     4254    +1     
  Branches       537      537           
========================================
+ Hits          3588     3589    +1     
  Misses         469      469           
  Partials       196      196           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[omad]

I like the idea of setting this automatically, but unfortunately this isn't a suitable way of doing it.

• The number of concurrent processes is limited by the database server, which is very often not on the same machine as the python process.
• CPU Scheduler Affinity is a rarely used mechanism for restricting a process to one or more specific CPU cores, which is different from how much CPU time a process can use. If we did need to know how much processor time we could use in this process we'd want to use either https://docs.python.org/3.13/library/os.html#os.process_cpu_count (Python 3.13+ only) or dask.system.cpu_count