OSError: [Errno 29] Illegal seek for jobs>1 & --output-format teamcity.pylint_reporter.TeamCityReporter

[cs3180]

Bug description

I have tried with different versions of pylint & dill. Issue can be reproduced in every one of them  -  The issue arises because dill is attempting to serialize an object that includes a file-like object or stream that does not support seeking. This is triggered when using pylint with the --output-format option set to teamcity.pylint_reporter.TeamCityReporter in a multi-job (--jobs) configuration.  
Root Cause
pylint uses dill to serialize the linter state for multiprocessing when --jobs is greater than 1.
The teamcity.pylint_reporter.TeamCityReporter output format likely involves a non-seekable stream, which dill cannot handle.


$ touch aa.py
$ python -m pylint --jobs 2 --output-format teamcity.pylint_reporter.TeamCityReporter aa.py

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\__main__.py", line 10, in <module>
    pylint.run_pylint()
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\__init__.py", line 36, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\lint\run.py", line 213, in __init__
    linter.check(args)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\lint\pylinter.py", line 681, in check
    check_parallel(
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\lint\parallel.py", line 145, in check_parallel
    max_workers=jobs, initializer=initializer, initargs=(dill.dumps(linter),)
                                                         ^^^^^^^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 263, in dumps
    dump(obj, file, protocol, byref, fmode, recurse, **kwds)#, strictio)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 235, in dump
    Pickler(file, protocol, **_kwds).dump(obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 394, in dump
    StockPickler.dump(self, obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 487, in dump
    self.save(obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 717, in save_reduce
    save(state)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 1186, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 972, in save_dict
    self._batch_setitems(obj.items())
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 998, in _batch_setitems
    save(v)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 717, in save_reduce
    save(state)
    self._batch_setitems(obj.items())
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 998, in _batch_setitems
    save(v)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 1336, in save_file
    f = _save_file(pickler, obj, open)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 1310, in _save_file
    position = obj.tell()
               ^^^^^^^^^^
io.UnsupportedOperation: seek

Configuration

works for --jobs set to 1

Error is readily reproduced for multi-job and output-format set to teamcity.pylint_reporter.TeamCityReporter

Command used

$ touch aa.py
$ python -m pylint --jobs 2 --output-format teamcity.pylint_reporter.TeamCityReporter aa.py

Pylint output

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\__main__.py", line 10, in <module>
    pylint.run_pylint()
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\__init__.py", line 36, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\lint\run.py", line 213, in __init__
    linter.check(args)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\lint\pylinter.py", line 681, in check
    check_parallel(
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\pylint\lint\parallel.py", line 145, in check_parallel
    max_workers=jobs, initializer=initializer, initargs=(dill.dumps(linter),)
                                                         ^^^^^^^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 263, in dumps
    dump(obj, file, protocol, byref, fmode, recurse, **kwds)#, strictio)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 235, in dump
    Pickler(file, protocol, **_kwds).dump(obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 394, in dump
    StockPickler.dump(self, obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 487, in dump
    self.save(obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 717, in save_reduce
    save(state)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 1186, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 972, in save_dict
    self._batch_setitems(obj.items())
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 998, in _batch_setitems
    save(v)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 717, in save_reduce
    save(state)
    self._batch_setitems(obj.items())
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 998, in _batch_setitems
    save(v)
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "REDACTED\python\win\Vpy311np124-502\Lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 1336, in save_file
    f = _save_file(pickler, obj, open)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "REDACTED\python\win\Vpy311np124-502\Lib\site-packages\dill\_dill.py", line 1310, in _save_file
    position = obj.tell()
               ^^^^^^^^^^
io.UnsupportedOperation: seek

Expected behavior

not OSError

Pylint version

pylint=2.16.3  , 3.3.5
dill = 0.3.6   , 0.4.0
python=3.11.5

OS / Environment

both Windows (10) and unix RHEL

Additional dependencies

[cs3180]

Same command works on my older setup:
python=3.6.7
pylint=2.2.2
no dill installed