pyls acceptance test failing on Ubuntu with Python 3.6 on master

[krassowski]

This was not an issue at the release time, but I noticed this on #459 and then re-triggered master to see if this is related to the PR or not (its not):

• Failure (re-triggerred): https://github.com/krassowski/jupyterlab-lsp/runs/1668978098
• Non-failure: https://github.com/krassowski/jupyterlab-lsp/runs/1657936485

All robot tests (except for Smoke and non-pyls tests) fail with:

Setup failed:
Element 'css:div.lsp-statusbar-item' did not get text 'Fully initialized' in 1 minute.

Also teardown failed:
Phrases found in /home/runner/work/jupyterlab-lsp/jupyterlab-lsp/atest/output/linux_36_4/lab.log: {'pyls_jsonrpc.endpoint - Failed to handle request': True}

All other versions seem ok.

Failing now:

python-language-server                         0.36.2  pyhd8ed1ab_0         conda-forge/noarch       Cached
python-jsonrpc-server                           0.4.0  pyh9f0ad1d_0         conda-forge/noarch       Cached

Non-failing (before):

python-language-server                         0.36.2  pyhd8ed1ab_0         conda-forge/noarch       Cached
python-jsonrpc-server                           0.4.0  pyh9f0ad1d_0         conda-forge/noarch       Cached

[krassowski]

Of note utest passes.

• Also, we need to update the badge in the readme.

[bollwyvl]

Smells like a missing build somewhere... will do some investigating of the versions... there's been a lot of churn in the last few days.

[krassowski]

Conda:

25c25
< certifi                   2020.12.5        py36h5fab9bb_0    conda-forge
---
> certifi                   2020.12.5        py36h5fab9bb_1    conda-forge
27d26
< chardet                   4.0.0            py36h5fab9bb_0    conda-forge
30c29
< coverage                  5.3.1            py36h8f6f2f9_0    conda-forge
---
> coverage                  5.3.1            py36h8f6f2f9_1    conda-forge
56c55
< idna                      2.10               pyh9f0ad1d_0    conda-forge
---
> idna                      3.1                pyhd3deb0d_0    conda-forge
72c71
< jupyter_server            1.1.4            py36h5fab9bb_0    conda-forge
---
> jupyter_server            1.2.0            py36h5fab9bb_0    conda-forge
75c74
< jupyterlab_server         2.0.0              pyhd8ed1ab_0    conda-forge
---
> jupyterlab_server         2.1.0              pyhd8ed1ab_0    conda-forge
80,81c79,80
< libblas                   3.9.0                6_openblas    conda-forge
< libcblas                  3.9.0                6_openblas    conda-forge
---
> libblas                   3.9.0                7_openblas    conda-forge
> libcblas                  3.9.0                7_openblas    conda-forge
93c92
< liblapack                 3.9.0                6_openblas    conda-forge
---
> liblapack                 3.9.0                7_openblas    conda-forge
181c180
< r-diffobj                 0.3.2             r36h9e2df91_1    conda-forge
---
> r-diffobj                 0.3.3             r36hcfec24a_0    conda-forge
205c204
< r-matrix                  1.3_0             r36he454529_0    conda-forge
---
> r-matrix                  1.3_2             r36he454529_0    conda-forge
251c250
< r-xfun                    0.19              r36h9e2df91_0    conda-forge
---
> r-xfun                    0.20              r36hcfec24a_0    conda-forge
257c256
< requests                  2.25.1             pyhd3deb0d_0    conda-forge
---
> requests                  2.13.0                   py36_0    conda-forge
267c266
< setuptools                49.6.0           py36h9880bd3_2    conda-forge
---
> setuptools                49.6.0           py36h5fab9bb_3    conda-forge
282c281
< tornado                   6.1              py36h1d69622_0    conda-forge
---
> tornado                   6.1              py36h8f6f2f9_1    conda-forge

[blink1073]

This may be related to jupyterlab/jupyterlab#9579

[krassowski]

Thanks for the heads up! Though the browsers does start for us currently. I was wrong to write that it fails for all tests - it fails for all pyls-tests (so vast majority). The browser does - at least for now - start for us:

If there is another connection, it might be beyond my current understanding.

[bollwyvl]

pyls, ipykernel, probably others (jupyter_client 😱) are presently very sensitive to jedi versioning...

[krassowski]

We have a pin for jedi already. Something downgraded requests from 2.25.1 to 2.13.0:

257c256
< requests                  2.25.1             pyhd3deb0d_0    conda-forge
---
> requests                  2.13.0                   py36_0    conda-forge

I pinned:

  - requests >=2.25.1
  - jupyterlab_server <=2.0.0
  - jupyter_server <=1.1.4

and triggered a run (after enabling workflow_dispatch) here: https://github.com/krassowski/jupyterlab-lsp/actions/runs/472872755 - it seems to pass again. I will now check which one of those causes trouble (I hope it's requests).

Edit: it is not requests.

[krassowski]

Pinning jupyterlab_server==2.0.0 fixes the issue, even when running with jupyter_server==1.2.1.

[krassowski]

So the jupyterlab_server 2.0 vs 2.1 seem to differ only by one commit jupyterlab/jupyterlab_server@eae924e which is translations.

Looking at the logs from terminal for the failure I find warnings coming from R:

1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_COLLATE failed, using "C" 
3: Setting LC_TIME failed, using "C" 
4: Setting LC_MESSAGES failed, using "C" 
5: Setting LC_MONETARY failed, using "C" 
6: Setting LC_PAPER failed, using "C" 
7: Setting LC_MEASUREMENT failed, using "C" 

and errors from pyls:

2021-01-08 13:55:23,265 UTC - ERROR - pyls_jsonrpc.endpoint - Failed to handle request 0
Traceback (most recent call last):
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls_jsonrpc/endpoint.py", line 113, in consume
    self._handle_request(message['id'], message['method'], message.get('params'))
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls_jsonrpc/endpoint.py", line 182, in _handle_request
    handler_result = handler(params)
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls_jsonrpc/dispatchers.py", line 23, in handler
    return method(**(params or {}))
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/python_ls.py", line 209, in m_initialize
    processId, _kwargs.get('capabilities', {}))
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/config/config.py", line 68, in __init__
    self._update_disabled_plugins()
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/config/config.py", line 154, in _update_disabled_plugins
    plugin for name, plugin in self.plugin_manager.list_name_plugin()
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/config/config.py", line 155, in <listcomp>
    if not self.settings().get('plugins', {}).get(name, {}).get('enabled', True)
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/config/config.py", line 129, in settings
    source_conf = source.project_config(document_path or self._root_path)
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/config/pycodestyle_conf.py", line 29, in project_config
    files = find_parents(self.root_path, document_path, PROJECT_CONFIGS)
  File "/usr/share/miniconda3/envs/test/lib/python3.6/site-packages/pyls/_utils.py", line 76, in find_parents
    existing = list(filter(os.path.exists, [os.path.join(search_dir, n) for n in names]))
  File "/usr/share/miniconda3/envs/test/lib/python3.6/genericpath.py", line 19, in exists
    os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 78: ordinal not in range(128)

It looks that there are some filenames added that have non-ascii characters which causes some of the pyls plugins to go awry when looking for config files (pcodestyle checks ['pycodestyle.cfg', 'setup.cfg', 'tox.ini']). They should not be touching these files in the first place, but their directory traversing code seems to do that.

Questions:

• what files is causing this (I am unable to replicate locally)
• what dependency is causing this file to materialize?
• what is the connection with jupyterlab_server?
• Edit: why it only happens on 3.6 and why only on Ubuntu?

[krassowski]

Over to you @bollwyvl, I am out of ideas (and cannot reproduce locally).

[bollwyvl]

Catching up now... semi-related to #348 i guess, as git diff is a fair bit nicer than having to scrape logs!

[bollwyvl]

I wonder if our PYTHONIOENCODING=utf-8 is getting clobbered someplace that we can easily fix (e.g. when starting the robot server)... it might be falling back to a strange locale...