Kubernetes setup and JupyterHub services

[betatim]

This looks cool! Two things I've been thinking about since discovering this:

• Have you thought about what it would take to make this also work for a kubernetes based setup?
• Would it make sense to run this as a JupyterHub service instead of hooking into the JupyterHub server directly?

I don't know the answers to the above questions so if you have some thoughts that would be great.

(If you have a good idea for a more descriptive title let me know, this isn't an ideal one.)

[giovannipizzi]

Thanks @betatim! Happy to know this is useful also for others.

To answer (even if partially) to your answers:

1. unfortunately we don't have a kubernetes cluster yet so we are not experienced enough to understand what it would take to extend. Indeed, this is (at this stage "on purpose", i.e. we stop if the spawner is not the expected one) limited to dockerspawner on the same machine. I was already starting to think how to extend this for e.g. docker-swarm or anyway for notebooks running on different machines.
   I think that one simply should do the following:

• tell the proxy to connect to a port to the IP address/machine where the docker is actually running (now we hardcode localhost in the piece of code we inherited from nbserverproxy)
• make sure the VM on which the dockers are running exposes these ports inside the network so the jupyterhub machine can see them (both the VM has the ports open, and when the docker images are launched they open the ports on 0.0.0.0 or at least on a general-enough interface to be visible outside the VM).
  From my limited understanding of kubernetes, I imagine this would work also in kubernetes if it is set up correctly to open ports etc.

2. Sure, just it's the first time I'm looking into JupyterHub (and I had to learn tornado, coroutines, ... :-) ) and I wasn't aware of JupyterHub services. If you have links/pointers to docs for them, I can give a look on how to change the code.

In general (maybe solved by point 2) I also would like to remove as much as possible hooking into the JupyterHub code. Beside how to write this, there are a couple of things that still bug me and maybe you have suggestions:

• is it possible to store the token without having to inherit/extend the DockerSpawner class? I don't like the idea of doing it, not only from a 'code cleanliness' point of view but also because if we want to support different spawners we need to start extending all of them.

• change a bit the dockerspawner code to avoid that we need to redefine variables, see e.g. my comment here:
  

  jhproxy/examples/jupyterhub_config.py

  Lines 26 to 35 in 4de55ac

  	# For how the DockerSpawner currently works, if we have to redefine
  	# explicitly the jupyter port inside the container if we define
  	# additional ports, otherwise JupyterHub will not be able to connect
  	# to the jupyter notebooks in the spawned containers
  	# Also use_internal_ip=False is neeed, otherwise port_bindings are overwritten
  	jupyter_port = 8888
  	host_ip = '127.0.0.1'
  	c.DockerSpawner.port = jupyter_port
  	c.DockerSpawner.use_internal_ip = False
  	c.DockerSpawner.host_ip = host_ip

  (I'm referring specifically to these lines, that overwrite the port bindings, while I think they could just extend them): https://github.com/jupyterhub/dockerspawner/blob/10d3a373370de4d1edfb6e9a092524d2ff768f31/dockerspawner/dockerspawner.py#L731

Probably, for the second issue, I can just open an issue to dockerspawner? For the first one instead I have no idea.

I'm happy to get any feedback on this - I just needed it and wrote it to make it work - then I thought it could be useful to more people and tried to make it general in a git repo by itself, but I know there are a few things that can still be improved!