Start jupyter from within my application and access its variables and functions

[henfri]

Hello,

In a software I contribute to, the user can write its own logics. In order to debug those easily, an interactive python shell is available:

        shell = code.InteractiveConsole(locals())
        shell.interact()

https://github.com/smarthomeNG/smarthome/blob/3828810efa87c59994407125691a858882ded69a/bin/smarthome.py#L1282

I would now like to have similar functionality, but with a jupyter notebook. I found this:
#4873

But it seemingly starts a jupyter notebook which is empty just like running jupyter notebook.

So what's missing is that the functions and variables of the existing code is available in jupyter.

Is this forseen/possible?

Regards,
Hendrik

[henfri]

Hm, no ideas?

[bollwyvl]

If your software is running on the asyncio event loop, it's feasible to start not only a kernel, but also a notebook server, all inside the same process. The bummer is if anything blocks, ever, it will block everything. Threads _might_ solve this. If it's not in the event loop (or a different one, e.g. qt) it would be a lot harder. Getting a new server is pretty easy from inside an ipykernel. https://gist.github.com/bollwyvl/bd56b58ba0a078534272043327c52bd1 A lot of the complexity is around logging, but the first couple cells do the job. It would then need to start a kernel, and overload the kernel session machinery to service requests in-loop. I have no example of this, and the chance for deadlock becomes very real. Less hacky: wrap the other way, where you allow the notebook server to start as normal, and treat your application as a kernel. https://jupyter-client.readthedocs.io/en/stable/wrapperkernels.html I'd probably recommend that approach, as you can repurpose 95% of ipython, which is probably what you want, and work with all kinds of clients. If you're ready for jupyterlab 3, you might be able to use xeus-python and the debugger, which might be even better for your users. Whichever route you take, it would make a great blog post!