Add command to enable Cylc uiserver in Jupyter launcher

[ScottWales]

An enable command appears to be necessary to have the UI server installed alongside Jupyter lab, add this to the instructions.

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
• Tests are included (or explain why tests are not needed).
• Changelog entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[oliver-sanders]

Curious, we run Jupyter Lab and Cylc UI Server together at our site without doing this.

• cylc-uiserver 1.5.x
• jupyter_server: 2.14.x
• jupyterlab: 4.2.x
• jupyterhub: 5.0.x

If you are using Jupyter Hub to launch your servers, are you using the cylc hub command to launch it? Or have you otherwise applied the Cylc config?

I'm not sure what the jupyter server extension enable command actually does? Presumably it adds some code to a configuration file? But which configuration file I have no idea.

[ScottWales]

This is for when launching with jupyter lab or jupyter hub - NCI has an open on demand based system that will launch jupyter lab on a compute node and present it in a web browser, so I was seeing if I could add the Cylc gui to this.

Without the extension enable jupyter lab does not start Cylc:

(cylc-test) swales@Freya:~$ jupyter lab
[I 2024-09-05 11:14:46.561 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-09-05 11:14:46.565 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-09-05 11:14:46.568 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-09-05 11:14:46.745 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-09-05 11:14:46.759 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-09-05 11:14:46.760 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-09-05 11:14:46.761 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-09-05 11:14:46.774 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-09-05 11:14:46.774 ServerApp] Serving notebooks from local directory: /home/swales

With extension enable the Cylc server is started as well:

(cylc-test) swales@Freya:~$ jupyter server extension enable --py cylc.uiserver
Enabling: cylc.uiserver
- Writing config: /home/swales/miniconda3/envs/cylc-test/etc/jupyter
    - Validating cylc.uiserver...
      cylc.uiserver  OK
    - Extension successfully enabled.
(cylc-test) swales@Freya:~$ jupyter lab
[I 2024-09-05 11:16:20.460 ServerApp] cylc.uiserver | extension was successfully linked.
[I 2024-09-05 11:16:20.460 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-09-05 11:16:20.463 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-09-05 11:16:20.467 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-09-05 11:16:20.584 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-09-05 11:16:20.600 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-09-05 11:16:20.601 CylcUIServer] Starting Cylc UI Server
[I 2024-09-05 11:16:20.601 CylcUIServer] Serving UI from: /home/swales/miniconda3/envs/cylc-test/lib/python3.9/site-packages/cylc/uiserver/ui/2.5.0
[I 2024-09-05 11:16:20.603 ServerApp] cylc.uiserver | extension was successfully loaded.
[I 2024-09-05 11:16:20.604 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-09-05 11:16:20.605 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-09-05 11:16:20.617 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-09-05 11:16:20.618 ServerApp] Serving notebooks from local directory: /home/swales

The enable command creates a json file in envs/cylc-test/etc/jupyter/jupyter_server_config.d:

{
  "ServerApp": {
    "jpserver_extensions": {
      "cylc.uiserver": true
    }
  }
}

This doesn't pick up the same configuration as cylc hub, you'd need to look at the rest of the readme section first so I don't suggest distributing this configuration file by default (I guess you could using false to disable it by default but still show up in jupyter server extension list).

[oliver-sanders]

Right, so you are adding Cylc UI Server to an existing Jupyter Hub configuration, but you are not changing the Spawner.cmd to cylc hubapp.

And for some reason, the Cylc UI Server extension is blocked by default from Jupyter Lab, whereas the Jupyter Lab extension is enabled by default from Cylc UI Server?!

I would suggest documenting the jpserver_extensions config and linking to the Jupyter Server config page for more info. We haven't use the jupyter server command for config in other docs. I'm not sure how much sense it makes for multi-user setups (user might not have write access to the env, site might be using another config file path, config file load order), but I'm happy to be wrong about this.

Here's my suggestion, feel free to re-work.

We should really shift this into cylc-doc at some point and re-organise the Cylc UI Server setup docs. Note, Since Jupyter Hub v4 if you are opening up multi-user access but you are not using the Cylc Hub configuration (either via the cylc hub command or by adding it into an existing Jupyter Hub setup) then this section is important:

https://cylc.github.io/cylc-doc/stable/html/reference/config/sharing-access-to-workflows.html#jupyter-server-authorisation

[oliver-sanders]

Suggested change

	* If the `spawner.cmd` is something other than `cylc hubapp` (e.g. `jupyter lab`), then you will need to use the Jupyter Server [jpserver_extensions](https://jupyter-server.readthedocs.io/en/latest/other/full-config.html) configuration to enable the Cylc UI Server i.e. `jpserver_extensions = {"cylc.uiserver": true}`. The Cylc UI will be available by replacing `/lab` in the URL with `/cylc`.

[oliver-sanders]

Suggested change

	Enable the Cylc UI Server in `jupyterhub` or `jupyter lab` by running
	```bash
	jupyter server extension enable --py cylc.uiserver
	```
	The Cylc UI will be available by replacing `/lab` in the URL with `/cylc`.

I think this command will fail (or create a file in a user-specific location) in many situations because it requires the user to have write access to the deployment which they might not have. For transparency we aught to document the configuration and leave them to decide where best to configure it.

Note, this applies to Jupyter Lab not Jupyter Hub.