Skip to content

VSCode

Microsoft VSCode is a common tool for software development, and can be used with Spartan.

Please note however that it can cause problems in its default configuration. First you will need to limit its behavior so that it does not get overloaded with the amount of files on the HPC system. Secondly, you must configure it to actually use Spartan's Slurm system. This will allow you to run directly on compute hardware, rather than on Spartan's login nodes... This is to prevent your processes being terminated when they exceed default limits, and to prevent user processes running into each other.

Index limiting

A few changes must be made to your settings.json file to ensure that VSCode does not try to index certain file locations, mostly because of their prohibitive sizes. If you do not do this step you may see VSCode become unresponsive as it attempts to read all the files in these locations and map them.

Important

These changes must be synced to "~/.vscode-server/data/Machine/settings.json" file on Spartan and not just in your desktop/laptop's settings.json. Choose the appropriate Remote SSH sync setting option(s) in VSCode to synchronize your exclude settings to the remote host.

Settings

"files.watcherExclude": {
  "**/.git/objects/**": true,
  "**/.git/subtree-cache/**": true,     
  "**/node_modules/*/**": true,
  "/apps/**": true,
  "/data/gpfs/**": true,
  "/data/projects/**": true,
  "/data/cephfs/**": true,
  "/data/scratch/**": true,
  "/scratch/**": true},

"search.followSymlinks": false,

"search.exclude": {     
  "**/.git/objects/**": true,
  "**/.git/subtree-cache/**": true,     
  "**/node_modules/*/**": true,
  "/apps/**": true,
  "/data/gpfs/**": true,
  "/data/projects/**": true,
  "/data/cephfs/**": true,
  "/data/scratch/**": true,
  "/scratch/**": true},

Connecting via slurm

You will first need to install the Remote SSH extension.

After Remote SSH is installed, press F1. Use the search interface to locate (Remote-SSH: Open SSH Configuration File) and add the following, replacing 'username' with your own Spartan account name.

Settings

Host spartanslurm
    HostName spartan.hpc.unimelb.edu.au
    User username
    RemoteCommand /apps/resplat/bin/vscode-shell-proxy.py --salloc-arg=--partition=interactive --salloc-arg=--time=10:00:00 --salloc-arg=--cpus-per-task=4
    RequestTTY yes

Important

Advanced users can alter the RemoteCommand --salloc-arg= settings to customise the Slurm session that is being requested. Note that requesting more resources may increase wait times when initiating your session. An example can be found below in the Using VSCode with GPU support section.

press F1 again, select Preferences: Open User Settings (JSON):

Settings

{
    "remote.SSH.useLocalServer": true,
    "remote.SSH.enableRemoteCommand": true,
    "remote.SSH.suppressWindowsSshWarning": true,
}

Important

You must alter the above file directly rather than using the VSCode Settings menu. There is a form entry at Settings>Extensions>Remote-SSH>useLocalServer that is currently not working in the Windows version of this plugin. Because of this and releated issues, you must also ake sure there are no entries in Settings>Extensions>Remote-SSH>RemotePlatform list.

  • Connecting to Spartan

    Press F1 and select Remote-SSH: Connect to Host. You will be presented with a list of locations, including the hostname set in your SSH configuration file. Assuming you have followed the above directions, you should see 'spartanslurm' in the list. Selecting this entry should start the connection process.

    After you have connected, open a terminal from the Terminal dropdown menu and enter the linux command hostname. You should see 'spartan-bm' suffixed with a number. If you instead see 'spartan-login' then something has gone wrong in the setup.

    Assuming you are connected correctly you should now be able to execute code on Spartan's compute nodes. If you are unable to connect as expected then please contact Spartan's admins.

Using Jupyter with VSCode

Jupyter is available as a VSCode extension. After installing it, you can create new Jupyter notebook instances and open existing ones.

It is highly recommended that you use Jupyter with a dedicated Anaconda environment. After creating or opening a Jupyter notebook you can select a 'kernel' by using the menu on the top right of the notebook window. This 'kernel' is any python installation with access to the ipykernel library. VSCode will prompt to automatically install the ipykernel library if it is not present.

Using VSCode with GPU support

Similarly to the [Connecting via Slurm] method, we will need to configue a remote host entry using the RemoteCommand parameter.

Settings

Host spartanslurmgpu
    HostName spartan.hpc.unimelb.edu.au
    User username
    RemoteCommand /apps/resplat/bin/vscode-shell-proxy.py --salloc-arg=--partition=gpu-a100 --salloc-arg=--time=10:00:00 --salloc-arg=--cpus-per-task=4 --salloc-args=--gpus=1
    RequestTTY yes

Important

It is highly recommended to increase your SSH timeout when using this method. If your connection attempt takes longer than your current SSH timeout (default 15 seconds) to launch an active shell session, your connection attempt will fail. SSH timeout can be found in VSCode's Settings->Extensions->RemoteSSH menu. We recommend raising this value to at least 90 seconds. Please note that if slurm cannot serve your current connection request (i.e. if you request too many resources, or simply if there is no GPU available to connect to), your connection will still fail.

Important

You MUST explicitly load CUDA drivers to allow VSCode to make use of your assigned GPU(s). This can be done by using a conda environment with the nvidia-cu* packages installed. A shortcut method for this is to install torch into your conda environment while using a GPU node, so that the Torch installer recognises that a GPU is present. Within a connected VSCode GPU enabled Spartan session, you can do: 

module load Anaconda3
eval "$(conda shell.bash hook)"
activate conda_env
pip install torch
We do not normally endorse the use of pip with conda due to problems between the package managers, but so far this works. Please note that when building custom software in your conda environment you may find that you still need to load a system level CUDA module to ensure that CUDA headers are available. The header files aren't included as part of the pip CUDA packages, but they are generally needed when building CUDA enabled software.