Search documentation… Ctrl K
esc
Kuwait Institute for Scientific Research

IKARUS HPC: User Documentation

Your complete guide to the IKARUS High-Performance Computing cluster: the IKARUS portal, interactive applications, job submission, and hardware specifications.

57
Compute Nodes
3
Partitions
SLURM
Scheduler
13
Interactive Apps
Live
Monitoring

What is IKARUS?

IKARUS is KISR's High-Performance Computing (HPC) cluster, giving researchers and engineers shared access to large-scale computational resources. It is accessible through two methods:

  • IKARUS Portal: browser-based access at khpc.kisr.edu.kw/ood/. No software installation required. Recommended for most users. This documentation is linked from the portal's Help menu.
  • SSH (terminal): traditional command-line access via hpc.kisr.edu.kw for advanced or automated workflows.

Through the portal you can browse and transfer files, submit and monitor batch jobs, open a browser terminal, and launch fully-featured applications like JupyterLab, RStudio, and MATLAB all without leaving your browser.

Quick Access

Getting Started

Accessing the IKARUS Portal

The IKARUS portal gives you full access to the cluster from any web browser. No SSH client or special software required.

System Requirements

RequirementDetails
BrowserChrome 90+, Firefox 88+, Edge 90+, or Safari 14+. Chrome or Edge recommended.
JavaScriptMust be enabled (on by default in all browsers).
Pop-upsAllow pop-ups from khpc.kisr.edu.kw, interactive app sessions open in new tabs.
CookiesMust be enabled for login session to be maintained.

Logging In

  1. Open your browser and go tohttps://khpc.kisr.edu.kw/ood/
  2. The IKARUS login page loads. Enter your SSH username and password provided by IKARUS administration.
  3. If it is your first time, you are asked to add Two-Factor Authentication to an authenticator app of your choice. Otherwise, enter the current code from your authenticator app.
  4. Click "Log In".
  5. On success you are redirected to the IKARUS Portal dashboard.
⚠
Do not share your credentials. Every user must log in with their own account. Sessions are tied to your identity for audit and resource tracking.

Dashboard Tour

Top Navigation Bar

The navigation bar is present on every page of the portal. It contains six menus:

MenuWhat it does
FilesDrop-down to open the file browser. Home Directory takes you to /NFS/scratch/homes/<username>.
JobsTwo sub-items: Active Jobs (live queue view) and Job Composer (script editor and submission).
ClustersContains IKARUS Shell Access: a browser-based terminal directly on the login node.
Interactive AppsLists all available applications: Desktop, JupyterLab, RStudio, MATLAB...etc.
ToolsAdditional research tools integrated into the portal, such as MLFlow for experiment tracking.
Help / LogoutRight side of the bar. Access documentation links and the logout option.
My Interactive Sessions

The main body of the dashboard shows the My Interactive Sessions panel: cards for all your active or recently completed interactive app sessions. From here you connect to running sessions, check their status, and delete completed ones.

A Delete All Sessions button clears all completed/failed session records in one click, keeping the dashboard tidy.

💡
Bookmark khpc.kisr.edu.kw/ood/ for quick access. Your session stays active while the browser is open; after a period of inactivity you will be prompted to log in again.
Files

File Browser

Manage files on cluster storage directly in your browser. No SFTP client needed for everyday transfers.

✓
Your home directory is /NFS/scratch/homes/<username>. This is the default location when you open the file browser. Store your scripts, data, and results here.

Navigating Directories

  1. In the top bar click Files → Home Directory. The file browser opens to your home directory.
  2. The path bar at the top shows your current location. Click any segment to jump there, or type a path and press Enter.
  3. Click a folder name in the listing to enter it. Use the ← back button or path bar to navigate up.

Show Hidden Files

Toggle Show Dotfiles in the toolbar to reveal files beginning with . (such as .bashrc and application config files).

Filter / Search

Use the Filter box in the toolbar to narrow the file listing by name within the current directory.

Uploading Files

Method 1: Drag and Drop

  1. Navigate to the destination directory in the file browser.
  2. Drag one or more files from your local file manager and drop them into the portal file browser window.
  3. A progress indicator appears. Wait until each file shows 100% before navigating away.

Method 2: Upload Button

  1. Navigate to the destination directory.
  2. Click "Upload" in the toolbar.
  3. Select files in the file picker that appears. Uploads begin immediately.
⚠
For very large files (multiple GB), the browser upload may time out. Use SFTP (WinSCP, FileZilla, or sftp) connected to hpc.kisr.edu.kw for large data transfers instead.

Downloading Files

  1. Navigate to the directory containing the files you want.
  2. Check the checkbox to the left of each file or folder you want to download.
  3. Click "Download" in the action toolbar that appears at the top. Files are saved to your browser's downloads folder.
💡
When downloading a directory, the portal packages it as a .zip archive first. For large directories, compress with tar in the shell terminal first, then download the single archive.

Create, Rename, Copy, Delete

New Directory

  1. Navigate to where you want the new folder.
  2. Click "New Dir" in the toolbar.
  3. Type the name and press Enter.

New File

  1. Navigate to the target directory.
  2. Click "New File" in the toolbar.
  3. Type the filename (including extension, e.g. job.sh) and press Enter.

Actions on Existing Files

Check the checkbox next to any file or directory to reveal the action toolbar:

ActionWhat it does
EditOpen the file in the built-in web editor (text files only)
Rename/MoveEnter a new name or path to rename or move the item
Copy/MoveDuplicate or relocate selected files to a destination you specify
DeletePermanently remove selected files/directories. Cannot be undone.
DownloadDownload selected items to your computer
⚠
There is no recycle bin. Deleted files are gone permanently. Double-check your selection before clicking Delete.

File Editor

The built-in web editor lets you edit job scripts, configuration files, and any text file directly in the browser.

  1. Check the checkbox next to the file, then click "Edit" or click the filename directly for text files.
  2. The editor opens in a new browser tab with syntax highlighting (detected from the file extension).
  3. Make your changes, then press "Ctrl+S" (or click Save) to save back to the cluster.
  4. Close the tab when done.
💡
For a full IDE experience with auto-complete, debugging, and extensions, use the VS Code interactive app instead.
Jobs

Active Jobs

A live, real-time view of all jobs in the SLURM queue across the entire cluster.

Click Jobs → Active Jobs in the navigation bar. Click Refresh to update, or enable Auto-refresh to poll automatically.

Reading the Queue

ColumnDescription
Job IDUnique SLURM identifier. Used with scancel, sacct, and sstat.
Job NameName set by --job-name in the script.
UserOwning user. You can see all users but can only cancel your own jobs.
PartitionQueue the job was submitted to (Res, def1, Dev).
StateRUNNING active · PENDING waiting · COMPLETED done · FAILED error · COMPLETING cleaning up
TimeElapsed run time for running jobs; wait time for pending jobs.
NodesNumber of nodes allocated.
Node List (Reason)Allocated node names, or why the job is pending (e.g. Resources, Priority).

Filtering

Use the Filter field to search by name, user, or ID. Toggle Your Jobs to show only your own activity.

Cancelling Jobs

  1. Find your job. Confirm the "User" column matches your username.
  2. Click the red trash icon to the right of the row.
  3. Confirm in the dialog that appears.
  4. The job moves to CANCELLED state and is removed from the queue.
⚠
You can only cancel your own jobs. Contact the IKARUS administrators if another user's job is causing problems.
Jobs

Job Composer

A graphical SLURM script editor and submission tool. Create, edit, save, and submit job scripts without using the terminal.

Access via Jobs → Job Composer. The page shows saved jobs on the left and a detail/editor panel on the right.

Writing a Job Script

Creating a New Job
  1. Click + New Job then choose "From Default Template".
  2. A new job entry appears. Click it to open the detail panel.
  3. Click "Open Editor" in the detail panel. The script opens in a new tab.
  4. Write your #SBATCH directives and commands. Save with "Ctrl+S".
  5. Close the editor tab and return to the Job Composer.
💡
Scripts are stored under your home directory in ondemand/data/sys/myjobs/ and can also be edited directly from the file browser.
Example Job Script
#!/bin/bash

######## SLURM Options ########
#SBATCH --job-name=my_analysis
#SBATCH --partition=Res
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=8
#SBATCH --mem=32G
#SBATCH --time=02:00:00
#SBATCH --output=%j_output.log
#SBATCH --error=%j_error.log
######## End of Options ########

module load python/3.10
cd /NFS/scratch/homes/${USER}/my_project
python3 analysis.py --input data.csv --output results/

Submitting a Job

  1. Select the job from the list.
  2. Optionally change the "Submit Directory", click the folder icon to browse.
  3. Click the green "Submit" button.
  4. On success a banner shows the assigned "Job ID". Monitor in Jobs → Active Jobs.
⚠
A red error banner on failure shows the SLURM error. Common causes: invalid partition, requesting more memory or cores than available, or a script syntax error.

Using Templates

Save as Template

  1. Select a working job, then click "Copy to Templates" (star icon).
  2. Give it a descriptive name (e.g. Python 8-core 32G Res) and save.

Create from Template

  1. Click + New Job → From Template.
  2. Select your template. A copy is created and you can edit specific parameters before submitting.
💡
Build a template library for your most common job types (serial Python, multi-core R, MPI...etc.). This saves time and eliminates directive typos.
Shell Access

Browser Terminal

A full Linux terminal inside your browser, connected directly to the IKARUS login node. No SSH client needed.

Opening the Terminal
  1. Click Clusters → IKARUS Shell Access in the navigation bar.
  2. A new tab opens with a full terminal emulator. You are automatically logged in as your IKARUS user on clavis.
  3. Prompt confirms connection:
[hpcdemo@clavis2 ~]$ 
What You Can Do
  • Navigate the filesystem and manage files
  • Load environment modules (module avail, module load)
  • Submit and monitor SLURM jobs (sbatch, squeue, scancel)
  • Start an interactive compute session with srun
  • Compile code and run quick tests
  • Transfer files with scp or rsync
squeue -me                                    # your jobs only
sbatch my_job.sh                              # submit a batch job
srun --partition=Res --cpus-per-task=4 \
     --mem=8G --time=01:00:00 bash -l        # interactive compute session
module avail                                  # list available modules
du -sh /NFS/scratch/homes/${USER}            # check disk usage
💡
Use right-click → Copy to copy selected text (Ctrl+C sends an interrupt to the running process). Paste with right-click → Paste or Ctrl+Shift+V.
⚠
The login node is shared: do not run heavy computations here. Submit all intensive work to compute nodes via sbatch or srun.
Interactive Apps

Interactive Applications

Thirteen applications running on real compute nodes, accessible in your browser with SLURM-allocated resources.

Desktop
Full Linux desktop via VNC
JupyterLab logo
JupyterLab
Notebooks · Python · R
RStudio logo
RStudio
IDE for R statistics
MATLAB logo
MATLAB
Numerical computing
ParaView logo
ParaView
Scientific visualisation
QGIS logo
QGIS
GIS & geospatial analysis
VS Code logo
VS Code
Full IDE in the browser
SIESTA logo
SIESTA
DFT electronic structure
Quickplot logo
Quickplot
Delft3D output visualisation
Delft3DFM logo
Delft3DFM
Ocean & coastal simulation
FVCOM logo
FVCOM
Coastal ocean circulation
WRF-Chem logo
WRF-Chem
Atmospheric chemistry model
OSRA logo
OSRA
Oil spill risk analysis

Launching an App

  1. Click Interactive Apps in the header and select the application you want.
  2. A launch form appears. Fill in the resource fields.
  3. Click "Launch".
  4. You are redirected to the My Interactive Sessions dashboard. A session card appears.
  5. Wait for the card to show "Running" (blue header).
  6. Click "Connect to [App Name]" to open the application in a new tab.
💡
If the app does not open in a new tab, check that pop-ups are allowed for khpc.kisr.edu.kw in your browser settings.

Resource Request Form

FieldDescriptionGuidance
PartitionSLURM queue. Options shown depend on your group membership.Use Res for standard work. Shorter time requests start sooner.
Number of HoursMax session duration. Session is terminated when this expires.Request only what you need, launch a new session if more time is needed.
Number of CPU CoresCores dedicated to the session.4–8 for most interactive work. Increase for parallel code.
Memory (GB)RAM allocated to the session.Match to your largest expected dataset. See individual app pages.
App-specific optionsSome apps have extra fields (e.g. conda environment, version).See the individual app page.

Session States

Queued / Starting

Waiting for SLURM to allocate a node, or the app is initialising.

Action: Wait typically 30 sec to a few minutes.

Running

Resources allocated, app is ready.

Action: Click the Connect button to open the application.

Completed / Failed

Session ended: time limit reached, you closed it, or an error occurred.

Action: Delete the card or review logs.

Managing Sessions

Reconnecting to a Running Session

If you close the app tab, the session continues running. Return to the IKARUS Portal dashboard, find your Running session card, and click "Connect" again.

Ending a Session

  1. Quit the application normally (e.g. File → Quit in MATLAB, close the Jupyter tab).
  2. Return to the IKARUS Portal dashboard. The session card should show "Completed".
  3. Click the "Delete" (trash) button on the card.
⚠
Closing your browser tab does not end the session, it continues running until the time limit expires, consuming resources. Always delete sessions you are no longer using.

Delete All Sessions Button

Removes all completed and failed session cards at once. Running sessions must be deleted individually after ending the application.

💡
Occasionally a session card may fail to delete with many errors. This is a known portal limitation, you can safely ignore it and go back. The underlying compute job has still ended; the card is only a display record.

Multiple Sessions

You can run multiple interactive sessions simultaneously (e.g. JupyterLab and a Desktop session at the same time), subject to your resource quota.

Interactive Apps

Linux Desktop (VNC)

A full graphical XFCE desktop running on a compute node, streamed to your browser via VNC. Ideal for GUI-based workflows and running graphical applications interactively.

Launch Options
FieldRecommendedNotes
PartitionResChoose based on your group access
Hours1–4 hExtend if you need longer work sessions
CPU Cores4–8Increase for apps you run inside the desktop
Memory8–32 GBIncrease for memory-intensive graphical apps
Using the Desktop
  1. Click Connect to Desktop on the session card. The desktop opens via noVNC in a new tab.
  2. An XFCE desktop is shown. Right-click the desktop for a context menu, or use the panel at the top.
  3. Open a Terminal Emulator (Applications → Terminal Emulator, or right-click → Open Terminal). Load modules and launch any application from here.
  4. Launch GUI apps with an ampersand to keep the terminal free: matlab &, paraview &

noVNC Controls

ControlAction
Ctrl+Alt+ShiftOpen the noVNC side panel (clipboard, settings, fullscreen toggle)
Clipboard (noVNC panel)Paste text from your local machine into the remote desktop
Fullscreen (noVNC panel)Enter/exit fullscreen mode
Scroll wheelScroll within the VNC window
💡
To copy text out of the VNC desktop to your local clipboard: select it in the VNC window, open the noVNC panel (Ctrl+Alt+Shift), and copy from the clipboard field there.
⚠
Unsaved work is lost when the time limit expires. Save frequently and monitor remaining time on the session card.
Interactive Apps

JupyterLab

Interactive notebook-based computing with Python, R, and other kernels. Run code, visualise results, and document your analysis in the browser.

Launch Options
FieldRecommendedNotes
PartitionRes
Hours2–8 hJupyter sessions can run long; plan ahead
CPU Cores4–16Increase for parallel Python or heavy data processing
Memory16–64 GBMatch to your largest expected dataset or array
Using JupyterLab
  1. After connecting, JupyterLab opens. The left panel shows a file browser rooted at your home directory. The right panel shows the Launcher.
  2. Click a kernel tile in the Launcher (e.g. Python 3) to open a new notebook, or use File → Open from Path to open an existing .ipynb file.
  3. Write code in cells and press Shift+Enter to run. Output appears directly below.
  4. Use the file browser (left panel) to navigate directories and open additional files.

Available Kernels

IKARUS provides six shared pre-configured kernels accessible to all users from /NFS/shared/libraries/jupyter/kernels/. These appear in the JupyterLab Launcher automatically; no installation or configuration required.

KernelLanguageNotes
Python 3Python 3 (Anaconda base)NumPy, SciPy, pandas, Matplotlib, scikit-learn and the full Anaconda scientific stack pre-loaded
RR 4.5.3 (IRkernel)Full R environment; install additional CRAN packages with install.packages() in a notebook cell
BashBash shellRun shell commands and SLURM submissions directly in notebook cells; useful for pipeline documentation
PyTorch (CPU)Python 3 + PyTorchtorch, torchvision, torchaudio (CPU-only build). Use for model development and inference without GPU allocation
TensorFlow (CPU)Python 3 + TensorFlowtensorflow-cpu (CPU-only build). Suitable for training small models and running inference on the compute nodes
pyPDAF 1.0.4Python 3 + pyPDAF + mpi4pyPython interface to the PDAF Parallel Data Assimilation Framework. Supports all PDAF filters via MPI (Intel MPI 2021.12). Import submodules directly: from pyPDAF import PDAF, PDAF3, PDAFomi
ℹ
The PyTorch, TensorFlow, and pyPDAF kernels are CPU-only; IKARUS compute nodes do not have GPUs. For GPU workloads, contact the administrators to discuss options.

Installing Packages

# In a notebook cell: installs to your home directory
!pip install --user package_name

# For R:
install.packages("ggplot2")
⚠
Packages installed with !pip install --user may not persist across different kernel environments. For reproducible work, request a dedicated Conda kernel from the administrators.
💡
JupyterLab includes a terminal (File → New → Terminal): a full shell on the compute node for running scripts, loading modules, and managing files.
Interactive Apps

RStudio

Full RStudio Server IDE for R statistical computing, running on a compute node and accessed in your browser.

Launch Options
FieldRecommendedNotes
PartitionRes
Hours2–8 h
CPU Cores4–8Increase for parallel R with parallel or future packages
Memory16–64 GBR loads data entirely into RAM; match to dataset size
Using RStudio
  1. After connecting, RStudio opens with four panels: Source (top-left), Console (bottom-left), Environment/History (top-right), Files/Plots/Help (bottom-right).
  2. Use the Console for interactive R commands, or open/create .R scripts in the Source panel.
  3. Set your working directory: setwd("/NFS/scratch/homes/username/my_project"), or use File → New Project.
  4. Browse cluster files in the Files tab (bottom-right).

Installing R Packages

install.packages("ggplot2")
install.packages(c("dplyr", "tidyr", "data.table"))

Packages install to your personal R library in your home directory and persist across sessions.

💡
Use RStudio Projects (.Rproj files): each project sets the working directory automatically and maintains a separate history and environment.
Interactive Apps

MATLAB

The full MATLAB desktop running on a compute node and displayed via VNC. Suitable for numerical computation, signal processing, image analysis, and simulations.

Launch Options
FieldRecommendedNotes
PartitionRes
Hours2–8 h
CPU Cores8–16Increase for Parallel Computing Toolbox usage
Memory32–128 GBAllocate enough for your largest arrays and datasets
Using MATLAB
  1. After connecting, the MATLAB desktop opens in VNC: Command Window, Workspace, and file browser are visible.
  2. Use the Command Window for interactive commands, or open .m files in the MATLAB Editor.
  3. Set your working directory: cd('/NFS/scratch/homes/username/my_project')
  4. Run scripts with the green Run button or by typing the script name.

Parallel Computing

% Create a local parallel pool using the cores you requested
parpool('local', 8);

% Use parfor for parallelised loops
parfor i = 1:100
    result(i) = my_function(i);
end
💡
If the VNC display feels slow, open the noVNC panel (Ctrl+Alt+Shift) and reduce the image quality setting for a more responsive session.
Interactive Apps

ParaView

Open-source scientific visualisation for large simulation and experimental datasets. Runs with software rendering on the compute node, displayed via VNC.

Launch Options
FieldRecommendedNotes
PartitionRes
Hours1–4 h
CPU Cores8–16More cores enable parallel rendering and processing
Memory32–128 GBLarge meshes require significant RAM; size to your dataset
Using ParaView
  1. After connecting, ParaView opens in VNC with its Pipeline Browser, Properties panel, and 3D render view.
  2. Open data: File → Open. Supported formats include VTK, EnSight, OpenFOAM, netCDF, HDF5, Exodus, CGNS, and more.
  3. Click Apply in the Properties panel to load and render the data.
  4. Use the Pipeline Browser to add filters (Clip, Threshold, Contour, Warp, Streamlines…).
  5. Navigate the 3D view: left-drag to rotate, middle-drag to pan, scroll to zoom.
💡
For datasets that exceed single-node RAM, contact the IKARUS administrators about a client-server ParaView configuration with multiple render servers.
Interactive Apps

QGIS

Professional GIS application for viewing, editing, and analysing geospatial data. Runs in a VNC session on the compute node.

Launch Options
FieldRecommendedNotes
PartitionRes
Hours2–6 h
CPU Cores4–8QGIS processing algorithms can use multiple cores
Memory16–64 GBIncrease for large rasters or complex vector datasets
Using QGIS
  1. After connecting, QGIS opens in VNC with its map canvas, Layers panel (left), and toolbars.
  2. Add data via Layer → Add Layer or drag files from the QGIS Browser panel. Supported: GeoTIFF, Shapefile, GeoPackage, WMS/WFS/WCS, PostGIS, and more.
  3. Use the Processing Toolbox (Processing → Toolbox) for hundreds of spatial analysis algorithms.
  4. Store your GIS data in /NFS/scratch/homes/username/ for best performance.
💡
For large rasters, pre-tile with GDAL before loading into QGIS: gdaladdo adds pyramid overviews that dramatically speed up rendering at different zoom levels.
Interactive Apps

VS Code (Code Server)

Visual Studio Code in your browser, connected directly to the cluster filesystem. Full IDE: IntelliSense, debugging, integrated terminal, Git, and extensions, all running on the compute node.

Launch Options
FieldRecommendedNotes
PartitionRes or DevDev is ideal for development/testing workflows
Hours4–8 h
CPU Cores4–8Increase if running code in the integrated terminal
Memory8–32 GBMatch to the data your code will process
Using VS Code
  1. After connecting, VS Code opens in a new browser tab, identical to the desktop application.
  2. Click File → Open Folder and navigate to your project at /NFS/scratch/homes/username/my_project.
  3. Edit files from the Explorer panel (left sidebar) with full syntax highlighting and IntelliSense.
  4. Open the Integrated Terminal (Ctrl+` or View → Terminal). This is a full shell on the compute node: load modules, run code, and submit jobs without leaving VS Code.

Recommended Extensions

Press Ctrl+Shift+X to browse and install. Extensions store in your home directory and persist across sessions:

  • Python: IntelliSense, linting, debugging
  • Jupyter: Run .ipynb notebooks inside VS Code
  • R: R language support and LSP integration
  • GitLens: Enhanced Git history, blame, and diff views
  • Remote-SSH: Connect to the cluster from your local VS Code outside the portal
💡
The integrated terminal runs on the compute node: you can module load, run scripts, and sbatch additional jobs all from within VS Code. It is the most complete all-in-one development environment on IKARUS.
Interactive Apps

SIESTA

SIESTA is a density functional theory (DFT) code for electronic structure calculations and ab initio molecular dynamics. The app runs your SIESTA calculation as an MPI job and then automatically opens XCrySDen to visualise the result, all inside the same VNC session.

⚠
The XCrySDen visualisation step currently does not launch successfully; this is a known limitation pending GPU hardware on IKARUS. SIESTA itself runs and completes normally regardless: your output files are written correctly and can be downloaded from the file browser even while this is unresolved.
Launch Options
FieldRecommendedNotes
Input FDF fileFull NFS pathYour main SIESTA input file. Must already exist on the cluster before launching, see below.
Pseudopotential / working directoryFull NFS pathA directory containing your pseudopotential files (*.psf or *.psml) and any other auxiliary input files
Nodes11–10 supported
MPI tasks per node4–20SIESTA's solver requires the orbital count to be at least the MPI task count; scale down for small test systems (see tip below)
Memory8–32 GB1–350 GB supported
PartitionRes
Hours1–4 h1–72 h supported
Staging Your Files

SIESTA does not accept file uploads through the launch form directly. Instead, you stage your files on the cluster first, then point the form at their paths:

  1. Upload your .fdf input file and all pseudopotential files to your home directory using the file browser.
  2. In the launch form, enter the full path to your .fdf file (e.g. /NFS/scratch/homes/username/my_calc/input.fdf).
  3. Enter the full path to the directory containing your pseudopotential files. Everything in that directory is copied into the job's working directory automatically, so pseudopotentials and any other auxiliary files can live together.
💡
SIESTA matches pseudopotential files to species automatically by filename, based on the species labels in your %block ChemicalSpeciesLabel. There's no need to name anything explicitly in the form beyond pointing to the directory.
Using SIESTA
  1. After launching, the session runs SIESTA as a blocking MPI job for the duration of your calculation.
  2. On completion, your output files (.out, .XV, and others) are written to a job-specific working directory under your home directory.
  3. Retrieve your results via the file browser, or open a terminal in the VNC session to inspect them directly.
💡
For small test or tutorial-scale systems, SIESTA's solver can fail with "too many processors for the system size" if MPI tasks exceed the orbital count. Reduce MPI tasks per node for small systems.
Interactive Apps

Quickplot

Quickplot (Deltares) is a visualisation and post-processing tool for Delft3D model output, supporting both structured and unstructured grid results. It opens as a standalone graphical application in a VNC session.

Launch Options
FieldRecommendedNotes
PartitionRes
Hours1–4 h
Memory8–16 GBIncrease for very large output files
Using Quickplot
  1. Stage your Delft3D output files on the cluster in advance, using the file browser.
  2. After connecting, Quickplot opens directly in the VNC session.
  3. Use Quickplot's own file open dialog to browse to and load your staged output file.
  4. Use Quickplot's plotting tools to inspect structured or unstructured grid results.
💡
Quickplot is also launched automatically by the Delft3DFM and FVCOM apps when you select it as your post-processing visualiser, so you often won't need to open it directly.
Interactive Apps

Delft3DFM

Delft3DFM is an ocean and coastal simulation pipeline. It runs your simulation across multiple compute nodes in the background while giving you a live desktop session to monitor progress, and launches Quickplot automatically to visualise results once the run finishes.

Launch Options
FieldRecommendedNotes
Model folderFull NFS pathMust contain dimr_config.xml and your dflowfm/<name>.mdu file. See requirements below.
MPI typeDepends on your buildMPICH or Intel MPI
Container versionLatest available
Nodes / tasks per nodeDepends on model sizeMulti-node simulations are supported
Memory16–64 GBMatch to your model's grid size
PartitionRes
HoursDepends on simulation length
Model Folder Requirements
File / FolderRequiredNotes
dimr_config.xmlYesMust exist at the model root
dflowfm/<name>.mduYesFilename is read automatically from dimr_config.xml
dflowfm/*.pli, *.nc, *.ext, *.bc, *.xynVariesReferenced internally by your MDU file as needed
dflowfm/output/RecommendedCreate this directory before running, or the simulation will error
Using Delft3DFM
  1. Prepare your model folder on the cluster with all required files, using the file browser.
  2. Fill in the launch form with your model folder path and desired resources, then launch.
  3. A desktop session opens with a live terminal showing simulation progress.
  4. On successful completion, Quickplot launches automatically for immediate visualisation.
  5. If the simulation fails, an error terminal opens showing the end of the error log so you can diagnose the issue; the session stays open for inspection.
💡
The session stays open as long as Quickplot is open. Closing Quickplot ends the session cleanly.
Interactive Apps

FVCOM

FVCOM (Finite Volume Community Ocean Model) is a coastal ocean circulation model. IKARUS provides several physics variants of FVCOM as separate executables, so you can select a build with the physics your simulation needs without having to compile it yourself.

Physics Variants
VariantUse Case
Base / General3D baroclinic simulations with water quality, data assimilation, heat flux, and dye tracers, all toggled at runtime
Wave-Current InteractionCoupled wave-current and vortex force simulations. Requires external SWAN wave model output as forcing.
SphericalLarge-domain or regional runs using geographic coordinates
Sediment Transport (ORIG)Suspended sediment transport, FVCOM's original scheme
Sediment Transport (CSTMS)Suspended sediment transport, community CSTMS scheme
2D BarotropicDepth-averaged runs only, no vertical stratification
Launch Options
FieldRecommendedNotes
Physics variantBase / General for most casesSee table above
Run directoryFull NFS pathMust contain your casename.nml file
CasenameMatches your .nml file
Physics togglesAs neededWater quality, dye release, data assimilation, and heat flux toggles, only shown for the Base variant
Post-processing visualiserQuickplot or ParaViewQuickplot for Deltares-format unstructured results; ParaView for general 3D visualisation
Nodes1–56
MPI tasks per nodeUp to 40Total tasks must equal Nprocs in your casename.nml
PartitionRes
Using FVCOM
  1. Upload your FVCOM input files (grid, forcing data, and casename.nml) to your home directory using the file browser.
  2. Set Nprocs in your casename.nml to match Nodes × MPI tasks per node exactly.
  3. Select the physics variant matching your simulation's requirements.
  4. If using the Base variant, set any physics toggles you need. The app patches your casename.nml automatically before launch.
  5. Choose a post-processing visualiser and optionally specify the output file to open automatically.
  6. Launch. A live terminal opens showing simulation progress.
  7. On success, your chosen visualiser opens automatically with the results. On failure, an error terminal opens showing the end of the error log.
⚠
Your domain decomposition file (casename_dep.dat) must already exist in your run directory before submitting, and its partition count must match your Nodes × tasks-per-node setting.
Interactive Apps

WRF-Chem

WRF-Chem is an atmospheric chemistry and weather simulation pipeline. Unlike the other interactive apps, WRF-Chem runs as a background batch job with no live desktop; you submit it and monitor progress through Active Jobs and the file browser, and it runs the full preprocessing and simulation pipeline automatically.

Launch Options
FieldRecommendedNotes
Run directoryFull NFS pathMust contain a pre-configured namelist.input and namelist.wps
Meteorological data pathFull NFS pathDirectory of GRIB files from your chosen global model
Global modelGFS, IFS, or AIFSDetermines boundary condition interval and available start hours
Start dateyyyy/mm/dd
Start hourModel-dependentGFS/AIFS: 00, 06, 12, 18 UTC. IFS: 00 or 12 UTC only.
Nodes2 or more
MPI tasks per node40
Memory64 GBUp to 350 GB
Hours24 hUp to 120 h
PartitionRes
Before You Submit
  • Run geogrid.exe once for your domain beforehand; the required geo_em.d01.nc (and additional domains if nested) must already be present in your run directory.
  • namelist.input must be fully configured for your domain (grid spacing, dimensions, physics, and chemistry options). The app only sets the start date/time and interval automatically.
  • namelist.wps must be configured for your domain and projection. The app sets the start date, interval, and met data source automatically.
  • GRIB input files from your chosen global model must already be staged in your met data directory.
Using WRF-Chem
  1. Complete the prerequisites above, staging your run directory and meteorological data via the file browser.
  2. Fill in the launch form and submit.
  3. The pipeline runs automatically: ungrib, then metgrid, then real.exe, then wrf.exe, in sequence.
  4. Monitor progress via Active Jobs. There is no live desktop or terminal for this app.
  5. When the job completes, retrieve your output files via the file browser.
Interactive Apps

OSRA

OSRA (Oil Spill Risk Analysis) runs an ensemble of particle-tracking oil spill simulations across the Arabian Gulf, producing a probabilistic hazard atlas showing where spilled oil is likely to beach under different seasonal conditions. On completion, QGIS opens automatically with the hazard atlas loaded.

Launch Options
FieldRecommendedNotes
Run typeTest, for explorationTest: a single simulation task, runs in around 10–30 minutes. Full: submits a full 200-task ensemble.
Source locationAny of 10 Gulf locationsTest runs only
SeasonSpring, summer, autumn, or winterTest runs only
Year2018–2022Test runs only
Number of particles500 for a quick test; 10,000 for standard runs
PartitionRes
Hours4 h
Memory32 GB
Using OSRA

Test Run

Choose a single source location, season, and year to run one simulation task interactively. A live terminal in the VNC session shows progress; on completion, trajectory plots open automatically.

Full Run

Submits a complete 200-task ensemble covering all source locations, seasons, and years. A monitoring terminal in the VNC session tracks task progress. Once the full ensemble and post-processing complete, QGIS opens automatically with the resulting hazard atlas loaded.

💡
A JupyterLab kernel named OSRA is also available, with example notebooks for submitting runs, inspecting individual trajectories, and exploring the hazard atlas interactively. Run the setup script once from a JupyterLab terminal (see the notebook README) to copy the example notebooks into your home directory.
Tools

Tools

Additional research tools integrated directly into the portal, separate from the Interactive Apps. Accessed from the Tools menu in the top navigation bar.

Accessing the Tools Menu
  1. Log in to the portal at khpc.kisr.edu.kw/ood/
  2. Click Tools in the top navigation bar.
  3. Select the tool you want from the drop-down.
ℹ
Unlike the Interactive Apps, Tools generally don't launch a Slurm session on a compute node. They're standing services you connect to directly, and your portal login carries over automatically, no separate sign-in required.

Available Tools

ToolPurpose
MLFlowExperiment tracking, model registry, and artifact storage for machine learning and computational workflows
Tools

MLFlow

MLFlow is an open-source platform for managing the full machine learning and computational experiment lifecycle: parameter and metric logging, model versioning, artifact storage, and reproducibility across runs.

What MLFlow Provides
CapabilityDescription
Experiment trackingLog parameters, metrics, and tags from any Python, R, or shell script. Compare runs across Slurm jobs without manual spreadsheets.
Artifact storeSave model files, plots, checkpoints, and datasets as versioned artifacts, stored on shared cluster storage and accessible from any node.
Model registryVersion, stage, and move models through a lifecycle (Staging → Production), giving you a central catalogue with lineage tracking.
Web UIBrowse experiments, compare runs side-by-side, and visualise metrics, accessible in your browser via the portal.
REST API & SDKFull REST and Python SDK for programmatic access. Slurm jobs log directly to the tracking server with no manual steps.
Language supportPython (mlflow package), R (mlflow R package), and REST for any other language.

Accessing MLFlow

InterfaceHow to reach it
MLFlow UIClick Tools → MLFlow in the portal, or go directly to khpc.kisr.edu.kw/mlflow/
REST APIhttps://khpc.kisr.edu.kw/mlflow/api/2.0/, for use with the Python SDK or your own scripts from outside a Slurm job
✓
Your portal login carries over automatically. There is no separate MLFlow login, if you're logged into the portal, MLFlow just works.

Logging Experiments from Slurm Jobs

Tracking Server URI

Use the same public tracking URI everywhere, whether inside a Slurm job script or from your own machine:

https://khpc.kisr.edu.kw/mlflow/
Python Example
import mlflow

# Set tracking server (or use env var MLFLOW_TRACKING_URI)
mlflow.set_tracking_uri("https://khpc.kisr.edu.kw/mlflow/")

# Create or reuse an experiment
mlflow.set_experiment("water_quality_model_v2")

with mlflow.start_run():
    # Log parameters
    mlflow.log_param("learning_rate", 0.001)
    mlflow.log_param("epochs", 100)
    mlflow.log_param("batch_size", 32)

    # --- your training code here ---

    # Log metrics per epoch
    for epoch in range(100):
        loss = train_one_epoch(...)
        mlflow.log_metric("loss", loss, step=epoch)

    # Log artifacts (model file, plots)
    mlflow.log_artifact("model.pkl")
    mlflow.log_artifact("training_curve.png")

    # Log the model itself with schema
    mlflow.sklearn.log_model(model, "random_forest_model")
R Example
library(mlflow)

mlflow_set_tracking_uri("https://khpc.kisr.edu.kw/mlflow/")
mlflow_set_experiment("air_quality_forecast")

with(mlflow_start_run(), {
  mlflow_log_param("ntree", 500)
  mlflow_log_param("mtry", 3)

  # --- your model training ---

  mlflow_log_metric("rmse", rmse_value)
  mlflow_log_artifact("forecast_plot.pdf")
})
Shell / Environment Variable Approach

For any language or tool that respects environment variables, set MLFLOW_TRACKING_URI in your Slurm job script. MLFlow's autologging can automatically capture parameters and metrics for supported frameworks (scikit-learn, TensorFlow, PyTorch, XGBoost, LightGBM, Keras):

export MLFLOW_TRACKING_URI="https://khpc.kisr.edu.kw/mlflow/"
export MLFLOW_EXPERIMENT_NAME="batch_simulation_run"

python3 -c "import mlflow; mlflow.autolog()"
Job Composer Template

A pre-configured MLFlow Experiment template is available in the Job Composer. It sets MLFLOW_TRACKING_URI and a starting experiment name automatically. Click + New Job → From Template and select it to get started without writing the boilerplate yourself, then edit the script path and parameters for your own run.

Naming Convention

Recommended Experiment Naming

MLFlow experiment names are global on the shared tracking server. Two researchers using the same experiment name will have their runs merged into the same experiment. Adopt a naming convention to avoid collisions:

ConventionExampleNotes
<department>/<project>/<descriptive_name>env/airquality/lstm_v3Hierarchical, maps to directory-like paths
<username>_<project>_<date>abduljalil_wq_20260501Simple, prevents collisions between users
<PI_name>/<project>hussain_lab/groundwater_modelGroups all experiments under a PI or group
⚠
Pick a unique experiment name before your first run. Renaming later doesn't merge or separate existing runs automatically.

Viewing Results

  1. Click Tools → MLFlow in the portal, or open khpc.kisr.edu.kw/mlflow/ directly.
  2. Select an experiment from the left sidebar to see all its runs.
  3. Click a run to see its logged parameters, metrics, and artifacts.
  4. Use the comparison view to select multiple runs and compare metrics side-by-side.
  5. Artifacts are served through MLFlow directly. Model files and plots can be downloaded from the browser.
Hardware

Cluster Hardware

Specifications for the IKARUS HPC compute nodes, storage systems, network, and partition structure.

Cluster at a Glance
Cluster NameIKARUS
OperatorKuwait Institute for Scientific Research (KISR)
Job SchedulerSLURM Workload Manager
Web PortalIKARUS Portal: khpc.kisr.edu.kw/ood/
SSH Accesshpc.kisr.edu.kw (port 22) → login nodes clavis1 / clavis2
Operating SystemRocky Linux 8 (RHEL-compatible)
Total Compute Nodes57 (meteor1–meteor56 + fortis)
PartitionsRes · def1 · Dev
Primary Shared FilesystemNFS: user homes at /NFS/scratch/homes/<username>
Software Modules/NFS/shared/modules/, loaded via the module command

Compute Nodes

meteor1 – meteor56  ·  56 nodes  ·  General Compute
Quantity56 nodes
Node namesmeteor1 through meteor56
Processor2× Intel Xeon Gold 6248 @ 2.50 GHz
Cores per node40 cores (2 sockets × 20 cores)
RAM per node350 GB
PartitionsRes def1
PurposePrimary general-purpose compute nodes for all standard batch and interactive workloads
fortis  ·  1 node  ·  Special-Purpose Node
Quantity1 node
Node namefortis
Processor4× Intel Xeon Gold 6252 @ 2.10 GHz
Cores96 cores (4 sockets × 24 cores)
RAM1.5 TB
PartitionsDev
PurposeSpecial-purpose node; contact administrators for use-case guidance
💡
To see the live state of all nodes, run sinfo -N -o "%-20N %-10c %-12m %-10T %-10P" in the shell terminal.

Storage Systems

Mount PointPurposeAccessNotes
/NFS/scratch/homes/<username>User home directory (Scratch)Personal to each user460 TB total scratch storage. Default working location, shared across all nodes via NFS. Store scripts, data, and job output here.
/NFS/shared/Shared software and librariesRead-only for regular users230 TB shared storage. Contains modules, Conda environments, shared libraries, and Jupyter kernels.
/NFS/shared/projects/Project storagePersonal to each project's membersExclusive for permenant project data. Each project is partitioned from each other. Requested from IKARUS administration.
⚠
IKARUS does not automatically back up user home directories. Back up important data to your local workstation or storage regularly.

Checking Your Disk Usage

du -sh /NFS/scratch/homes/${USER}           # total home directory usage
du -sh /NFS/scratch/homes/${USER}/*/        # breakdown by subdirectory
find /NFS/scratch/homes/${USER} -type f \
  -printf '%s %p\n' | sort -rn | head -20  # find largest files

Network

Inter-node fabricInfiniBand / 1GbE
Internet Routing

Partitions & Access Levels

Which partitions you can access depends on your account. The Interactive Apps launch forms automatically show only the partitions available to you.

PartitionNodesCoresRAMMax WalltimeAccess
Res6 meteor nodes2402.1 TB36 hoursAll users
def150 meteor nodes2,00017.5 TB4 hoursproject, developer
Devfortis (1 node)1921.5 TB72 hoursdeveloper
PartitionDescription & Objective
ResDedicated to experimental, early-stage, or small-scale scientific projects and proof-of-concept simulations. Uses less than 10% of total cluster resources. Aimed at providing a flexible, accessible platform for application development and testing before scaling to production, without impacting high-priority workloads.
def1Dedicated to KISR projects (for the duration of the project) and collaborators at other research institutions (limited lifespan). Uses approximately 85% of total cluster resources. Aimed at supporting integrated applications, ongoing KISR projects, and related workloads.
DevDedicated to experimental or small-scale scientific developments and resource assessment. Uses less than 5% of total cluster resources. Aimed at providing a long-runtime platform for application development and testing before scaling to production, without impacting high-priority workloads.
💡
Contact the IKARUS administrators to request access to additional partitions. Partition access is tied to your KISR account type and project assignment.
System Status

System Status

A live snapshot of overall cluster availability and load, visible directly on your portal dashboard.

Where to Find It

The IKARUS Cluster Status panel is shown in two places, both displaying the same live information:

  • Automatically on the portal dashboard, directly below the Recently Used Apps tiles, every time you log in.
  • By clicking Clusters → System Status in the top navigation bar from anywhere in the portal.

What It Shows

Cluster Status Fields

The panel gives a quick, at-a-glance read of how busy the cluster currently is:

FieldDescription
Nodes AvailableTotal number of compute nodes in the cluster (57), with a bar showing the percentage currently allocated to jobs
Processors AvailableTotal CPU cores across the cluster (2,432), with a bar showing the percentage currently in use
GPUs AvailableAlways 0. IKARUS compute nodes do not have GPUs, so this bar has nothing to divide by and will show NaN% in use. This is expected and not an error.
Jobs RunningNumber of jobs currently executing on compute nodes across the whole cluster
Jobs QueuedNumber of jobs submitted and waiting for resources to become available
💡
Check Nodes Available and Processors Available before submitting a large job. If utilisation is already close to 100% and Jobs Queued is high, expect a longer wait before your job starts.
CLI Reference

SSH Access

Traditional terminal access, for advanced users and automated workflows.

Host: hpc.kisr.edu.kw  ·  Port: 22  ·  Username: your KISR username

Windows (MobaXterm)

  1. Download MobaXterm (Portable edition) from mobaxterm.mobatek.net.
  2. Unzip and launch MobaXterm_Personal_*.exe.
  3. Click Session → SSH. Set Remote host: hpc.kisr.edu.kw, Username: your KISR username, Port: 22.
  4. Click OK and enter your password when prompted (input is hidden).
  5. A successful login shows a prompt like:
[hpcdemo@clavis1 ~]$ 

Connecting on Mac / Linux

  1. Open Terminal (Mac: Applications → Utilities → Terminal).
  2. Run the SSH command with X11 forwarding enabled:
$ ssh -Y hpcdemo@hpc.kisr.edu.kw
  1. On first connection, type yes to accept the RSA fingerprint.
  2. Enter your password when prompted (not displayed on screen).
  3. On success you will see the shell prompt: [hpcdemo@clavis1 ~]$
💡
The -Y flag enables X11 forwarding, allowing GUI applications launched in the terminal to render on your local screen.
CLI Reference

Linux Commands

Essential shell commands for working on IKARUS. The default shell is BASH (Bourne Again Shell). Linux paths and filenames are case-sensitive.

Navigation

pwd                                      # print current directory path
cd /NFS/scratch/homes/hpcdemo/jobs       # navigate to a directory
cd ~                                     # return to home directory
cd ..                                    # go up one level (parent dir)
cd -                                     # return to previous directory
ls                                       # list directory contents
ls -l                                    # long listing (permissions, size, date)
ls -a                                    # include hidden files (dot files)
ls -la                                   # combine long + hidden
ls *.py                                  # list all Python files (wildcard)

File Operations

mkdir my_dir                             # create a new directory
mkdir -p project/data/raw                # create nested directories at once
touch file.txt                           # create an empty file
cp file.txt backup.txt                   # copy to a new filename
cp file.txt /path/to/dest/               # copy to a different directory
cp *.csv /data/                          # copy all CSV files
cp -r project/ project_backup/           # copy directory recursively
mv file.txt newname.txt                  # rename a file
mv file.txt /path/to/dest/               # move a file
mv old_dir/ new_dir/                     # rename a directory
rm file.txt                              # delete a file (PERMANENT)
rm -rf directory/                        # delete directory + contents (PERMANENT)
⚠
There is no recycle bin on Linux. rm is immediate and cannot be undone. Always double-check before deleting.

Viewing File Contents

cat file.txt                             # print entire file to screen
cat file.txt >> dest.txt                 # append file to another
head file.txt                            # show first 10 lines
head -n 30 file.txt                      # show first 30 lines
tail file.txt                            # show last 10 lines
tail -n 30 file.txt                      # show last 30 lines
tail -f output.log                       # follow file live (Ctrl+C to stop)
tail -F app.log                          # follow + handle log rotation
less file.txt                            # paginated viewer (q to quit, / to search)
more file.txt                            # simpler pager (Space=next page, q=quit)
more -5 file.log                         # show 5 lines at a time
more +10 file.log                        # start from line 10
💡
tail -f output.log is invaluable for watching a running job's output file in real time. Press Ctrl+C to stop following.

Searching

grep "error" output.log                  # find lines containing "error"
grep -i "warning" output.log             # case-insensitive search
grep -r "pattern" ./src/                 # search recursively in directory
grep -v "debug" output.log               # exclude lines containing "debug"
grep -n "error" output.log               # show line numbers with matches
find . -name "results"                   # find a file/dir named "results"
find . -name "*.py"                      # find all Python files
find . -type f -name "*.log"             # find only files (not dirs) matching *.log
wc -l file.txt                           # count lines
wc -w file.txt                           # count words
wc -c file.txt                           # count bytes

Other Useful Commands

CommandDescriptionExample
dateDisplay current date, time, and timezonedate
calShow calendar for the current monthcal
timeMeasure how long a command takestime python3 script.py
sleepPause for N seconds (useful in shell scripts)sleep 5
sortSort lines of a text filesort results.txt
fileDetermine file type from contents (not extension)file data.bin
tacPrint file contents in reverse line ordertac output.log
echoPrint text to screen or redirect to fileecho "done" >> status.txt
whichShow full path of an executablewhich python3
envShow all environment variablesenv | grep PATH
historyShow recent command historyhistory | tail -20
# Redirect and pipe examples
sort file.txt > sorted.txt               # sort and save to new file
sort file.txt >> sorted.txt              # sort and append to existing file
cat file.txt | grep "error" | wc -l      # count error lines using pipes
ls -la | less                            # browse long directory listings
CLI Reference

File Permissions & Compression

Understanding and changing Linux file permissions, and working with compressed archives.

Permission Notation

Every file and directory has a 10-character permission string shown by ls -l. For example: drwxr-xr-x

PositionCharactersMeaning
1std or -d = directory · - = regular file
2nd–4thrwxOwner (user) permissions
5th–7thr-xGroup permissions
8th–10thr-xOthers (everyone else) permissions

Each permission letter:

  • r (Read): view or copy the file, or list directory contents
  • w (Write): modify the file, or create/delete files in a directory
  • x (Execute): run the file as a program, or enter a directory
  • –: permission not granted

chmod: Changing Permissions

Format: chmod [who][action][permission] file

WhoActionPermission
u: owner (user)+ addr read
g: group- removew write
o: others= set exactlyx execute
a: all (u+g+o)
chmod u+x script.sh                      # make script executable by owner
chmod a+x script.sh                      # executable by everyone
chmod u+rwx,g+rx file.sh                 # owner: full access; group: read+execute
chmod o-x *                              # remove execute from others on all files
chmod g-w sensitive.txt                  # remove group write permission
chmod 755 script.sh                      # octal: rwxr-xr-x
chmod 644 data.csv                       # octal: rw-r--r--
⚠
Permission errors appear as Permission denied. Run ls -l to inspect permissions and ownership, and chmod to correct them.

Compression: tar, gzip

gzip / gunzip: Single File Compression

gzip filename.c                          # compress → filename.c.gz
gunzip filename.c.gz                     # decompress

tar: Archive and Compress Directories

FlagMeaning
-cCreate a new archive
-xExtract files from archive
-tList contents without extracting
-zUse gzip compression (.tar.gz)
-vVerbose: show files being processed
-fSpecify archive filename (always last flag)
tar -czvf archive.tar.gz my_project/    # create compressed archive of directory
tar -xzvf archive.tar.gz                # extract archive here
tar -xzvf archive.tar.gz -C /target/    # extract to a specific directory
tar -tzvf archive.tar.gz                # list contents without extracting
gzip -l archive.tar.gz                  # show compression ratio and sizes
💡
Before downloading a large directory, compress it with tar -czvf mydata.tar.gz data/ in the shell terminal first. Then download the single archive file; much faster and more reliable than downloading many individual files.
CLI Reference

Environment Modules

IKARUS uses a module system so multiple versions of the same software can coexist without conflicts. Loading a module configures your environment for that software.

All modules are stored at /NFS/shared/modules/ and are accessible on every node.

Module Commands

module avail                             # list ALL available modules
module avail python                      # search for modules matching "python"
module avail jasper                      # find all versions of jasper
module show jasper/2.0.14                # see what the module sets (PATH, libs…)
module add jasper                        # load default version (same as module load)
module load jasper/2.0.14                # load a specific version
module list                              # show currently loaded modules
module rm jasper                         # unload a specific module
module rm jasper/2.0.14                  # unload a specific version
module purge                             # unload ALL loaded modules
💡
module add and module load are identical, both load a module. Without specifying a version, the highest alphanumeric version (or the tagged default, shown with D in module avail) is loaded.

Using Modules in SLURM Scripts

Modules loaded in your interactive terminal session are not inherited by batch jobs. Always include module load commands inside your submission script:

#!/bin/bash
#SBATCH --job-name=python_job
#SBATCH --partition=Res
#SBATCH --cpus-per-task=8
#SBATCH --mem=32G
#SBATCH --time=01:00:00

# Load modules here (inside the script)
module purge                             # start from a clean state
module load python/3.10
module load openmpi/4.1.5

# Your code
python3 my_script.py
⚠
A common cause of batch job failures is forgetting to load modules in the script. The compute node starts with a clean environment, anything you need must be explicitly loaded.
CLI Reference

SLURM Commands

SLURM is the workload manager on IKARUS. These commands are used from the login node (clavis) to submit, monitor, and manage jobs.

Submitting Jobs

sbatch: Submit a Batch Script

Submits a script to run asynchronously on compute nodes. Returns immediately with the assigned Job ID.

[hpcdemo@clavis2 ~]$ sbatch my_job.sh
Submitted batch job 568

srun: Interactive Session on a Compute Node

Requests resources and gives you a shell on a compute node. Blocks until resources are available.

[hpcdemo@clavis2 ~]$ srun --partition=Res --nodes=1 \
  --cpus-per-task=4 --mem=8G --time=01:00:00 bash -l

Monitoring Jobs

squeue: View the Queue

squeue                                                # all jobs across all users
squeue -me                                            # your jobs only
squeue -u hpcdemo                                     # jobs for a specific user
squeue -p Res                                         # jobs in the Res partition
squeue --start                                        # estimated start times for pending jobs
squeue -o "%.10i %.9P %.20j %.8u %.8T %.10M %.6D %R"  # custom output format

squeue State Codes

CodeStateMeaning
RRUNNINGJob is actively executing on compute nodes
PDPENDINGWaiting for resources or priority; reason shown in last column
CGCOMPLETINGJob finishing, cleaning up processes
CDCOMPLETEDFinished successfully (exit code 0)
FFAILEDExited with non-zero status
CACANCELLEDCancelled by user or administrator
TOTIMEOUTExceeded the requested wall-clock time
OOMOUT_OF_MEMORYJob exceeded the requested memory allocation

sacct: Job History and Accounting

Query completed and historical jobs from the SLURM accounting database.

sacct --user=$USER --starttime=today                           # your jobs from today
sacct --user=$USER --starttime=2025-08-01                      # since a specific date
sacct --jobs=568                                               # details for job 568
sacct --jobs=568 --format=JobID,State,ExitCode,Elapsed,MaxRSS  # custom fields

scancel: Cancel a Job

scancel 568                              # cancel job 568 (must be your job)
scancel -u $USER                         # cancel ALL your jobs
scancel -u $USER -t PENDING              # cancel only your pending jobs

sinfo: Cluster State

sinfo                                    # partition and node summary
sinfo -N -o "%-20N %-10c %-12m %-10P"    # per-node: name, CPUs, RAM(MB), partition
sinfo -p Res                             # show only the Res partition

sstat: Live Resource Usage of a Running Job

sstat --jobs=568                         # all resource stats for job 568
sstat --jobs=568 --format=JobID,AveCPU,AveRSS,MaxRSS

Minimal Submission Script

#!/bin/bash

######## SLURM Directives ########
#SBATCH --job-name=my_job
#SBATCH --partition=Res
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=8
#SBATCH --mem=32G
#SBATCH --time=02:00:00
#SBATCH --output=%j.out         # %j is replaced by Job ID
#SBATCH --error=%j.err
######## End of Directives ########

module purge
module load python/3.10

cd /NFS/scratch/homes/${USER}/my_project
python3 my_script.py

#SBATCH Directive Reference

DirectiveExample ValueDescription
--job-namemy_simName shown in squeue output
--partitionResTarget queue: Res, def1, or Dev
--nodes1Minimum number of nodes to allocate
--ntasks16Total number of MPI tasks
--ntasks-per-node8MPI tasks per allocated node
--cpus-per-task4CPU threads per MPI task (default: 1)
--mem64GRAM per node, units: K, M, G, T
--mem-per-cpu4GRAM per CPU core (alternative to --mem)
--time12:00:00Max wall-clock time (D-HH:MM:SS or HH:MM:SS)
--array1-50Submit as job array with indices 1 through 50
--output%j.outStandard output file (%j = Job ID, %a = array task ID)
--error%j.errStandard error file
--mail-typeEND,FAILSend email on END, FAIL, BEGIN, or ALL
--mail-useruser@kisr.edu.kwEmail address for notifications
--test-only(no value)Validate script and show estimated start time without submitting
--dependencyafterok:567Hold this job until job 567 completes successfully

Job Array Example

Job arrays submit the same script N times, each with a unique SLURM_ARRAY_TASK_ID. Useful for running the same analysis on many input files.

#!/bin/bash
#SBATCH --job-name=array_example
#SBATCH --partition=Res
#SBATCH --array=1-16               # creates 16 jobs, IDs 1..16
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=4G
#SBATCH --time=00:30:00
#SBATCH --output=logs/job_%a.out   # %a = array task ID

module load python/3.10

# Use SLURM_ARRAY_TASK_ID to select input
INPUT_FILE="data/input_${SLURM_ARRAY_TASK_ID}.csv"
OUTPUT_FILE="results/output_${SLURM_ARRAY_TASK_ID}.csv"

echo "Processing task ${SLURM_ARRAY_TASK_ID}: ${INPUT_FILE}"
python3 process.py --input ${INPUT_FILE} --output ${OUTPUT_FILE}

MPI Parallel Job Example

#!/bin/bash
#SBATCH --job-name=mpi_job
#SBATCH --partition=Res
#SBATCH --nodes=2
#SBATCH --ntasks=40               # 20 tasks per node × 2 nodes
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=4G
#SBATCH --time=00:30:00

module purge
module load openmpi/4.1.5
module load python/3.10

cd ${SLURM_SUBMIT_DIR}

# Run 5 parallel MPI jobs in background, each with 4 tasks
mpirun -n 4 python3 pi-digits.py 10  &
mpirun -n 4 python3 pi-digits.py 15  &
mpirun -n 4 python3 pi-digits.py 20  &
mpirun -n 4 python3 pi-digits.py 25  &
mpirun -n 4 python3 pi-digits.py 1000 &

# Wait for ALL background tasks to finish
wait

# Then run the aggregation step
mpirun -n 20 python3 sum-digit.py

exit 0
💡
The wait command is critical in the MPI example. Without it, the aggregation step starts before all parallel tasks have completed.