Getting Started with AI and HPC
I’m new to AI or HPC. Where do I start?
UBC ARC offers resources to help researchers get started with AI and high-performance computing (HPC).
- Get familiar with the computing environment. Start with the Sockeye QuickStart Guide for an introduction to using Sockeye, then explore the AI/ML Resources Hub for information on AI research tools and resources.
- Try an interactive environment. Jupyter notebooks are available through UBC ARC OnDemand, allowing researchers to explore and test AI workflows without extensive command-line experience.
- Start with a small workload. Initial testing can be done using CPU resources or a single GPU before scaling up to more demanding workloads. The Your First GPU Job guide provides step-by-step instructions for submitting GPU jobs on Sockeye.
ARC Support can assist researchers in selecting appropriate computing resources, software tools, and workflows based on their research needs.
Do I need a GPU for my research?
Not all AI or machine learning workloads require GPUs. GPUs can accelerate computationally intensive tasks, but only when the software is designed to use GPU acceleration, such as CUDA-enabled applications.
If you're unsure, start with CPU resources. For GPU-enabled workloads, begin with a small test on a single GPU, monitor its utilization, and scale up as needed. For step-by-step instructions, see Your First GPU Job.
What AI and machine learning software is available on Sockeye?
Sockeye supports machine learning frameworks and tools such as PyTorch, TensorFlow, JAX, scikit-learn, and MATLAB. Researchers can access available software modules or set up custom Python environments and containers to support their research workflows.
Where should I store my datasets and model files?
Sockeye provides project and scratch storage to support different research needs. For more information, see the Choosing Storage on Sockeye documentation.
Running AI and Large Language Models on Sockeye
Does UBC ARC offer hosted language models or an API?
UBC ARC does not currently provide a centrally hosted language model service or API, but a hosted service is in development.
In the meantime, researchers can run supported open-weight language models on Sockeye or explore UBC-approved AI tools, depending on their research needs and data requirements
Can Sockeye run large language models (LLMs)?
Yes. Researchers can run supported open-weight language models on Sockeye’s NVIDIA V100 GPUs. Tools such as Ollama and OpenWebUI support interactive model testing, while vLLM supports multi-GPU inference. Parameter-efficient fine-tuning using techniques such as QLoRA is also possible for suitable smaller models.
How large a language model can I run on Sockeye?
The maximum model size depends on the GPU resources allocated to the job. For 4-bit quantized models running on V100 GPUs, approximate limits are:
| GPU allocation | Approximate model size | Example |
| 1 × 16 GB | Up to 14B parameters | Qwen3 14B (9.3 GB) |
| 1 × 32 GB | Up to 32B parameters | Qwen3 32B (20 GB) |
| 2 × 32 GB | Up to 70B parameters | Llama 3.3 70B (43 GB) |
| 4 × 32 GB (full node) | Up to 100B parameters | Llama 3.3 70B at 8-bit (75 GB) |
Estimates based on Ollama model download sizes as of October 2026. Actual GPU memory requirements depend on model configuration, context length, and runtime overhead. Visit the AI/ML Resources Hub for additional resources and recommendations.
Can I run very large language models or train models from scratch on Sockeye?
Sockeye is not suitable for models that exceed its available GPU memory, or for training large language models from scratch. For workloads requiring larger models or training from scratch, UBC researchers can access additional computing resources through the Digital Research Alliance of Canada (the Alliance).
Before using Alliance resources, researchers should review their data classification and applicable security and privacy requirements. ARC can help determine whether Alliance resources are appropriate for a given research project and data type.
How can I download AI models or access external web services on Sockeye?
Sockeye compute nodes have restricted internet access. Researchers can load the http_proxy module to access ARC-permitted external services or download models and datasets in advance through a login node. For more information, see Accessing Web-Based Services in Your Jobs.
Examples of external services include:
- Hugging Face
- Weights & Biases / Comet
- ColabFold
- OpenAI
Researchers can contact ARC Support to request access to additional external services. Access is subject to ARC's network restrictions, service availability, and applicable research data security and privacy requirements.
AI Security, Privacy, and Responsible Use
Which AI tool can I use for my research?
Different tools have different approved uses and data-handling requirements. Visit the Safe and Responsible AI in Research page for more information about AI Requirements at UBC.
Additional Computing Resources and Support
What if I need more GPUs or computing capacity than Sockeye provides?
UBC ARC is modernizing its digital research infrastructure, with new computing resources and capabilities expected to become available in 2027.
UBC researchers can also access additional computing resources through the Digital Research Alliance of Canada (the Alliance). Default access is available through a Rapid Access Service (RAS) allocation for resources such as compute and storage, while larger allocations may be obtained through the Alliance's annual Resource Allocation Competition (RAC). ARC can assist with resource selection and allocation applications.
Before using Alliance resources, researchers should review their data classification and applicable security and privacy requirements. ARC can help determine whether Alliance resources are appropriate for a given research project and data type.
Page last updated on November 13, 2026