Julia GPU Computing Guide — CUDA, AMDGPU, and Parallel GPU Programming

In this tutorial, you will learn about Julia GPU Computing Guide. We cover key concepts, practical examples, and best practices to help you master this topic.

Julia GPU computing offers CUDA.jl for NVIDIA GPUs with CuArray operations, custom kernels with @cuda, and integration with Flux.jl for Deep Learning -- while KernelAbstractions.jl provides portable GPU kernels across CUDA, AMD, and oneAPI backends.

CuArray Basics

using CUDA

# Create GPU array
a = CuArray([1, 2, 3, 4, 5])
b = CuArray{Float32}(rand(100, 100))

# Operations work the same
c = a .* 2
d = a + b

# Matrix multiplication
A = CUDA.rand(100, 100)
B = CUDA.rand(100, 100)
C = A * B

# Move between CPU and GPU
cpu_arr = Array(a)     # download to CPU
gpu_arr = CuArray(cpu_arr)  # upload to GPU

Custom CUDA Kernels

using CUDA

# Define CUDA kernel
function vadd!(c, a, b)
    i = threadIdx().x + (blockIdx().x - 1) * blockDim().x
    if i <= length(a)
        @inbounds c[i] = a[i] + b[i]
    end
    return nothing
end

# Launch kernel
a = CuArray{Float32}(rand(1000))
b = CuArray{Float32}(rand(1000))
c = similar(a)

@cuda blocks=10 threads=100 vadd!(c, a, b)

GPU Memory Management

# Memory pool
CUDA.memory_status()

# Pin memory for faster transfers
using CUDA: CuStream
stream = CuStream()
src = a
dst = CuArray{Float32}(undef, size(a))
CUDA.copy!(dst, src, stream)

# Out-of-core processing
using GPUArrays
# Process data in chunks
for chunk in partition(data, 1024)
    gpu_data = CuArray(chunk)
    result = gpu_function(gpu_data)
    # ...
end

KernelAbstractions

using KernelAbstractions

@kernel function vadd_kernel(@Const(a), @Const(b), c)
    i = @index(Global, Linear)
    if i <= length(a)
        @inbounds c[i] = a[i] + b[i]
    end
end

# Works on any backend
backend = CUDABackend()  # or CPUBackend()
a = CUDA.rand(1000)
b = CUDA.rand(1000)
c = similar(a)

kernel = vadd_kernel(backend, 256)
kernel(a, b, c, ndrange=length(a))
synchronize(backend)

Common Mistakes

1. Not synchronizing

GPU operations are asynchronous. Use synchronize() before reading results. Array(gpu_array) automatically syncs.

2. Excessive GPU-CPU transfers

Data transfer is the bottleneck. Minimize transfers. Keep data on GPU as long as possible.

3. Launching too few threads

GPU needs many threads for occupancy. Launch at least 256 threads per block, enough blocks to fill all SMs.

Practice Questions

1. How do you create an array on the GPU? CuArray([1, 2, 3]) or CUDA.rand(100, 100) creates arrays in GPU memory.

2. How do you write a custom CUDA kernel? Define a function, mark with @cuda launch: @cuda threads=256 vadd!(c, a, b).

3. How do you move data between CPU and GPU? Array(gpu_array) downloads to CPU. CuArray(cpu_array) uploads to GPU.

FAQ

{{< faq question="Do I need to install CUDA separately?" >}} CUDA.jl includes the CUDA toolkit. Just ] add CUDA and Julia handles the rest.

{{< faq question="Can I use multiple GPUs?" >}} Yes. Use CUDA.devices() to list GPUs. device!(i) selects GPU i. Distribute work manually or use MPI.jl.

{{< faq question="What is the performance compared to Python CUDA?" >}} Julia's GPU code compiles to native PTX. Performance is comparable to or better than Python for compute-intensive kernels. {{< /faq >}}

What's Next

Now learn about interoperability.