Senior AI Compute Infrastructure Engineer

About the role

Kraken is building a dedicated AI Compute and Infrastructure team to power the next generation of model training, inference, evaluation, and experimentation across its exchange. As a Senior AI Compute Infrastructure Engineer you will own and operate GPU and accelerator clusters, ensuring fast, reliable, and cost‑efficient AI workloads.

Key responsibilities

• Operate and maintain GPU/accelerator clusters, including drivers, runtimes, kernels, device plugins, and scheduling primitives.
• Design and improve scheduling, orchestration, quota management, and utilization systems for heterogeneous accelerator environments.
• Optimize inference pipelines for latency, throughput, reliability, and cost using frameworks such as vLLM, Triton Inference Server, or TensorRT.
• Partner with ML engineers and researchers to remove bottlenecks in training, batch inference, online inference, and deployment workflows.
• Build observability for GPU utilization, memory pressure, queue depth, latency, and spend, and drive incident response and reliability improvements.

Required profile

• 5+ years of infrastructure engineering experience with significant exposure to GPU compute, ML infrastructure, or large‑scale production platforms.
• Hands‑on experience operating GPU clusters in production, including scheduling, orchestration, monitoring, and cost optimization.
• Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, and distributed runtimes.
• Proven ability to communicate technical trade‑offs to researchers, product teams, and leadership.

Required skills

• Python for automation and tooling.
• Kubernetes and container orchestration.
• ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, or Ray Serve.
• Linux systems, networking, and storage.
• Experience with GPU/accelerator hardware and drivers.