AI Infrastructure Engineer

About the role

42dot's AI Infrastructure Engineer manages high-performance AI infrastructure that orchestrates thousands of GPUs across several data centers. The role contributes to scaling, monitoring and operational optimization so that a robust, world-class computing environment is maintained. It is a full-time position based in Pangyo.

Key responsibilities

• Operate and maintain large-scale GPU clusters of thousands of nodes across multiple data centers using Kubernetes and Slurm.
• Monitor and diagnose failures in the GPU hardware and software stack to ensure high availability and fast recovery.
• Develop automation tools and scripts in Python or Shell to streamline repetitive infrastructure work.
• Manage GPU resource quotas and support ML researchers so computing resources are used optimally.
• Take part in designing and tuning distributed training environments for large-scale autonomous driving models.

Required profile

• Deep understanding of Linux, including kernel operation, process management and system security.
• Practical experience with containerization (Docker) and orchestration (Kubernetes), including building, managing and troubleshooting containerized environments.
• Ability to write clean, maintainable Python or Shell scripts for automation and system administration.
• Logical problem-solving approach to find root causes in large, complex systems.
• Good communication skills to work with cross-functional teams and partners.

Required skills

• Networking basics (TCP/IP, HTTP(S)) and basic network troubleshooting.
• Preferred: observability stacks with Prometheus, Grafana and Datadog.
• Preferred: public cloud infrastructure (AWS, GCP).
• Preferred: NVIDIA stack (drivers, CUDA, NCCL), ML training lifecycle, PyTorch or TensorFlow.
• Preferred: Kubernetes or Slurm, and IaC tools such as Terraform.

The process includes an application screening, an online coding test, a first virtual interview of about an hour, a second interview of about three hours, and then offer discussion and onboarding. A 3-month probation may apply.