Rust Jobs for Rustaceans

Key Responsibilities:

• Architect and implement distributed storage solutions tailored for massive AI workloads, ensuring seamless integration with compute infrastructure.
• Optimize storage performance for high-throughput and low-latency access, leveraging cutting-edge hardware (NVMe, SSD) and software (Ceph, ZFS).
• Collaborate with infrastructure teams to enhance deployment pipelines using Infrastructure-as-Code (IaC) best practices, ensuring robust and secure storage delivery.
• Monitor and maintain storage systems across on-premise clusters and cloud environments, proactively addressing bottlenecks and failures.
• Contribute to capacity planning and data durability strategies to support xAI’s growing computational needs.

Requirements:

• Designing and operating distributed storage systems (e.g., Ceph, Lustre, or ZFS) at scale for high-performance computing (HPC) or AI workloads.
• Hands-on experience with storage hardware (NVMe, SSD, HDD) and tuning I/O performance in compute-intensive environments.
• Writing scalable, high-performance code in Rust or Go for storage-related applications or tooling.
• Managing storage infrastructure with IaC tools like Pulumi, Terraform, or Ansible.
• Familiarity with Kubernetes storage primitives (e.g., Persistent Volumes, CSI drivers) and integrating storage with containerized workloads.
• Bonus: Experience with AI/ML data pipelines, including handling large datasets for training and inference.

Tech Stack

• Kubernetes
• Pulumi
• Rust and Go
• Ceph, ZFS, Minio, S3, GCS, or similar distributed storage systems
• NVMe, SSD, and HDD storage technologies