Rust Jobs for Rustaceans

Key Responsibilities:
We are looking for an AI Systems Engineer to join our growing team. Our current stack is Python, Rust, C++, PyTorch, Triton, CUDA, Kubernetes. You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference.

• Develop robust APIs for AI inference used by both internal and external customers
• Design, deploy, and maintain scalable, reliable infrastructure for deploying machine learning models
• Benchmark system performance, diagnose bottlenecks, and implement improvements across the inference stack
• Enhance system reliability and observability by integrating modern monitoring and alerting tools
• Respond swiftly to system outages and collaborate across teams to maintain high uptime and performance

Requirements:

• Experience in developing APIs and managing distributed systems
• Strong understanding of Kubernetes and container orchestration
• Experience with deploying reliable, distributed, real-time systems at scale
• High level familiarity with LLM architecture, and the key pieces (Multi-Head, Multi/Grouped-Query, as well as common Layers)