AI Inference Engineer

About F5 and Role Overview

At F5, we are dedicated to enhancing the digital experience worldwide by empowering organizations to create, secure, and operate innovative applications. Our commitment spans cybersecurity efforts to protect consumers and enable companies to innovate confidently. We prioritize a diverse and inclusive community where everyone can thrive.

The AI Inference Engineer position plays a vital role in bridging the gap between advanced model development and efficient deployment. Focusing on optimizing Large Language Models (LLMs) for inference across environments—from GPU-heavy data centers to resource-limited edge devices—this role aims to maximize throughput, minimize latency, and maintain model fidelity. You will help advance F5’s AI capabilities by leveraging hardware acceleration, designing scalable infrastructures, and monitoring system performance to ensure enterprise-grade reliability.

Key Responsibilities

• Develop and sustain high-performance inference engines using technologies like vLLM, Text Generation Inference (TGI), and NVIDIA Triton to achieve scalable, efficient AI serving.
• Optimize model deployment to enable low-latency AI service delivery across diverse business applications.
• Profile and fine-tune models for specific hardware platforms including NVIDIA GPUs (CUDA/TensorRT), Apple Silicon (CoreML), and AI accelerators such as TPUs and LPUs.
• Collaborate with hardware teams to enhance utilization and performance across multiple computational setups.
• Design and implement scalable, auto-scaling inference architectures for both real-time and batch processes utilizing Kubernetes for routing and orchestration.
• Ensure robust performance and reliability during peak traffic through optimized software solutions.
• Establish observability systems to monitor metrics like Time to First Token (TTFT), token throughput, and memory bandwidth aligned with service-level agreements.
• Develop and execute performance and load testing to detect bottlenecks and guarantee reliable operation at scale.

Technical Requirements

• Proficient in programming languages such as Python, C++, Rust, or Golang aimed at high-performance AI workflows.
• Experienced with inference tools including vLLM, TensorRT, Llama.cpp, and Ollama to build and enhance inference solutions.
• Well-versed in infrastructure technologies like Docker, Kubernetes, and cloud platforms including AWS, GCP, and Azure.
• In-depth knowledge of GPU and AI hardware profiling and optimization techniques, especially for NVIDIA GPUs and TPUs.

Preferred Experience

• Hands-on experience deploying LLMs using advanced methods like Speculative Decoding or PagedAttention.
• Contributions to open-source inference libraries or hardware kernel development (CUDA, Triton kernels).
• Background in MLOps or Site Reliability Engineering ensuring high-performance AI endpoints and dependable operation under load.
• Skills in designing scalable architectures for inference systems optimized to handle traffic surges efficiently.

Success Metrics

• Reduce latency focusing on minimal Time to First Token and maximizing token processing rates.
• Enhance cost efficiency by lowering "cost per 1K tokens" through superior resource and hardware optimization.
• Uphold system stability and consistency during traffic variations to assure scalable performance.
• Maximize throughput by tailoring models to fully utilize hardware capabilities.

Why Work at F5?

• Engage with cutting-edge AI hardware and technologies supporting real-time applications.
• Grow your career in a dynamic, innovative environment emphasizing scalability and problem-solving.
• Lead transformative projects delivering dependable AI services worldwide with cost-effective and efficient solutions.
• Work on state-of-the-art MLOps systems that facilitate enterprise-wide AI deployment at scale.

Measures of Success

• Apply technical expertise and creative problem solving to deliver scalable, efficient AI prediction systems with low latency.
• Work collaboratively with cross-disciplinary teams contributing to system improvement and knowledge exchange.
• Demonstrate initiative by optimizing hardware, tooling, and orchestration while balancing immediate needs and long-term architecture.
• Translate sophisticated AI inference challenges into actionable solutions aligned with strategic priorities.

Additional Information

The job description is a general representation and may evolve. Please note F5 contacts candidates solely via official F5 email domains.

Equal Employment Opportunity

F5 is committed to providing equal employment opportunities to all candidates regardless of race, religion, sex, age, disability, or other protected classifications. Reasonable accommodations are available upon request during the application process.