Location: Bengaluru, KA, India
Job Type: Full-time
Salary: $90K - $110K
Equity: 0.50% - 1.00%
Experience: 3+ years
Skills: Python, Distributed Systems, Machine Learning, CUDA
We’re hiring a Founding ML Infrastructure Engineer in Bengaluru to own the systems that train and serve our Speech models: the distributed training stack, Realtime inference engine, and the cloud services. You’ll work in a small team with tons of compute and see your ideas making it to production in days.
What you’ll do
Own the training stack: FSDP/HSDP and Megatron-style tensor parallelism in our PyTorch framework
Make realtime inference fast: time-to-first-audio and throughput on our vLLM-based engine, Quantization, model compilation and load testing under high concurrency.
Build data pipelines at scale: Work on creating in-house data pipelines that process Millions of hours of audio and feed training at high throughput.
Run the production path: API gateway and streaming WebSocket APIs, deploy tooling, logging, monitoring.
What we’re looking for
3+ years building ML systems or high-performance backend infrastructure.
Hands-on depth in at least one of: distributed training (FSDP/TP/PP), LLM serving (vLLM/SGLang/TensorRT-LLM), High volume data processing, or low-latency streaming systems.
Comfort operating in ambiguity: you can spec, build, debug, and ship.
Speech/audio experience is a plus, not required.
Summary
Own the systems that train and serve Speech models, including the distributed training stack, realtime inference engine, and cloud services. Requires 3+ years building ML systems or high-performance backend infrastructure with hands-on depth in distributed training, LLM serving, data processing, or streaming systems.
Responsibilities
Own the training stack: FSDP/HSDP and Megatron-style tensor parallelism. Make realtime inference fast. Build data pipelines at scale. Run the production path: API gateway and streaming WebSocket APIs, deploy tooling, logging, monitoring.
Qualifications
3+ years building ML systems or high-performance backend infrastructure.
Experience requirements
3+ years building ML systems or high-performance backend infrastructure. Hands-on depth in at least one of: distributed training (FSDP/TP/PP), LLM serving (vLLM/SGLang/TensorRT-LLM), high volume data processing, or low-latency streaming systems. Comfort operating in ambiguity.
Benefits
Equity: 0.50% - 1.00%