Employer description & source details
Quadric's GPNPU runs both neural networks and C++ code on a single programmable architecture. As a member of our AI Application team, you will make the GPNPU a first-class backend in the frameworks developers already use, such as llama.cpp, PyTorch, and vLLM. That way, customers can move their applications from GPUs to Quadric with minimal changes. You'll own backend integration from memory and dispatch through serving features, working closely with our kernel, runtime and compiler teams. Integrate GPNPU backend into AI compute frameworks (llama.cpp, PyTorch, vLLM, Optimum) Implement framework backend interface, memory allocation, compute dispatch, registration and operators/kernels Implement model compute pipelines including prefill, decode, batching, multi-sequence and multi-modal Support important framework serving features including KV-cache management, paged attention, continuous batching, speculative decoding Optimize framework performance and accuracy for GPNPU backend Reduce host-device dispatch overhead, implement graph capture/replay, integrate efficient kernels, minimize latency Support quantization formats such as GGUF (Q4/Q8), GPTQ/AWQ, int8/fp16 Validate accuracy against reference backends (perplexity, task benchmarks) and drive improvements Within your first 6-12 months, you’ll: Bring up a new LLM/VLM family end to end on the GPNPU backend, with accuracy matching the reference Deliver measurable tokens/s or latency gains on a frontier model Ship one serving feature (e.g. paged KV cache) in a production release 3+ years building or extending ML frameworks or inference runtimes Solid understanding of framework internals: tensor memory management, graph scheduling, operator dispatch Solid understanding of AI compute performance analysis and optimization Flue
Extracted skills: python, c++