Employer description & source details
The bottleneck in building reliable and useful AI agents isn't writing code anymore. It's knowing what good looks like, measuring it, and steadily closing the gap. Glean is building the next generation of enterprise AI assistants and autonomous agents that reason, plan, and act across complex, multi-step workflows. This role works at the intersection of applied research, quality, and product engineering to make those systems better. The focus is on quality improvements across Glean's models and agent harnesses. That means hill-climbing with evals, evolving evalsets and agentic verifiers, post-training models for use cases like model routing, and training task-specific models for enterprise workflows. It also means bringing emerging research into the product and working closely with product teams and customers, turning learnings from evals and production usage into better outcomes through quality improvements. Use eval results and production traces to improve the entire stack, including the agent harness, model routing, prompts, and retrieval, while weighing quality against latency and cost. Work on post-training models with supervised fine-tuning (SFT) and preference- or reward-based optimization (e.g., DPO, PPO, GRPO), including generating the data for training. Build RL environments for simulating enterprise workflows, including relevant evalsets, harness support, and LLM-based verifiers for hill-climbing and for internal and external benchmarking. Improve the quality of representative evalsets, LLM judges and agentic verifiers, and calibrate them against human graders so the scores can be trusted in release decisions and model training. Work with product, platform, and research teams across Glean to land quality improvements in production. 5+ years of industry experi
Extracted skills: typescript, python, go, machine learning