Find The Best
AI Jobs

Senior Machine Learning Engineer - Model Evaluations, Public Sector The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, and multimodal pipelines—into mission-critical government environments. We build evaluation frameworks that ensure these models operate reliably, safely, and effectively under real-world constraints. As an ML Engineer, you will design, implement, and scale automated evaluation pipelines that help customers trust and operationalize advanced AI systems across defense, intelligence, and federal missions. You will: - Develop and maintain automated evaluation pipelines for ML models across functional, performance, robustness, and safety metrics, including LLM-judge–based evaluations. - Design test datasets and benchmarks to measure generalization, bias, explainability, and failure modes. - Build evaluation frameworks for LLM agents, including infrastructure for scenario-based and environment-based testing. - Conduct comparative analyses of model architectures, training procedures, and evaluation outcomes. - Implement tools for continuous monitoring, regression testing, and quality assurance for ML systems. - Design and execute stress tests and red-teaming workflows to uncover vulnerabilities and edge cases. - Collaborate with operations teams and subject matter experts to produce high-quality evaluation datasets. - Comfortable with light travel (approximately 10%) for customer interaction and team needs. This role will require an active security clearance or the ability to obtain a security clearance. Ideally you’d have: - Experience in computer vision, deep learning, reinforcement learning, or NLP in production settings. - Strong programming skills in Python; experience with TensorFlow or PyTorch. - Background in algorithms, data structures, and object-oriented programming. - Experience with LLM pipelines, simulation environments, or automated evaluation systems. - Ability to convert research insights into measurable evaluation criteria. Nice to haves: - Graduate degree in CS, ML, or AI. - Cloud experience (AWS, GCP) and model deployment experience. - Experience with LLM evaluation, CV robustness, or RL validation. - Knowledge of interpretability, adversarial robustness, or AI safety frameworks. - Familiarity with ML evaluation frameworks and agentic model design. - Experience in regulated, classified, or mission-critical ML domains. Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position and may be inclusive of several career levels at Scale; it will be determined during the interview process based on work location and additional factors, including job-related skills, experience, qualifications, interview performance, and relevant education or training. Scale employees in eligible roles are also granted equity based compensation, subject to Board of Director approval. Your recruiter can share more about the specific salary range for your preferred location during the hiring process, and confirm whether the hired role will be elig

AI is becoming vitally important in every function of our society. At Scale, our mission is to accelerate the development of AI applications. For 9 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including generative AI, defense applications, and autonomous vehicles. With our recent investment from Meta, we are doubling down on building out state of the art post-training algorithms to reach the performance necessary for complex agents in enterprises around the world.  The Enterprise ML Research Lab works on the front lines of this AI revolution. We are working on an arsenal of proprietary research and resources that serve all of our enterprise clients. As an ML Sys Research Engineer, you’ll work on building out the algorithms for our next-gen Agent RL training platform, support large scale training, and research and integrate state-of-the-art technologies to optimize our ML system. Your customer will be other MLREs and AAIs on the Enterprise AI team who are taking the training algorithms and applying them to client use-cases ranging from next-generation AI cybersecurity firewall LLMs to training foundation healthtech search models. If you are excited about shaping the future of the modern AI movement, we would love to hear from you! You will:  - Build, profile and optimize our training and inference framework. - Post-train state of the art models, developed both internally and from the community, to define stable post-training recipes for our enterprise engagements.  - Collaborate with ML teams to accelerate their research and development, and enable them to develop the next generation of models and data curation.. - Create a next-gen agent training algorithm for multi-agent/multi-tool rollouts. Ideally you’d have: - At least 1-3 years of LLM training in a production environment - Passionate about system optimization - Experience with post-training methods like RLHF/RLVR and related algorithms like PPO/GRPO etc. - Ability to demonstrate know-how on how to operate the architecture of the modern GPU cluster - Experience with multi-node LLM training and inference - Strong software engineering skills, proficient in frameworks and tools such as CUDA, Pytorch, transformers, flash attention, etc.  - Strong written and verbal communication skills to operate in a cross functional team environment. - PhD or Masters in Computer Science or a related field Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position and may be inclusive of several career levels at Scale; it will be determined during the interview process based on work location and additional factors, including job-related skills, experience, qualifications, interview performance, and relevant education or training. Scale employees in eligible roles are also granted equity based compensation, subject to Board of Director approval. Your recruiter can share more about the specific salary range for your preferred location during the hiring process, and confirm whether the hired role will be eligible for equity grant. You'll also receive benefits including, but not limited to: comprehensive health, dental and vision coverage, retirement benefits, a learning and development stipend, and generous PTO. Additionally, this role may be eligibl

About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, fueling the most exciting advancements in AI, including generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re amplifying access to high-quality data to drive progress toward Artificial General Intelligence (AGI). Building on our history of model evaluation with enterprise and government customers, we are expanding our capabilities to set new standards for both public and private evaluations. About This Role This role operates at the forefront of AI research and real-world implementation, with a strong focus on reasoning within large language models (LLMs). The ideal candidate will study the data types critical for advancing LLM-based agents, including browser and software engineering (SWE) agents. You will play a key role in shaping Scale’s data strategy by identifying the most effective data sources and methodologies for improving LLM reasoning. Success in this role requires a deep understanding of LLMs, planning algorithms, and novel approaches to agentic reasoning, as well as creativity in tackling challenges related to data generation, model interaction, and evaluation. You will contribute to impactful research on language model reasoning , collaborate with external researchers, and work closely with engineering teams to bring state-of-the-art advancements into scalable, real-world solutions. Ideally, you’d have: - Practical experience working with LLMs, with proficiency in frameworks like PyTorch, JAX, or TensorFlow. You should also be skilled at rapidly interpreting research literature and turning new ideas into working prototypes. - A track record of published research in top ML and NLP venues (e.g., ACL, EMNLP, NAACL, NeurIPS, ICML, ICLR, CoLLM, etc.). - At least three years of experience solving complex ML challenges, either in a research setting or product development, particularly in areas related to LLM capabilities and reasoning. - Strong written and verbal communication skills, along with the ability to work effectively across teams. Nice to have: - Hands-on experience fine-tuning open-source LLMs or leading bespoke LLM fine-tuning projects using PyTorch/JAX. - Research and practical experience in building applications and evaluations related to LLM-based agents, including tool-use, text-to-SQL, browser agents, coding agents, and GUI agents. - Experience with agent frameworks such as OpenHands, Swarm, LangGraph, or similar. - Familiarity with advanced agentic reasoning techniques such as STaR and PLANSEARCH. - Proficiency in cloud-based ML development, with experience in AWS or GCP environments. Our research interviews are designed to assess candidates' ability to prototype and debug ML models, their depth of understanding in research concepts, and their alignment with our organizational culture. We do not conduct LeetCode-style problem-solving assessments. Compensation packages at Scale for eligible roles include base salary, equity, and benef

About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re accelerating the abundance of frontier data to pave the road to Artificial General Intelligence (AGI), and building upon our prior model evaluation work with enterprise customers and governments, to deepen our capabilities and offerings for both public and private evaluations. About the ACE team  The Agent Capabilities & Environments (ACE) team, part of Scale’s Research organization, brings together customer-facing Researchers and Applied AI Engineers. Our core mission includes research on agent environments and RL reward signals, benchmarking autonomous agent performance across real-world scenarios and environments, creating robust data programs to improve Large Language Models (LLMs) agentic capabilities and building foundational tools and frameworks for evaluating models as agents. ACE focuses on autonomous agents that dynamically interact with diverse external environments, including code repositories, GUI interfaces, browsers, and more.  About This Role This role is at the intersection of cutting-edge AI research and practical application, with a focus on studying the data types essential for building state-of-the-art agents, such as browser and SWE agents. The ideal candidate will explore the data landscape needed to advance intelligent, adaptable AI agents, guiding the data strategy at Scale to drive innovation. This position requires not only expertise in LLM agents and planning algorithms but also creativity in addressing novel challenges related to data, interaction, and evaluation. You will contribute to impactful research publications on agents, collaborate with customer researchers, and work alongside the engineering team to translate these advancements into real-world, scalable solutions. Ideally you’d have:  - Practical experience working with LLMs, with proficiency in frameworks like Pytorch, Jax, or Tensorflow. You should also be adept at interpreting research literature and quickly turning new ideas into prototypes. - A track record of published research in top ML venues (e.g., ACL, EMNLP, NAACL, NeurIPS, ICML, ICLR, COLM, etc.) - At least three years of experience addressing sophisticated ML problems, either in a research setting or product development. - Strong written and verbal communication skills and the ability to operate cross-functionally.  Nice to have: - Hands-on experience with open source LLM fine-tuning or involvement in bespoke LLM fine-tuning projects using Pytorch/Jax.  - Hands-on experience and publications in building applications and evaluations related to AI agents such as tool-use, text2SQL, browser agents, coding agents and GUI agents. - Hands-on experience with agent frameworks such as OpenHands, Swarm, LangGraph, etc. - Familiarity with agentic reasoning methods such as STaR and PLANSEARCH  - Experience working with cloud technology stack (eg. AWS or GCP) and developing machine learning models in a cloud environment. Our research interviews are crafted to assess candidates' skills in practical ML prototyping and debugging, their grasp of research concepts, and their alignment with our organizational culture. We will not ask any Lee

The goal of a Senior Machine Learning Engineer at Scale is to leverage techniques in the fields of generative AI, computer vision, reinforcement learning, and agentic AI to improve Scale's products and customer experience in production environments. Our machine learning engineers take advantage of robust internal infrastructure and  unique access to massive datasets to deliver improvements to our customers. Our Public Sector Machine Learning team is focused on deploying cutting-edge models to mission-critical government systems through products like Donovan and Thunderforge . Our work spans multiple modalities, with a strong focus on both large language models and computer vision. On the LLM side, we are developing agentic systems that help solve complex operational and planning challenges for government partners. This includes building agent frameworks that integrate with custom retrieval pipelines and production APIs, as well as evaluation tools to benchmark and refine agent behavior. We're also advancing research in areas like reinforcement learning for agentic LLMs, with successful deployment into real-world operational environments. On the computer vision front, we're training advanced models to increase labeling throughput and automate perception tasks. Our efforts include building large-scale fine-tuning pipelines, training models across multiple modalities, and developing generalizable vision foundation models to support a wide range of defense applications. You will: - Take state of the art models developed internally and from the community, use them in production to solve problems for our customers and taskers - Improve and maintain production models through retraining, hyperparameter tuning, and architectural updates, while preserving core performance characteristics - Collaborate with product and research teams to identify and prototype ML-driven product enhancements, including for upcoming product lines - Work with massive datasets to develop both generic models as well as fine tune models for specific products - Build scalable machine learning infrastructure to automate and optimize our ML services - Serve as a cross-functional representative and advocate for machine learning techniques across engineering and product organizations - Be comfortable learning new technologies quickly and managing multiple priorities in a fast-paced environment - Comfortable with light travel (approximately 10%) for customer interaction and team needs - This role will require an active security clearance or the ability to obtain a security clearance Ideally You’d Have: - Extensive experience with GenAI, Agentic AI, natural language processing, deep learning and deep reinforcement learning, or computer vision in a production environment - Solid background in algorithms, data structures, and object-oriented programming - Strong programing skills in Python, experience in Tensorflow or PyTorch Nice to Haves: - Graduate degree in Computer Science, Machine Learning or Artificial Intelligence specialization - Experience working with cloud platforms (eg. AWS or GCP) and deploying machine learning models in cloud environments - Experience with computer vision, generative AI models, large language models, or agentic systems - Familiarity with ML evaluation frameworks and agentic model design

Scale AI is seeking a highly technical and strategic Staff / Senior Staff Machine Learning Engineer to act as the Tech Lead (TL) for our next generation of deep research agents for the Enterprise. This high-impact role will drive the technical direction and oversight for Deep Research Agent Development , translating cutting-edge research in Generative AI, Large Language Models (LLMs), and Agentic Frameworks into robust, scalable, and high-impact production systems that enhance enterprise operations, analytics, and core efficiency. The ideal candidate thrives in a fast-paced environment, has a passion for both deep technical work and mentoring, and is capable of setting a long-term technical strategy for a critical domain while maintaining a strong, hands-on delivery focus. Responsibilities Technical Leadership & Vision - Set the Technical Roadmap: Define and own the technical strategy, architecture, and roadmap for Deep Research Agents for the Enterprise, ensuring alignment with Scale AI’s overall AI strategy and business goals. - Drive Breakthrough Research to Production: Lead the end-to-end development, from initial research to production deployment, to landing on customer impact, with a focus on integrating diverse data modalities . - Core Agent Capabilities Development: - Advanced Knowledge Retrieval: Architect and implement state-of-the-art retrieval systems to ensure the agents provide accurate and comprehensive answers from public and proprietary data sources from enterprises. - Data analysis: Design and champion the development of data analysis agents that accurately translate complex natural language queries into executable SQL/code against diverse enterprise data schemas. - Multimodal Intelligence: Lead the integration of Multimodal AI capabilities to process and extract structured information from visual documents, tables, and forms, enriching the agent's knowledge base. - Architecture & Design: Design and champion highly scalable, reliable, and low-latency infrastructure and frameworks for building, orchestrating, and evaluating multi-agent systems at enterprise scale. - Technical Excellence: Serve as the technical authority for the team, leading design reviews, defining ML engineering best practices, and ensuring code quality, security, and operational excellence for all agent systems. Team Leadership & Mentorship - Lead and Mentor: Technically lead and mentor a team of Machine Learning Engineers and Research Scientists, fostering a culture of innovation, rigorous engineering, rapid iteration, and technical depth. - Recruiting & Growth: Partner with management to hire, onboard, and grow top-tier talent, helping to shape the long-term structure and capabilities of the team. - Cross-Functional Influence: Collaborate effectively with Product Managers, Data Scientists, and other engineering/science teams to translate ambiguous, high-level business problems into concrete, executable technical specifications and impactful agent sol