Annotation Program Manager
About Us
Are you ready to build the future of supply chain? At Gather AI, we're not just creating software, we're pioneering a new era of warehouse intelligence. We've developed a groundbreaking, vision-powered platform that uses autonomous drones and existing equipment to capture real-time data, completely digitizing workflows that have historically been manual and error-prone. This means facilities operate smarter, safer, and more efficiently, ultimately redefining "on-time, in full" delivery.
If you're looking for an opportunity to contribute to truly transformative technology and make a significant impact in a vital industry, Gather AI is the place for you. We're leading the charge in the rapidly evolving robotics industry, and we invite you to join us in reshaping the global supply chain, one intelligent warehouse at a time.
About the Team
Our engineering organization spans autonomy, computer vision and machine learning, embedded and hardware systems, full-stack, and cloud, all working in parallel across multiple active product lines tied to live customer deployments. It's a technically deep, fast-moving team where individual contributors carry real accountability and the work shows up directly in customer operations. We're at the stage where the complexity of running concurrent programs across disciplines has outpaced informal coordination, and we're ready to fix that.
About the Role
We are looking for an experienced Annotation Program Manager to own and scale our production annotation operations supporting computer vision and machine learning initiatives. This role is responsible for managing external annotation partners, improving annotation quality, maintaining documentation, and ensuring high-quality labeled datasets are delivered on schedule to support model development.
You will work closely with Machine Learning Engineers, Data Engineers, and Operations teams while serving as the primary owner of our annotation programs and vendor relationship. Success in this role requires a combination of technical problem-solving, operational excellence, and strong project management.
What You'll Do- Own day-to-day management of our annotation vendor relationship, including workload planning, throughput monitoring, and SLA management.
- Coordinate annotation batches, scheduling, and delivery timelines across multiple production programs.
- Monitor annotation quality and continuously improve labeling accuracy through structured QA processes.
- Develop, maintain, and improve annotation SOPs, taxonomies, and documentation.
- Build and maintain Python-based workflows for annotation data preparation, API integrations, reporting, and validation.
- Partner with ML teams to prioritize annotation work that maximizes model performance.
- Track work across Jira and communicate project status, risks, and capacity needs to stakeholders.
- Drive operational improvements that reduce annotation turnaround time while maintaining high quality.
Within your first three months, you will be expected to:
Own Annotation Operations
- Take over day-to-day coordination with CloudFactory from existing program owners.
- Own weekly turnaround time (TAT) reporting, batch scheduling, and vendor escalation management.
- Become the primary operational contact for annotation delivery.
Become Fluent in Existing Annotation Programs
Develop working expertise in both active production annotation programs:
- MHEV forklift event labeling (PICK_UP / DROP_OFF) using the internal MHEV annotation application
- Drone Human-in-the-Loop (HIL) annotation pipeline supporting case counting and empty detection through the CloudFactory API
Improve Annotation Quality
- Audit completed annotation batches across both programs.
- Run inter-annotator agreement (IAA) analysis against gold-set datasets.
- Identify common annotation errors and present recommendations to ML leads.
- Lead calibration sessions with the annotation vendor as needed.
Own Documentation
Take ownership of:
- MHEV Event Annotation SOP
- Barcode + Location Root Cause Analysis (RCA) SOP
Responsibilities include:
- Version control
- Closing documentation gaps
- Adding edge-case guidance
- Ensuring all annotators are using current documentation
- Annotation vendor management, including onboarding, batch management, SLA tracking, and calibration sessions
- Python scripting for data preparation, automation, reporting, validation, and API integrations
- Writing annotation SOPs, taxonomies, and detailed labeling guidelines
- Annotation quality assurance using inter-annotator agreement (IAA), gold-set validation, and structured quality programs
- Cloud storage and data handling using Azure Blob Storage or AWS S3, along with JSON and CSV data formats
- Strong Jira project management and task organization across multiple concurrent annotation programs
- Computer vision annotation experience including video event labeling, bounding boxes, polygon segmentation, or similar workflows
- Experience with annotation platforms such as Labelbox, Scale AI, CVAT, Hasty, or V7 Darwin
- Familiarity with active learning or ML-assisted annotation prioritization
Required
- 3–6 years of experience in data operations, annotation program management, ML data engineering, or a closely related role
- Experience managing an external annotation vendor operating under production SLAs
- Strong Python programming skills for data preparation and API integrations
- Experience authoring annotation guidelines or standard operating procedures
- Understanding of data privacy and confidentiality requirements when working with customer operational data
Preferred
- Experience with CloudFactory
- Experience in warehouse automation, robotics, or computer vision applications
- Experience with additional annotation platforms and APIs
- Python
- REST APIs (CloudFactory API or equivalent annotation platform APIs)
- Azure Blob Storage or AWS S3
- Jira
We're looking for someone who is:
- Highly organized and able to manage multiple concurrent workstreams
- Collaborative, respectful, and an excellent cross-functional partner
- Detail-oriented with a strong focus on annotation quality
- Resourceful and comfortable building processes where they don't yet exist
- Persistent in driving vendor accountability and meeting SLAs
- Comfortable managing external teams and influencing without direct authority
- Analytical, with the ability to identify root causes of annotation quality issues and implement lasting improvements
Explore more remote jobs
United States
Mexico