What Is Data Annotation and Why Do AI Teams Struggle with It?
Data annotation is the process of labelling raw data—images, text, audio, or video—so machine learning models can learn from it. Without annotated data, AI models cannot be trained effectively.
The types of annotation tasks include: image labelling (assigning objects to categories), bounding boxes (drawing rectangles around objects), semantic segmentation (pixel-level classification), text classification (assigning sentiment, intent, or topic to sentences), named entity recognition (identifying people, places, organisations), and audio transcription (converting speech to text with timestamps).
Why do UK AI teams struggle? Volume is the first challenge. A single model training project can require 10,000–100,000+ annotated samples. In-house teams cannot scale fast enough. Cost is the second challenge. Hiring full-time annotators at UK salary rates (£25,000–£30,000 annually) is expensive for short-term or variable-volume projects. Quality consistency is the third. Maintaining inter-annotator agreement across a growing team is labour-intensive.
Real-World Volume Scenario
A UK fintech company building a fraud-detection model needed 50,000 labelled transaction screenshots. At UK salaries, this would cost £8,000+ in labour alone (at 16p per annotation). Outsourcing to Kenya reduced costs to £2,000 (at 4p per annotation) whilst maintaining 92% inter-annotator agreement.
Annotation Types and Complexity Levels
Annotation tasks range from simple to highly technical. Understanding the complexity tier helps you estimate timelines and vendor capability.
Comparison
| Line Item | UK (London) | Treba (Nairobi) | Saving |
|---|---|---|---|
| Annotation Type | Complexity | UK Cost per 1,000 | Kenya Cost per 1,000 |
| Image Classification | Low | £180 | £45 |
| Bounding Box (1–3 objects) | Medium | £400 | £100 |
| Semantic Segmentation | High | £800 | £200 |
| Text Classification (sentiment) | Low | £150 | £40 |
| Named Entity Recognition (NER) | Medium | £350 | £85 |
| Audio Transcription + Timestamps | High | £600 | £150 |
Cost differences reflect not just wages, but productivity. Kenyan annotators trained by Treba average 35–40% higher throughput due to structured workflows and tool familiarity.
Quality Control Frameworks and Inter-Annotator Agreement
Outsourcing does not mean sacrificing quality. Professional providers use structured QA frameworks to maintain consistency.
Four-Layer Quality Control
Layer 1: Annotation Guidelines. Detailed written rules (e.g., "label all cars in profile view; omit toy cars and drawings") prevent ambiguity. Guidelines should include 10–20 visual examples.
Layer 2: Gold Standard Sets. A small subset (5–10%) of annotated samples validated by a subject-matter expert. New annotators calibrate against these before production work.
Layer 3: Inter-Annotator Agreement (IAA) Measurement. Cohen's Kappa or Fleiss' Kappa measures agreement across annotators. Industry standard: Kappa > 0.80 (80% agreement). Treba maintains 0.85–0.92 Kappa across most projects.
Layer 4: Spot-Check Audits. Random 5–10% sample review by a senior annotator or QA lead. Failed samples trigger retraining or task reassignment.
Example: Image Bounding Box QA
A UK insurance company needed 40,000 vehicle damage photos labelled with bounding boxes. Treba implemented: (1) 15-page annotation guide with 40 example images, (2) 500-sample gold standard set, (3) bi-weekly IAA audits (target Kappa > 0.88), (4) 10% spot-check audits. Result: 98% first-pass accuracy, project completed in 6 weeks.
Annotation Tools: CVAT, Labelbox, Scale AI, Prodigy
Your choice of tool impacts cost, speed, and quality. Here's a quick comparison:
Comparison
| Line Item | UK (London) | Treba (Nairobi) | Saving |
|---|---|---|---|
| Tool | Best For | Strengths | UK Cost/mo |
| CVAT (Open Source) | Image, video, 3D | Free, on-prem, full control | £0–£200 |
| Labelbox | Image, text, multi-modal | Collaborative, auto-labelling | £400–£2,000 |
| Scale AI | Enterprise, managed service | Outsourced QA, complex tasks | £3,000+ |
| Prodigy | NLP, text, active learning | Fast iteration, weak supervision | £50–£300/mo |
For UK teams outsourcing to Kenya, we recommend CVAT or Prodigy deployed on a shared server accessible from Nairobi. This gives you cost control and transparency.
Team Structure and Cost Breakdown
A typical outsourced annotation project follows this structure:
Comparison
| Role | Responsibility | Typical Cost (Kenya) |
|---|---|---|
| Role | Responsibility | UK Annual Cost |
| Annotation Lead / QA Manager | Guideline creation, QA audits, vendor management, retraining | £35,000–£45,000 |
| Data Annotators (team of 5) | Label data per guidelines, flagging edge cases | £125,000–£150,000 (5 × £25–£30k) |
| Project Manager (part-time, 0.5 FTE) | Timeline tracking, communication with vendor, sample review | £20,000–£25,000 (0.5 FTE) |
| Subject Matter Expert (ad hoc) | Gold standard validation, difficult cases, process improvement | £15,000–£20,000 |
Total cost (5 annotators, UK in-house): £195,000–£240,000 annually. Total cost (5 annotators, Kenya outsourced): £39,000–£57,000 annually. Saving: 75–80%.
Vendor Selection and SLA Frameworks
Not all outsourcing providers are equal. Look for:
Key Selection Criteria
- 1. Domain expertise. Have they annotated for computer vision, NLP, or your specific use case before? Ask for 2–3 references.
- 2. Scalability proof. Can they reliably deliver 5,000+ annotations per week? Request a small pilot project (500–1,000 samples) before committing to full volume.
- 3. Quality guarantees. Do they commit to a minimum Kappa score? Offer penalties for falling below target?
- 4. Pricing transparency. Fixed per-task rates are better than hourly billing. Example: £0.04 per image classification, £0.12 per bounding box.
- 5. Security and compliance. GDPR data processing agreement (DPA), ISO 27001 certification, and encrypted communication channels.
Sample SLA Framework
- Turnaround time: 70% of tasks delivered within 5 business days of submission.
- Quality: Cohen's Kappa ≥ 0.85 for 95% of projects.
- Availability: Minimum 95% team attendance month-on-month.
- Escalation: Edge cases escalated to senior annotator within 24 hours.
Key takeaways
• Data annotation is labour-intensive; outsourcing cuts costs by 60–75% whilst maintaining quality. • Quality control relies on annotation guidelines, gold standard sets, inter-annotator agreement (Kappa > 0.80), and spot-check audits. • Tool choice (CVAT, Labelbox, Prodigy) impacts cost and speed; consider shared-server deployments for overseas teams. • Team structure: 1 QA lead, 5 annotators, 0.5 project manager = £39,000–£57,000/year outsourced vs. £195,000–£240,000 in-house. • Vendor selection should prioritise domain expertise, scalability proof (pilot projects), quality guarantees, and GDPR compliance.
Written by
Treba Research
Treba editorial team — expert analysis on outsourcing, compliance, and building distributed UK–Kenya teams.
