Active Learning in Machine Learning

The Labeling Cost Crisis

Labeled data is the most expensive ingredient in supervised machine learning:

• Medical imaging: A single chest X-ray annotation by a radiologist costs $5-15. A pneumonia detector needs 50,000+ images —$250K-750K in annotation alone.
• NLP for Indian languages: Named entity labeling in Hindi-Marathi code-mixed text requires rare bilingual annotators at ₹800-2,000/hour. A 100K-sentence NER dataset could cost ₹40-80 lakhs.
• Autonomous driving: LiDAR 3D bounding box annotation costs $6-8 per frame. Annotation budgets routinely exceed$10M.

The key insight: 60-80% of labeled samples are redundant. They fall in well-separated regions where the model is already confident. The remaining 20-40% near decision boundaries, in ambiguous regions, or representing rare classes — these are the samples that actually teach the model.

Why Random Sampling Fails

1. Redundant labeling: The oracle labels easy samples the model already handles correctly.
2. Class imbalance blindness: Rare classes are undersampled proportionally. A 95/5 class split gives you 95 easy labels for every 5 informative ones.
3. Boundary neglect: Decision boundary regions — where labeling provides maximum gradient signal — are sampled at the same rate as trivial regions.

Active learning solves all three by directing the oracle's attention to the most informative samples. Empirically, it achieves the same accuracy as random sampling with 2-10x fewer labeled examples.

The Modern Catalyst: LLM Fine-Tuning

Active learning has gained renewed importance for LLMs. Fine-tuning or training reward models for RLHF requires high-quality human labels costing $0.50-5.00 per example. Active learning helps select the most impactful fine-tuning examples, reducing costs by 3-5x.

The Analogy: A Student Choosing What to Study

Imagine a medical student with 10,000 practice questions but time for only 1,000. A passive student picks randomly. A smart student:

1. Takes a diagnostic test to identify weak areas
2. Focuses on topics where they are most uncertain — cardiology at 50% accuracy, not dermatology at 95%
3. Seeks out edge cases — tricky differential diagnoses, not textbook presentations
4. Periodically reassesses as weaknesses shift

This is exactly how active learning works. The model trains on a small set, identifies uncertainty regions, asks the oracle about the hardest cases, and repeats.

The Information-Theoretic View

Every unlabeled sample carries expected information gain about the model's parameters. A 99%-confident sample carries almost no information. A 50/50-uncertain sample carries maximum information. Active learning labels the highest-information samples first — analogous to optimal experimental design in statistics.

Why It Works: The Decision Boundary Argument

For classification, accuracy depends on the decision boundary. Points far from the boundary are easy — any reasonable boundary classifies them correctly. Points near the boundary determine its shape. Active learning preferentially labels boundary points, providing maximal gradient signal. This is why uncertainty sampling — choosing points where the model is least certain — is the simplest yet most effective strategy.

Problem Setup

Let $\mathcal{X}$ be the input space, $\mathcal{Y}$ the label space. We have a small labeled set $\mathcal{L} = \{(x_i, y_i)\}_{i=1}^{n_0}$, a large unlabeled pool $\mathcal{U} = \{x_j\}_{j=1}^{N}$ where $N \gg n_0$, an oracle $\mathcal{O}: \mathcal{X} \to \mathcal{Y}$, and budget $B$.

The Active Learning Loop

At each iteration $t$: (1) Train model $\theta_t$, (2) Score unlabeled samples via acquisition function $a(x)$, (3) Select batch $\mathcal{S}_t$ of size $b$, (4) Query oracle, (5) Update $\mathcal{L}_{t+1} = \mathcal{L}_t \cup \{(x, \mathcal{O}(x)) : x \in \mathcal{S}_t\}$.

Acquisition Functions

Uncertainty Sampling:

• Least Confidence: $a_{LC}(x) = 1 - \max_y P(y|x; \theta)$
• Margin Sampling: $a_{M}(x) = 1 - (P(y_1|x) - P(y_2|x))$ for top-2 classes
• Entropy: $a_{H}(x) = -\sum_y P(y|x) \log P(y|x)$

Query-by-Committee (QBC) — Committee of models $\{f_{\theta_c}\}_{c=1}^C$, select by vote entropy: $a_{QBC}(x) = -\sum_y \frac{V(y|x)}{C} \log \frac{V(y|x)}{C}$

Expected Model Change (EMC) — Expected gradient length: $a_{EMC}(x) = \sum_y P(y|x; \theta) \|\nabla_\theta \ell(f_\theta(x), y)\|$

BALD — Mutual information between parameters and label: $a_{BALD}(x) = H(y|x, \mathcal{L}) - \mathbb{E}_{P(\theta|\mathcal{L})}[H(y|x, \theta)]$

First term = total entropy (epistemic + aleatoric), second = expected entropy (aleatoric only). Difference = pure epistemic uncertainty.

Batch Mode Active Learning

Core-Set: Minimize maximum distance from any unlabeled point to nearest labeled point (k-center problem).

BatchBALD: Jointly maximize mutual information for the entire batch: $I(y_{\mathcal{S}}; \theta | x_{\mathcal{S}}, \mathcal{L})$. Uses greedy submodular approximation.

Scenarios

• Pool-based: Access to full pool, score all, select top-b. Most common.
• Stream-based: Samples arrive one at a time; query if $a(x) > \tau$.
• Membership query synthesis: Generate synthetic inputs. Rarely used with human oracles.

What We Covered

Active learning is a training paradigm where the model selects the most informative unlabeled samples for a human oracle to label, achieving target accuracy with 2-10x fewer labels than random sampling. The core mechanism is the acquisition function — a scoring method that estimates each sample's expected information gain.

Key Acquisition Strategies

• Uncertainty Sampling (entropy, least confidence, margin): simplest and most widely used. Works well with calibrated models.
• Query-by-Committee / BALD: multiple models (or MC Dropout) select samples with maximum disagreement. BALD disentangles epistemic from aleatoric uncertainty.
• Expected Model Change / BADGE: select samples causing the largest parameter update. BADGE uses gradient embeddings for joint uncertainty-diversity.
• Core-Set Selection: maximize feature space coverage. Pure diversity, no uncertainty.

Three Scenarios

Pool-based (most common): score all unlabeled samples, select top-b. Stream-based: decide per sample as it arrives. Membership query synthesis: generate synthetic samples to query.

Critical Success Factors

1. Calibration: uncalibrated models give bad uncertainty estimates. Use temperature scaling, MC Dropout, or ensembles.
2. Batch diversity: greedy selection creates redundant batches. Use hybrid uncertainty+diversity or BADGE.
3. Oracle quality: AL selects the hardest samples. Use multi-annotator, agreement checks, gold standards.
4. Cold start: early iterations have unreliable uncertainty. Use transfer learning, larger seed sets, or random blending.
5. Baseline comparison: always compare against random sampling.

Production Pattern

Integrate AL with an annotation tool (Label Studio, Labelbox, Prodigy). The ML backend scores unlabeled tasks. The annotation tool presents highest-priority tasks first. Run on a daily/weekly cycle: retrain → score → annotate → repeat.