Skip to content

fix(filters): allow FastTextClassifierFilter without keep_labels or remove_labels - #563

Merged
davanstrien merged 3 commits into
huggingface:mainfrom
RaphaelFakhri:fix/fasttext-annotate-only
Nov 4, 2026
Merged

davanstrien merged 3 commits into
huggingface:mainfrom
RaphaelFakhri:fix/fasttext-annotate-only

Conversation

@RaphaelFakhri

@RaphaelFakhri RaphaelFakhri commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

The FastTextClassifierFilter docstring says that if you supply neither keep_labels nor remove_labels, the block annotates each document with the label scores. In that mode the first document raises a TypeError when the model loads:

TypeError: can only concatenate list (not "NoneType") to list

The label check in the model property is written as self.keep_labels or [] + self.remove_labels or []. Python parses this as self.keep_labels or ([] + self.remove_labels) or [], so when both are None it evaluates [] + None.

This change adds parentheses so both lists default to [] before they are joined. The behavior with keep_labels or remove_labels set does not change.

Tests

  • TestFastTextClassifierFilter.test_annotates_without_keep_or_remove_labels: runs the filter with neither label list and checks that the document is kept and the scores land in its metadata. It fails on main with the TypeError above and passes with this change.
  • TestFastTextClassifierFilter.test_rejects_unknown_label: checks that an unknown label in remove_labels still raises ValueError.

Both tests replace the fastText model with a small stub, so they need no model download or training.

pytest tests/pipeline/filters/test_filters.py -k FastText
make quality

Fixes #562


Note

Low Risk
Single-expression fix in filter setup plus mocked unit tests; no auth, data, or pipeline behavior change beyond fixing the crash.

Overview
Fixes a TypeError when FastTextClassifierFilter runs in annotate-only mode (neither keep_labels nor remove_labels). Label validation used self.keep_labels or [] + self.remove_labels or [], which Python grouped as [] + None when both were unset; it now iterates over [*(self.keep_labels or []), *(self.remove_labels or [])], so the check is a no-op in that mode and unchanged when label lists are provided.

Adds TestFastTextClassifierFilter with a stubbed FastText model: metadata annotation without label lists, tuple-of-tuples keep_labels, and ValueError for unknown remove_labels.

Reviewed by Cursor Bugbot for commit af6371a. Bugbot is set up for automated code reviews on this repo. Configure here.

davanstrien and others added 2 commits November 4, 2026 10:42
Concatenating the two label lists raised TypeError when keep_labels
was a tuple of tuples, which worked before this change. Unpacking
handles None, lists and tuples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@davanstrien
davanstrien merged commit 417cf6d into huggingface:main Nov 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FastTextClassifierFilter raises TypeError when neither keep_labels nor remove_labels is set

2 participants