moma/renamer/test/datasets

Test Datasets

This directory contains organized test data for the Renamer test suite.

Directory Structure

datasets/
├── README.md                     # This file
├── filenames/
│   └── filename_patterns.json   # Comprehensive filename test cases (46+ cases)
├── mediainfo/
│   └── frame_class_tests.json   # Frame class detection test cases
├── sample_mediafiles/           # Generated test files (in .gitignore)
│   └── *.mkv, *.mp4, etc.       # Empty files created from filename_patterns.json
└── expected_results/            # Reserved for future use

Note: The sample_mediafiles/ directory is generated by running fill_sample_mediafiles.py and is excluded from git. Run uv run python renamer/test/fill_sample_mediafiles.py to create these files.

Dataset Files

filenames/filename_patterns.json

Version: 2.0 Test Cases: 46+

Comprehensive dataset of media filenames with their expected extracted metadata.

Categories:

• simple (2 cases): Basic filenames with minimal metadata
• order (5 cases): Files with order numbers in various formats ([01], 01., 1.1, etc.)
• year_formats (2 cases): Different year positioning (parentheses, dots, standalone)
• database_id (3 cases): Files with TMDB/IMDB identifiers
• special_edition (4 cases): Director's Cut, Extended Edition, Remastered, etc.
• multi_audio (3 cases): Multiple audio track counts (2ukr, 4eng, 3ukr, etc.)
• cyrillic (3 cases): Non-Latin character sets (Russian, Ukrainian)
• multilingual_title (2 cases): Titles with alternative names or translations
• hdr (2 cases): HDR/SDR metadata
• resolution_formats (3 cases): Different resolution formats (1080p, 720p, 4K, 8K)
• sources (4 cases): Various source types (BDRip, WEB-DL, DVDRip, etc.)
• series (2 cases): TV series episodes
• complex (2 cases): Filenames with all metadata fields
• edge_cases (9 cases): Edge cases and unusual formatting

Format:

{
  "description": "Comprehensive test dataset for filename metadata extraction",
  "version": "2.0",
  "test_cases": [
    {
      "testname": "simple-001",
      "filename": "Movie Title (2020) BDRip [1080p,ukr,eng].mkv",
      "expected": {
        "order": null,
        "title": "Movie Title",
        "year": "2020",
        "source": "BDRip",
        "frame_class": "1080p",
        "hdr": null,
        "movie_db": null,
        "special_info": null,
        "audio_langs": "ukr,eng",
        "extension": "mkv"
      },
      "category": "simple",
      "description": "Basic filename with standard metadata"
    }
  ],
  "categories": {
    "simple": "Basic filename with minimal metadata",
    "order": "Files with order numbers in various formats",
    "year_formats": "Different year positioning formats",
    "database_id": "Contains TMDB/IMDB identifiers",
    "special_edition": "Director's Cut, Extended, Remastered, etc.",
    "multi_audio": "Multiple audio track counts",
    "cyrillic": "Non-Latin character sets (Russian, Ukrainian)",
    "multilingual_title": "Titles with alternative names or translations",
    "hdr": "HDR/SDR metadata",
    "resolution_formats": "Different resolution formats and positions",
    "sources": "Various source types (BDRip, WEB-DL, DVDRip, etc.)",
    "series": "TV series episodes",
    "complex": "Filename with multiple metadata fields",
    "edge_cases": "Edge cases and unusual formatting"
  }
}

Key Test Cases:

• Order formats: [01], 01., 1.1, [01.1]
• Year formats: (2020), 2020, .2020.
• Database IDs: [tmdbid-12345], {imdb-tt1234567}
• Special editions: [Director's Cut], [Ultimate Extended Edition], [Remastered]
• Multi-audio: 2ukr,eng, 3ukr,eng, rus,ukr,4eng
• Cyrillic titles: 12 стульев, Бриллиантовая рука
• Multilingual titles: Il racconto dei racconti (Tale of Tales)
• HDR: [2160p,HDR,ukr,eng], 2160p HDR Ukr Eng
• Resolutions: 1080p, 720p, 2160p, 4K, 8K, 4320p
• Sources: BDRip, WEB-DL, DVDRip, WEB-DLRip
• Series: S01E01, Season 1 Episode 1
• Edge cases: Title starting with number (2001 A Space Odyssey), no year, multipart (pt1), dots in title

mediainfo/frame_class_tests.json

Test cases for frame class (resolution) detection from video dimensions.

Format:

[
  {
    "testname": "test-1080p-standard",
    "resolution": [1920, 1080],
    "interlaced": "No",
    "expected_frame_class": "1080p",
    "description": "Standard 1080p Full HD"
  }
]

Note: The expected_results/ directory is reserved for future use. It may contain expected extraction results for integration testing across multiple extractors.

Usage in Tests

Using conftest.py Fixtures

The test suite provides convenient fixtures for loading datasets:

import pytest

# Use the load_filename_patterns fixture
def test_with_filename_patterns(load_filename_patterns):
    """Test using the filename patterns dataset."""
    test_cases = load_filename_patterns
    assert len(test_cases) >= 46

    for case in test_cases:
        filename = case['filename']
        expected = case['expected']
        # ... your test logic

# Use the load_frame_class_tests fixture
def test_with_frame_class_data(load_frame_class_tests):
    """Test using the frame class dataset."""
    test_cases = load_frame_class_tests
    # ... your test logic

Loading Datasets Manually

import json
from pathlib import Path

def load_dataset(dataset_name):
    """Load a dataset file from datasets directory."""
    from renamer.test.conftest import load_dataset
    return load_dataset(dataset_name)

# Load filename patterns
data = load_dataset("filename_patterns")
test_cases = data["test_cases"]

# Load frame class tests
frame_tests = load_dataset("frame_class_tests")

Parametrized Tests

import pytest
from pathlib import Path
from renamer.extractors.filename_extractor import FilenameExtractor

# Load test cases at module level
def load_test_cases():
    dataset_file = Path(__file__).parent / "datasets" / "filenames" / "filename_patterns.json"
    with open(dataset_file, 'r', encoding='utf-8') as f:
        data = json.load(f)
        return data['test_cases']

@pytest.mark.parametrize("test_case", load_test_cases(), ids=lambda tc: tc['testname'])
def test_filename_extraction(test_case):
    """Test filename extraction with all test cases."""
    extractor = FilenameExtractor(Path(test_case["filename"]))

    expected = test_case["expected"]
    assert extractor.extract_title() == expected["title"]
    assert extractor.extract_year() == expected["year"]
    assert extractor.extract_source() == expected["source"]
    assert extractor.extract_frame_class() == expected["frame_class"]
    assert extractor.extract_audio_langs() == expected["audio_langs"]

Filtering by Category

def test_order_patterns(load_filename_patterns):
    """Test only order-related patterns."""
    order_cases = [
        case for case in load_filename_patterns
        if case['category'] == 'order'
    ]

    for case in order_cases:
        # Test order extraction
        pass

def test_cyrillic_titles(load_filename_patterns):
    """Test only Cyrillic title patterns."""
    cyrillic_cases = [
        case for case in load_filename_patterns
        if case['category'] == 'cyrillic'
    ]

    for case in cyrillic_cases:
        # Test Cyrillic handling
        pass

Using Sample Files

from renamer.test.conftest import get_test_file_path

# Get path to a sample file from the dataset
sample_file = get_test_file_path("Movie Title (2020) BDRip [1080p,ukr,eng].mkv")
assert sample_file.exists()

# Sample files in sample_mediafiles/ are empty placeholder files
# generated from filename_patterns.json for testing file system operations

Generating Sample Media Files

The sample_mediafiles/ directory contains empty files for all test cases in filename_patterns.json. These files are generated automatically and should not be committed to git.

Generate files:

# From project root
uv run python renamer/test/fill_sample_mediafiles.py

Output:

Creating sample media files in: /path/to/renamer/test/datasets/sample_mediafiles
Test cases in dataset: 46

  ✅ Created: Movie Title (2020) BDRip [1080p,ukr,eng].mkv
  ✅ Created: [01] Movie Title (2020) BDRip [1080p,ukr,eng].mkv
  ...

Summary:
  Created: 46 files
  Skipped (already exist): 0 files
  Errors: 0 files

Note: These files are in .gitignore and will not be committed. Run the script after cloning the repository to generate them for local testing.

Adding New Test Data

Adding Filename Patterns

Edit filenames/filename_patterns.json:

{
  "testname": "your-test-name",
  "filename": "Your Movie Title (2024) [1080p].mkv",
  "expected": {
    "order": null,
    "title": "Your Movie Title",
    "year": "2024",
    "source": null,
    "frame_class": "1080p",
    "hdr": null,
    "movie_db": null,
    "special_info": null,
    "audio_langs": "",
    "extension": "mkv"
  },
  "category": "simple",
  "description": "Brief description of what this tests"
}

Adding MediaInfo Tests

Edit mediainfo/frame_class_tests.json:

{
  "testname": "your-test-name",
  "resolution": [1920, 1080],
  "interlaced": "No",
  "expected_frame_class": "1080p",
  "description": "What resolution/format this tests"
}

Adding Sample Files

Create empty files in filenames/sample_files/:

touch filenames/sample_files/"Your Movie Title (2024) [1080p].mkv"

Test Coverage by Category

Order Patterns (5 cases)

• [01] - Square bracket order
• 01. - Dot order
• 1.1 - Decimal order
• [01.1] - Complex bracketed decimal
• 9. - Single digit order

Year Formats (2 cases)

• (2020) - Standard parentheses (most common)
• 2020 - Standalone year
• .2020. - Dot-separated year

Database IDs (3 cases)

• [tmdbid-12345] - TMDB with square brackets
• {imdb-tt1234567} - IMDB with curly braces
• Multiple IDs in single filename

Audio Languages (3 cases)

• 2ukr,eng - Multiple tracks of same language
• rus,ukr,4eng - Mixed languages with counts
• 3ukr,eng - Three Ukrainian tracks

Cyrillic (3 cases)

• Full Cyrillic titles
• Cyrillic with numbers
• Mixed Cyrillic/Latin

Edge Cases (9 cases)

• Title starting with number (2001, 9)
• Title with colons, dashes, apostrophes
• Title with dots
• No brackets around metadata
• No year present
• Multipart films (pt1, pt2)
• Remastered versions
• Multiple resolution indicators
• Series episodes

Data Quality Guidelines

When adding test data:

1. Completeness: Include all expected fields, use null for missing values
2. Accuracy: Verify expected values match actual extractor output
3. Coverage: Include edge cases and corner cases
4. Categorization: Assign appropriate category
5. Documentation: Provide clear description
6. Naming: Use descriptive testname (e.g., order-001, cyrillic-002)
7. Realism: Use realistic filenames from actual use cases

Maintenance

• Keep datasets in sync with test requirements
• Document expected behavior in descriptions
• Use consistent naming conventions (lowercase, hyphens)
• Group related test cases together (same category)
• Update README when adding new dataset types
• Run tests after adding new data to validate
• Remove obsolete test cases when extractors change

Version History

• v2.0: Comprehensive reorganization with 46+ test cases across 14 categories

  • Added testname field for better test identification
  • Added category field for test organization
  • Expanded coverage to include all extractor fields
  • Added edge cases and special formatting tests

• v1.0: Initial dataset with basic test cases