moma/renamer/test/datasets/README.md

# Test Datasets

This directory contains organized test data for the Renamer test suite.

## Directory Structure

```
datasets/
├── README.md                     # This file
├── filenames/
│   └── filename_patterns.json   # Comprehensive filename test cases (46+ cases)
├── mediainfo/
│   └── frame_class_tests.json   # Frame class detection test cases
├── sample_mediafiles/           # Generated test files (in .gitignore)
│   └── *.mkv, *.mp4, etc.       # Empty files created from filename_patterns.json
└── expected_results/            # Reserved for future use
```

**Note**: The `sample_mediafiles/` directory is generated by running `fill_sample_mediafiles.py`
and is excluded from git. Run `uv run python renamer/test/fill_sample_mediafiles.py` to create these files.

## Dataset Files

### filenames/filename_patterns.json

**Version**: 2.0
**Test Cases**: 46+

Comprehensive dataset of media filenames with their expected extracted metadata.

**Categories**:
- `simple` (2 cases): Basic filenames with minimal metadata
- `order` (5 cases): Files with order numbers in various formats ([01], 01., 1.1, etc.)
- `year_formats` (2 cases): Different year positioning (parentheses, dots, standalone)
- `database_id` (3 cases): Files with TMDB/IMDB identifiers
- `special_edition` (4 cases): Director's Cut, Extended Edition, Remastered, etc.
- `multi_audio` (3 cases): Multiple audio track counts (2ukr, 4eng, 3ukr, etc.)
- `cyrillic` (3 cases): Non-Latin character sets (Russian, Ukrainian)
- `multilingual_title` (2 cases): Titles with alternative names or translations
- `hdr` (2 cases): HDR/SDR metadata
- `resolution_formats` (3 cases): Different resolution formats (1080p, 720p, 4K, 8K)
- `sources` (4 cases): Various source types (BDRip, WEB-DL, DVDRip, etc.)
- `series` (2 cases): TV series episodes
- `complex` (2 cases): Filenames with all metadata fields
- `edge_cases` (9 cases): Edge cases and unusual formatting

**Format**:
```json
{
  "description": "Comprehensive test dataset for filename metadata extraction",
  "version": "2.0",
  "test_cases": [
    {
      "testname": "simple-001",
      "filename": "Movie Title (2020) BDRip [1080p,ukr,eng].mkv",
      "expected": {
        "order": null,
        "title": "Movie Title",
        "year": "2020",
        "source": "BDRip",
        "frame_class": "1080p",
        "hdr": null,
        "movie_db": null,
        "special_info": null,
        "audio_langs": "ukr,eng",
        "extension": "mkv"
      },
      "category": "simple",
      "description": "Basic filename with standard metadata"
    }
  ],
  "categories": {
    "simple": "Basic filename with minimal metadata",
    "order": "Files with order numbers in various formats",
    "year_formats": "Different year positioning formats",
    "database_id": "Contains TMDB/IMDB identifiers",
    "special_edition": "Director's Cut, Extended, Remastered, etc.",
    "multi_audio": "Multiple audio track counts",
    "cyrillic": "Non-Latin character sets (Russian, Ukrainian)",
    "multilingual_title": "Titles with alternative names or translations",
    "hdr": "HDR/SDR metadata",
    "resolution_formats": "Different resolution formats and positions",
    "sources": "Various source types (BDRip, WEB-DL, DVDRip, etc.)",
    "series": "TV series episodes",
    "complex": "Filename with multiple metadata fields",
    "edge_cases": "Edge cases and unusual formatting"
  }
}
```

**Key Test Cases**:
- Order formats: `[01]`, `01.`, `1.1`, `[01.1]`
- Year formats: `(2020)`, `2020`, `.2020.`
- Database IDs: `[tmdbid-12345]`, `{imdb-tt1234567}`
- Special editions: `[Director's Cut]`, `[Ultimate Extended Edition]`, `[Remastered]`
- Multi-audio: `2ukr,eng`, `3ukr,eng`, `rus,ukr,4eng`
- Cyrillic titles: `12 стульев`, `Бриллиантовая рука`
- Multilingual titles: `Il racconto dei racconti (Tale of Tales)`
- HDR: `[2160p,HDR,ukr,eng]`, `2160p HDR Ukr Eng`
- Resolutions: `1080p`, `720p`, `2160p`, `4K`, `8K`, `4320p`
- Sources: `BDRip`, `WEB-DL`, `DVDRip`, `WEB-DLRip`
- Series: `S01E01`, `Season 1 Episode 1`
- Edge cases: Title starting with number (`2001 A Space Odyssey`), no year, multipart (`pt1`), dots in title

### mediainfo/frame_class_tests.json

Test cases for frame class (resolution) detection from video dimensions.

**Format**:
```json
[
  {
    "testname": "test-1080p-standard",
    "resolution": [1920, 1080],
    "interlaced": "No",
    "expected_frame_class": "1080p",
    "description": "Standard 1080p Full HD"
  }
]
```

**Note**: The `expected_results/` directory is reserved for future use. It may contain
expected extraction results for integration testing across multiple extractors.

## Usage in Tests

### Using conftest.py Fixtures

The test suite provides convenient fixtures for loading datasets:

```python
import pytest

# Use the load_filename_patterns fixture
def test_with_filename_patterns(load_filename_patterns):
    """Test using the filename patterns dataset."""
    test_cases = load_filename_patterns
    assert len(test_cases) >= 46

    for case in test_cases:
        filename = case['filename']
        expected = case['expected']
        # ... your test logic

# Use the load_frame_class_tests fixture
def test_with_frame_class_data(load_frame_class_tests):
    """Test using the frame class dataset."""
    test_cases = load_frame_class_tests
    # ... your test logic
```

### Loading Datasets Manually

```python
import json
from pathlib import Path

def load_dataset(dataset_name):
    """Load a dataset file from datasets directory."""
    from renamer.test.conftest import load_dataset
    return load_dataset(dataset_name)

# Load filename patterns
data = load_dataset("filename_patterns")
test_cases = data["test_cases"]

# Load frame class tests
frame_tests = load_dataset("frame_class_tests")
```

### Parametrized Tests

```python
import pytest
from pathlib import Path
from renamer.extractors.filename_extractor import FilenameExtractor

# Load test cases at module level
def load_test_cases():
    dataset_file = Path(__file__).parent / "datasets" / "filenames" / "filename_patterns.json"
    with open(dataset_file, 'r', encoding='utf-8') as f:
        data = json.load(f)
        return data['test_cases']

@pytest.mark.parametrize("test_case", load_test_cases(), ids=lambda tc: tc['testname'])
def test_filename_extraction(test_case):
    """Test filename extraction with all test cases."""
    extractor = FilenameExtractor(Path(test_case["filename"]))

    expected = test_case["expected"]
    assert extractor.extract_title() == expected["title"]
    assert extractor.extract_year() == expected["year"]
    assert extractor.extract_source() == expected["source"]
    assert extractor.extract_frame_class() == expected["frame_class"]
    assert extractor.extract_audio_langs() == expected["audio_langs"]
```

### Filtering by Category

```python
def test_order_patterns(load_filename_patterns):
    """Test only order-related patterns."""
    order_cases = [
        case for case in load_filename_patterns
        if case['category'] == 'order'
    ]

    for case in order_cases:
        # Test order extraction
        pass

def test_cyrillic_titles(load_filename_patterns):
    """Test only Cyrillic title patterns."""
    cyrillic_cases = [
        case for case in load_filename_patterns
        if case['category'] == 'cyrillic'
    ]

    for case in cyrillic_cases:
        # Test Cyrillic handling
        pass
```

### Using Sample Files

```python
from renamer.test.conftest import get_test_file_path

# Get path to a sample file from the dataset
sample_file = get_test_file_path("Movie Title (2020) BDRip [1080p,ukr,eng].mkv")
assert sample_file.exists()

# Sample files in sample_mediafiles/ are empty placeholder files
# generated from filename_patterns.json for testing file system operations
```

### Generating Sample Media Files

The `sample_mediafiles/` directory contains empty files for all test cases in `filename_patterns.json`.
These files are generated automatically and should not be committed to git.

**Generate files:**
```bash
# From project root
uv run python renamer/test/fill_sample_mediafiles.py
```

**Output:**
```
Creating sample media files in: /path/to/renamer/test/datasets/sample_mediafiles
Test cases in dataset: 46

  ✅ Created: Movie Title (2020) BDRip [1080p,ukr,eng].mkv
  ✅ Created: [01] Movie Title (2020) BDRip [1080p,ukr,eng].mkv
  ...

Summary:
  Created: 46 files
  Skipped (already exist): 0 files
  Errors: 0 files
```

**Note:** These files are in `.gitignore` and will not be committed. Run the script after cloning
the repository to generate them for local testing.

## Adding New Test Data

### Adding Filename Patterns

Edit `filenames/filename_patterns.json`:

```json
{
  "testname": "your-test-name",
  "filename": "Your Movie Title (2024) [1080p].mkv",
  "expected": {
    "order": null,
    "title": "Your Movie Title",
    "year": "2024",
    "source": null,
    "frame_class": "1080p",
    "hdr": null,
    "movie_db": null,
    "special_info": null,
    "audio_langs": "",
    "extension": "mkv"
  },
  "category": "simple",
  "description": "Brief description of what this tests"
}
```

### Adding MediaInfo Tests

Edit `mediainfo/frame_class_tests.json`:

```json
{
  "testname": "your-test-name",
  "resolution": [1920, 1080],
  "interlaced": "No",
  "expected_frame_class": "1080p",
  "description": "What resolution/format this tests"
}
```

### Adding Sample Files

Create empty files in `filenames/sample_files/`:

```bash
touch filenames/sample_files/"Your Movie Title (2024) [1080p].mkv"
```

## Test Coverage by Category

### Order Patterns (5 cases)
- `[01]` - Square bracket order
- `01.` - Dot order
- `1.1` - Decimal order
- `[01.1]` - Complex bracketed decimal
- `9.` - Single digit order

### Year Formats (2 cases)
- `(2020)` - Standard parentheses (most common)
- `2020` - Standalone year
- `.2020.` - Dot-separated year

### Database IDs (3 cases)
- `[tmdbid-12345]` - TMDB with square brackets
- `{imdb-tt1234567}` - IMDB with curly braces
- Multiple IDs in single filename

### Audio Languages (3 cases)
- `2ukr,eng` - Multiple tracks of same language
- `rus,ukr,4eng` - Mixed languages with counts
- `3ukr,eng` - Three Ukrainian tracks

### Cyrillic (3 cases)
- Full Cyrillic titles
- Cyrillic with numbers
- Mixed Cyrillic/Latin

### Edge Cases (9 cases)
- Title starting with number (`2001`, `9`)
- Title with colons, dashes, apostrophes
- Title with dots
- No brackets around metadata
- No year present
- Multipart films (pt1, pt2)
- Remastered versions
- Multiple resolution indicators
- Series episodes

## Data Quality Guidelines

When adding test data:

1. **Completeness**: Include all expected fields, use `null` for missing values
2. **Accuracy**: Verify expected values match actual extractor output
3. **Coverage**: Include edge cases and corner cases
4. **Categorization**: Assign appropriate category
5. **Documentation**: Provide clear description
6. **Naming**: Use descriptive testname (e.g., `order-001`, `cyrillic-002`)
7. **Realism**: Use realistic filenames from actual use cases

## Maintenance

- Keep datasets in sync with test requirements
- Document expected behavior in descriptions
- Use consistent naming conventions (lowercase, hyphens)
- Group related test cases together (same category)
- Update README when adding new dataset types
- Run tests after adding new data to validate
- Remove obsolete test cases when extractors change

## Version History

- **v2.0**: Comprehensive reorganization with 46+ test cases across 14 categories
  - Added testname field for better test identification
  - Added category field for test organization
  - Expanded coverage to include all extractor fields
  - Added edge cases and special formatting tests

- **v1.0**: Initial dataset with basic test cases