cursebreaker-parser-rust/REFACTORING_SUMMARY.md

# Database Schema Refactoring Summary

## Overview

Successfully refactored the minimap tiles system to use a single unified table with a `zoom` column instead of separate tables and multiple WebP columns.

## Changes Made

### 1. Database Schema

#### Before:
```sql
-- Two separate tables
CREATE TABLE minimap_tiles (
    webp_512 BLOB,
    webp_256 BLOB,
    webp_128 BLOB,
    webp_64 BLOB,
    webp_512_size INTEGER,
    webp_256_size INTEGER,
    webp_128_size INTEGER,
    webp_64_size INTEGER,
    ...
);

CREATE TABLE merged_tiles (
    zoom_level INTEGER,
    webp_data BLOB,
    ...
);
```

#### After:
```sql
-- Single unified table
CREATE TABLE minimap_tiles (
    x INTEGER NOT NULL,
    y INTEGER NOT NULL,
    zoom INTEGER NOT NULL,              -- 0, 1, or 2
    width INTEGER NOT NULL,            -- Always 512
    height INTEGER NOT NULL,           -- Always 512
    image BLOB NOT NULL,               -- Single WebP column
    image_size INTEGER NOT NULL,
    original_file_size INTEGER,        -- Only for zoom=2
    source_path TEXT NOT NULL,
    processed_at TIMESTAMP NOT NULL,
    UNIQUE(x, y, zoom)
);
```

### 2. Data Model (`minimap_models.rs`)

**Before:**
```rust
pub struct MinimapTileRecord {
    pub webp_512: Vec<u8>,
    pub webp_256: Vec<u8>,
    pub webp_128: Vec<u8>,
    pub webp_64: Vec<u8>,
    pub webp_512_size: i32,
    // ... more fields
}
```

**After:**
```rust
pub struct MinimapTileRecord {
    pub x: i32,
    pub y: i32,
    pub zoom: i32,          // NEW: Zoom level (0, 1, 2)
    pub width: i32,
    pub height: i32,
    pub image: Vec<u8>,     // UNIFIED: Single image column
    pub image_size: i32,
    pub original_file_size: Option<i32>,
    pub source_path: String,
    pub processed_at: String,
}
```

### 3. ImageProcessor (`image_processor.rs`)

**Added new methods:**

```rust
impl ImageProcessor {
    /// Encode image to lossless WebP
    pub fn encode_webp_lossless(img: &RgbaImage)
        -> Result<Vec<u8>, ImageProcessingError>

    /// Create a black tile of specified size
    pub fn create_black_tile(size: u32) -> RgbaImage

    /// Merge multiple tiles into a single image
    pub fn merge_tiles(
        tiles: &HashMap<(i32, i32), Vec<u8>>,
        grid_x: i32,
        grid_y: i32,
        tile_size: u32,
        output_size: u32,
    ) -> Result<RgbaImage, ImageProcessingError>
}
```

### 4. MinimapDatabase (`minimap_database.rs`)

**Completely rewritten to:**
- Process all PNG files at zoom level 2 (original, lossless WebP)
- Automatically generate zoom level 1 tiles (2×2 merged)
- Automatically generate zoom level 0 tiles (4×4 merged)
- Store all zoom levels in a single `minimap_tiles` table

**Key method:**
```rust
pub fn load_from_directory() -> Result<usize, MinimapDatabaseError> {
    // Step 1: Load all PNGs → zoom level 2
    // Step 2: Generate zoom level 1 (2×2 merged)
    // Step 3: Generate zoom level 0 (4×4 merged)
}
```

### 5. Image Parser (`image-parser.rs`)

**Simplified significantly:**
- No longer needs to be run separately from merge process
- Single command now generates **all** zoom levels
- Updated output to show statistics per zoom level

**Usage:**
```bash
cd cursebreaker-parser
cargo run --bin image-parser --release
```

**Output includes:**
- Tile counts per zoom level (0, 1, 2)
- Storage size per zoom level
- Total compression ratio
- Map bounds

### 6. Map Server (`cursebreaker-map/src/main.rs`)

**Updated to use new schema:**

**Before:**
```rust
// Queried merged_tiles table
use cursebreaker_parser::schema::merged_tiles::dsl::*;
merged_tiles
    .filter(zoom_level.eq(z))
    .select(webp_data)
```

**After:**
```rust
// Queries unified minimap_tiles table
use cursebreaker_parser::schema::minimap_tiles::dsl::*;
minimap_tiles
    .filter(zoom.eq(z))
    .select(image)
```

### 7. Removed Files

- ❌ **`merge-tiles.rs`** - No longer needed (merged into image-parser)
- ❌ **`merged_tiles` table** - Replaced by unified minimap_tiles

## Benefits

1. **Simpler Schema**: One table instead of two
2. **Cleaner Code**: Single column for images instead of multiple
3. **Single Command**: One tool (`image-parser`) generates all zoom levels
4. **Maintainability**: Easier to understand and modify
5. **Consistency**: All tiles stored in the same way

## Migration Path

### For Existing Databases:

1. **Run migration** (automatically done):
   ```bash
   diesel migration run
   ```
   This will:
   - Drop old `minimap_tiles` and `merged_tiles` tables
   - Create new unified `minimap_tiles` table

2. **Regenerate tiles**:
   ```bash
   cd cursebreaker-parser
   cargo run --bin image-parser --release
   ```

3. **Start map server**:
   ```bash
   cd ../cursebreaker-map
   cargo run --release
   ```

## Technical Details

### Zoom Level Mapping

| Zoom | Description | Merge Factor | Typical Count |
|------|-------------|--------------|---------------|
| 0 | Most zoomed out | 4×4 | ~31 tiles |
| 1 | Medium zoom | 2×2 | ~105 tiles |
| 2 | Full detail (original) | 1×1 | ~345 tiles |

### Coordinate System

- **Zoom 2**: Uses original tile coordinates (e.g., x=5, y=10)
- **Zoom 1**: Uses divided coordinates (e.g., x=2, y=5 for 2×2 grid starting at 4,10)
- **Zoom 0**: Uses divided coordinates (e.g., x=1, y=2 for 4×4 grid starting at 4,8)

### WebP Encoding

All tiles use **lossless WebP** compression:
- No quality loss
- Smaller than PNG
- Faster to decode than PNG
- Browser-native format

## Testing

After running the refactored system:

1. **Check tile counts**:
   ```sql
   SELECT zoom, COUNT(*) as count
   FROM minimap_tiles
   GROUP BY zoom;
   ```
   Expected: ~31 for zoom 0, ~105 for zoom 1, ~345 for zoom 2

2. **Verify storage**:
   ```sql
   SELECT
       zoom,
       COUNT(*) as tiles,
       SUM(image_size) / 1048576 as mb
   FROM minimap_tiles
   GROUP BY zoom;
   ```

3. **Test map viewer**:
   - Open `http://127.0.0.1:3000`
   - Zoom in/out to verify all levels load correctly
   - Check browser DevTools network tab for tile requests

## Performance

Tile generation time:
- **Old approach**: Run image-parser (~30s) + run merge-tiles (~90s) = ~2 minutes total
- **New approach**: Run image-parser once (~90s) = ~1.5 minutes total
- **Improvement**: Simpler workflow, one less step

Database storage:
- Similar total size (~111 MB)
- Cleaner schema with single image column
- Indexed by (zoom, x, y) for fast queries

## Files Modified

```
cursebreaker-parser/
├── migrations/
│   └── 2026-01-10-122732-0000_restructure_minimap_tiles/
│       ├── up.sql              # NEW
│       └── down.sql            # NEW
├── src/
│   ├── bin/
│   │   ├── image-parser.rs     # MODIFIED
│   │   └── merge-tiles.rs      # DELETED
│   ├── databases/
│   │   └── minimap_database.rs # REWRITTEN
│   ├── image_processor.rs      # MODIFIED (added merge methods)
│   ├── types/
│   │   └── minimap_models.rs   # MODIFIED (new schema)
│   └── schema.rs               # REGENERATED
└── Cargo.toml                  # MODIFIED (removed merge-tiles bin)

cursebreaker-map/
└── src/
    └── main.rs                 # MODIFIED (use new schema)
```

## Backward Compatibility

⚠️ **Breaking Changes:**
- Old database will be wiped by migration
- Must re-run `image-parser` to regenerate tiles
- `merge-tiles` command no longer exists

✅ **No Breaking Changes:**
- Map viewer API unchanged (`/api/tiles/:z/:x/:y`)
- Frontend code unchanged
- Tile coordinates same at each zoom level