cursebreaker-parser-rust/REFACTORING_SUMMARY.md

Database Schema Refactoring Summary

Overview

Successfully refactored the minimap tiles system to use a single unified table with a zoom column instead of separate tables and multiple WebP columns.

Changes Made

1. Database Schema

Before:

-- Two separate tables
CREATE TABLE minimap_tiles (
    webp_512 BLOB,
    webp_256 BLOB,
    webp_128 BLOB,
    webp_64 BLOB,
    webp_512_size INTEGER,
    webp_256_size INTEGER,
    webp_128_size INTEGER,
    webp_64_size INTEGER,
    ...
);

CREATE TABLE merged_tiles (
    zoom_level INTEGER,
    webp_data BLOB,
    ...
);

After:

-- Single unified table
CREATE TABLE minimap_tiles (
    x INTEGER NOT NULL,
    y INTEGER NOT NULL,
    zoom INTEGER NOT NULL,              -- 0, 1, or 2
    width INTEGER NOT NULL,            -- Always 512
    height INTEGER NOT NULL,           -- Always 512
    image BLOB NOT NULL,               -- Single WebP column
    image_size INTEGER NOT NULL,
    original_file_size INTEGER,        -- Only for zoom=2
    source_path TEXT NOT NULL,
    processed_at TIMESTAMP NOT NULL,
    UNIQUE(x, y, zoom)
);

2. Data Model (minimap_models.rs)

Before:

pub struct MinimapTileRecord {
    pub webp_512: Vec<u8>,
    pub webp_256: Vec<u8>,
    pub webp_128: Vec<u8>,
    pub webp_64: Vec<u8>,
    pub webp_512_size: i32,
    // ... more fields
}

After:

pub struct MinimapTileRecord {
    pub x: i32,
    pub y: i32,
    pub zoom: i32,          // NEW: Zoom level (0, 1, 2)
    pub width: i32,
    pub height: i32,
    pub image: Vec<u8>,     // UNIFIED: Single image column
    pub image_size: i32,
    pub original_file_size: Option<i32>,
    pub source_path: String,
    pub processed_at: String,
}

3. ImageProcessor (image_processor.rs)

Added new methods:

impl ImageProcessor {
    /// Encode image to lossless WebP
    pub fn encode_webp_lossless(img: &RgbaImage)
        -> Result<Vec<u8>, ImageProcessingError>

    /// Create a black tile of specified size
    pub fn create_black_tile(size: u32) -> RgbaImage

    /// Merge multiple tiles into a single image
    pub fn merge_tiles(
        tiles: &HashMap<(i32, i32), Vec<u8>>,
        grid_x: i32,
        grid_y: i32,
        tile_size: u32,
        output_size: u32,
    ) -> Result<RgbaImage, ImageProcessingError>
}

4. MinimapDatabase (minimap_database.rs)

Completely rewritten to:

• Process all PNG files at zoom level 2 (original, lossless WebP)
• Automatically generate zoom level 1 tiles (2×2 merged)
• Automatically generate zoom level 0 tiles (4×4 merged)
• Store all zoom levels in a single minimap_tiles table

Key method:

pub fn load_from_directory() -> Result<usize, MinimapDatabaseError> {
    // Step 1: Load all PNGs → zoom level 2
    // Step 2: Generate zoom level 1 (2×2 merged)
    // Step 3: Generate zoom level 0 (4×4 merged)
}

5. Image Parser (image-parser.rs)

Simplified significantly:

• No longer needs to be run separately from merge process
• Single command now generates all zoom levels
• Updated output to show statistics per zoom level

Usage:

cd cursebreaker-parser
cargo run --bin image-parser --release

Output includes:

• Tile counts per zoom level (0, 1, 2)
• Storage size per zoom level
• Total compression ratio
• Map bounds

6. Map Server (cursebreaker-map/src/main.rs)

Updated to use new schema:

Before:

// Queried merged_tiles table
use cursebreaker_parser::schema::merged_tiles::dsl::*;
merged_tiles
    .filter(zoom_level.eq(z))
    .select(webp_data)

After:

// Queries unified minimap_tiles table
use cursebreaker_parser::schema::minimap_tiles::dsl::*;
minimap_tiles
    .filter(zoom.eq(z))
    .select(image)

7. Removed Files

• ❌ merge-tiles.rs - No longer needed (merged into image-parser)
• ❌ merged_tiles table - Replaced by unified minimap_tiles

Benefits

1. Simpler Schema: One table instead of two
2. Cleaner Code: Single column for images instead of multiple
3. Single Command: One tool (image-parser) generates all zoom levels
4. Maintainability: Easier to understand and modify
5. Consistency: All tiles stored in the same way

Migration Path

For Existing Databases:

1. Run migration (automatically done):

   diesel migration run
   

   This will:

   • Drop old minimap_tiles and merged_tiles tables
   • Create new unified minimap_tiles table

2. Regenerate tiles:

   cd cursebreaker-parser
   cargo run --bin image-parser --release
   

3. Start map server:

   cd ../cursebreaker-map
   cargo run --release
   

Technical Details

Zoom Level Mapping

Zoom	Description	Merge Factor	Typical Count
0	Most zoomed out	4×4	~31 tiles
1	Medium zoom	2×2	~105 tiles
2	Full detail (original)	1×1	~345 tiles

Coordinate System

• Zoom 2: Uses original tile coordinates (e.g., x=5, y=10)
• Zoom 1: Uses divided coordinates (e.g., x=2, y=5 for 2×2 grid starting at 4,10)
• Zoom 0: Uses divided coordinates (e.g., x=1, y=2 for 4×4 grid starting at 4,8)

WebP Encoding

All tiles use lossless WebP compression:

• No quality loss
• Smaller than PNG
• Faster to decode than PNG
• Browser-native format

Testing

After running the refactored system:

1. Check tile counts:

   SELECT zoom, COUNT(*) as count
   FROM minimap_tiles
   GROUP BY zoom;
   

   Expected: ~31 for zoom 0, ~105 for zoom 1, ~345 for zoom 2

2. Verify storage:

   SELECT
       zoom,
       COUNT(*) as tiles,
       SUM(image_size) / 1048576 as mb
   FROM minimap_tiles
   GROUP BY zoom;
   

3. Test map viewer:

   • Open http://127.0.0.1:3000
   • Zoom in/out to verify all levels load correctly
   • Check browser DevTools network tab for tile requests

Performance

Tile generation time:

• Old approach: Run image-parser (~30s) + run merge-tiles (~90s) = ~2 minutes total
• New approach: Run image-parser once (~90s) = ~1.5 minutes total
• Improvement: Simpler workflow, one less step

Database storage:

• Similar total size (~111 MB)
• Cleaner schema with single image column
• Indexed by (zoom, x, y) for fast queries

Files Modified

cursebreaker-parser/
├── migrations/
│   └── 2026-01-10-122732-0000_restructure_minimap_tiles/
│       ├── up.sql              # NEW
│       └── down.sql            # NEW
├── src/
│   ├── bin/
│   │   ├── image-parser.rs     # MODIFIED
│   │   └── merge-tiles.rs      # DELETED
│   ├── databases/
│   │   └── minimap_database.rs # REWRITTEN
│   ├── image_processor.rs      # MODIFIED (added merge methods)
│   ├── types/
│   │   └── minimap_models.rs   # MODIFIED (new schema)
│   └── schema.rs               # REGENERATED
└── Cargo.toml                  # MODIFIED (removed merge-tiles bin)

cursebreaker-map/
└── src/
    └── main.rs                 # MODIFIED (use new schema)

Backward Compatibility

⚠️ Breaking Changes:

• Old database will be wiped by migration
• Must re-run image-parser to regenerate tiles
• merge-tiles command no longer exists

✅ No Breaking Changes:

• Map viewer API unchanged (/api/tiles/:z/:x/:y)
• Frontend code unchanged
• Tile coordinates same at each zoom level