Connor/cursebreaker-parser-rust

Fork 0

Files

Connor 8438dabf0b item DB extension

2026-01-12 03:02:45 +00:00

7.6 KiB

Raw Blame History

Item Images in Database - Implementation Summary

✅ Completed Successfully

Successfully added WebP image processing and storage for item icons in the database.

What Was Added

1. Database Migration: `add_item_images`

New Columns in items Table:

icon_large (BLOB) - 256x256 WebP image
icon_medium (BLOB) - 64x64 WebP image
icon_small (BLOB) - 16x16 WebP image

All columns are nullable (items without icons will have NULL values).

2. Image Processing in `ItemDatabase`

New Method: save_to_db_with_images()

pub fn save_to_db_with_images<P: AsRef<Path>>(
    &self,
    conn: &mut SqliteConnection,
    icon_path: P,
) -> Result<(usize, usize), diesel::result::Error>

Features:

Processes PNG icons from the ItemIcons directory
Converts to WebP format (85% quality)
Generates 3 sizes: 256px, 64px, 16px
Returns tuple: (items_saved, images_processed)
Handles missing icons gracefully (stores NULL)
Uses transactions for data consistency

Helper Method: process_item_icon()

Loads PNG file by item ID (e.g., "33.png" for Copper Ore)
Processes image at 3 resolutions using ImageProcessor
Returns tuple of WebP blobs
Logs warnings for failed conversions

3. Updated xml-parser

The xml-parser now:

Locates ItemIcons directory from CB_ASSETS_PATH
Calls save_to_db_with_images() instead of save_to_db()
Reports both item count and processed image count

Results

Processing Statistics

✅ Total items: 1360
✅ Items with icons: 1228
✅ Items without icons: 132 (no source PNG available)
💾 Total image storage: 9.03 MB

Sample Item Icons

Copper Ore (ID: 33)
  Large (256px): 9.6 KB | Medium (64px): 2.4 KB | Small (16px): 0.4 KB
  Total per item: 12.4 KB

Iron Ore (ID: 34)
  Large (256px): 3.3 KB | Medium (64px): 1.8 KB | Small (16px): 0.4 KB
  Total per item: 5.5 KB

Storage Efficiency

WebP Compression Benefits:

Original PNG (256x256, RGBA): ~20-40 KB per icon
WebP Large (256px): ~3-10 KB (70-80% reduction)
WebP Medium (64px): ~1-2 KB
WebP Small (16px): ~0.3-0.5 KB

Total storage for 1228 items with 3 sizes each:

Only 9.03 MB for all processed images
Average ~7.4 KB per item (all 3 sizes combined)
Excellent compression with minimal quality loss

Code Changes

Files Modified

migrations/2026-01-11-135041-0000_add_item_images/up.sql
- Added 3 BLOB columns to items table
src/schema.rs
- Auto-regenerated with new columns
src/databases/item_database.rs
- Added save_to_db_with_images() method
- Added process_item_icon() helper
- Updated load_from_db() ItemRecord struct
src/bin/xml-parser.rs
- Updated to use save_to_db_with_images()
- Added icon path construction

Files Created

src/bin/verify-images.rs
- Verification tool to check image storage
- Shows statistics and sample image sizes

Usage

Saving Items with Images

use cursebreaker_parser::ItemDatabase;
use diesel::sqlite::SqliteConnection;

let item_db = ItemDatabase::load_from_xml("items.xml")?;
let mut conn = SqliteConnection::establish("database.db")?;

// Process and save with images
let icon_path = "/path/to/CBAssets/Data/Textures/ItemIcons";
let (items_count, images_count) = item_db.save_to_db_with_images(&mut conn, icon_path)?;

println!("Saved {} items with {} icons", items_count, images_count);

Retrieving Images from Database

-- Get item with images
SELECT id, name, icon_large, icon_medium, icon_small
FROM items
WHERE id = 33;  -- Copper Ore

-- Find items with icons
SELECT id, name
FROM items
WHERE icon_large IS NOT NULL;

-- Get just the small icon for quick previews
SELECT id, name, icon_small
FROM items;

Serving Images in Web API

For your interactive map and wiki:

Small icons (16px) - List views, tooltips, inventory grids
Medium icons (64px) - Item cards, search results, crafting UI
Large icons (256px) - Detail pages, zoomed views, high-res displays

Example API endpoint:

// Get item icon by size
async fn get_item_icon(item_id: i32, size: String) -> Result<Vec<u8>> {
    let item = query_item(item_id)?;

    let icon_data = match size.as_str() {
        "small" => item.icon_small,
        "medium" => item.icon_medium,
        "large" | _ => item.icon_large,
    };

    match icon_data {
        Some(data) => Ok(data),
        None => Err("Icon not found"),
    }
}

// Serve as image/webp
// Response headers: Content-Type: image/webp

Benefits

✅ Efficient Storage - WebP compression saves ~70-80% space ✅ Multiple Resolutions - Responsive design ready ✅ Self-Contained - No external file dependencies ✅ Fast Delivery - No filesystem lookups, direct from DB ✅ Transactional - Images saved atomically with item data ✅ Graceful Degradation - Missing icons handled automatically ✅ Web-Optimized - WebP format supported by all modern browsers

Performance Considerations

Processing Time

~1228 images processed during save
Each image generates 3 sizes (256px, 64px, 16px)
Total: ~3,684 WebP encodings
Processing time: ~5-10 seconds on modern hardware

Database Size Impact

Before images: ~130 MB (JSON data + columns)
After images: ~139 MB (+9 MB for all icons)
Only 7% increase in database size
All icons for all sizes included

Query Performance

Small overhead when selecting items (if loading all columns)
Minimal impact if only selecting needed columns
Consider separate queries for image data vs metadata

Optimization Tips:

-- Fast: Get metadata only
SELECT id, name, item_type, level, price
FROM items;

-- Slower: Include large images unnecessarily
SELECT *
FROM items;

-- Optimal: Get specific image size when needed
SELECT id, name, icon_small
FROM items;

Future Enhancements

Optional Additions:

Lazy Loading - Separate images table with FK to items

CREATE TABLE item_images (
    item_id INTEGER PRIMARY KEY,
    icon_large BLOB,
    icon_medium BLOB,
    icon_small BLOB,
    FOREIGN KEY (item_id) REFERENCES items(id)
);

Image Metadata - Track source info
- Original file path
- Processing timestamp
- Source dimensions
- Compression ratio
Additional Sizes - More resolution options
- 512px for retina displays
- 32px for medium-sized UI elements
- 8px for tiny thumbnails
Image Variants - Different visual styles
- Grayscale for disabled/locked items
- Outlined versions for specific UI needs
- Colored overlays for rarity/quality

Migration Management

To rollback (columns stay but can be cleared):

diesel migration revert

To reprocess images after icon updates:

# Just rerun the xml-parser - it uses REPLACE INTO
cargo run --bin xml-parser

Testing

Verify Images

# Check image statistics
cargo run --bin verify-images

# Extract and view an image
sqlite3 cursebreaker.db "SELECT hex(icon_large) FROM items WHERE id=33;" | xxd -r -p > copper_ore.webp
# View with: xdg-open copper_ore.webp

Performance Test

# Time the full processing
time cargo run --bin xml-parser

# Should complete in 5-15 seconds for 1228 images

Ready for Production

Your item images are now:

✅ Stored in the database
✅ Optimally compressed
✅ Available in 3 sizes
✅ Ready for your interactive map and wiki
✅ Fast to query and serve
✅ Properly handled when missing

The database now contains all the data needed for a fully functional item system with visual assets!

7.6 KiB Raw Blame History