cursebreaker-parser-rust/cursebreaker-parser/DATABASE_MIGRATION_GUIDE.md

Database Migration Guide

This guide shows how to update all databases to use actual SQL storage with Diesel instead of just prepare_for_sql().

Status

✅ Completed: ItemDatabase ✅ Completed: Database tables created (migration) ✅ Completed: Main.rs integration example

⏳ Remaining: 9 databases need the same updates

Pattern to Follow

For each database file in src/databases/, follow this pattern (using ItemDatabase as the reference):

Step 1: Add Diesel Imports

At the top of the file, add:

use diesel::prelude::*;
use diesel::sqlite::SqliteConnection;

Step 2: Add save_to_db() Method

Replace or add after the prepare_for_sql() method:

/// Save all [items/npcs/quests/etc] to SQLite database
pub fn save_to_db(&self, conn: &mut SqliteConnection) -> Result<usize, diesel::result::Error> {
    use crate::schema::TABLE_NAME;  // Replace TABLE_NAME

    let records: Vec<_> = self
        .ITEMS_FIELD  // Replace with actual field name (e.g., items, npcs, quests)
        .iter()
        .map(|item| {
            let json = serde_json::to_string(item).unwrap_or_else(|_| "{}".to_string());
            (
                TABLE_NAME::id.eq(item.ID_FIELD),      // Replace ID_FIELD
                TABLE_NAME::name.eq(&item.NAME_FIELD), // Replace NAME_FIELD
                TABLE_NAME::data.eq(json),
            )
        })
        .collect();

    let mut count = 0;
    for record in records {
        diesel::insert_into(TABLE_NAME::table)
            .values(&record)
            .execute(conn)?;
        count += 1;
    }

    Ok(count)
}

Step 3: Add load_from_db() Method

/// Load all [items/npcs/quests/etc] from SQLite database
pub fn load_from_db(conn: &mut SqliteConnection) -> Result<Self, diesel::result::Error> {
    use crate::schema::TABLE_NAME::dsl::*;  // Replace TABLE_NAME

    #[derive(Queryable)]
    struct Record {
        id: Option<i32>,      // Or Option<String> for text keys
        name: String,         // Adjust based on schema
        data: String,
    }

    let records = TABLE_NAME.load::<Record>(conn)?;  // Replace TABLE_NAME

    let mut loaded_items = Vec::new();
    for record in records {
        if let Ok(item) = serde_json::from_str::<TYPE>(&record.data) {  // Replace TYPE
            loaded_items.push(item);
        }
    }

    let mut db = Self::new();
    db.add_ITEMS(loaded_items);  // Replace add_ITEMS with actual method
    Ok(db)
}

Step 4: Mark prepare_for_sql() as Deprecated

#[deprecated(note = "Use save_to_db() to save directly to SQLite database")]
pub fn prepare_for_sql(&self) -> Vec<...> {
    // existing implementation
}

Database-Specific Mappings

Simple Databases (id: i32, name: String, data: String)

Database	Table	Items Field	ID Field	Name Field	Type
NpcDatabase	npcs	npcs	type_id	npc_name	Npc
QuestDatabase	quests	quests	id	name	Quest
HarvestableDatabase	harvestables	harvestables	type_id	name	Harvestable

Example for NpcDatabase:

pub fn save_to_db(&self, conn: &mut SqliteConnection) -> Result<usize, diesel::result::Error> {
    use crate::schema::npcs;

    let records: Vec<_> = self
        .npcs
        .iter()
        .map(|npc| {
            let json = serde_json::to_string(npc).unwrap_or_else(|_| "{}".to_string());
            (
                npcs::id.eq(npc.type_id),
                npcs::name.eq(&npc.npc_name),
                npcs::data.eq(json),
            )
        })
        .collect();

    let mut count = 0;
    for record in records {
        diesel::insert_into(npcs::table)
            .values(&record)
            .execute(conn)?;
        count += 1;
    }

    Ok(count)
}

Text-Key Databases

Database	Table	Primary Key Field	Type
LootDatabase	loot_tables	table_id: String	LootTable
MapDatabase	maps	scene_id: String	Map

Example for LootDatabase:

pub fn save_to_db(&self, conn: &mut SqliteConnection) -> Result<usize, diesel::result::Error> {
    use crate::schema::loot_tables;

    let records: Vec<_> = self
        .loot_tables  // Check actual field name
        .iter()
        .map(|loot| {
            let json = serde_json::to_string(loot).unwrap_or_else(|_| "{}".to_string());
            (
                loot_tables::table_id.eq(&loot.table_id),
                loot_tables::npc_id.eq(loot.npc_id.as_ref()),  // Optional field
                loot_tables::data.eq(json),
            )
        })
        .collect();

    let mut count = 0;
    for record in records {
        diesel::insert_into(loot_tables::table)
            .values(&record)
            .execute(conn)?;
        count += 1;
    }

    Ok(count)
}

Complex Databases (Multiple Columns)

Database	Table	Additional Columns	Notes
FastTravelDatabase	fast_travel_locations	map_name: String	Has map reference
PlayerHouseDatabase	player_houses	map_id: i32	Has map ID
TraitDatabase	traits	description: Option<String>, trainer_id: Option<i32>	Multiple optional fields
ShopDatabase	shops	unique_items: bool, item_count: usize	Has metadata columns

Example for ShopDatabase:

pub fn save_to_db(&self, conn: &mut SqliteConnection) -> Result<usize, diesel::result::Error> {
    use crate::schema::shops;

    let records: Vec<_> = self
        .shops
        .iter()
        .map(|shop| {
            let json = serde_json::to_string(shop).unwrap_or_else(|_| "{}".to_string());
            (
                shops::id.eq(shop.id),
                shops::name.eq(&shop.name),
                shops::unique_items.eq(if shop.unique_items { 1 } else { 0 }),
                shops::item_count.eq(shop.items.len() as i32),
                shops::data.eq(json),
            )
        })
        .collect();

    let mut count = 0;
    for record in records {
        diesel::insert_into(shops::table)
            .values(&record)
            .execute(conn)?;
        count += 1;
    }

    Ok(count)
}

Usage in main.rs

After loading all databases from XML, save them to SQL:

// Establish database connection
let mut conn = SqliteConnection::establish("cursebreaker.db")?;

// Save each database
match item_db.save_to_db(&mut conn) {
    Ok(count) => info!("✅ Saved {} items to database", count),
    Err(e) => warn!("⚠️  Failed to save items: {}", e),
}

match npc_db.save_to_db(&mut conn) {
    Ok(count) => info!("✅ Saved {} NPCs to database", count),
    Err(e) => warn!("⚠️  Failed to save NPCs: {}", e),
}

// ... repeat for all databases

Testing

After implementing for each database:

1. Build: cargo build - Should compile without errors
2. Run: cargo run - Should show save confirmations
3. Verify: Check cursebreaker.db contains data

Implementation Order Recommendation

1. ✅ ItemDatabase (DONE)
2. NpcDatabase (simple, same as items)
3. QuestDatabase (simple, same as items)
4. HarvestableDatabase (simple, same as items)
5. MapDatabase (text key, medium)
6. LootDatabase (text key with optional field, medium)
7. FastTravelDatabase (multiple columns, complex)
8. PlayerHouseDatabase (multiple columns, complex)
9. TraitDatabase (optional columns, complex)
10. ShopDatabase (boolean + count columns, complex)

Schema Reference

The migration created these tables (see src/schema.rs):

• items(id, name, data)
• npcs(id, name, data)
• quests(id, name, data)
• harvestables(id, name, data)
• loot_tables(table_id, npc_id, data)
• maps(scene_id, name, data)
• fast_travel_locations(id, name, map_name, data)
• player_houses(id, name, map_id, data)
• traits(id, name, description, trainer_id, data)
• shops(id, name, unique_items, item_count, data)

All data columns store the full JSON-serialized object for complete data preservation.