cursebreaker-parser-rust/cursebreaker-parser/README.md

# Cursebreaker Parser

A Rust library for parsing and managing game data from the Cursebreaker game. This crate provides tools to extract, load, and query game data from Unity scenes and XML files.

## Overview

Cursebreaker Parser is designed to:
- Parse Unity scenes and extract game objects using the unity-parser library
- Load game data from XML files (Items, NPCs, Quests, Harvestables, Loot tables, Maps, Fast Travel, Player Houses, Traits, Shops)
- Process and compress minimap tiles and item images
- Provide in-memory databases for efficient querying of game data
- Serialize game data to SQL format for database storage

## Features

- **Item Database**: Load and query items with support for filtering by ID, category, slot, and other attributes
- **NPC Database**: Manage NPC data including stats, levels, animations, and quest markers
- **Quest Database**: Handle quest definitions, phases, and rewards
- **Harvestable Database**: Track harvestable resources and their drop tables
- **Loot Database**: Manage loot tables and drop configurations
- **Map Database**: Handle map data and navigation
- **Fast Travel Database**: Manage fast travel locations and connections
- **Player House Database**: Track player houses and their locations
- **Trait Database**: Handle character traits and their effects
- **Shop Database**: Manage shop inventories and pricing
- **Minimap Database**: Process and manage minimap tiles with multiple zoom levels
- **XML Parsing**: Robust XML parsing with error handling
- **SQL Export**: Prepare data for SQL database insertion
- **Image Processing**: Process and compress minimap tiles and item icons
- **Unity Scene Parsing**: Extract game objects and world resources from Unity scenes

## Binaries

The project provides multiple binaries to handle different parsing tasks. This allows you to run only the parts you need, avoiding long load times for unnecessary operations.

### Available Binaries

1. **xml-parser** - Loads game data from XML files and populates the SQLite database
   - Fast execution
   - Run this when XML files change
   ```bash
   cargo run --bin xml-parser
   ```

2. **scene-parser** - Parses Unity scenes and extracts world resource locations
   - Slow execution (Unity project initialization)
   - Extracts InteractableResource components and their positions
   - Saves to `world_resources` table (harvestable_id and 2D coordinates)
   - Processes item icons for harvestables:
     - Looks up the first item drop for each harvestable from `harvestable_drops` table
     - Loads the icon from `Data/Textures/ItemIcons/{item_id}.png`
     - Applies white outline (1px) and resizes to 64x64
     - Converts to WebP and stores in `resource_icons` table
   - Run this when scene files change
   ```bash
   cargo run --bin scene-parser
   ```

3. **image-parser** - Processes minimap tiles
   - Slow execution (image processing and compression)
   - Run this when minimap images change
   ```bash
   cargo run --bin image-parser
   ```

4. **cursebreaker-parser** - All-in-one binary (runs all parsers)
   - Slowest execution (runs everything)
   - Use when you need to regenerate the entire database
   ```bash
   cargo run --bin cursebreaker-parser
   # or simply
   cargo run
   ```

5. **verify-db** - Verifies database contents and shows basic statistics
   ```bash
   cargo run --bin verify-db
   ```

6. **verify-expanded-db** - Verifies expanded database schema with items, recipes, and stats
   ```bash
   cargo run --bin verify-expanded-db
   ```

7. **verify-images** - Verifies item images and shows storage statistics
   ```bash
   cargo run --bin verify-images
   ```

8. **verify-stats** - Verifies item stats and shows breakdown by type
   ```bash
   cargo run --bin verify-stats
   ```

9. **verify-resource-icons** - Verifies resource icons for harvestables
   ```bash
   cargo run --bin verify-resource-icons
   ```

### Building for Production

Build specific binaries for release:
```bash
cargo build --release --bin xml-parser
cargo build --release --bin scene-parser
cargo build --release --bin image-parser
```

The compiled binaries will be in `target/release/`.

## Configuration

### Environment Variables

Set the `CB_ASSETS_PATH` environment variable to the path of your CurseBreaker assets directory:

```bash
export CB_ASSETS_PATH="/path/to/CBAssets"
```

If not set, the default fallback is `/home/connor/repos/CBAssets`.

## Usage

### Loading Items from XML

```rust
use cursebreaker_parser::ItemDatabase;

// Load all items from XML
let item_db = ItemDatabase::load_from_xml("Data/XMLs/Items/Items.xml")?;
println!("Loaded {} items", item_db.len());

// Get item by ID
if let Some(item) = item_db.get_by_id(150) {
    println!("Found: {}", item.name);
}

// Query items by category
let weapons = item_db.get_by_category("bow");
println!("Found {} bows", weapons.len());

// Query items by slot
let consumables = item_db.get_by_slot("consumable");
for item in consumables {
    println!("Consumable: {}", item.name);
}
```

### Preparing Data for SQL

```rust
use cursebreaker_parser::ItemDatabase;

let item_db = ItemDatabase::load_from_xml("Data/XMLs/Items/Items.xml")?;

// Prepare data for SQL insertion
// Returns Vec<(id, name, json_data)>
let sql_data = item_db.prepare_for_sql();

for (id, name, json) in sql_data.iter().take(5) {
    println!("INSERT INTO items VALUES ({}, '{}', '{}')", id, name, json);
}
```

### Querying World Resources

```rust
use diesel::prelude::*;
use diesel::sqlite::SqliteConnection;

// Connect to database
let mut conn = SqliteConnection::establish("../cursebreaker.db")?;

// Define the structure
#[derive(Queryable, Debug)]
struct WorldResource {
    item_id: i32,
    pos_x: f32,
    pos_y: f32,
}

// Query resources by item ID
use cursebreaker_parser::schema::world_resources::dsl::*;

let copper_ore = world_resources
    .filter(item_id.eq(2))
    .load::<WorldResource>(&mut conn)?;

println!("Found {} copper ore nodes", copper_ore.len());
for resource in copper_ore {
    println!("  Position: ({:.2}, {:.2})", resource.pos_x, resource.pos_y);
}
```

See `examples/query_world_resources.rs` for a complete example.

### Querying Resource Icons

```rust
use diesel::prelude::*;
use diesel::sqlite::SqliteConnection;

// Connect to database
let mut conn = SqliteConnection::establish("../cursebreaker.db")?;

// Define the structure
#[derive(Queryable, Debug)]
struct ResourceIcon {
    item_id: i32,      // Harvestable ID
    name: String,      // Harvestable name
    icon_64: Vec<u8>,  // WebP image data (64x64 with white border)
}

// Query icon for a specific harvestable
use cursebreaker_parser::schema::resource_icons::dsl::*;

let copper_icon = resource_icons
    .filter(item_id.eq(2))  // Harvestable ID for Copper Ore
    .first::<ResourceIcon>(&mut conn)?;

println!("Found icon for: {}", copper_icon.name);
println!("Icon size: {} bytes (WebP format)", copper_icon.icon_64.len());

// Save to file if needed
std::fs::write("copper_ore.webp", &copper_icon.icon_64)?;
```

See `examples/resource_icons_example.rs` for a complete example.

### Additional Databases

Similar APIs are available for other game data types:

```rust
use cursebreaker_parser::{
    MapDatabase, FastTravelDatabase, PlayerHouseDatabase,
    TraitDatabase, ShopDatabase, MinimapDatabase
};

// Load maps, fast travel points, player houses, etc.
let map_db = MapDatabase::load_from_xml("Data/XMLs/Maps/Map.xml")?;
// ... similar usage patterns
```

See the examples directory for usage of each database type.

### Database Verification

After parsing data, you can verify the database contents using the verification binaries:

```bash
# Basic database verification
cargo run --bin verify-db

# Verify expanded schema with recipes and stats
cargo run --bin verify-expanded-db

# Check item images and storage usage
cargo run --bin verify-images

# Analyze item stats breakdown
cargo run --bin verify-stats
```

### Examples

The project includes several example programs demonstrating different aspects of the parser:

- **game_data_demo.rs** - Comprehensive demo loading and querying all game data types (Items, NPCs, Quests, Harvestables, Loot)
- **item_database_demo.rs** - Focused on item database operations
- **query_world_resources.rs** - Querying world resource locations from the database
- **resource_icons_example.rs** - Querying processed harvestable icons with white borders
- **fast_travel_example.rs** - Working with fast travel locations
- **maps_example.rs** - Map data handling
- **player_houses_example.rs** - Player house management
- **shops_example.rs** - Shop inventory and pricing
- **traits_example.rs** - Character traits and effects

Run any example with:
```bash
cargo run --example <example_name>
```

## Project Structure

```
cursebreaker-parser/
├── src/
│   ├── lib.rs                    # Library entry point and public API
│   ├── main.rs                   # Main binary (all-in-one parser)
│   ├── bin/                      # Separate parser binaries
│   │   ├── xml-parser.rs         # XML parsing only
│   │   ├── scene-parser.rs       # Unity scene parsing only
│   │   ├── image-parser.rs       # Image processing only
│   │   ├── verify-db.rs          # Database verification
│   │   ├── verify-expanded-db.rs # Expanded database verification
│   │   ├── verify-images.rs      # Image verification
│   │   ├── verify-stats.rs       # Stats verification
│   │   └── verify-resource-icons.rs # Resource icons verification
│   ├── xml_parser.rs             # XML parsing utilities
│   ├── image_processor.rs        # Image processing utilities
│   ├── item_loader.rs            # Item loading logic
│   ├── schema.rs                 # Database schema definitions
│   ├── databases/                # Database implementations
│   │   ├── item_database.rs
│   │   ├── npc_database.rs
│   │   ├── quest_database.rs
│   │   ├── harvestable_database.rs
│   │   ├── loot_database.rs
│   │   ├── map_database.rs
│   │   ├── fast_travel_database.rs
│   │   ├── player_house_database.rs
│   │   ├── trait_database.rs
│   │   ├── shop_database.rs
│   │   └── minimap_database.rs
│   └── types/                    # Type definitions
│       ├── cursebreaker/         # Game-specific types (Items, NPCs, Quests, etc.)
│       └── monobehaviours/       # Unity MonoBehaviour types
├── examples/                     # Example usage
│   ├── fast_travel_example.rs
│   ├── game_data_demo.rs
│   ├── item_database_demo.rs
│   ├── maps_example.rs
│   ├── player_houses_example.rs
│   ├── query_world_resources.rs
│   ├── shops_example.rs
│   └── traits_example.rs
├── migrations/                   # Database migrations
├── Cargo.toml                   # Package configuration
├── XML_PARSING.md               # XML parsing documentation
└── README.md                    # This file

```

## Database Schema

The parser uses Diesel for database operations with SQLite. Database migrations are located in the `migrations/` directory and handle:

- Item data with stats, images, and crafting recipes
- NPC information and loot tables
- Quest definitions and phases
- Harvestable resources and drop tables
- World resource locations from Unity scenes
- Resource icons for harvestables (64x64 WebP with white borders)
- Minimap tiles and metadata
- Shop inventories and pricing
- Player houses and locations
- Fast travel points
- Character traits

## Dependencies

- **unity-parser**: For parsing Unity scene files
- **quick-xml**: XML parsing
- **serde**: Serialization/deserialization
- **serde_json**: JSON support
- **serde_yaml**: YAML support
- **sparsey**: ECS (Entity Component System) support
- **diesel**: SQL database support with SQLite
- **image**: Image processing and WebP compression
- **thiserror**: Error handling

## Building

```bash
# Build the library
cargo build

# Run tests
cargo test

# Build specific binaries
cargo build --bin xml-parser
cargo build --bin scene-parser
cargo build --bin image-parser
cargo build --bin verify-db

# Run examples
cargo run --example game_data_demo
cargo run --example item_database_demo

# Build for release
cargo build --release
```

## Documentation

For detailed XML parsing information, see [XML_PARSING.md](XML_PARSING.md).

Generate API documentation:
```bash
cargo doc --open
```