cursebreaker-parser-rust/cursebreaker-parser/XML_PARSING.md

# XML Parsing in Cursebreaker Parser

This document describes the XML parsing functionality added to the cursebreaker-parser project.

## Overview

The parser now supports loading game data from Cursebreaker's XML files and storing them in efficient data structures for runtime access and SQL database serialization.

## Features

- ✅ Parse Items, NPCs, Quests, and Harvestables XML files with full attribute and nested element support
- ✅ In-memory databases with fast lookups by ID, name, and various filters
- ✅ JSON serialization for SQL database storage
- ✅ Type-safe data structures with serde support
- ✅ Easy-to-use API with query methods
- ✅ Cross-referencing support between different data types

## Quick Start

### Loading Items

```rust
use cursebreaker_parser::ItemDatabase;

let item_db = ItemDatabase::load_from_xml("Data/XMLs/Items/Items.xml")?;
println!("Loaded {} items", item_db.len());
```

### Querying Items

```rust
// Get by ID
if let Some(item) = item_db.get_by_id(150) {
    println!("Found: {}", item.name);
}

// Get by category
let bows = item_db.get_by_category("bow");

// Get by slot
let weapons = item_db.get_by_slot("weapon");

// Get by skill requirement
let magic_items = item_db.get_by_skill("magic");

// Get all items
for item in item_db.all_items() {
    println!("{}: {}", item.id, item.name);
}
```

### SQL Serialization

```rust
// Prepare items for SQL insertion
let sql_data = item_db.prepare_for_sql();

for (id, name, json_data) in sql_data {
    // INSERT INTO items (id, name, data) VALUES (?, ?, ?)
    // Use your preferred SQL library to insert
}
```

## Data Structures

### Item

The main `Item` struct contains all item attributes from the XML:

```rust
pub struct Item {
    // Required
    pub id: i32,
    pub name: String,

    // Optional attributes
    pub level: Option<i32>,
    pub description: Option<String>,
    pub price: Option<i32>,
    pub slot: Option<String>,
    pub category: Option<String>,
    pub skill: Option<String>,

    // ... many more fields

    // Nested elements
    pub stats: Vec<ItemStat>,
    pub crafting_recipes: Vec<CraftingRecipe>,
    pub animations: Option<AnimationSet>,
    pub generate_rules: Vec<GenerateRule>,
}
```

### ItemStat

Represents item statistics:

```rust
pub struct ItemStat {
    // Damage
    pub damagephysical: Option<i32>,
    pub damagemagical: Option<i32>,
    pub damageranged: Option<i32>,

    // Accuracy
    pub accuracyphysical: Option<i32>,
    pub accuracymagical: Option<i32>,
    pub accuracyranged: Option<i32>,

    // Resistance
    pub resistancephysical: Option<i32>,
    pub resistancemagical: Option<i32>,
    pub resistanceranged: Option<i32>,

    // Core stats
    pub health: Option<i32>,
    pub mana: Option<i32>,
    pub manaregen: Option<i32>,
    pub healing: Option<i32>,

    // Harvesting
    pub harvestingspeedwoodcutting: Option<i32>,
}
```

## Example Programs

Run the demos to see all features in action:

```bash
# Items only
cargo run --example item_database_demo

# All game data (Items, NPCs, Quests, Harvestables)
cargo run --example game_data_demo
```

## Loading NPCs

```rust
use cursebreaker_parser::NpcDatabase;

let npc_db = NpcDatabase::load_from_xml("Data/XMLs/Npcs/NPCInfo.xml")?;
println!("Loaded {} NPCs", npc_db.len());
```

### Querying NPCs

```rust
// Get by ID
if let Some(npc) = npc_db.get_by_id(1) {
    println!("Found: {}", npc.name);
}

// Get hostile NPCs
let hostile = npc_db.get_hostile();

// Get interactable NPCs
let interactable = npc_db.get_interactable();

// Get NPCs by tag
let undead = npc_db.get_by_tag("Undead");

// Get shopkeepers
let shops = npc_db.get_shopkeepers();
```

## Loading Quests

```rust
use cursebreaker_parser::QuestDatabase;

let quest_db = QuestDatabase::load_from_xml("Data/XMLs/Quests/Quests.xml")?;
println!("Loaded {} quests", quest_db.len());
```

### Querying Quests

```rust
// Get by ID
if let Some(quest) = quest_db.get_by_id(1) {
    println!("Quest: {}", quest.name);
    println!("Phases: {}", quest.phase_count());
}

// Get main quests
let main_quests = quest_db.get_main_quests();

// Get side quests
let side_quests = quest_db.get_side_quests();

// Get hidden quests
let hidden = quest_db.get_hidden_quests();
```

## Loading Harvestables

```rust
use cursebreaker_parser::HarvestableDatabase;

let harvestable_db = HarvestableDatabase::load_from_xml("Data/XMLs/Harvestables/HarvestableInfo.xml")?;
println!("Loaded {} harvestables", harvestable_db.len());
```

### Querying Harvestables

```rust
// Get by type ID
if let Some(harvestable) = harvestable_db.get_by_typeid(1) {
    println!("Found: {}", harvestable.name);
}

// Get by skill
let woodcutting = harvestable_db.get_by_skill("Woodcutting");
let mining = harvestable_db.get_by_skill("mining");
let fishing = harvestable_db.get_by_skill("Fishing");

// Get trees (harvestables with tree=1)
let trees = harvestable_db.get_trees();

// Get by tool requirement
let hatchet_nodes = harvestable_db.get_by_tool("hatchet");
let pickaxe_nodes = harvestable_db.get_by_tool("pickaxe");

// Get by level range
let beginner = harvestable_db.get_by_level_range(1, 10);
let advanced = harvestable_db.get_by_level_range(50, 100);
```

## Loading Loot Tables

```rust
use cursebreaker_parser::LootDatabase;

let loot_db = LootDatabase::load_from_xml("Data/XMLs/Loot/Loot.xml")?;
println!("Loaded {} loot tables", loot_db.len());
```

### Querying Loot Tables

```rust
// Get all loot tables for a specific NPC
let npc_id = 45;
let tables = loot_db.get_tables_for_npc(npc_id);

// Get all drops for a specific NPC
let drops = loot_db.get_drops_for_npc(npc_id);
for drop in drops {
    println!("Item ID: {}, Rate: {:?}", drop.item, drop.rate);
}

// Find which NPCs drop a specific item
let item_id = 180;
let npcs = loot_db.get_npcs_dropping_item(item_id);
println!("Item {} drops from {} NPCs", item_id, npcs.len());

// Get all tables with conditional drops (checks field)
let conditional = loot_db.get_conditional_tables();

// Get all tables with guaranteed drops (rate = 1)
let guaranteed = loot_db.get_tables_with_guaranteed_drops();

// Get all unique item IDs that can drop
let droppable_items = loot_db.get_all_droppable_items();

// Get all NPCs that have loot tables
let npcs_with_loot = loot_db.get_all_npcs_with_loot();
```

## Cross-referencing Data

```rust
// Find items rewarded by quests
for quest in quest_db.all_quests() {
    for reward in &quest.rewards {
        if let Some(item_id) = reward.item {
            if let Some(item) = item_db.get_by_id(item_id) {
                println!("Quest '{}' rewards: {}", quest.name, item.name);
            }
        }
    }
}

// Find NPCs that give quests
for npc in npc_db.all_npcs() {
    if !npc.questmarkers.is_empty() {
        println!("NPC '{}' has {} quest markers", npc.name, npc.questmarkers.len());
    }
}

// Find items that drop from harvestables
for harvestable in harvestable_db.all_harvestables() {
    for drop in &harvestable.drops {
        if let Some(item) = item_db.get_by_id(drop.id) {
            println!("'{}' drops: {} (rate: {})",
                     harvestable.name, item.name, drop.droprate.unwrap_or(0));
        }
    }
}

// Find what items an NPC drops
let npc_id = 45;
if let Some(npc) = npc_db.get_by_id(npc_id) {
    let drops = loot_db.get_drops_for_npc(npc_id);
    println!("NPC '{}' drops {} items:", npc.name, drops.len());
    for drop in drops {
        if let Some(item) = item_db.get_by_id(drop.item) {
            println!("  - {}", item.name);
        }
    }
}

// Find which NPCs drop a specific item
let item_id = 180;
if let Some(item) = item_db.get_by_id(item_id) {
    let npcs = loot_db.get_npcs_dropping_item(item_id);
    println!("Item '{}' drops from:", item.name);
    for npc_id in npcs {
        if let Some(npc) = npc_db.get_by_id(npc_id) {
            println!("  - {}", npc.name);
        }
    }
}
```

## Statistics from XML Files

When loaded from `/home/connor/repos/CBAssets/Data/XMLs/`:

### Items.xml
- **Total Items**: 1,360
- **Weapons**: 166
- **Armor**: 148
- **Consumables**: 294
- **Trinkets**: 59
- **Bows**: 18
- **Magic Items**: 76

### NPCs/NPCInfo.xml
- **Total NPCs**: 1,242
- **Hostile NPCs**: 328
- **Interactable NPCs**: 512
- **Undead**: 71
- **Predators**: 13
- **Quest Givers**: 108

### Quests/Quests.xml
- **Total Quests**: 108
- **Main Quests**: 19
- **Side Quests**: 89
- **Hidden Quests**: 2
- **Unique Quest Reward Items**: 70

### Harvestables/HarvestableInfo.xml
- **Total Harvestables**: 96
- **Trees**: 9
- **Woodcutting**: 10
- **Mining**: 11
- **Fishing**: 11
- **Alchemy**: 50
- **Level 1-10**: 31
- **Level 11-50**: 37
- **Level 51-100**: 28
- **Unique Items from Harvestables**: 98

### Loot/Loot.xml
- **Total Loot Tables**: 175
- **NPCs with Loot**: 267
- **Droppable Items**: 405
- **Tables with Conditional Drops**: 33
- **Tables with Guaranteed Drops**: Multiple tables include guaranteed (rate=1) drops

## File Structure

```
cursebreaker-parser/
├── src/
│   ├── lib.rs                    # Library exports
│   ├── main.rs                   # Main binary (Unity + XML parsing)
│   ├── types/
│   │   ├── mod.rs
│   │   ├── item.rs               # Item data structures
│   │   ├── npc.rs                # NPC data structures
│   │   ├── quest.rs              # Quest data structures
│   │   ├── harvestable.rs        # Harvestable data structures
│   │   ├── loot.rs               # Loot table data structures
│   │   └── interactable_resource.rs
│   ├── xml_parser.rs             # XML parsing logic (all types)
│   ├── item_database.rs          # ItemDatabase for runtime access
│   ├── npc_database.rs           # NpcDatabase for runtime access
│   ├── quest_database.rs         # QuestDatabase for runtime access
│   ├── harvestable_database.rs   # HarvestableDatabase for runtime access
│   └── loot_database.rs          # LootDatabase for runtime access
└── examples/
    ├── item_database_demo.rs     # Items usage example
    └── game_data_demo.rs         # Full game data example
```

## Dependencies Added

```toml
quick-xml = "0.37"              # XML parsing
serde = { version = "1.0", features = ["derive"] }  # Serialization
serde_json = "1.0"              # JSON serialization
diesel = { version = "2.2", features = ["sqlite"], optional = true }  # SQL (optional)
thiserror = "1.0"               # Error handling
```

## Completed Features

- ✅ Items (`/XMLs/Items/Items.xml`)
- ✅ NPCs (`/XMLs/Npcs/NPCInfo.xml`)
- ✅ Quests (`/XMLs/Quests/Quests.xml`)
- ✅ Harvestables (`/XMLs/Harvestables/HarvestableInfo.xml`)
- ✅ Loot tables (`/XMLs/Loot/Loot.xml`)

## Future Enhancements

The same pattern can be extended to parse other XML files:

- [ ] Maps (`/XMLs/Maps/*.xml`)
- [ ] Dialogue (`/XMLs/Dialogue/*.xml`)
- [ ] Events (`/XMLs/Events/*.xml`)
- [ ] Achievements (`/XMLs/Achievements/*.xml`)
- [ ] Traits (`/XMLs/Traits/*.xml`)
- [ ] Shops (`/XMLs/Shops/*.xml`)

Each follows the same pattern:
1. Define data structures in `src/types/`
2. Create parser in `src/xml_parser.rs`
3. Create database wrapper for runtime access
4. Add to `lib.rs` exports

## Integration with Unity Parser

The main binary (`src/main.rs`) demonstrates integration of both systems:

1. Load game data from XML files (Items, etc.)
2. Parse Unity scenes for game objects
3. Cross-reference data (e.g., item IDs in loot spawners)

This creates a complete game data pipeline from source files to runtime.