Connor/cursebreaker-parser-rust
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

resource icons DB

Browse Source
This commit is contained in:
Connor
2026-01-12 07:19:38 +00:00
parent 1072186ff1
commit 3720b6ad80
17 changed files with 545 additions and 2405 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
cursebreaker-parser/README.md
View File

@@ -45,7 +45,12 @@ The project provides multiple binaries to handle different parsing tasks. This a
2. **scene-parser** - Parses Unity scenes and extracts world resource locations
- Slow execution (Unity project initialization)
- Extracts InteractableResource components and their positions
- Saves to `world_resources` table (item_id and 2D coordinates)
- Saves to `world_resources` table (harvestable_id and 2D coordinates)
- Processes item icons for harvestables:
- Looks up the first item drop for each harvestable from `harvestable_drops` table
- Loads the icon from `Data/Textures/ItemIcons/{item_id}.png`
- Applies white outline (1px) and resizes to 64x64
- Converts to WebP and stores in `resource_icons` table
- Run this when scene files change
```bash
cargo run --bin scene-parser
@@ -87,6 +92,11 @@ The project provides multiple binaries to handle different parsing tasks. This a
cargo run --bin verify-stats
```
9. **verify-resource-icons** - Verifies resource icons for harvestables
```bash
cargo run --bin verify-resource-icons
```
### Building for Production
Build specific binaries for release:
@@ -185,6 +195,39 @@ for resource in copper_ore {
See `examples/query_world_resources.rs` for a complete example.
### Querying Resource Icons
```rust
use diesel::prelude::*;
use diesel::sqlite::SqliteConnection;
// Connect to database
let mut conn = SqliteConnection::establish("../cursebreaker.db")?;
// Define the structure
#[derive(Queryable, Debug)]
struct ResourceIcon {
item_id: i32, // Harvestable ID
name: String, // Harvestable name
icon_64: Vec<u8>, // WebP image data (64x64 with white border)
}
// Query icon for a specific harvestable
use cursebreaker_parser::schema::resource_icons::dsl::*;
let copper_icon = resource_icons
.filter(item_id.eq(2)) // Harvestable ID for Copper Ore
.first::<ResourceIcon>(&mut conn)?;
println!("Found icon for: {}", copper_icon.name);
println!("Icon size: {} bytes (WebP format)", copper_icon.icon_64.len());
// Save to file if needed
std::fs::write("copper_ore.webp", &copper_icon.icon_64)?;
```
See `examples/resource_icons_example.rs` for a complete example.
### Additional Databases
Similar APIs are available for other game data types:
@@ -227,6 +270,7 @@ The project includes several example programs demonstrating different aspects of
- **game_data_demo.rs** - Comprehensive demo loading and querying all game data types (Items, NPCs, Quests, Harvestables, Loot)
- **item_database_demo.rs** - Focused on item database operations
- **query_world_resources.rs** - Querying world resource locations from the database
- **resource_icons_example.rs** - Querying processed harvestable icons with white borders
- **fast_travel_example.rs** - Working with fast travel locations
- **maps_example.rs** - Map data handling
- **player_houses_example.rs** - Player house management
@@ -252,7 +296,8 @@ cursebreaker-parser/
│ │ ├── verify-db.rs # Database verification
│ │ ├── verify-expanded-db.rs # Expanded database verification
│ │ ├── verify-images.rs # Image verification
│ │ └── verify-stats.rs # Stats verification
│ │ ├── verify-stats.rs # Stats verification
│ │ └── verify-resource-icons.rs # Resource icons verification
│ ├── xml_parser.rs # XML parsing utilities
│ ├── image_processor.rs # Image processing utilities
│ ├── item_loader.rs # Item loading logic
@@ -297,6 +342,7 @@ The parser uses Diesel for database operations with SQLite. Database migrations
- Quest definitions and phases
- Harvestable resources and drop tables
- World resource locations from Unity scenes
- Resource icons for harvestables (64x64 WebP with white borders)
- Minimap tiles and metadata
- Shop inventories and pricing
- Player houses and locations

442
cursebreaker-parser/XML_PARSING.md
View File

@@ -1,442 +0,0 @@
# XML Parsing in Cursebreaker Parser
This document describes the XML parsing functionality added to the cursebreaker-parser project.
## Overview
The parser now supports loading game data from Cursebreaker's XML files and storing them in efficient data structures for runtime access and SQL database serialization.
## Features
- ✅ Parse Items, NPCs, Quests, and Harvestables XML files with full attribute and nested element support
- ✅ In-memory databases with fast lookups by ID, name, and various filters
- ✅ JSON serialization for SQL database storage
- ✅ Type-safe data structures with serde support
- ✅ Easy-to-use API with query methods
- ✅ Cross-referencing support between different data types
## Quick Start
### Loading Items
```rust
use cursebreaker_parser::ItemDatabase;
let item_db = ItemDatabase::load_from_xml("Data/XMLs/Items/Items.xml")?;
println!("Loaded {} items", item_db.len());
```
### Querying Items
```rust
// Get by ID
if let Some(item) = item_db.get_by_id(150) {
println!("Found: {}", item.name);
}
// Get by category
let bows = item_db.get_by_category("bow");
// Get by slot
let weapons = item_db.get_by_slot("weapon");
// Get by skill requirement
let magic_items = item_db.get_by_skill("magic");
// Get all items
for item in item_db.all_items() {
println!("{}: {}", item.id, item.name);
}
```
### SQL Serialization
```rust
// Prepare items for SQL insertion
let sql_data = item_db.prepare_for_sql();
for (id, name, json_data) in sql_data {
// INSERT INTO items (id, name, data) VALUES (?, ?, ?)
// Use your preferred SQL library to insert
}
```
## Data Structures
### Item
The main `Item` struct contains all item attributes from the XML:
```rust
pub struct Item {
// Required
pub id: i32,
pub name: String,
// Optional attributes
pub level: Option<i32>,
pub description: Option<String>,
pub price: Option<i32>,
pub slot: Option<String>,
pub category: Option<String>,
pub skill: Option<String>,
// ... many more fields
// Nested elements
pub stats: Vec<ItemStat>,
pub crafting_recipes: Vec<CraftingRecipe>,
pub animations: Option<AnimationSet>,
pub generate_rules: Vec<GenerateRule>,
}
```
### ItemStat
Represents item statistics:
```rust
pub struct ItemStat {
// Damage
pub damagephysical: Option<i32>,
pub damagemagical: Option<i32>,
pub damageranged: Option<i32>,
// Accuracy
pub accuracyphysical: Option<i32>,
pub accuracymagical: Option<i32>,
pub accuracyranged: Option<i32>,
// Resistance
pub resistancephysical: Option<i32>,
pub resistancemagical: Option<i32>,
pub resistanceranged: Option<i32>,
// Core stats
pub health: Option<i32>,
pub mana: Option<i32>,
pub manaregen: Option<i32>,
pub healing: Option<i32>,
// Harvesting
pub harvestingspeedwoodcutting: Option<i32>,
}
```
## Example Programs
Run the demos to see all features in action:
```bash
# Items only
cargo run --example item_database_demo
# All game data (Items, NPCs, Quests, Harvestables)
cargo run --example game_data_demo
```
## Loading NPCs
```rust
use cursebreaker_parser::NpcDatabase;
let npc_db = NpcDatabase::load_from_xml("Data/XMLs/Npcs/NPCInfo.xml")?;
println!("Loaded {} NPCs", npc_db.len());
```
### Querying NPCs
```rust
// Get by ID
if let Some(npc) = npc_db.get_by_id(1) {
println!("Found: {}", npc.name);
}
// Get hostile NPCs
let hostile = npc_db.get_hostile();
// Get interactable NPCs
let interactable = npc_db.get_interactable();
// Get NPCs by tag
let undead = npc_db.get_by_tag("Undead");
// Get shopkeepers
let shops = npc_db.get_shopkeepers();
```
## Loading Quests
```rust
use cursebreaker_parser::QuestDatabase;
let quest_db = QuestDatabase::load_from_xml("Data/XMLs/Quests/Quests.xml")?;
println!("Loaded {} quests", quest_db.len());
```
### Querying Quests
```rust
// Get by ID
if let Some(quest) = quest_db.get_by_id(1) {
println!("Quest: {}", quest.name);
println!("Phases: {}", quest.phase_count());
}
// Get main quests
let main_quests = quest_db.get_main_quests();
// Get side quests
let side_quests = quest_db.get_side_quests();
// Get hidden quests
let hidden = quest_db.get_hidden_quests();
```
## Loading Harvestables
```rust
use cursebreaker_parser::HarvestableDatabase;
let harvestable_db = HarvestableDatabase::load_from_xml("Data/XMLs/Harvestables/HarvestableInfo.xml")?;
println!("Loaded {} harvestables", harvestable_db.len());
```
### Querying Harvestables
```rust
// Get by type ID
if let Some(harvestable) = harvestable_db.get_by_typeid(1) {
println!("Found: {}", harvestable.name);
}
// Get by skill
let woodcutting = harvestable_db.get_by_skill("Woodcutting");
let mining = harvestable_db.get_by_skill("mining");
let fishing = harvestable_db.get_by_skill("Fishing");
// Get trees (harvestables with tree=1)
let trees = harvestable_db.get_trees();
// Get by tool requirement
let hatchet_nodes = harvestable_db.get_by_tool("hatchet");
let pickaxe_nodes = harvestable_db.get_by_tool("pickaxe");
// Get by level range
let beginner = harvestable_db.get_by_level_range(1, 10);
let advanced = harvestable_db.get_by_level_range(50, 100);
```
## Loading Loot Tables
```rust
use cursebreaker_parser::LootDatabase;
let loot_db = LootDatabase::load_from_xml("Data/XMLs/Loot/Loot.xml")?;
println!("Loaded {} loot tables", loot_db.len());
```
### Querying Loot Tables
```rust
// Get all loot tables for a specific NPC
let npc_id = 45;
let tables = loot_db.get_tables_for_npc(npc_id);
// Get all drops for a specific NPC
let drops = loot_db.get_drops_for_npc(npc_id);
for drop in drops {
println!("Item ID: {}, Rate: {:?}", drop.item, drop.rate);
}
// Find which NPCs drop a specific item
let item_id = 180;
let npcs = loot_db.get_npcs_dropping_item(item_id);
println!("Item {} drops from {} NPCs", item_id, npcs.len());
// Get all tables with conditional drops (checks field)
let conditional = loot_db.get_conditional_tables();
// Get all tables with guaranteed drops (rate = 1)
let guaranteed = loot_db.get_tables_with_guaranteed_drops();
// Get all unique item IDs that can drop
let droppable_items = loot_db.get_all_droppable_items();
// Get all NPCs that have loot tables
let npcs_with_loot = loot_db.get_all_npcs_with_loot();
```
## Cross-referencing Data
```rust
// Find items rewarded by quests
for quest in quest_db.all_quests() {
for reward in &quest.rewards {
if let Some(item_id) = reward.item {
if let Some(item) = item_db.get_by_id(item_id) {
println!("Quest '{}' rewards: {}", quest.name, item.name);
}
}
}
}
// Find NPCs that give quests
for npc in npc_db.all_npcs() {
if !npc.questmarkers.is_empty() {
println!("NPC '{}' has {} quest markers", npc.name, npc.questmarkers.len());
}
}
// Find items that drop from harvestables
for harvestable in harvestable_db.all_harvestables() {
for drop in &harvestable.drops {
if let Some(item) = item_db.get_by_id(drop.id) {
println!("'{}' drops: {} (rate: {})",
harvestable.name, item.name, drop.droprate.unwrap_or(0));
}
}
}
// Find what items an NPC drops
let npc_id = 45;
if let Some(npc) = npc_db.get_by_id(npc_id) {
let drops = loot_db.get_drops_for_npc(npc_id);
println!("NPC '{}' drops {} items:", npc.name, drops.len());
for drop in drops {
if let Some(item) = item_db.get_by_id(drop.item) {
println!(" - {}", item.name);
}
}
}
// Find which NPCs drop a specific item
let item_id = 180;
if let Some(item) = item_db.get_by_id(item_id) {
let npcs = loot_db.get_npcs_dropping_item(item_id);
println!("Item '{}' drops from:", item.name);
for npc_id in npcs {
if let Some(npc) = npc_db.get_by_id(npc_id) {
println!(" - {}", npc.name);
}
}
}
```
## Statistics from XML Files
When loaded from `/home/connor/repos/CBAssets/Data/XMLs/`:
### Items.xml
- **Total Items**: 1,360
- **Weapons**: 166
- **Armor**: 148
- **Consumables**: 294
- **Trinkets**: 59
- **Bows**: 18
- **Magic Items**: 76
### NPCs/NPCInfo.xml
- **Total NPCs**: 1,242
- **Hostile NPCs**: 328
- **Interactable NPCs**: 512
- **Undead**: 71
- **Predators**: 13
- **Quest Givers**: 108
### Quests/Quests.xml
- **Total Quests**: 108
- **Main Quests**: 19
- **Side Quests**: 89
- **Hidden Quests**: 2
- **Unique Quest Reward Items**: 70
### Harvestables/HarvestableInfo.xml
- **Total Harvestables**: 96
- **Trees**: 9
- **Woodcutting**: 10
- **Mining**: 11
- **Fishing**: 11
- **Alchemy**: 50
- **Level 1-10**: 31
- **Level 11-50**: 37
- **Level 51-100**: 28
- **Unique Items from Harvestables**: 98
### Loot/Loot.xml
- **Total Loot Tables**: 175
- **NPCs with Loot**: 267
- **Droppable Items**: 405
- **Tables with Conditional Drops**: 33
- **Tables with Guaranteed Drops**: Multiple tables include guaranteed (rate=1) drops
## File Structure
```
cursebreaker-parser/
├── src/
│ ├── lib.rs # Library exports
│ ├── main.rs # Main binary (Unity + XML parsing)
│ ├── types/
│ │ ├── mod.rs
│ │ ├── item.rs # Item data structures
│ │ ├── npc.rs # NPC data structures
│ │ ├── quest.rs # Quest data structures
│ │ ├── harvestable.rs # Harvestable data structures
│ │ ├── loot.rs # Loot table data structures
│ │ └── interactable_resource.rs
│ ├── xml_parser.rs # XML parsing logic (all types)
│ ├── item_database.rs # ItemDatabase for runtime access
│ ├── npc_database.rs # NpcDatabase for runtime access
│ ├── quest_database.rs # QuestDatabase for runtime access
│ ├── harvestable_database.rs # HarvestableDatabase for runtime access
│ └── loot_database.rs # LootDatabase for runtime access
└── examples/
├── item_database_demo.rs # Items usage example
└── game_data_demo.rs # Full game data example
```
## Dependencies Added
```toml
quick-xml = "0.37" # XML parsing
serde = { version = "1.0", features = ["derive"] } # Serialization
serde_json = "1.0" # JSON serialization
diesel = { version = "2.2", features = ["sqlite"], optional = true } # SQL (optional)
thiserror = "1.0" # Error handling
```
## Completed Features
- ✅ Items (`/XMLs/Items/Items.xml`)
- ✅ NPCs (`/XMLs/Npcs/NPCInfo.xml`)
- ✅ Quests (`/XMLs/Quests/Quests.xml`)
- ✅ Harvestables (`/XMLs/Harvestables/HarvestableInfo.xml`)
- ✅ Loot tables (`/XMLs/Loot/Loot.xml`)
## Future Enhancements
The same pattern can be extended to parse other XML files:
- [ ] Maps (`/XMLs/Maps/*.xml`)
- [ ] Dialogue (`/XMLs/Dialogue/*.xml`)
- [ ] Events (`/XMLs/Events/*.xml`)
- [ ] Achievements (`/XMLs/Achievements/*.xml`)
- [ ] Traits (`/XMLs/Traits/*.xml`)
- [ ] Shops (`/XMLs/Shops/*.xml`)
Each follows the same pattern:
1. Define data structures in `src/types/`
2. Create parser in `src/xml_parser.rs`
3. Create database wrapper for runtime access
4. Add to `lib.rs` exports
## Integration with Unity Parser
The main binary (`src/main.rs`) demonstrates integration of both systems:
1. Load game data from XML files (Items, etc.)
2. Parse Unity scenes for game objects
3. Cross-reference data (e.g., item IDs in loot spawners)
This creates a complete game data pipeline from source files to runtime.

53
cursebreaker-parser/examples/resource_icons_example.rs Normal file
View File

@@ -0,0 +1,53 @@
//! Example: Query resource icons from the database
//!
//! This example shows how to retrieve processed resource icons for harvestables.
//! Icons are 64x64 WebP images with white borders.
use diesel::prelude::*;
use diesel::sqlite::SqliteConnection;
use std::env;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Connect to database
let database_url = env::var("DATABASE_URL").unwrap_or_else(|_| "../cursebreaker.db".to_string());
let mut conn = SqliteConnection::establish(&database_url)?;
// Define the structure
#[derive(Queryable, Debug)]
struct ResourceIcon {
item_id: i32,
name: String,
icon_64: Vec<u8>,
}
// Import schema
use cursebreaker_parser::schema::resource_icons::dsl::*;
// Query all resource icons
let icons = resource_icons.load::<ResourceIcon>(&mut conn)?;
println!("📦 Resource Icons Database");
println!("========================\n");
println!("Total icons: {}\n", icons.len());
for icon in icons {
println!("Harvestable ID: {}", icon.item_id);
println!(" Name: {}", icon.name);
println!(" Icon size: {} bytes (WebP format, 64x64 with white border)", icon.icon_64.len());
println!();
}
// Example: Get icon for a specific harvestable
println!("\n🔍 Looking up Copper Ore (harvestable_id = 2):");
let copper_icon = resource_icons
.filter(item_id.eq(2))
.first::<ResourceIcon>(&mut conn)?;
println!(" Name: {}", copper_icon.name);
println!(" Icon size: {} bytes", copper_icon.icon_64.len());
// You can save the icon to a file for testing:
// std::fs::write("copper_ore.webp", &copper_icon.icon_64)?;
Ok(())
}

10
cursebreaker-parser/migrations/2026-01-12-063636-0000_expand_harvestables/down.sql Normal file
View File

@@ -0,0 +1,10 @@
-- Revert to the simple harvestables table
DROP TABLE IF EXISTS harvestable_drops;
DROP TABLE IF EXISTS harvestables;
CREATE TABLE harvestables (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
data TEXT NOT NULL
);

39
cursebreaker-parser/migrations/2026-01-12-063636-0000_expand_harvestables/up.sql Normal file
View File

@@ -0,0 +1,39 @@
-- Restructure harvestables table to store expanded data
DROP TABLE IF EXISTS harvestables;
CREATE TABLE harvestables (
id INTEGER PRIMARY KEY NOT NULL,
name TEXT NOT NULL,
description TEXT NOT NULL,
comment TEXT NOT NULL,
level INTEGER NOT NULL,
skill TEXT NOT NULL,
tool TEXT NOT NULL,
min_health INTEGER NOT NULL,
max_health INTEGER NOT NULL,
harvesttime INTEGER NOT NULL,
hittime INTEGER NOT NULL,
respawntime INTEGER NOT NULL
);
-- Create harvestable_drops table
CREATE TABLE harvestable_drops (
id INTEGER PRIMARY KEY AUTOINCREMENT,
harvestable_id INTEGER NOT NULL,
item_id INTEGER NOT NULL,
minamount INTEGER NOT NULL,
maxamount INTEGER NOT NULL,
droprate INTEGER NOT NULL,
droprateboost INTEGER NOT NULL,
amountboost INTEGER NOT NULL,
comment TEXT NOT NULL,
FOREIGN KEY (harvestable_id) REFERENCES harvestables(id),
FOREIGN KEY (item_id) REFERENCES items(id)
);
CREATE INDEX idx_harvestable_drops_harvestable_id ON harvestable_drops(harvestable_id);
CREATE INDEX idx_harvestable_drops_item_id ON harvestable_drops(item_id);
CREATE INDEX idx_harvestables_skill ON harvestables(skill);
CREATE INDEX idx_harvestables_tool ON harvestables(tool);
CREATE INDEX idx_harvestables_level ON harvestables(level);

72
cursebreaker-parser/src/bin/scene-parser.rs
View File

@@ -122,19 +122,19 @@ fn process_item_icons(
conn: &mut SqliteConnection,
scene: &unity_parser::UnityScene,
) -> Result<(), Box<dyn std::error::Error>> {
use cursebreaker_parser::schema::{resource_icons, items};
use cursebreaker_parser::schema::{resource_icons, items, harvestables, harvestable_drops};
// Collect unique item IDs from resources
let mut unique_items: HashMap<i32, String> = HashMap::new();
// Collect unique harvestable IDs from resources
let mut unique_harvestables: HashMap<i32, String> = HashMap::new();
scene.world
.query_all::<(&InteractableResource, &unity_parser::GameObject)>()
.for_each(|(resource, object)| {
unique_items.entry(resource.type_id as i32)
unique_harvestables.entry(resource.type_id as i32)
.or_insert_with(|| object.name.to_string());
});
info!(" Found {} unique resource types", unique_items.len());
info!(" Found {} unique harvestable types", unique_harvestables.len());
// Clear existing resource icons (regenerated each run)
diesel::delete(resource_icons::table).execute(conn)?;
@@ -146,22 +146,46 @@ fn process_item_icons(
let mut processed_count = 0;
let mut failed_count = 0;
// Process each unique item
for (item_id, default_name) in unique_items.iter() {
// Try to get the actual item name from the items table
let item_name: String = items::table
.filter(items::id.eq(item_id))
.select(items::name)
// Process each unique harvestable
for (harvestable_id, default_name) in unique_harvestables.iter() {
// Get the harvestable name
let harvestable_name: String = harvestables::table
.filter(harvestables::id.eq(harvestable_id))
.select(harvestables::name)
.first(conn)
.unwrap_or_else(|_| default_name.clone());
// Construct icon path
// Get the first item drop for this harvestable
let item_id_result: Result<i32, _> = harvestable_drops::table
.filter(harvestable_drops::harvestable_id.eq(harvestable_id))
.select(harvestable_drops::item_id)
.order(harvestable_drops::id.asc())
.first(conn);
let item_id = match item_id_result {
Ok(id) => id,
Err(_) => {
warn!(" ⚠️ No drops found for harvestable {} ({})", harvestable_id, harvestable_name);
failed_count += 1;
continue;
}
};
// Get the item name
let item_name: String = items::table
.filter(items::id.eq(&item_id))
.select(items::name)
.first(conn)
.unwrap_or_else(|_| format!("Item {}", item_id));
// Construct icon path using the item_id from the drop
let icon_path = PathBuf::from(cb_assets_path)
.join("Data/Textures/ItemIcons")
.join(format!("{}.png", item_id));
if !icon_path.exists() {
warn!(" ⚠️ Icon not found for item {} ({}): {}", item_id, item_name, icon_path.display());
warn!(" ⚠️ Icon not found for harvestable {} ({}) -> item {} ({}): {}",
harvestable_id, harvestable_name, item_id, item_name, icon_path.display());
failed_count += 1;
continue;
}
@@ -170,38 +194,38 @@ fn process_item_icons(
match processor.process_image(&icon_path, &[64], None, Some(&outline_config)) {
Ok(processed) => {
if let Some(icon_data) = processed.get(64) {
// Insert into database
// Insert into database using harvestable_id as the key
match diesel::insert_into(resource_icons::table)
.values((
resource_icons::item_id.eq(item_id),
resource_icons::name.eq(&item_name),
resource_icons::item_id.eq(harvestable_id),
resource_icons::name.eq(&harvestable_name),
resource_icons::icon_64.eq(icon_data.as_slice()),
))
.execute(conn)
{
Ok(_) => {
info!(" ✓ Processed icon for item {} ({}): {} bytes",
item_id, item_name, icon_data.len());
info!(" ✓ Harvestable {} ({}) -> Item {} ({}): {} bytes",
harvestable_id, harvestable_name, item_id, item_name, icon_data.len());
processed_count += 1;
}
Err(e) => {
warn!(" ⚠️ Failed to insert icon for item {} ({}): {}",
item_id, item_name, e);
warn!(" ⚠️ Failed to insert icon for harvestable {} ({}): {}",
harvestable_id, harvestable_name, e);
failed_count += 1;
}
}
}
}
Err(e) => {
warn!(" ⚠️ Failed to process icon for item {} ({}): {}",
item_id, item_name, e);
warn!(" ⚠️ Failed to process icon for harvestable {} ({}) -> item {} ({}): {}",
harvestable_id, harvestable_name, item_id, item_name, e);
failed_count += 1;
}
}
}
info!("✅ Processed {} item icons ({} succeeded, {} failed)",
unique_items.len(), processed_count, failed_count);
info!("✅ Processed {} harvestable icons ({} succeeded, {} failed)",
unique_harvestables.len(), processed_count, failed_count);
Ok(())
}

2
cursebreaker-parser/src/bin/xml-parser.rs
View File

@@ -5,7 +5,7 @@
//! - Populating the SQLite database with the parsed data
//! - Generating statistics about the loaded data
use cursebreaker_parser::ItemDatabase;
use cursebreaker_parser::{ItemDatabase, HarvestableDatabase};
use log::{info, warn, LevelFilter};
use unity_parser::log::DedupLogger;
use diesel::prelude::*;

241
cursebreaker-parser/src/databases/harvestable_database.rs
View File

@@ -62,27 +62,21 @@ impl HarvestableDatabase {
/// Get harvestables by skill
pub fn get_by_skill(&self, skill: &str) -> Vec<&Harvestable> {
use crate::types::SkillType;
let skill_type = skill.parse::<SkillType>().unwrap_or(SkillType::None);
self.harvestables
.iter()
.filter(|h| {
h.skill
.as_ref()
.map(|s| s.eq_ignore_ascii_case(skill))
.unwrap_or(false)
})
.filter(|h| h.skill == skill_type)
.collect()
}
/// Get harvestables that require a specific tool
pub fn get_by_tool(&self, tool: &str) -> Vec<&Harvestable> {
use crate::types::Tool;
let tool_type = tool.parse::<Tool>().unwrap_or(Tool::None);
self.harvestables
.iter()
.filter(|h| {
h.tool
.as_ref()
.map(|t| t.eq_ignore_ascii_case(tool))
.unwrap_or(false)
})
.filter(|h| h.tool == tool_type)
.collect()
}
@@ -106,11 +100,7 @@ impl HarvestableDatabase {
pub fn get_by_level_range(&self, min_level: i32, max_level: i32) -> Vec<&Harvestable> {
self.harvestables
.iter()
.filter(|h| {
h.level
.map(|l| l >= min_level && l <= max_level)
.unwrap_or(false)
})
.filter(|h| h.level >= min_level && h.level <= max_level)
.collect()
}
@@ -126,38 +116,136 @@ impl HarvestableDatabase {
/// Prepare harvestables for SQL insertion (deprecated - use save_to_db instead)
#[deprecated(note = "Use save_to_db() to save directly to SQLite database")]
pub fn prepare_for_sql(&self) -> Vec<(i32, String, String)> {
#[allow(deprecated)]
pub fn prepare_for_sql(&self) -> Vec<(i32, String, String, String, i32, String, String, i32, i32, i32, i32, i32)> {
use crate::types::{SkillType, Tool};
self.harvestables
.iter()
.map(|harvestable| {
let json = serde_json::to_string(harvestable).unwrap_or_else(|_| "{}".to_string());
(harvestable.typeid, harvestable.name.clone(), json)
let skill_str = match harvestable.skill {
SkillType::None => "none",
SkillType::Swordsmanship => "swordsmanship",
SkillType::Archery => "archery",
SkillType::Magic => "magic",
SkillType::Defence => "defence",
SkillType::Mining => "mining",
SkillType::Woodcutting => "woodcutting",
SkillType::Fishing => "fishing",
SkillType::Cooking => "cooking",
SkillType::Carpentry => "carpentry",
SkillType::Blacksmithy => "blacksmithy",
SkillType::Tailoring => "tailoring",
SkillType::Alchemy => "alchemy",
}.to_string();
let tool_str = match harvestable.tool {
Tool::None => "none",
Tool::Pickaxe => "pickaxe",
Tool::Hatchet => "hatchet",
Tool::Scythe => "scythe",
Tool::Hammer => "hammer",
Tool::Shears => "shears",
Tool::FishingRod => "fishingrod",
}.to_string();
(
harvestable.typeid,
harvestable.name.clone(),
harvestable.desc.clone(),
harvestable.comment.clone(),
harvestable.level,
skill_str,
tool_str,
harvestable.min_health,
harvestable.max_health,
harvestable.harvesttime,
harvestable.hittime,
harvestable.respawntime,
)
})
.collect()
}
/// Save all harvestables to SQLite database
pub fn save_to_db(&self, conn: &mut SqliteConnection) -> Result<usize, diesel::result::Error> {
use crate::schema::harvestables;
use crate::schema::{harvestables, harvestable_drops};
use crate::types::{SkillType, Tool};
let records: Vec<_> = self
.harvestables
.iter()
.map(|harvestable| {
let json = serde_json::to_string(harvestable).unwrap_or_else(|_| "{}".to_string());
(
harvestables::id.eq(harvestable.typeid),
harvestables::name.eq(&harvestable.name),
harvestables::data.eq(json),
)
})
.collect();
// Clear existing data
diesel::delete(harvestable_drops::table).execute(conn)?;
diesel::delete(harvestables::table).execute(conn)?;
let mut count = 0;
for record in records {
for harvestable in &self.harvestables {
// Convert enums to strings for database storage
let skill_str = match harvestable.skill {
SkillType::None => "none",
SkillType::Swordsmanship => "swordsmanship",
SkillType::Archery => "archery",
SkillType::Magic => "magic",
SkillType::Defence => "defence",
SkillType::Mining => "mining",
SkillType::Woodcutting => "woodcutting",
SkillType::Fishing => "fishing",
SkillType::Cooking => "cooking",
SkillType::Carpentry => "carpentry",
SkillType::Blacksmithy => "blacksmithy",
SkillType::Tailoring => "tailoring",
SkillType::Alchemy => "alchemy",
};
let tool_str = match harvestable.tool {
Tool::None => "none",
Tool::Pickaxe => "pickaxe",
Tool::Hatchet => "hatchet",
Tool::Scythe => "scythe",
Tool::Hammer => "hammer",
Tool::Shears => "shears",
Tool::FishingRod => "fishingrod",
};
// Insert harvestable
diesel::insert_into(harvestables::table)
.values(&record)
.values((
harvestables::id.eq(harvestable.typeid),
harvestables::name.eq(&harvestable.name),
harvestables::description.eq(&harvestable.desc),
harvestables::comment.eq(&harvestable.comment),
harvestables::level.eq(harvestable.level),
harvestables::skill.eq(skill_str),
harvestables::tool.eq(tool_str),
harvestables::min_health.eq(harvestable.min_health),
harvestables::max_health.eq(harvestable.max_health),
harvestables::harvesttime.eq(harvestable.harvesttime),
harvestables::hittime.eq(harvestable.hittime),
harvestables::respawntime.eq(harvestable.respawntime),
))
.execute(conn)?;
// Insert drops
for drop in &harvestable.drops {
// Try to insert, but skip if foreign key constraint fails (item doesn't exist)
let insert_result = diesel::insert_into(harvestable_drops::table)
.values((
harvestable_drops::harvestable_id.eq(harvestable.typeid),
harvestable_drops::item_id.eq(drop.id),
harvestable_drops::minamount.eq(drop.minamount),
harvestable_drops::maxamount.eq(drop.maxamount),
harvestable_drops::droprate.eq(drop.droprate),
harvestable_drops::droprateboost.eq(drop.droprateboost),
harvestable_drops::amountboost.eq(drop.amountboost),
harvestable_drops::comment.eq(&drop.comment),
))
.execute(conn);
// Log warning if insert failed but continue
if let Err(e) = insert_result {
eprintln!("Warning: Failed to insert drop for harvestable {} (item {}): {}",
harvestable.typeid, drop.id, e);
}
}
count += 1;
}
@@ -166,22 +254,91 @@ impl HarvestableDatabase {
/// Load all harvestables from SQLite database
pub fn load_from_db(conn: &mut SqliteConnection) -> Result<Self, diesel::result::Error> {
use crate::schema::harvestables::dsl::*;
use crate::schema::{harvestables, harvestable_drops};
use crate::types::{Harvestable, HarvestableDrop, SkillType, Tool};
use diesel::prelude::*;
#[derive(Queryable)]
struct HarvestableRecord {
id: Option<i32>,
id: i32,
name: String,
data: String,
description: String,
comment: String,
level: i32,
skill: String,
tool: String,
min_health: i32,
max_health: i32,
harvesttime: i32,
hittime: i32,
respawntime: i32,
}
let records = harvestables.load::<HarvestableRecord>(conn)?;
#[derive(Queryable)]
struct HarvestableDropRecord {
id: Option<i32>,
harvestable_id: i32,
item_id: i32,
minamount: i32,
maxamount: i32,
droprate: i32,
droprateboost: i32,
amountboost: i32,
comment: String,
}
let harv_records = harvestables::table.load::<HarvestableRecord>(conn)?;
let drop_records = harvestable_drops::table.load::<HarvestableDropRecord>(conn)?;
let mut loaded_harvestables = Vec::new();
for record in records {
if let Ok(harvestable) = serde_json::from_str::<Harvestable>(&record.data) {
loaded_harvestables.push(harvestable);
for record in harv_records {
let mut harvestable = Harvestable {
typeid: record.id,
name: record.name,
actionname: String::new(),
desc: record.description,
comment: record.comment,
level: record.level,
skill: record.skill.parse().unwrap_or(SkillType::None),
tool: record.tool.parse().unwrap_or(Tool::None),
min_health: record.min_health,
max_health: record.max_health,
harvesttime: record.harvesttime,
hittime: record.hittime,
respawntime: record.respawntime,
harvestsfx: String::new(),
endsfx: String::new(),
receiveitemsfx: String::new(),
animation: String::new(),
takehitanimation: String::new(),
endgfx: String::new(),
tree: false,
hidemilestone: false,
nohighlight: false,
hideminimap: false,
noleftclickinteract: false,
interactdistance: String::new(),
drops: Vec::new(),
};
// Add drops for this harvestable
for drop_rec in &drop_records {
if drop_rec.harvestable_id == record.id {
harvestable.drops.push(HarvestableDrop {
id: drop_rec.item_id,
minamount: drop_rec.minamount,
maxamount: drop_rec.maxamount,
droprate: drop_rec.droprate,
droprateboost: drop_rec.droprateboost,
amountboost: drop_rec.amountboost,
checks: String::new(),
comment: drop_rec.comment.clone(),
dontconsumehealth: false,
});
}
}
loaded_harvestables.push(harvestable);
}
let mut db = Self::new();

30
cursebreaker-parser/src/schema.rs
View File

@@ -31,10 +31,33 @@ diesel::table! {
}
diesel::table! {
harvestables (id) {
harvestable_drops (id) {
id -> Nullable<Integer>,
harvestable_id -> Integer,
item_id -> Integer,
minamount -> Integer,
maxamount -> Integer,
droprate -> Integer,
droprateboost -> Integer,
amountboost -> Integer,
comment -> Text,
}
}
diesel::table! {
harvestables (id) {
id -> Integer,
name -> Text,
data -> Text,
description -> Text,
comment -> Text,
level -> Integer,
skill -> Text,
tool -> Text,
min_health -> Integer,
max_health -> Integer,
harvesttime -> Integer,
hittime -> Integer,
respawntime -> Integer,
}
}
@@ -174,12 +197,15 @@ diesel::table! {
diesel::joinable!(crafting_recipe_items -> crafting_recipes (recipe_id));
diesel::joinable!(crafting_recipe_items -> items (item_id));
diesel::joinable!(crafting_recipes -> items (product_item_id));
diesel::joinable!(harvestable_drops -> harvestables (harvestable_id));
diesel::joinable!(harvestable_drops -> items (item_id));
diesel::joinable!(item_stats -> items (item_id));
diesel::allow_tables_to_appear_in_same_query!(
crafting_recipe_items,
crafting_recipes,
fast_travel_locations,
harvestable_drops,
harvestables,
item_stats,
items,

117
cursebreaker-parser/src/types/cursebreaker/harvestable.rs
View File

@@ -1,4 +1,5 @@
use serde::{Deserialize, Serialize};
use super::item::{SkillType, Tool};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Harvestable {
@@ -7,40 +8,41 @@ pub struct Harvestable {
pub name: String,
// Basic attributes
pub actionname: Option<String>,
pub desc: Option<String>,
pub comment: Option<String>,
pub level: Option<i32>,
pub skill: Option<String>,
pub tool: Option<String>,
pub actionname: String,
pub desc: String,
pub comment: String,
pub level: i32,
pub skill: SkillType,
pub tool: Tool,
// Health (can be range like "3-5" or single value)
pub health: Option<String>,
// Health
pub min_health: i32,
pub max_health: i32,
// Timing
pub harvesttime: Option<i32>,
pub hittime: Option<i32>,
pub respawntime: Option<i32>,
pub harvesttime: i32,
pub hittime: i32,
pub respawntime: i32,
// Audio
pub harvestsfx: Option<String>,
pub endsfx: Option<String>,
pub receiveitemsfx: Option<String>,
pub harvestsfx: String,
pub endsfx: String,
pub receiveitemsfx: String,
// Visuals
pub animation: Option<String>,
pub takehitanimation: Option<String>,
pub endgfx: Option<String>,
pub animation: String,
pub takehitanimation: String,
pub endgfx: String,
// Behavior flags
pub tree: Option<i32>,
pub hidemilestone: Option<i32>,
pub nohighlight: Option<i32>,
pub hideminimap: Option<i32>,
pub noleftclickinteract: Option<i32>,
pub tree: bool,
pub hidemilestone: bool,
pub nohighlight: bool,
pub hideminimap: bool,
pub noleftclickinteract: bool,
// Interaction
pub interactdistance: Option<String>,
pub interactdistance: String,
// Drops
pub drops: Vec<HarvestableDrop>,
@@ -49,14 +51,14 @@ pub struct Harvestable {
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HarvestableDrop {
pub id: i32,
pub minamount: Option<i32>,
pub maxamount: Option<i32>,
pub droprate: Option<i32>,
pub droprateboost: Option<i32>,
pub amountboost: Option<i32>,
pub checks: Option<String>,
pub comment: Option<String>,
pub dontconsumehealth: Option<i32>,
pub minamount: i32,
pub maxamount: i32,
pub droprate: i32,
pub droprateboost: i32,
pub amountboost: i32,
pub checks: String,
pub comment: String,
pub dontconsumehealth: bool,
}
impl Harvestable {
@@ -64,45 +66,46 @@ impl Harvestable {
Self {
typeid,
name,
actionname: None,
desc: None,
comment: None,
level: None,
skill: None,
tool: None,
health: None,
harvesttime: None,
hittime: None,
respawntime: None,
harvestsfx: None,
endsfx: None,
receiveitemsfx: None,
animation: None,
takehitanimation: None,
endgfx: None,
tree: None,
hidemilestone: None,
nohighlight: None,
hideminimap: None,
noleftclickinteract: None,
interactdistance: None,
actionname: String::new(),
desc: String::new(),
comment: String::new(),
level: 0,
skill: SkillType::None,
tool: Tool::None,
min_health: 0,
max_health: 0,
harvesttime: 0,
hittime: 0,
respawntime: 0,
harvestsfx: String::new(),
endsfx: String::new(),
receiveitemsfx: String::new(),
animation: String::new(),
takehitanimation: String::new(),
endgfx: String::new(),
tree: false,
hidemilestone: false,
nohighlight: false,
hideminimap: false,
noleftclickinteract: false,
interactdistance: String::new(),
drops: Vec::new(),
}
}
/// Check if this is a tree
pub fn is_tree(&self) -> bool {
self.tree == Some(1)
self.tree
}
/// Check if this requires a tool
pub fn requires_tool(&self) -> bool {
self.tool.is_some()
!matches!(self.tool, Tool::None)
}
/// Get the skill associated with this harvestable
pub fn get_skill(&self) -> Option<&str> {
self.skill.as_deref()
pub fn get_skill(&self) -> SkillType {
self.skill
}
/// Get all item IDs that can drop from this harvestable

99
cursebreaker-parser/src/xml_parser.rs
View File

@@ -9,6 +9,7 @@ use crate::types::{
PlayerHouse,
Trait, TraitTrainer,
Shop, ShopItem,
SkillType, Tool,
};
use quick_xml::events::Event;
use quick_xml::reader::Reader;
@@ -180,6 +181,20 @@ fn parse_stat(attrs: &HashMap<String, String>) -> ItemStat {
}
}
/// Parse health range string like "3-5" or "3" into (min, max)
fn parse_health_range(health_str: &str) -> (i32, i32) {
if let Some(dash_pos) = health_str.find('-') {
let min_str = &health_str[..dash_pos];
let max_str = &health_str[dash_pos + 1..];
let min = min_str.trim().parse().unwrap_or(0);
let max = max_str.trim().parse().unwrap_or(0);
(min, max)
} else {
let val = health_str.trim().parse().unwrap_or(0);
(val, val)
}
}
// ============================================================================
// NPC Parser
// ============================================================================
@@ -549,45 +564,49 @@ pub fn parse_harvestables_xml<P: AsRef<Path>>(path: P) -> Result<Vec<Harvestable
let mut harvestable = Harvestable::new(typeid, name);
// Parse optional attributes
if let Some(v) = attrs.get("actionname") { harvestable.actionname = Some(v.clone()); }
if let Some(v) = attrs.get("desc") { harvestable.desc = Some(v.clone()); }
if let Some(v) = attrs.get("comment") { harvestable.comment = Some(v.clone()); }
if let Some(v) = attrs.get("level") { harvestable.level = v.parse().ok(); }
if let Some(v) = attrs.get("skill") { harvestable.skill = Some(v.clone()); }
if let Some(v) = attrs.get("tool") { harvestable.tool = Some(v.clone()); }
if let Some(v) = attrs.get("health") { harvestable.health = Some(v.clone()); }
if let Some(v) = attrs.get("harvesttime") { harvestable.harvesttime = v.parse().ok(); }
if let Some(v) = attrs.get("hittime") { harvestable.hittime = v.parse().ok(); }
if let Some(v) = attrs.get("respawntime") { harvestable.respawntime = v.parse().ok(); }
// Parse optional attributes with defaults
if let Some(v) = attrs.get("actionname") { harvestable.actionname = v.clone(); }
if let Some(v) = attrs.get("desc") { harvestable.desc = v.clone(); }
if let Some(v) = attrs.get("comment") { harvestable.comment = v.clone(); }
if let Some(v) = attrs.get("level") { harvestable.level = v.parse().unwrap_or(0); }
if let Some(v) = attrs.get("skill") { harvestable.skill = v.parse().unwrap_or(SkillType::None); }
if let Some(v) = attrs.get("tool") { harvestable.tool = v.parse().unwrap_or(Tool::None); }
if let Some(v) = attrs.get("health") {
let (min, max) = parse_health_range(v);
harvestable.min_health = min;
harvestable.max_health = max;
}
if let Some(v) = attrs.get("harvesttime") { harvestable.harvesttime = v.parse().unwrap_or(0); }
if let Some(v) = attrs.get("hittime") { harvestable.hittime = v.parse().unwrap_or(0); }
if let Some(v) = attrs.get("respawntime") { harvestable.respawntime = v.parse().unwrap_or(0); }
// Audio (handle both cases: harvestSfx and harvestsfx)
if let Some(v) = attrs.get("harvestSfx").or_else(|| attrs.get("harvestsfx")) {
harvestable.harvestsfx = Some(v.clone());
if let Some(v) = attrs.get("harvestSfx").or_else(|| attrs.get("harvestsfx")) {
harvestable.harvestsfx = v.clone();
}
if let Some(v) = attrs.get("endSfx").or_else(|| attrs.get("endsfx")) {
harvestable.endsfx = Some(v.clone());
if let Some(v) = attrs.get("endSfx").or_else(|| attrs.get("endsfx")) {
harvestable.endsfx = v.clone();
}
if let Some(v) = attrs.get("receiveItemSfx").or_else(|| attrs.get("receiveitemsfx")) {
harvestable.receiveitemsfx = Some(v.clone());
if let Some(v) = attrs.get("receiveItemSfx").or_else(|| attrs.get("receiveitemsfx")) {
harvestable.receiveitemsfx = v.clone();
}
if let Some(v) = attrs.get("animation") { harvestable.animation = Some(v.clone()); }
if let Some(v) = attrs.get("takehitanimation") { harvestable.takehitanimation = Some(v.clone()); }
if let Some(v) = attrs.get("endgfx") { harvestable.endgfx = Some(v.clone()); }
if let Some(v) = attrs.get("tree") { harvestable.tree = v.parse().ok(); }
if let Some(v) = attrs.get("hidemilestone") { harvestable.hidemilestone = v.parse().ok(); }
if let Some(v) = attrs.get("nohighlight") { harvestable.nohighlight = v.parse().ok(); }
if let Some(v) = attrs.get("animation") { harvestable.animation = v.clone(); }
if let Some(v) = attrs.get("takehitanimation") { harvestable.takehitanimation = v.clone(); }
if let Some(v) = attrs.get("endgfx") { harvestable.endgfx = v.clone(); }
if let Some(v) = attrs.get("tree") { harvestable.tree = v.parse().unwrap_or(0) == 1; }
if let Some(v) = attrs.get("hidemilestone") { harvestable.hidemilestone = v.parse().unwrap_or(0) == 1; }
if let Some(v) = attrs.get("nohighlight") { harvestable.nohighlight = v.parse().unwrap_or(0) == 1; }
// Handle both cases: hideMinimap and hideminimap
if let Some(v) = attrs.get("hideMinimap").or_else(|| attrs.get("hideminimap")) {
harvestable.hideminimap = v.parse().ok();
if let Some(v) = attrs.get("hideMinimap").or_else(|| attrs.get("hideminimap")) {
harvestable.hideminimap = v.parse().unwrap_or(0) == 1;
}
if let Some(v) = attrs.get("noLeftClickInteract").or_else(|| attrs.get("noleftclickinteract")) {
harvestable.noleftclickinteract = v.parse().ok();
if let Some(v) = attrs.get("noLeftClickInteract").or_else(|| attrs.get("noleftclickinteract")) {
harvestable.noleftclickinteract = v.parse().unwrap_or(0) == 1;
}
if let Some(v) = attrs.get("interactDistance").or_else(|| attrs.get("interactdistance")) {
harvestable.interactdistance = Some(v.clone());
if let Some(v) = attrs.get("interactDistance").or_else(|| attrs.get("interactdistance")) {
harvestable.interactdistance = v.clone();
}
current_harvestable = Some(harvestable);
@@ -599,14 +618,14 @@ pub fn parse_harvestables_xml<P: AsRef<Path>>(path: P) -> Result<Vec<Harvestable
if let Ok(id) = id_str.parse::<i32>() {
let drop = HarvestableDrop {
id,
minamount: attrs.get("minamount").and_then(|v| v.parse().ok()),
maxamount: attrs.get("maxamount").and_then(|v| v.parse().ok()),
droprate: attrs.get("droprate").and_then(|v| v.parse().ok()),
droprateboost: attrs.get("droprateboost").and_then(|v| v.parse().ok()),
amountboost: attrs.get("amountboost").and_then(|v| v.parse().ok()),
checks: attrs.get("checks").cloned(),
comment: attrs.get("comment").cloned(),
dontconsumehealth: attrs.get("dontconsumehealth").and_then(|v| v.parse().ok()),
minamount: attrs.get("minamount").and_then(|v| v.parse().ok()).unwrap_or(0),
maxamount: attrs.get("maxamount").and_then(|v| v.parse().ok()).unwrap_or(0),
droprate: attrs.get("droprate").and_then(|v| v.parse().ok()).unwrap_or(0),
droprateboost: attrs.get("droprateboost").and_then(|v| v.parse().ok()).unwrap_or(0),
amountboost: attrs.get("amountboost").and_then(|v| v.parse().ok()).unwrap_or(0),
checks: attrs.get("checks").cloned().unwrap_or_default(),
comment: attrs.get("comment").cloned().unwrap_or_default(),
dontconsumehealth: attrs.get("dontconsumehealth").and_then(|v| v.parse().ok()).unwrap_or(0) == 1,
};
harvestable.drops.push(drop);
}