pokemon-battle-engine/tools/data/README.md

Pokemon Data Downloader

A comprehensive tool for downloading Pokemon battle data from the PokeAPI using the pokebase library. This tool supports segmented downloading, data validation, and exports data in JSON format optimized for the C++ Pokemon battle simulator.

Features

• Segmented Downloads: Download specific ranges of Pokemon or moves for testing and incremental data collection
• Concurrent Processing: Multi-threaded downloads with configurable worker counts
• Data Validation: Built-in JSON schema validation for data integrity
• Rate Limiting: Respectful API usage with automatic rate limiting
• Progress Tracking: Beautiful progress bars and detailed logging
• Battle-Ready Data: Exports complete Pokemon stats, moves, types, and effectiveness data
• CLI Interface: Easy-to-use command-line interface with multiple commands

Installation

1. Install dependencies:

cd /testbed
pip install -r tools/requirements.txt

2. The tool is ready to use! No additional setup required.

Quick Start

Download a Small Set of Pokemon (Testing)

# Download first 5 Pokemon with their moves
python -m tools.data.pokemon_downloader download-pokemon --start 1 --end 5 --include-moves

# Download specific Pokemon (Pikachu)
python -m tools.data.pokemon_downloader download-pokemon --start 25 --end 25 --include-moves

Download Specific Moves

# Download first 10 moves
python -m tools.data.pokemon_downloader download-moves --move-ids "1,2,3,4,5,6,7,8,9,10"

Download Type Effectiveness Data

# Download complete type effectiveness chart
python -m tools.data.pokemon_downloader download-types

Download Complete Gen 1 Dataset

# Download all Gen 1 Pokemon (1-151) with moves and type data
python -m tools.data.pokemon_downloader download-complete --start 1 --end 151

CLI Commands

Global Options

• --output-dir: Directory to save downloaded data (default: data)
• --cache-dir: Directory for API response caching (default: .cache)
• --no-validation: Disable data validation before saving

Commands

download-pokemon

Download Pokemon data for a specific ID range.

python -m tools.data.pokemon_downloader download-pokemon [OPTIONS]

Options:

• --start: Starting Pokemon ID (default: 1)
• --end: Ending Pokemon ID (default: 10)
• --workers: Number of concurrent workers (default: 5)
• --include-moves: Also download moves for these Pokemon

download-moves

Download specific moves by ID.

python -m tools.data.pokemon_downloader download-moves --move-ids "1,2,3,4,5"

Options:

• --move-ids: Comma-separated list of move IDs
• --workers: Number of concurrent workers (default: 5)

download-types

Download type effectiveness data.

python -m tools.data.pokemon_downloader download-types

download-complete

Download complete dataset (Pokemon, moves, and type effectiveness).

python -m tools.data.pokemon_downloader download-complete [OPTIONS]

Options:

• --start: Starting Pokemon ID (default: 1)
• --end: Ending Pokemon ID (default: 151 for Gen 1)
• --workers: Number of concurrent workers (default: 5)

Data Structure

Pokemon Data Format

{
  "1": {
    "id": 1,
    "name": "bulbasaur",
    "types": ["grass", "poison"],
    "base_stats": {
      "hp": 45,
      "attack": 49,
      "defense": 49,
      "special_attack": 65,
      "special_defense": 65,
      "speed": 45
    },
    "abilities": ["overgrow", "chlorophyll"],
    "moves": [1, 2, 3, 4, ...],
    "weight": 69,
    "height": 7,
    "base_experience": 64
  }
}

Move Data Format

{
  "1": {
    "id": 1,
    "name": "pound",
    "type": "normal",
    "power": 40,
    "accuracy": 100,
    "pp": 35,
    "priority": 0,
    "damage_class": "physical",
    "effect_id": null,
    "effect_chance": null,
    "target": "selected-pokemon",
    "description": "Inflicts regular damage."
  }
}

Type Effectiveness Format

[
  {
    "attacking_type": "fire",
    "defending_type": "grass",
    "damage_factor": 2.0
  },
  {
    "attacking_type": "water",
    "defending_type": "fire", 
    "damage_factor": 2.0
  }
]

Examples

Basic Usage (Python API)

from pathlib import Path
from tools.data.pokemon_downloader import PokemonDownloader

# Initialize downloader
downloader = PokemonDownloader(
    output_dir=Path("my_pokemon_data"),
    validate_data=True
)

# Download first 10 Pokemon
pokemon_data = downloader.download_pokemon_batch(1, 10)
downloader.save_pokemon_data(pokemon_data, "starter_pokemon.json")

# Download some moves
move_ids = [1, 2, 3, 4, 5]  # Pound, Karate Chop, etc.
moves_data = downloader.download_moves_batch(move_ids)
downloader.save_moves_data(moves_data, "basic_moves.json")

# Download type effectiveness
effectiveness = downloader.download_type_effectiveness()
downloader.save_type_effectiveness(effectiveness, "types.json")

Testing Small Segments

# Test with just 3 Pokemon
python tools/data/test_downloader.py

Custom Data Validation

from tools.data.schemas import DataValidator

validator = DataValidator()

# Validate Pokemon data
errors = validator.validate_pokemon_collection(pokemon_data)
if errors:
    print("Validation errors:", errors)

Performance Considerations

• Rate Limiting: The tool implements 100ms delays between API calls to be respectful
• Concurrent Workers: Default of 5 workers balances speed with API courtesy
• Caching: API responses are cached to avoid redundant requests
• Memory Usage: Large datasets are processed in batches to manage memory

Recommended Usage Patterns

For Development/Testing

# Start small - download just a few Pokemon
python -m tools.data.pokemon_downloader download-pokemon --start 1 --end 3 --include-moves

# Test specific Pokemon you're interested in
python -m tools.data.pokemon_downloader download-pokemon --start 25 --end 25 --include-moves  # Pikachu

For Production Data

# Download by generations or batches
python -m tools.data.pokemon_downloader download-pokemon --start 1 --end 50 --include-moves
python -m tools.data.pokemon_downloader download-pokemon --start 51 --end 100 --include-moves
python -m tools.data.pokemon_downloader download-pokemon --start 101 --end 151 --include-moves

# Always download type effectiveness data
python -m tools.data.pokemon_downloader download-types

For Complete Gen 1 Dataset

# One command for everything (will take several minutes)
python -m tools.data.pokemon_downloader download-complete --start 1 --end 151

Data Validation

The tool includes comprehensive JSON schema validation:

• Pokemon Data: Validates stats, types, abilities, and structure
• Move Data: Validates power, accuracy, PP, and damage classes
• Type Effectiveness: Validates damage multipliers and type names
• Generation 1 Focus: Ensures only valid Gen 1 types and data

Validation errors are displayed during save operations but don't prevent saving (warnings only).

Integration with C++ Battle Simulator

The exported JSON files are designed to be easily consumed by the C++ battle simulator:

1. Pokemon Data: Direct mapping to Pokemon class properties
2. Move Data: Complete move information for battle calculations
3. Type Effectiveness: Lookup table for damage calculations
4. Consistent IDs: All data uses consistent Pokemon and move IDs

Troubleshooting

Common Issues

1. Network Errors: The tool retries failed requests automatically
2. Rate Limiting: Built-in delays prevent API rate limiting
3. Memory Usage: Large downloads are processed in batches
4. Validation Warnings: Usually safe to ignore, indicate minor data inconsistencies

Getting Help

• Run tests: python tools/data/test_downloader.py
• Check logs: The tool provides detailed logging for debugging
• Validate data: Use --no-validation flag if validation is too strict

Contributing

To extend the downloader:

1. Add new data structures to pokemon_downloader.py
2. Update validation schemas in schemas.py
3. Add tests to test_downloader.py
4. Update this documentation

License

This tool is part of the Pokemon Battle Simulator project and follows the same license terms.