GDL (Game Data Library)

Game-agnostic Rust microservice for syncing, storing, and serving game data. Communicates with the NestJS API via gRPC.

Overview

GDL separates game data (raids, dungeons, housing items, talent trees, character stats) from application data (users, guilds, memberships, events). The NestJS API remains the frontend-facing server; GDL handles all external game API communication and game data storage.

Architecture

• Language: Rust (tokio async runtime)
• Communication: gRPC (tonic server, NestJS client via @nestjs/microservices)
• Databases: Two PostgreSQL databases
  • sanctum_catalog — game reference data (journal, housing, reputation, talent trees, icons)
  • sanctum_characters — player enrichment data (stats, gear, encounters, reputations, professions, PvP, achievements)
• Adapter pattern: GameAdapter trait enables multi-game support. Blizzard/WoW is the first adapter.
• Patch versioning: All data tagged with patch_id. Historical data preserved across game patches.

Two-Database Strategy

	Catalog DB (sanctum_catalog)	Character DB (sanctum_characters)
Purpose	What exists in the game	What players have done
Growth	Bounded (finite game content)	Unbounded (scales with users)
Writes	Rare (patch syncs only)	Frequent (every enrichment)
Cacheable	Highly (same for all users)	Not (per-character)
Reconstructable	Yes (re-sync from API)	No (user-specific history)
Auth	Client credentials	User OAuth tokens

gRPC Services

Service	Purpose
GdlService	Game and patch management (register patches, set current)
CatalogService	Query game data by category, locale, patch
SyncService	Trigger syncs, subscribe to progress streams
EnrichmentService	Character enrichment with user OAuth tokens

Data Categories (WoW)

• wow.journal — Expansions, raids, dungeons, encounters
• wow.housing — Decor, fixtures, rooms
• wow.reputation — Faction hierarchy
• wow.talent_trees — Class/spec talent trees
• wow.icons — Class and spec icon cache

NestJS Integration

The GdlModule (apps/api/src/common/gdl/) provides four service classes:

• GdlCoreService — game/patch CRUD
• GdlCatalogService — game data queries
• GdlSyncService — sync triggers + progress streaming
• GdlEnrichmentService — character enrichment delegation

Existing WebSocket gateways relay sync progress from GDL gRPC streams to the frontend.

JSON Denormalization

GDL flattens many JSON blobs from the current schema into proper relational columns:

• CharacterStatistics.stats → ~40 individual columns on wow_character_stats
• CharacterItem.weaponStats → weapon_min_damage, weapon_max_damage, weapon_speed, weapon_dps
• CharacterItem.stats/enchantments/gems → normalized child tables
• CharacterPvpSummary.brackets/pvpMapStats → wow_character_pvp_brackets, wow_character_pvp_map_stats
• CharacterAchievement.achievements → one row per achievement in wow_character_achievements

JSON is retained only for genuinely dynamic structures: CharacterSpecialization.specializations and talent tree node definitions.

Development

# Start GDL with databases
docker compose -f docker-compose.yml -f docker-compose.dev.yml up gdl gdl-postgres

# Build locally
cd services/gdl && cargo build

# Run migrations manually
sqlx migrate run --source migrations/catalog
sqlx migrate run --source migrations/characters

Project Structure

services/gdl/
  proto/           — gRPC protocol buffer definitions
  migrations/
    catalog/       — sqlx migrations for sanctum_catalog
    characters/    — sqlx migrations for sanctum_characters
  src/
    core/          — adapter trait, registry, sync engine, progress
    db/            — database pool management
    grpc/          — tonic gRPC service implementations
    adapters/
      blizzard/    — WoW/Blizzard API adapter
        game_data/ — catalog sync (journal, housing, reputation, etc.)
        profile/   — character enrichment
        db/        — SQL queries for catalog and character DBs

Adding a New Game

1. Create src/adapters/{game}/ with adapter, client, and sync modules
2. Implement the GameAdapter trait
3. Add {game}_* migrations to both databases as needed
4. Register the adapter in main.rs
5. No changes needed to core engine, gRPC layer, or NestJS