Memory leak: new TCGdex SDK instance created per card in fetch_card.py #5

New Issue

llabeyrie · 2026-03-19T17:29:52Z

llabeyrie commented

2026-03-19 17:29:52 +00:00

Description

In fetch_card.py line 81, a new TCGdex SDK instance is created for every single card:

def process_card(card_id: str, set_dir: Path) -> str | None:
    sdk = TCGdex(Language.EN)   # new instance per card!
    card = sdk.card.getSync(card_id)
    ...

This function runs inside a ThreadPoolExecutor with 8 workers (line 122), potentially processing thousands of cards.

Problems

Each instance may allocate memory, open HTTP sessions, or load internal caches
With thousands of cards, this creates thousands of unnecessary SDK instances
Potential thread-safety issues if the SDK shares mutable global state

Fix

Pass a shared SDK instance or use a thread-local pattern:

import threading
_thread_local = threading.local()

def _get_sdk():
    if not hasattr(_thread_local, "sdk"):
        _thread_local.sdk = TCGdex(Language.EN)
    return _thread_local.sdk

## Description In `fetch_card.py` line 81, a **new `TCGdex` SDK instance is created for every single card**: ```python def process_card(card_id: str, set_dir: Path) -> str | None: sdk = TCGdex(Language.EN) # new instance per card! card = sdk.card.getSync(card_id) ... ``` This function runs inside a `ThreadPoolExecutor` with 8 workers (line 122), potentially processing thousands of cards. ### Problems - Each instance may allocate memory, open HTTP sessions, or load internal caches - With thousands of cards, this creates thousands of unnecessary SDK instances - Potential thread-safety issues if the SDK shares mutable global state ### Fix Pass a shared SDK instance or use a thread-local pattern: ```python import threading _thread_local = threading.local() def _get_sdk(): if not hasattr(_thread_local, "sdk"): _thread_local.sdk = TCGdex(Language.EN) return _thread_local.sdk ```

llabeyrie added the performance priority: high labels 2026-03-19 17:31:41 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: llabeyrie/Juicepyter#5