---
description: Python Performance
alwaysApply: false
---

# Python Performance

Profile before optimizing. Reach for the right tool when raw speed matters.

## Profile First

```bash
py-spy record -o profile.svg -- python myapp.py  # Production profiling
python -m cProfile -s cumulative script.py        # Function-level
# line_profiler for line-by-line, memory_profiler for memory
```

## Data Structures

- `dict` — O(1) lookup, insertion, deletion
- `set` — O(1) membership testing
- `deque` — O(1) append/pop from both ends
- `heapq` — O(log n) push/pop for priority queues
- `bisect` — O(log n) search in sorted lists

## Common Bottlenecks

```python
# String concatenation — use join, not +=
result = "".join(chunks)  # O(n), not O(n²)

# Generator expressions when you don't need the full list
total = sum(x * x for x in range(1_000_000))  # No intermediate list

# Local variable lookup is faster in hot loops
append = result.append
for item in items:
    append(transform(item))
```

## Concurrency Models

```python
# I/O-bound: threading or asyncio
with ThreadPoolExecutor(max_workers=10) as executor:
    results = list(executor.map(fetch_url, urls))

# CPU-bound: multiprocessing (bypasses GIL)
with ProcessPoolExecutor() as executor:
    results = list(executor.map(cpu_work, data_chunks))
```

## Caching

```python
from functools import lru_cache, cache

@lru_cache(maxsize=256)
def fibonacci(n: int) -> int:
    if n < 2: return n
    return fibonacci(n - 1) + fibonacci(n - 2)

@cache  # 3.9+, unlimited
def load_schema(name: str) -> Schema:
    return Schema.from_file(f"schemas/{name}.json")
```

## Key Rules

- `slots=True` on dataclasses for memory efficiency
- `"".join()` over `+=` for string building
- NumPy/Polars for vectorized numeric operations
- Never optimize without profiling first
- A 10% speedup that makes code unmaintainable is a net loss