# Troubleshooting Guide

Common issues and solutions for the Slidev system in production.

**Production URL:** https://slide-system-production-5813.up.railway.app

---

## Table of Contents

1. [Quick Diagnosis](#quick-diagnosis)
2. [Generation Issues](#generation-issues)
3. [Control Command Issues](#control-command-issues)
4. [Performance Issues](#performance-issues)
5. [Database Issues](#database-issues)
6. [Railway Deployment Issues](#railway-deployment-issues)
7. [Supabase Realtime Issues](#supabase-realtime-issues)

---

## Quick Diagnosis

### Is the API Running?

```bash
# Check if API responds
curl -I https://slide-system-production-5813.up.railway.app/

# Expected: HTTP 200 OK
# If fails: API is down, check Railway status
```

### Check Recent Errors

```bash
# View last 50 log lines
railway logs --tail 50 | grep ERROR

# If errors found: Investigate specific error messages
```

### Check Railway Status

```bash
# Check if service is running
railway status

# Expected: "Service is running"
# If not: Restart service with `railway service restart`
```

---

## Generation Issues

### Issue: "Session not found" (HTTP 404)

**Symptom:**
```json
{
  "error": "Session not found",
  "details": "No session exists with ID: xxx"
}
```

**Root Cause:** Session ID doesn't exist in Supabase `conversation_sessions` table.

**Solution:**

1. **Create session first:**
   ```bash
   curl -X POST https://slide-system-production-5813.up.railway.app/demo-sessions \
     -H "Content-Type: application/json" \
     -d '{"session_type":"test","environment":"prod"}' \
     | jq -r '.session_id'
   ```

2. **Use returned session_id** in generation request

3. **Verify session exists:**
   ```bash
   # Query Supabase
   curl "https://your-project.supabase.co/rest/v1/conversation_sessions?id=eq.SESSION_ID" \
     -H "apikey: YOUR_SUPABASE_SERVICE_KEY" \
     -H "Authorization: Bearer YOUR_SUPABASE_SERVICE_KEY"
   ```

### Issue: "Missing required fields" (HTTP 400)

**Symptom:**
```json
{
  "error": "Validation failed",
  "details": "Missing required fields: sessionId, templateId, and data"
}
```

**Root Cause:** Request payload is incomplete or malformed.

**Solution:**

Ensure request includes all required fields:

```json
{
  "sessionId": "uuid-string",  // Required
  "templateId": "starter",      // Required
  "data": {                     // Required (can be empty object)
    "company": "Example Corp"
  }
}
```

### Issue: Generation Takes Too Long (> 90s)

**Symptom:** Request times out or takes longer than expected.

**Root Cause:** Slow deck build or database operations.

**Investigation:**

1. **Check Railway metrics:**
   - CPU usage > 80%? → Consider upgrading plan
   - Memory usage > 80%? → Optimize code or upgrade

2. **Check logs for slow operations:**
   ```bash
   railway logs | grep "duration"
   ```

3. **Test Supabase performance:**
   ```bash
   time curl "https://your-project.supabase.co/rest/v1/conversation_sessions?limit=1" \
     -H "apikey: YOUR_SUPABASE_SERVICE_KEY"
   # Should be < 500ms
   ```

**Solution:**

- If CPU bound: Optimize build process or upgrade Railway plan
- If database slow: Optimize queries or upgrade Supabase plan
- If network slow: Check Railway-Supabase connectivity

### Issue: "SUPABASE_URL or SUPABASE_SERVICE_KEY not configured" (HTTP 500)

**Symptom:**
```json
{
  "error": "Failed to generate deck",
  "details": "SUPABASE_URL or SUPABASE_SERVICE_KEY not configured"
}
```

**Root Cause:** Environment variables missing in Railway.

**Solution:**

```bash
# Set environment variables
railway variables set SUPABASE_URL=https://your-project.supabase.co
railway variables set SUPABASE_SERVICE_KEY=your-service-key-here

# Redeploy
railway up

# Verify
railway variables | grep SUPABASE
```

---

## Control Command Issues

### Issue: "Invalid or missing API key" (HTTP 401)

**Symptom:**
```json
{
  "error": "Invalid or missing API key"
}
```

**Root Cause:** `x-api-key` header missing or doesn't match Railway environment variable.

**Solution:**

1. **Check API key is set in Railway:**
   ```bash
   railway variables | grep SLIDEV_API_KEY
   ```

2. **If not set:**
   ```bash
   railway variables set SLIDEV_API_KEY=your-api-key-here
   railway up
   ```

3. **Verify request includes header:**
   ```bash
   curl -X POST https://slide-system-production-5813.up.railway.app/api/slidev/control \
     -H "Content-Type: application/json" \
     -H "x-api-key: YOUR_API_KEY" \
     -d '{"deckId":"test","command":"next"}'
   ```

### Issue: "Deck not found" (HTTP 404)

**Symptom:**
```json
{
  "error": "Deck not found",
  "details": "No deck exists with ID: xxx"
}
```

**Root Cause:** Deck ID doesn't exist in Supabase `slidev_decks` table.

**Solution:**

1. **Verify deck was generated:**
   ```bash
   curl "https://your-project.supabase.co/rest/v1/slidev_decks?deck_id=eq.DECK_ID" \
     -H "apikey: YOUR_SUPABASE_SERVICE_KEY"
   ```

2. **If not found:** Generate deck first with `/api/slidev/generate`

3. **Check deck_id format:** Should be `xxxx-xxxx-xxxx` (12 chars, lowercase alphanumeric)

### Issue: Commands Not Executing (Realtime Not Working)

**Symptom:** Control API returns success, but slides don't advance.

**Root Cause:** Supabase Realtime not configured or client not subscribed.

**Investigation:**

1. **Check Realtime is enabled on table:**
   - Go to Supabase Dashboard → Database → Replication
   - Verify `slide_control_commands` table is enabled for Realtime

2. **Check browser console** (if using web client):
   ```javascript
   // Should see connection message
   [Slidev] Realtime subscription status: SUBSCRIBED
   ```

3. **Verify command was inserted:**
   ```bash
   curl "https://your-project.supabase.co/rest/v1/slide_control_commands?deck_id=eq.DECK_ID&order=timestamp.desc&limit=1" \
     -H "apikey: YOUR_SUPABASE_SERVICE_KEY"
   ```

**Solution:**

- Enable Realtime on `slide_control_commands` table
- Verify client is subscribed to correct channel: `deck:{deckId}`
- Check Supabase Realtime logs for connection issues

---

## Performance Issues

### Issue: High Response Time (> 5s)

**Symptom:** API responses consistently slow.

**Investigation:**

```bash
# Measure API response time
time curl https://slide-system-production-5813.up.railway.app/demo-sessions \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"session_type":"test","environment":"prod"}'

# Check Railway metrics
railway metrics

# Check logs for slow operations
railway logs | grep "slow" | grep "duration"
```

**Solution:**

1. **If Railway CPU/memory high:**
   - Upgrade Railway plan
   - Optimize code (caching, async operations)

2. **If database slow:**
   - Add database indexes
   - Optimize queries
   - Upgrade Supabase plan

3. **If network issues:**
   - Check Railway-Supabase network latency
   - Consider co-locating services in same region

### Issue: High Error Rate

**Symptom:** Many 500 errors in logs.

**Investigation:**

```bash
# Count errors in last 100 log lines
railway logs --tail 100 | grep -c "ERROR"

# View error details
railway logs | grep ERROR | tail -20
```

**Common Causes:**

- Database connection issues
- Missing environment variables
- Bugs in code (check stack traces)
- External service (Supabase) downtime

**Solution:**

1. **Check external services:**
   - Supabase status: https://status.supabase.com
   - GitHub status: https://www.githubstatus.com

2. **Review recent deployments:**
   ```bash
   railway deployments
   # If errors started after recent deploy, rollback
   railway rollback --previous
   ```

3. **Check environment variables:**
   ```bash
   railway variables
   # Verify all required variables are set
   ```

### Issue: Memory Leaks / OOM Errors

**Symptom:** Service restarts frequently, memory usage climbing.

**Investigation:**

```bash
# Check Railway metrics for memory usage
railway metrics

# Check logs for OOM errors
railway logs | grep "out of memory"
```

**Solution:**

1. **Short-term:** Restart service
   ```bash
   railway service restart
   ```

2. **Medium-term:** Upgrade Railway plan for more memory

3. **Long-term:** Profile code for memory leaks
   - Review for unclosed connections
   - Check for large objects in memory
   - Implement proper cleanup in error handlers

---

## Database Issues

### Issue: "Failed to verify session" (HTTP 500)

**Symptom:** Database queries failing.

**Root Cause:** Supabase connection issues or table doesn't exist.

**Investigation:**

1. **Test Supabase connectivity:**
   ```bash
   curl "https://your-project.supabase.co/rest/v1/" \
     -H "apikey: YOUR_SUPABASE_SERVICE_KEY"
   # Expected: {"message":"success"}
   ```

2. **Verify tables exist:**
   - Go to Supabase Dashboard → Table Editor
   - Check for: `conversation_sessions`, `slidev_decks`, `slide_control_commands`

3. **Check SERVICE_KEY is correct:**
   ```bash
   railway variables | grep SUPABASE_SERVICE_KEY
   ```

**Solution:**

- If tables missing: Run database migrations
- If key wrong: Update Railway environment variable
- If Supabase down: Check https://status.supabase.com

### Issue: Slow Database Queries

**Symptom:** API responses slow, logs show long query times.

**Investigation:**

1. **Check query performance in Supabase:**
   - Go to Supabase Dashboard → Database → Query Performance
   - Identify slow queries

2. **Check for missing indexes:**
   ```sql
   -- In Supabase SQL Editor
   SELECT * FROM pg_indexes WHERE tablename IN ('conversation_sessions', 'slidev_decks', 'slide_control_commands');
   ```

**Solution:**

Add indexes for frequently queried columns:

```sql
-- Index on session lookup
CREATE INDEX IF NOT EXISTS idx_conversation_sessions_id ON conversation_sessions(id);

-- Index on deck lookup
CREATE INDEX IF NOT EXISTS idx_slidev_decks_deck_id ON slidev_decks(deck_id);
CREATE INDEX IF NOT EXISTS idx_slidev_decks_session_id ON slidev_decks(session_id);

-- Index on control commands
CREATE INDEX IF NOT EXISTS idx_slide_control_commands_deck_id ON slide_control_commands(deck_id);
CREATE INDEX IF NOT EXISTS idx_slide_control_commands_timestamp ON slide_control_commands(timestamp DESC);
```

---

## Railway Deployment Issues

### Issue: Deployment Failed

**Symptom:** Railway shows "Failed" deployment status.

**Investigation:**

```bash
# View deployment logs
railway logs

# Common issues:
# - TypeScript compilation errors
# - Missing dependencies
# - Environment variable issues
```

**Solution:**

1. **Fix compilation errors locally:**
   ```bash
   cd apps/slide-system
   pnpm install
   pnpx tsc --noEmit  # Check for TypeScript errors
   ```

2. **Fix and redeploy:**
   ```bash
   git add .
   git commit -m "fix: deployment issues"
   git push origin main
   ```

3. **If still failing:** Deploy manually via CLI
   ```bash
   railway up
   ```

### Issue: Service Not Responding After Deployment

**Symptom:** Deployment succeeds but API returns errors or timeouts.

**Investigation:**

```bash
# Check service status
railway status

# View recent logs
railway logs --tail 100

# Test health endpoint
curl -I https://slide-system-production-5813.up.railway.app/
```

**Solution:**

1. **If service crashed:** Check logs for errors
   ```bash
   railway logs | grep "ERROR\|FATAL\|crashed"
   ```

2. **If environment issues:** Verify variables
   ```bash
   railway variables
   ```

3. **If persistent:** Rollback and investigate
   ```bash
   railway rollback --previous
   ```

### Issue: Old Code Still Running After Deploy

**Symptom:** Changes not reflected in production.

**Investigation:**

```bash
# Check latest deployment
railway deployments | head -5

# Verify deployment succeeded
railway status
```

**Solution:**

1. **Clear Railway cache and redeploy:**
   ```bash
   railway up --force
   ```

2. **Or trigger new deployment:**
   ```bash
   git commit --allow-empty -m "chore: trigger redeploy"
   git push origin main
   ```

---

## Supabase Realtime Issues

### Issue: Realtime Subscriptions Not Working

**Symptom:** Control commands don't reach presentation.

**Investigation:**

1. **Check Realtime is enabled:**
   - Supabase Dashboard → Database → Replication
   - Verify `slide_control_commands` table is enabled

2. **Check browser console for errors:**
   ```
   Failed to subscribe to channel
   WebSocket connection failed
   ```

3. **Test Realtime directly:**
   ```javascript
   const { createClient } = supabase;
   const client = createClient(SUPABASE_URL, SUPABASE_ANON_KEY);

   client.channel('test')
     .on('broadcast', { event: 'test' }, (payload) => {
       console.log('Received:', payload);
     })
     .subscribe((status) => {
       console.log('Status:', status);
     });
   ```

**Solution:**

- Enable Realtime on table via Supabase Dashboard
- Check Supabase plan supports Realtime
- Verify WebSocket connections not blocked by firewall

### Issue: Delayed Command Execution

**Symptom:** Commands execute but with 2-3 second delay.

**Root Cause:** Normal Realtime latency or network issues.

**Investigation:**

```bash
# Test command latency
time curl -X POST https://slide-system-production-5813.up.railway.app/api/slidev/control \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{"deckId":"test","command":"next"}'
```

**Expected:** < 500ms for API response, < 1s for client execution

**Solution:**

- If > 1s consistently: Check Supabase Realtime performance
- If network latency high: Co-locate services in same region
- If acceptable latency: This is normal for Realtime (100-500ms is typical)

---

## Emergency Procedures

### Complete Service Outage

If the service is completely down:

```bash
# 1. Check Railway status
railway status

# 2. View logs for critical errors
railway logs --tail 50 | grep "ERROR\|FATAL"

# 3. Restart service
railway service restart

# 4. If still down, rollback
railway rollback --previous

# 5. Test
curl -I https://slide-system-production-5813.up.railway.app/
```

### Data Corruption

If database data appears corrupted:

```sql
-- Check recent slidev_decks for anomalies
SELECT * FROM slidev_decks ORDER BY created_at DESC LIMIT 10;

-- Check for duplicate deck_ids
SELECT deck_id, COUNT(*) FROM slidev_decks GROUP BY deck_id HAVING COUNT(*) > 1;

-- If corruption found, investigate and possibly delete bad records
-- DELETE FROM slidev_decks WHERE id = 'bad-record-id';
```

---

## Getting Help

**Internal Documentation:**
- API Documentation: `docs/api/slidev-api.md`
- Architecture: `docs/architecture/slidev-system.md`
- Deployment Guide: `docs/deployment/railway-deployment.md`
- Monitoring Guide: `docs/operations/monitoring.md`

**External Resources:**
- Railway Status: https://status.railway.app
- Supabase Status: https://status.supabase.com
- GitHub Status: https://www.githubstatus.com

**Support Channels:**
- Railway Support: https://railway.app/help
- Supabase Support: https://supabase.com/support

---

**Last Updated:** 2025-10-09