# Workflow Schedule Module

A comprehensive NestJS module for scheduling and executing workflows using Bull queues and Redis.

## Features

- ✅ Schedule workflows with cron expressions
- ✅ Timezone support (IANA timezone format)
- ✅ Pause/resume schedules
- ✅ Manual trigger execution
- ✅ Comprehensive execution logging
- ✅ Retry with exponential backoff
- ✅ Batch processing (100 records per batch)
- ✅ Multiple action types:
  - Send Email
  - Update Records
  - Create Tasks
  - Send Notifications

## Installation

### 1. Install Required Dependencies

```bash
npm install @nestjs/bull bull redis cron-parser
npm install --save-dev @types/bull
```

Note: `cron-parser` includes its own TypeScript definitions, so no separate `@types` package is needed.

### 2. Configure Redis

Add Redis configuration to your `.env` file:

```env
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
REDIS_DB=0
```

### 3. Run Migrations

The module includes three migrations:

1. **AddWorkflowTypeColumn** - Adds `workflow_type` to `cr_wf_master` table
2. **CreateScheduledWorkflowsTable** - Creates `cr_wf_scheduled_workflow` table
3. **CreateWorkflowExecutionLogsTable** - Creates `cr_wf_execution_log` table

To run migrations:

```bash
# Using TypeORM CLI
npx typeorm migration:run -d src/config/database.config.ts

# Or using npm script (if configured)
npm run migration:run
```

### 4. Start Redis Server

Make sure Redis is running:

```bash
# If using Docker
docker run -d -p 6379:6379 redis

# If installed locally
redis-server
```

## Module Structure

```
src/module/workflow-schedule/
├── constants/
│   └── schedule.constants.ts          # Constants and enums
├── interfaces/
│   └── schedule-job-data.interface.ts # TypeScript interfaces
├── entities/
│   ├── scheduled-workflow.entity.ts   # ScheduledWorkflow entity
│   └── workflow-execution-log.entity.ts # WorkflowExecutionLog entity
├── dto/
│   ├── create-schedule.dto.ts         # Create schedule DTO
│   ├── update-schedule.dto.ts         # Update schedule DTO
│   └── get-execution-logs.dto.ts      # Query execution logs DTO
├── processors/
│   └── schedule.processor.ts          # Bull job processor
├── service/
│   └── workflow-schedule.service.ts   # Business logic (9 methods)
├── controller/
│   └── workflow-schedule.controller.ts # REST API (9 endpoints)
└── workflow-schedule.module.ts        # Module definition
```

## API Endpoints

### 1. Create Schedule
```http
POST /workflow-schedule/create
Authorization: Bearer <token>

{
  "workflow_id": 1,
  "workflow_name": "Daily Lead Assignment",
  "name": "Assign leads every morning",
  "description": "Automatically assign new leads to sales team",
  "cron_expression": "0 9 * * 1-5",
  "timezone": "Asia/Kolkata",
  "actions": [
    {
      "actionType": "UPDATE_RECORDS",
      "targetEntityType": "LEAD",
      "actionConfig": {
        "updateFields": {
          "status": "ASSIGNED"
        }
      }
    }
  ]
}
```

### 2. Update Schedule
```http
PUT /workflow-schedule/update

{
  "id": 1,
  "cron_expression": "0 10 * * 1-5"
}
```

### 3. Get Schedule by ID
```http
GET /workflow-schedule/:id
```

### 4. Get All Schedules
```http
GET /workflow-schedule/list?page=1&size=10&schedule_status=ACTIVE
```

### 5. Pause Schedule
```http
POST /workflow-schedule/:id/pause
```

### 6. Resume Schedule
```http
POST /workflow-schedule/:id/resume
```

### 7. Delete Schedule
```http
DELETE /workflow-schedule/:id
```

### 8. Manual Trigger
```http
POST /workflow-schedule/:id/trigger

{
  "metadata": {
    "triggered_reason": "Testing"
  }
}
```

### 9. Get Execution Logs
```http
POST /workflow-schedule/execution-logs

{
  "schedule_id": 1,
  "execution_status": "COMPLETED",
  "page": 1,
  "size": 10
}
```

### 10. Get Execution Statistics
```http
GET /workflow-schedule/:id/stats
```

## Service Methods

The `WorkflowScheduleService` provides 9 core methods:

1. **createSchedule** - Create a new scheduled workflow
2. **updateSchedule** - Update an existing schedule
3. **getScheduleById** - Get schedule by ID
4. **getAllSchedules** - Get all schedules with pagination
5. **pauseSchedule** - Pause a running schedule
6. **resumeSchedule** - Resume a paused schedule
7. **deleteSchedule** - Delete (soft delete) a schedule
8. **triggerManualExecution** - Manually trigger execution
9. **getExecutionLogs** - Query execution logs with filters

## Cron Expression Examples

```javascript
// Every day at 9 AM
"0 9 * * *"

// Every weekday at 9 AM
"0 9 * * 1-5"

// Every hour
"0 * * * *"

// Every 30 minutes
"*/30 * * * *"

// First day of every month at midnight
"0 0 1 * *"

// Every Monday at 8 AM
"0 8 * * 1"
```

## Action Types

### 1. Send Email
```json
{
  "actionType": "SEND_EMAIL",
  "targetEntityType": "USR",
  "actionConfig": {
    "subject": "Weekly Report",
    "template": "weekly-report-template",
    "from": "noreply@example.com"
  },
  "filterCriteria": {
    "status": "ACTIVE"
  }
}
```

### 2. Update Records
```json
{
  "actionType": "UPDATE_RECORDS",
  "targetEntityType": "LEAD",
  "actionConfig": {
    "updateFields": {
      "status": "FOLLOW_UP",
      "last_contacted": "NOW()"
    }
  },
  "filterCriteria": {
    "status": "NEW"
  }
}
```

### 3. Create Task
```json
{
  "actionType": "CREATE_TASK",
  "targetEntityType": "LEAD",
  "actionConfig": {
    "taskName": "Follow up with lead",
    "taskDescription": "Scheduled follow-up task"
  },
  "filterCriteria": {
    "status": "QUALIFIED"
  }
}
```

### 4. Send Notification
```json
{
  "actionType": "SEND_NOTIFICATION",
  "targetEntityType": "USR",
  "actionConfig": {
    "eventType": "WORKFLOW_SCHEDULED",
    "message": "Your weekly report is ready"
  }
}
```

## Retry Configuration

```json
{
  "retry_config": {
    "maxRetries": 3,
    "retryDelay": 5000,
    "backoffMultiplier": 2
  }
}
```

This will retry failed jobs:
- 1st retry: after 5 seconds
- 2nd retry: after 10 seconds (5 * 2)
- 3rd retry: after 20 seconds (10 * 2)

## Monitoring

### Check Queue Status

Bull provides a web UI for monitoring queues:

```bash
npm install bull-board
```

### View Execution Logs

Use the execution logs API to monitor:
- Success rate
- Failed executions
- Average execution time
- Records processed

```http
GET /workflow-schedule/:id/stats
```

## Database Schema

### cr_wf_scheduled_workflow
- Stores schedule configuration
- Includes cron expression, timezone, retry config
- Tracks execution count and next execution time

### cr_wf_execution_log
- Logs every execution
- Tracks success/failure status
- Records processing metrics
- Stores error details

## Environment Variables

```env
# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
REDIS_DB=0

# Database Configuration
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASS=password
DB_NAME=core
```

## Testing

### Manual Testing

1. Create a schedule with a simple cron (e.g., every minute: `* * * * *`)
2. Check execution logs after 1-2 minutes
3. Test pause/resume functionality
4. Test manual trigger
5. Verify retry mechanism by creating failing actions

### Unit Testing

```typescript
import { Test } from '@nestjs/testing';
import { WorkflowScheduleService } from './workflow-schedule.service';

describe('WorkflowScheduleService', () => {
  let service: WorkflowScheduleService;

  beforeEach(async () => {
    const module = await Test.createTestingModule({
      providers: [WorkflowScheduleService],
    }).compile();

    service = module.get<WorkflowScheduleService>(WorkflowScheduleService);
  });

  it('should create a schedule', async () => {
    // Test implementation
  });
});
```

## Troubleshooting

### Redis Connection Issues
- Verify Redis is running: `redis-cli ping`
- Check Redis connection settings in `.env`

### Jobs Not Processing
- Check Bull queue status
- Verify cron expression is valid
- Check schedule is enabled and active

### Migration Issues
- Ensure database user has CREATE/ALTER permissions
- Run migrations in order (1, 2, 3)

## Performance Considerations

- **Batch Size**: Default is 100 records. Adjust based on your use case.
- **Retry Strategy**: Balance between retry attempts and system load.
- **Redis Memory**: Monitor Redis memory usage for large queues.
- **Database Indexes**: Migrations include optimized indexes.

## Security

- All endpoints are protected with `JwtAuthGuard`
- Organization-level data isolation
- Soft delete for schedules (audit trail)
- Execution logs retention (configurable)

## Support

For issues or questions:
1. Check the logs: `src/module/workflow-schedule/`
2. Review Bull queue status
3. Check Redis connection
4. Verify database migrations ran successfully