Migrate send_event_reminders to Celery Task

[ad-m-ss]

Migrate send_event_reminders to Celery Task

Overview

Migrate the send_event_reminders management command to a Celery task with robust error handling and email delivery management. This task sends daily email reminders for upcoming events to users.

Current Implementation Analysis

Command Location

• File: poradnia/events/management/commands/send_event_reminders.py
• Current Schedule: Daily at 12:00 ("0 12 * * *")
• Execution: Via cron with file locking (.contrib/docker/cron/run_locked.sh)

Task Complexity

• Purpose: Send email reminders for upcoming events
• Operations: Event query processing, email composition and sending
• Dependencies: Event models, User models, Django email backend
• Runtime: Medium (depends on number of events and email backend performance)
• Failure Points: Email backend issues, SMTP connectivity, database queries

Implementation Tasks

1. Create Celery Task

• Create or update poradnia/events/tasks.py
• Convert management command logic to @shared_task decorated function
• Maintain existing event reminder functionality
• Preserve email template and content logic

2. Error Handling & Email Reliability

• Implement retry logic for email sending failures (max 3 retries)
• Add specific exception handling for:
  • SMTP connection errors
  • Email backend failures
  • Database query issues
  • Template rendering errors
• Batch email processing to handle large volumes
• Individual email failure handling (don't fail entire task for single email)

3. Logging & Monitoring

• Structured logging with email sending progress
• Track successful/failed email deliveries
• Log email recipient counts and event details
• Performance metrics (emails sent, processing time)
• Integration with Celery result backend

4. Email Processing Optimization

• Batch email sending to improve performance
• Connection pooling for email backend
• Memory-efficient event processing for large datasets
• Progress tracking for long-running email operations

5. Scheduling Configuration

• Configure Celery beat periodic task (daily at 12:00)
• Use database-backed scheduling (django-celery-beat)
• Allow runtime schedule modifications
• Maintain current timing during transition

Files to Modify/Create

New/Updated Files

• poradnia/events/tasks.py - Celery task implementation (create or update)

Modified Files

• poradnia/settings/base.py - Add event reminders to Celery beat schedule
• poradnia/events/management/commands/send_event_reminders.py - Update or deprecate
• docs/celery.rst – Documentation updates

Configuration

# poradnia/settings/base.py - Add to CELERY_BEAT_SCHEDULE
'send-event-reminders': {
    'task': 'poradnia.events.tasks.send_event_reminders',
    'schedule': crontab(hour=12, minute=0),  # Daily at 12:00
},

Dependencies

• BLOCKED BY: Phase 1: Celery Infrastructure Setup #1828 - Phase 1: Celery Infrastructure Setup
• REQUIRES: Redis service, Celery worker service, email backend configuration
• PARENT: Phase 2: Background Task Migration to Celery (Umbrella Issue) #1829 - Phase 2: Background Task Migration to Celery (Umbrella Issue)

This issue cannot begin until the Celery infrastructure from #1828 is fully operational.

Related Issues

• Infrastructure: Phase 1: Celery Infrastructure Setup #1828 - Phase 1: Celery Infrastructure Setup (must be completed first)
• Umbrella: Phase 2: Background Task Migration to Celery (Umbrella Issue) #1829 - Phase 2: Background Task Migration to Celery (tracks overall progress)
• Parallel Tasks:
  • Migrate run_court_session_parser to Celery Task #1830 - Migrate run_court_session_parser to Celery Task
  • Migrate send_old_cases_reminder to Celery Task #1832 - Migrate send_old_cases_reminder to Celery Task
  • Migrate Django clearsessions to Celery Task #1833 - Migrate Django clearsessions to Celery Task

Can be developed in parallel with other task migrations once infrastructure is ready.

Testing Requirements

Unit Tests

• Test task execution with mock events data
• Test email sending functionality
• Test error handling for email failures
• Test retry logic for transient failures
• Test batch processing logic

Integration Tests

• Test full Celery task execution with real email backend
• Test scheduling via Celery beat
• Test email delivery with various email providers
• Test task monitoring and result tracking

Email Testing

• Test email template rendering
• Test recipient list processing
• Test email backend failover (if configured)
• Test large volume email processing

Implementation Example Structure

# poradnia/events/tasks.py
from celery import shared_task
from celery.utils.log import get_task_logger
from django.core.mail import send_mail
from django.core.mail.backends.base import BaseEmailBackend

logger = get_task_logger(__name__)

@shared_task(bind=True, autoretry_for=(ConnectionError, SMTPException), retry_kwargs={'max_retries': 3, 'countdown': 300})
def send_event_reminders(self):
    """
    Send email reminders for upcoming events.
    Migrated from management command with enhanced error handling.
    """
    try:
        logger.info("Starting event reminders task")
        
        # Get upcoming events
        events = get_upcoming_events()
        
        email_count = 0
        failed_count = 0
        
        for event in events:
            try:
                # Send individual reminder
                send_event_reminder_email(event)
                email_count += 1
                logger.info(f"Sent reminder for event {event.id}")
            except Exception as email_exc:
                failed_count += 1
                logger.error(f"Failed to send reminder for event {event.id}: {email_exc}")
        
        result = {
            "status": "completed",
            "emails_sent": email_count,
            "emails_failed": failed_count,
            "events_processed": len(events)
        }
        
        logger.info(f"Event reminders completed: {result}")
        return result
        
    except Exception as exc:
        logger.error(f"Event reminders task failed: {exc}")
        raise self.retry(exc=exc)

Acceptance Criteria

• Celery task successfully sends event reminder emails
• Task runs on the same daily schedule (12:00)
• All current email functionality is preserved
• Email template and content remain unchanged
• Error handling improves email delivery reliability
• Individual email failures don't crash entire task
• Task execution can be monitored through Celery
• Email delivery metrics are tracked and logged
• Retry logic handles transient email backend failures

Email-Specific Considerations

• Respect email rate limits if configured
• Handle email backend connection pooling
• Preserve email headers and formatting
• Maintain email delivery preferences and opt-outs
• Handle bounce processing if implemented

Rollback Plan

• Keep original management command during transition
• Maintain cron job as backup initially
• Monitor email delivery rates for regressions
• Document rollback procedure

Success Metrics

• Email Delivery: 95%+ delivery success rate with retry handling
• Performance: Execution time comparable to original command
• Reliability: Improved handling of email backend failures
• Monitoring: Full visibility into email delivery status and failures

References

• Current management command: poradnia/events/management/commands/send_event_reminders.py
• Event models: poradnia/events/models.py
• Email configuration: Django email settings
• Cron schedule: .contrib/docker/cron/set_crontab.sh