feat: Bring up separate attachment delivery worker to handle file attachments instead of blocking the middleware (#19)

Author: devadathanmb

.tool-versions

@@ -0,0 +1 @@
+nodejs 20

CLAUDE.md

@@ -29,6 +29,7 @@ docker compose -f docker-compose.dev.yaml up ktu-bot-app --build
 docker compose -f docker-compose.dev.yaml up announcements-notify-worker --build
 docker compose -f docker-compose.dev.yaml up data-sync-worker --build
 docker compose -f docker-compose.dev.yaml up broadcasts-worker --build
+docker compose -f docker-compose.dev.yaml up attachment-delivery-worker --build
 
 # Local development without Docker
 pnpm dev              # Start bot with tsx hot reload
@@ -62,6 +63,7 @@ pnpm format:check     # Check formatting
 pnpm workers:announcements-notify-worker-dev    # Dev mode with hot reload
 pnpm workers:broadcasts-worker-dev
 pnpm workers:data-sync-worker-dev
+pnpm workers:attachment-delivery-worker-dev
 pnpm services:bull-board-dev                    # Bull Board dashboard
 ```
 
@@ -173,12 +175,14 @@ export class MyWorker extends BaseWorker<JobData> {
 - `DATA_SYNC_QUEUE` - Syncs KTU data (concurrency: 3, rate limited)
 - `ANNOUNCEMENTS_NOTIFY_QUEUE` - New announcement notifications (concurrency: 1)
 - `BROADCASTS_QUEUE` - Message delivery (concurrency: 1)
+- `ATTACHMENT_DELIVERY_QUEUE` - Non-blocking file downloads (concurrency: 2)
 
 **Worker Implementations:**
 
 - [src/workers/announcements/notify/](src/workers/announcements/notify/) - Monitors for new announcements, filters by course, creates broadcast jobs
 - [src/workers/broadcasts/](src/workers/broadcasts/) - Generic message delivery with rate limiting and error handling
 - [src/workers/data-sync/](src/workers/data-sync/) - Syncs KTU data to local DB for full-text search
+- [src/workers/attachment-delivery/](src/workers/attachment-delivery/) - Downloads and sends files asynchronously, prevents bot blocking
 
 **Shared Utilities:**
 
@@ -273,7 +277,7 @@ Middleware sequence is critical in [src/bot/bot.ts](src/bot/bot.ts:37):
 Each service exposes health endpoints:
 
 - Bot: port 3000
-- Workers: ports 3001, 3002, 3003
+- Workers: ports 3001, 3002, 3003, 3004
 - Bull Board: port 3010
 - Check database and Redis connectivity
 

README.md

@@ -56,6 +56,7 @@ The bot follows a microservices architecture where each component handles a spec
 | **Announcements Notify Worker** | Background worker   | Monitors for new announcements using BullMQ scheduled jobs and sends filtered alerts to users |
 | **Broadcasts Worker**           | Background worker   | Handles queued broadcast message delivery                                                     |
 | **Data Sync Worker**            | Background worker   | Periodically syncs KTU data to local DB via BullMQ scheduled jobs to power full-text search   |
+| **Attachment Delivery Worker**  | Background worker   | Downloads and sends files asynchronously to prevent bot blocking                              |
 | **Bull Board Service**          | Monitoring service  | Web dashboard for real-time queue monitoring and job management                               |
 | **PostgreSQL**                  | Database            | Stores all data with Drizzle ORM for type-safe queries                                        |
 | **Redis**                       | Queue               | Powers BullMQ jobs                                                                            |
@@ -147,6 +148,9 @@ docker compose -f docker/compose.dev.yaml up data-sync-worker --build
 
 # Broadcasts worker
 docker compose -f docker/compose.dev.yaml up broadcasts-worker --build
+
+# Attachment delivery worker
+docker compose -f docker/compose.dev.yaml up attachment-delivery-worker --build
 ```
 
 > [!TIP]

docker/compose.dev.yaml

@@ -216,6 +216,54 @@ services:
       retries: 3
       start_period: 30s
 
+  attachment-delivery-worker:
+    build:
+      context: ..
+      dockerfile: docker/Dockerfile.dev
+    container_name: attachment-delivery-worker
+    env_file:
+      - ../env/dev/bot.env
+      - ../env/dev/db.env
+      - ../env/dev/redis.env
+      - ../env/dev/api.env
+      - ../env/dev/attachment-delivery-worker.env
+      - ../env/dev/logging.env
+      - ../env/dev/.env
+    ports:
+      - "3004:3004"
+    volumes:
+      - ..:/app
+      - /app/node_modules
+    depends_on:
+      ktu-bot-postgres:
+        condition: service_healthy
+        restart: true
+      ktu-bot-redis:
+        condition: service_healthy
+        restart: true
+      ktu-bot-db-migrations:
+        condition: service_completed_successfully
+    networks:
+      - ktu-bot-network
+    command: ["pnpm", "workers:attachment-delivery-worker-dev"]
+    restart: unless-stopped
+    healthcheck:
+      test:
+        [
+          "CMD",
+          "curl",
+          "-f",
+          "-s",
+          "-S",
+          "--max-time",
+          "5",
+          "http://localhost:3004/health",
+        ]
+      interval: 60s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
+
   bull-board-service:
     build:
       context: ..
@@ -232,7 +280,7 @@ services:
       - ../env/dev/broadcasts-worker.env
       - ../env/dev/announcements-notify-worker.env
       - ../env/dev/data-sync-worker.env
-      - ../env/dev/logging.env
+      - ../env/dev/attachment-delivery-worker.env
       - ../env/dev/.env
     ports:
       - "3010:3010"

docker/compose.staging.yaml

@@ -192,6 +192,51 @@ services:
         max-size: "10m"
         max-file: "3"
 
+  attachment-delivery-worker:
+    image: ktu-bot:latest
+    container_name: attachment-delivery-worker
+    env_file:
+      - ../env/staging/.env
+      - ../env/staging/attachment-delivery-worker.env
+    networks:
+      - ktu-bot-network
+    depends_on:
+      ktu-bot-db-migrations:
+        condition: service_completed_successfully
+    ports:
+      - "3004:3004"
+    command: ["pnpm", "workers:attachment-delivery-worker"]
+    restart: unless-stopped
+    deploy:
+      resources:
+        limits:
+          memory: 512M
+          cpus: "0.5"
+        reservations:
+          memory: 256M
+          cpus: "0.25"
+    healthcheck:
+      test:
+        [
+          "CMD",
+          "curl",
+          "-f",
+          "-s",
+          "-S",
+          "--max-time",
+          "5",
+          "http://localhost:3004/health",
+        ]
+      interval: 60s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "10m"
+        max-file: "3"
+
   bull-board-service:
     image: ktu-bot:latest
     container_name: bull-board-service
@@ -206,6 +251,8 @@ services:
         condition: service_healthy
       data-sync-worker:
         condition: service_healthy
+      attachment-delivery-worker:
+        condition: service_healthy
     ports:
       - "3010:3010"
     command: ["pnpm", "services:bull-board"]
@@ -239,3 +286,7 @@ services:
       options:
         max-size: "10m"
         max-file: "3"
+
+networks:
+  ktu-bot-network:
+    driver: bridge

docker/compose.yaml

@@ -192,6 +192,51 @@ services:
         max-size: "10m"
         max-file: "3"
 
+  attachment-delivery-worker:
+    image: ktu-bot:latest
+    container_name: attachment-delivery-worker
+    env_file:
+      - ../env/prod/.env
+      - ../env/prod/attachment-delivery-worker.env
+    networks:
+      - ktu-bot-network
+    depends_on:
+      ktu-bot-db-migrations:
+        condition: service_completed_successfully
+    ports:
+      - "3004:3004"
+    command: ["pnpm", "workers:attachment-delivery-worker"]
+    restart: unless-stopped
+    deploy:
+      resources:
+        limits:
+          memory: 512M
+          cpus: "0.5"
+        reservations:
+          memory: 256M
+          cpus: "0.25"
+    healthcheck:
+      test:
+        [
+          "CMD",
+          "curl",
+          "-f",
+          "-s",
+          "-S",
+          "--max-time",
+          "5",
+          "http://localhost:3004/health",
+        ]
+      interval: 60s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "10m"
+        max-file: "3"
+
   bull-board-service:
     image: ktu-bot:latest
     container_name: bull-board-service
@@ -206,6 +251,8 @@ services:
         condition: service_healthy
       data-sync-worker:
         condition: service_healthy
+      attachment-delivery-worker:
+        condition: service_healthy
     ports:
       - "3010:3010"
     command: ["pnpm", "services:bull-board"]

docs/working.md

@@ -34,7 +34,8 @@ graph TB
     TG -->|Webhook/Long Polling| Bot
     Bot -->|Fetch Data| KTU
     Bot -->|Store/Retrieve| DB
-    Bot -->|Cache/Queue| Redis
+    Bot -->|Cache| Redis
+    Bot -->|Queue Attachments| Queue
     Bot -->|Response| TG
     TG -->|Messages| User
 
@@ -65,6 +66,7 @@ graph TB
             NotifyWorker[📢 Announcements<br/>Notify Worker<br/>Port: 3001]
             BroadcastWorker[📨 Broadcasts<br/>Worker<br/>Port: 3002]
             SyncWorker[🔄 Data Sync<br/>Worker<br/>Port: 3003]
+            AttachmentWorker[📦 Attachment<br/>Delivery Worker<br/>Port: 3004]
         end
 
         subgraph "Message Queue"
@@ -92,12 +94,16 @@ graph TB
     NotifyWorker -->|Add Jobs| Queue
 
     Queue -->|Process Jobs| BroadcastWorker
+    Queue -->|Process Attachments| AttachmentWorker
     BroadcastWorker -->|Send Messages| TG
     BroadcastWorker -->|Update Status| DB
 
     SyncWorker -->|Fetch All Data| KTU
     SyncWorker -->|Sync| DB
 
+    Bot -->|Queue Attachments| Queue
+    AttachmentWorker -->|Download & Send| TG
+
     Queue -.->|Uses| Redis
 
     style User fill:#4a90e2,stroke:#2e5c8a,color:#fff
@@ -198,9 +204,32 @@ When users perform inline searches, their queries hit this local copy instead of
 
 The BullMQ-based approach makes this much more resilient than traditional cron scheduling. Check out [`src/workers/data-sync/`](../src/workers/data-sync/) for the implementation.
 
+### Attachment Delivery Worker 📦
+
+This worker handles file downloads and deliveries asynchronously, preventing the bot from being blocked by large file downloads. When users request attachments (calendars, timetables, announcements), the bot immediately queues the request and returns to serving other users. Here's how it works:
+
+1. **Job Creation**: When a user requests files, the bot creates a job containing all attachment metadata and immediately returns
+   - Bot sends a friendly status message like _"⏳ Preparing your calendar... I'll send it shortly!"_
+   - The bot doesn't wait for downloads to complete
+2. **Background Processing**: The worker picks up jobs from the queue and downloads all attachments
+   - Uses **all-or-nothing delivery** - if any file fails to download, nothing is sent to ensure users get complete data
+   - Downloads happen asynchronously without blocking other users
+3. **Media Group Delivery**: Once all files are ready, sends them as media groups (batches of up to 10 files per Telegram API call)
+   - Much faster than individual file delivery
+   - Single caption listing all attachment names for clarity
+4. **Graceful Error Handling**: Handles various failure scenarios without crashing
+   - If user blocks the bot, marks their chat as kicked and removes subscriptions
+   - If user deactivates account, cleans up their data from the database
+   - If rate limited by Telegram, pauses the entire queue for the specified duration, then resumes processing
+5. **Status Updates**: Deletes the loading message once delivery is complete (or on failure)
+   - No chat pollution - the status message disappears after completion
+   - User receives their files cleanly without extra messages
+
+This worker has a `concurrency` of `2` to prevent overwhelming Telegram's rate limits while still processing requests efficiently. All implementation lives in [`src/workers/attachment-delivery/`](../src/workers/attachment-delivery/)
+
 ## Health Checks and Monitoring
 
-Each service exposes a health check endpoint (bot on port `3000`, workers on `3001-3003`) that verifies the service is running and can connect to its dependencies like the database and Redis. This enables zero-downtime deployments and automatic restarts if something goes wrong. The health check utility is in [`src/utils/healthCheck.ts`](../src/utils//healthCheck.ts) if you want to see how it works.
+Each service exposes a health check endpoint (bot on port `3000`, workers on `3001-3004`) that verifies the service is running and can connect to its dependencies like the database and Redis. This enables zero-downtime deployments and automatic restarts if something goes wrong. The health check utility is in [`src/utils/healthCheck.ts`](../src/utils//healthCheck.ts) if you want to see how it works.
 
 ### Queue Monitoring with Bull Board 📊
 

env/dev/attachment-delivery-worker.env

@@ -0,0 +1,2 @@
+# Attachment Delivery Worker configs
+ATTACHMENT_DELIVERY_WORKER_HEALTHCHECK_PORT=3004

env/prod/attachment-delivery-worker.env

@@ -0,0 +1,2 @@
+# Attachment Delivery Worker configs
+ATTACHMENT_DELIVERY_WORKER_HEALTHCHECK_PORT=3004

env/staging/attachment-delivery-worker.env

@@ -0,0 +1,2 @@
+# Attachment Delivery Worker configs
+ATTACHMENT_DELIVERY_WORKER_HEALTHCHECK_PORT=3004