feat(infra): Migrate from HPA to KEDA for all Services

[justin-tahara]

Description

[Provide a brief description of the changes in this PR]
Introducing KEDA to all services. Historically we have had HPA's that were configured for the pods but this is a simple setup that can only be used for Memory and CPU related issues.

With KEDA we should be able to allow for custom triggers and this will allow us to autoscale more properly and be more prepared for Production use cases.

How Has This Been Tested?

[Describe the tests you ran to verify your changes]
Locally ran helm template . commands to test and validate that the charts were created properly.

We added default values and got rid of or moved the templates that were disabled and not used anymore.

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

• This PR should be backported (make sure to check that the backport attempt succeeds)
• [Optional] Override Linear Check

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
internal-search	Ready	Preview	Comment	Sep 16, 2025 8:53pm

[justin-tahara]

@greptileai

[cubic-dev-ai]

11 issues found across 12 files

_{React with 👍 or 👎 to teach cubic. You can also tag @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[justin-tahara]

@greptileai

[greptile-apps]

Greptile Summary

This PR implements a comprehensive migration from Kubernetes HPA (Horizontal Pod Autoscaler) to KEDA (Kubernetes Event-Driven Autoscaler) across all services in the Onyx application. The migration replaces basic CPU/memory-based HPA configurations with more sophisticated KEDA ScaledObjects that support custom triggers and advanced scaling behaviors.

The changes include:

• Moving existing HPA templates (webserver-hpa.yaml and api-hpa.yaml) to the templates_disabled/ directory as backup references
• Creating new KEDA ScaledObject templates for all services: webserver, API, and six Celery worker services (primary, heavy, monitoring, docprocessing, light, user-files-indexing, docfetching)
• Adding KEDA-specific configuration options to values.yaml including pollingInterval (30s), cooldownPeriod (300s), idleReplicaCount (1), failureThreshold (3), fallbackReplicas (1), and a customTriggers array
• Incrementing the Helm chart version from 0.2.11 to 0.2.12

KEDA provides significant advantages over HPA by supporting custom scaling triggers beyond CPU/memory metrics, such as queue depth, external metrics, and event-driven scaling. The new ScaledObjects include enhanced configuration options like idle replica management, fallback mechanisms, and customizable polling/cooldown periods. The migration maintains backward compatibility by preserving existing HPA configuration fields while adding KEDA-specific parameters. The customTriggers array enables future configuration of advanced scaling scenarios without requiring template changes.

Confidence score: 2/5

• This PR introduces significant breaking changes with critical configuration issues that will prevent proper autoscaling functionality
• Score lowered due to missing required KEDA fields, inconsistent default values, deprecated configurations, and potential empty triggers sections across multiple ScaledObject templates
• Pay close attention to all KEDA ScaledObject template files, particularly scaleTargetRef configurations and trigger definitions

_{13 files reviewed, 1 comment}

_{Edit Code Review Bot Settings | Greptile}

[greptile-apps]

logic: Empty triggers section will cause KEDA ScaledObject validation failure - ensure at least one trigger is always configured or add validation