-
Notifications
You must be signed in to change notification settings - Fork 1.9k
feat: Easy Install #5461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: Easy Install #5461
Conversation
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR introduces an 'Easy Install' feature that enables one-command deployment of Onyx through significant infrastructure changes and comprehensive rebranding. The changes span three main areas:
Docker Compose Consolidation: Multiple docker-compose files (gpu-dev, dev, prod variants) have been consolidated into a single unified docker-compose.yml
with comprehensive documentation and production guidance. A new installation script (install.sh
) provides automated setup with colorized output, health checks, and resource validation. The new env.template
centralizes all configuration variables with sensible defaults, replacing scattered environment variable declarations.
Systematic Rebranding: All configuration variables, environment settings, and internal references have been updated from 'DANSWER' to 'ONYX' prefixes. This includes Slack bot configurations (DANSWER_BOT_*
→ ONYX_BOT_*
), model interaction logging (LOG_DANSWER_MODEL_INTERACTIONS
→ LOG_ONYX_MODEL_INTERACTIONS
), and related imports across backend services. Some emoji configurations retain the old naming, creating minor inconsistencies.
Configuration Simplification: Several frontend build arguments have been removed (NEXT_PUBLIC_DISABLE_STREAMING
, NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA
) to reduce deployment complexity. Nginx configurations have been streamlined with production-ready templates (app.conf.template.prod
) that include SSL/TLS support, header mapping, and security considerations. The QA_PROMPT_OVERRIDE configuration has been deprecated and removed.
The changes integrate by creating a cohesive deployment experience where users can run a single installation script that downloads the unified docker-compose configuration, sets up the environment file with working defaults, and launches all services with proper dependencies and health monitoring.
Confidence score: 2/5
- This PR has several critical issues that will prevent successful deployment, particularly malformed YAML syntax and broken file references
- Score reflects serious deployment-breaking issues in CloudFormation templates and potential missing nginx configuration files
- Pay close attention to AWS CloudFormation templates, nginx template references, and the installation script's hardcoded branch references
28 files reviewed, 12 comments
Environment: | ||
- Name: NEXT_PUBLIC_DISABLE_STREAMING | ||
Value: "false" | ||
- Name: NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA | ||
Value: "false" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Syntax error: orphaned 'Value: "false"' without corresponding 'Name' field. This will cause CloudFormation deployment to fail.
Environment: | |
- Name: NEXT_PUBLIC_DISABLE_STREAMING | |
Value: "false" | |
- Name: NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA | |
Value: "false" | |
Environment: | |
- Name: INTERNAL_URL |
deployment/docker_compose/install.sh
Outdated
print_step "Setting up nginx configuration" | ||
|
||
# Base URL for nginx files | ||
NGINX_BASE_URL="https://raw.githubusercontent.com/onyx-dot-app/onyx/docker-compose-easy/deployment/data/nginx" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Same hardcoded branch issue here. Consider using a variable for branch name.
NGINX_BASE_URL="https://raw.githubusercontent.com/onyx-dot-app/onyx/docker-compose-easy/deployment/data/nginx" | |
NGINX_BASE_URL="https://raw.githubusercontent.com/onyx-dot-app/onyx/main/deployment/data/nginx" |
deployment/docker_compose/install.sh
Outdated
cd onyx_data/deployment && $COMPOSE_CMD -f docker-compose.yml pull && cd ../.. | ||
|
||
# Start services | ||
print_step "Starting Onyx services" | ||
print_info "Launching containers..." | ||
echo "" | ||
cd onyx_data/deployment && $COMPOSE_CMD -f docker-compose.yml up -d && cd ../.. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Directory changes without proper error handling. If cd fails, subsequent commands run in wrong location.
max-file: "6" | ||
# The specified script waits for the api_server to start up. | ||
# Without this we've seen issues where nginx shows no error logs but | ||
# does not recieve any traffic |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Typo in comment: 'recieve' should be 'receive'
# does not recieve any traffic | |
# does not receive any traffic |
|
||
ARG NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA | ||
ENV NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA=${NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Unnecessary empty lines left from removed environment variables. Clean up whitespace.
|
||
ARG NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA | ||
ENV NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA=${NEXT_PUBLIC_NEW_CHAT_DIRECTS_TO_SAME_PERSONA} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Unnecessary empty lines left from removed environment variables. Clean up whitespace.
################################################################################ | ||
## Database Configuration | ||
POSTGRES_USER=postgres | ||
POSTGRES_PASSWORD=password |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Using 'password' as default for production template poses security risk
MODEL_SERVER_HOST=inference_model_server | ||
INDEXING_MODEL_SERVER_HOST=indexing_model_server |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Duplicate MODEL_SERVER_HOST assignment - line 214 also sets this variable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
21 issues found across 30 files
Prompt for AI agents (all 21 issues)
Understand the root cause of the following 21 issues and fix them.
<file name="deployment/docker_compose/docker-compose.prod-no-letsencrypt.yml">
<violation number="1" location="deployment/docker_compose/docker-compose.prod-no-letsencrypt.yml:233">
Nonexistent nginx template referenced; container will fail to generate config</violation>
</file>
<file name="backend/onyx/onyxbot/slack/handlers/handle_message.py">
<violation number="1" location="backend/onyx/onyxbot/slack/handlers/handle_message.py:55">
Guarding with `not` allows negative values; a negative reminder offset results in a past timestamp. Use `<= 0` to disable on non-positive values, or clamp at parse time for safety.
(Based on your team's feedback about validating env-derived timeout multipliers at parse time and providing a safe default.)</violation>
</file>
<file name="backend/onyx/configs/app_configs.py">
<violation number="1" location="backend/onyx/configs/app_configs.py:670">
Breaking change: env var renamed without backward-compatible fallback; existing LOG_DANSWER_MODEL_INTERACTIONS will be ignored.</violation>
</file>
<file name="backend/onyx/onyxbot/slack/utils.py">
<violation number="1" location="backend/onyx/onyxbot/slack/utils.py:92">
Negative response-limit values will block all messages; treat <= 0 as unlimited instead of only == 0.</violation>
<violation number="2" location="backend/onyx/onyxbot/slack/utils.py:107">
Global message counter is updated without synchronization, leading to race conditions under concurrent requests.</violation>
<violation number="3" location="backend/onyx/onyxbot/slack/utils.py:211">
Guard retry tries to be >= 1 so a misconfigured env (0/negative) doesn’t disable retries.</violation>
<violation number="4" location="backend/onyx/onyxbot/slack/utils.py:648">
Normalize max_qpm: treat negatives as unlimited (None) or clamp to a positive value to avoid permanently blocking requests.</violation>
<violation number="5" location="backend/onyx/onyxbot/slack/utils.py:649">
Clamp max_wait_time to non-negative to avoid immediate TimeoutError with a misconfigured negative value.</violation>
<violation number="6" location="backend/onyx/onyxbot/slack/utils.py:709">
Strip whitespace before lowercasing to handle env values with accidental spaces.</violation>
</file>
<file name="deployment/aws_ecs_fargate/cloudformation/services/onyx_nginx_service_template.yaml">
<violation number="1" location="deployment/aws_ecs_fargate/cloudformation/services/onyx_nginx_service_template.yaml:160">
curl download lacks --fail; HTTP 4xx/5xx won't break the chain, risking invalid config being used.</violation>
</file>
<file name="deployment/docker_compose/docker-compose.yml">
<violation number="1" location="deployment/docker_compose/docker-compose.yml:159">
Case-sensitive check prevents disabling model server when DISABLE_MODEL_SERVER=true; compare to "true" (lowercase) to match env defaults.</violation>
<violation number="2" location="deployment/docker_compose/docker-compose.yml:295">
Avoid floating image tag for MinIO; pin to a specific release for reproducibility and safer upgrades.</violation>
</file>
<file name="deployment/docker_compose/install.sh">
<violation number="1" location="deployment/docker_compose/install.sh:58">
Hard-coding a feature branch (docker-compose-easy) makes installs fragile and non-reproducible; use a stable ref such as main or a pinned tag/commit.</violation>
<violation number="2" location="deployment/docker_compose/install.sh:202">
Downloading from a moving feature branch risks breakage; point to a stable ref like main or a pinned tag/commit.</violation>
<violation number="3" location="deployment/docker_compose/install.sh:253">
Using cd ... && ... && cd ../.. with set -e can leave the script in the wrong directory if the middle command fails; wrap the operation in a subshell to avoid side effects and ensure proper failure behavior.</violation>
</file>
<file name="deployment/helm/charts/onyx/values.yaml">
<violation number="1" location="deployment/helm/charts/onyx/values.yaml:859">
Added env var ONYX_BOT_DISABLE_COT appears unused; consider removing it or wiring it into the app to avoid configuration drift.</violation>
</file>
<file name="deployment/data/nginx/app.conf.template">
<violation number="1" location="deployment/data/nginx/app.conf.template:38">
Overwrites X-Forwarded-* with $scheme/$host/$server_port, dropping upstream values; behind a TLS-terminating proxy this can misreport https→http (and 443→80), causing incorrect redirects/cookie/security behavior. Reintroduce fallback to $http_x_forwarded_* (via map) or otherwise preserve upstream headers.</violation>
</file>
<file name="deployment/data/nginx/app.conf.template.prod">
<violation number="1" location="deployment/data/nginx/app.conf.template.prod:56">
Do not trust client-supplied X-Forwarded-Proto on public :80; set it from $scheme or restrict to trusted proxies.</violation>
<violation number="2" location="deployment/data/nginx/app.conf.template.prod:57">
Avoid propagating client X-Forwarded-Host on :80; set from $host instead.</violation>
<violation number="3" location="deployment/data/nginx/app.conf.template.prod:58">
Do not trust client X-Forwarded-Port on :80; set from $server_port.</violation>
</file>
<file name="deployment/docker_compose/env.template">
<violation number="1" location="deployment/docker_compose/env.template:155">
Duplicate environment variable definition for MODEL_SERVER_HOST; later entry overrides the earlier one. Remove the duplicate to avoid confusion.</violation>
</file>
Since this is your first cubic review, here's how it works:
- cubic automatically reviews your code and comments on bugs and improvements
- Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
- Ask questions if you need clarification on any suggestion
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai
to give feedback, ask questions, or re-run the review.
command: > | ||
/bin/sh -c "dos2unix /etc/nginx/conf.d/run-nginx.sh | ||
&& /etc/nginx/conf.d/run-nginx.sh app.conf.template.no-letsencrypt" | ||
&& /etc/nginx/conf.d/run-nginx.sh app.conf.template.prod.no-letsencrypt" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nonexistent nginx template referenced; container will fail to generate config
Prompt for AI agents
Address the following comment on deployment/docker_compose/docker-compose.prod-no-letsencrypt.yml at line 233:
<comment>Nonexistent nginx template referenced; container will fail to generate config</comment>
<file context>
@@ -232,7 +230,7 @@ services:
command: >
/bin/sh -c "dos2unix /etc/nginx/conf.d/run-nginx.sh
- && /etc/nginx/conf.d/run-nginx.sh app.conf.template.no-letsencrypt"
+ && /etc/nginx/conf.d/run-nginx.sh app.conf.template.prod.no-letsencrypt"
env_file:
- .env.nginx
</file context>
&& /etc/nginx/conf.d/run-nginx.sh app.conf.template.prod.no-letsencrypt" | |
&& /etc/nginx/conf.d/run-nginx.sh app.conf.template.no-letsencrypt" |
logger = setup_logger(extra={SLACK_CHANNEL_ID: details.channel_to_respond}) | ||
|
||
if not DANSWER_BOT_FEEDBACK_REMINDER: | ||
if not ONYX_BOT_FEEDBACK_REMINDER: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Guarding with not
allows negative values; a negative reminder offset results in a past timestamp. Use <= 0
to disable on non-positive values, or clamp at parse time for safety.
(Based on your team's feedback about validating env-derived timeout multipliers at parse time and providing a safe default.)
Prompt for AI agents
Address the following comment on backend/onyx/onyxbot/slack/handlers/handle_message.py at line 55:
<comment>Guarding with `not` allows negative values; a negative reminder offset results in a past timestamp. Use `<= 0` to disable on non-positive values, or clamp at parse time for safety.
(Based on your team's feedback about validating env-derived timeout multipliers at parse time and providing a safe default.)</comment>
<file context>
@@ -52,7 +52,7 @@ def schedule_feedback_reminder(
logger = setup_logger(extra={SLACK_CHANNEL_ID: details.channel_to_respond})
- if not DANSWER_BOT_FEEDBACK_REMINDER:
+ if not ONYX_BOT_FEEDBACK_REMINDER:
logger.info("Scheduled feedback reminder disabled...")
return None
</file context>
def get_feedback_visibility() -> FeedbackVisibility: | ||
try: | ||
return FeedbackVisibility(DANSWER_BOT_FEEDBACK_VISIBILITY.lower()) | ||
return FeedbackVisibility(ONYX_BOT_FEEDBACK_VISIBILITY.lower()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strip whitespace before lowercasing to handle env values with accidental spaces.
Prompt for AI agents
Address the following comment on backend/onyx/onyxbot/slack/utils.py at line 709:
<comment>Strip whitespace before lowercasing to handle env values with accidental spaces.</comment>
<file context>
@@ -706,7 +706,7 @@ def waiter(self, func_randid: int) -> None:
def get_feedback_visibility() -> FeedbackVisibility:
try:
- return FeedbackVisibility(DANSWER_BOT_FEEDBACK_VISIBILITY.lower())
+ return FeedbackVisibility(ONYX_BOT_FEEDBACK_VISIBILITY.lower())
except ValueError:
return FeedbackVisibility.PRIVATE
</file context>
return FeedbackVisibility(ONYX_BOT_FEEDBACK_VISIBILITY.lower()) | |
return FeedbackVisibility(ONYX_BOT_FEEDBACK_VISIBILITY.strip().lower()) |
self.max_qpm: int | None = DANSWER_BOT_MAX_QPM | ||
self.max_wait_time = DANSWER_BOT_MAX_WAIT_TIME | ||
self.max_qpm: int | None = ONYX_BOT_MAX_QPM | ||
self.max_wait_time = ONYX_BOT_MAX_WAIT_TIME |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clamp max_wait_time to non-negative to avoid immediate TimeoutError with a misconfigured negative value.
Prompt for AI agents
Address the following comment on backend/onyx/onyxbot/slack/utils.py at line 649:
<comment>Clamp max_wait_time to non-negative to avoid immediate TimeoutError with a misconfigured negative value.</comment>
<file context>
@@ -645,8 +645,8 @@ def slack_usage_report(action: str, sender_id: str | None, client: WebClient) ->
- self.max_qpm: int | None = DANSWER_BOT_MAX_QPM
- self.max_wait_time = DANSWER_BOT_MAX_WAIT_TIME
+ self.max_qpm: int | None = ONYX_BOT_MAX_QPM
+ self.max_wait_time = ONYX_BOT_MAX_WAIT_TIME
self.active_question = 0
self.last_reset_time = time.time()
</file context>
self.max_wait_time = ONYX_BOT_MAX_WAIT_TIME | |
self.max_wait_time = max(0, ONYX_BOT_MAX_WAIT_TIME) |
proxy_set_header X-Forwarded-Proto $forwarded_proto; | ||
proxy_set_header X-Forwarded-Host $forwarded_host; | ||
proxy_set_header X-Forwarded-Port $forwarded_port; | ||
proxy_set_header X-Forwarded-Proto $scheme; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overwrites X-Forwarded-* with $scheme/$host/$server_port, dropping upstream values; behind a TLS-terminating proxy this can misreport https→http (and 443→80), causing incorrect redirects/cookie/security behavior. Reintroduce fallback to $http_x_forwarded_* (via map) or otherwise preserve upstream headers.
Prompt for AI agents
Address the following comment on deployment/data/nginx/app.conf.template at line 38:
<comment>Overwrites X-Forwarded-* with $scheme/$host/$server_port, dropping upstream values; behind a TLS-terminating proxy this can misreport https→http (and 443→80), causing incorrect redirects/cookie/security behavior. Reintroduce fallback to $http_x_forwarded_* (via map) or otherwise preserve upstream headers.</comment>
<file context>
@@ -53,9 +35,9 @@ server {
- proxy_set_header X-Forwarded-Proto $forwarded_proto;
- proxy_set_header X-Forwarded-Host $forwarded_host;
- proxy_set_header X-Forwarded-Port $forwarded_port;
+ proxy_set_header X-Forwarded-Proto $scheme;
+ proxy_set_header X-Forwarded-Host $host;
+ proxy_set_header X-Forwarded-Port $server_port;
</file context>
proxy_set_header X-Forwarded-Host $host; | ||
proxy_set_header X-Forwarded-Port $server_port; | ||
proxy_set_header X-Forwarded-Proto $forwarded_proto; | ||
proxy_set_header X-Forwarded-Host $forwarded_host; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Avoid propagating client X-Forwarded-Host on :80; set from $host instead.
Prompt for AI agents
Address the following comment on deployment/data/nginx/app.conf.template.prod at line 57:
<comment>Avoid propagating client X-Forwarded-Host on :80; set from $host instead.</comment>
<file context>
@@ -35,9 +53,9 @@ server {
- proxy_set_header X-Forwarded-Host $host;
- proxy_set_header X-Forwarded-Port $server_port;
+ proxy_set_header X-Forwarded-Proto $forwarded_proto;
+ proxy_set_header X-Forwarded-Host $forwarded_host;
+ proxy_set_header X-Forwarded-Port $forwarded_port;
proxy_set_header Host $host;
</file context>
proxy_set_header X-Forwarded-Host $forwarded_host; | |
proxy_set_header X-Forwarded-Host $host; |
proxy_set_header X-Forwarded-Proto $scheme; | ||
proxy_set_header X-Forwarded-Host $host; | ||
proxy_set_header X-Forwarded-Port $server_port; | ||
proxy_set_header X-Forwarded-Proto $forwarded_proto; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not trust client-supplied X-Forwarded-Proto on public :80; set it from $scheme or restrict to trusted proxies.
Prompt for AI agents
Address the following comment on deployment/data/nginx/app.conf.template.prod at line 56:
<comment>Do not trust client-supplied X-Forwarded-Proto on public :80; set it from $scheme or restrict to trusted proxies.</comment>
<file context>
@@ -35,9 +53,9 @@ server {
- proxy_set_header X-Forwarded-Proto $scheme;
- proxy_set_header X-Forwarded-Host $host;
- proxy_set_header X-Forwarded-Port $server_port;
+ proxy_set_header X-Forwarded-Proto $forwarded_proto;
+ proxy_set_header X-Forwarded-Host $forwarded_host;
+ proxy_set_header X-Forwarded-Port $forwarded_port;
</file context>
proxy_set_header X-Forwarded-Proto $forwarded_proto; | |
proxy_set_header X-Forwarded-Proto $scheme; |
proxy_set_header X-Forwarded-Port $server_port; | ||
proxy_set_header X-Forwarded-Proto $forwarded_proto; | ||
proxy_set_header X-Forwarded-Host $forwarded_host; | ||
proxy_set_header X-Forwarded-Port $forwarded_port; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not trust client X-Forwarded-Port on :80; set from $server_port.
Prompt for AI agents
Address the following comment on deployment/data/nginx/app.conf.template.prod at line 58:
<comment>Do not trust client X-Forwarded-Port on :80; set from $server_port.</comment>
<file context>
@@ -35,9 +53,9 @@ server {
- proxy_set_header X-Forwarded-Port $server_port;
+ proxy_set_header X-Forwarded-Proto $forwarded_proto;
+ proxy_set_header X-Forwarded-Host $forwarded_host;
+ proxy_set_header X-Forwarded-Port $forwarded_port;
proxy_set_header Host $host;
</file context>
proxy_set_header X-Forwarded-Port $forwarded_port; | |
proxy_set_header X-Forwarded-Port $server_port; |
# USE_SEMANTIC_KEYWORD_EXPANSIONS_BASIC_SEARCH= | ||
|
||
## Model Configuration | ||
MODEL_SERVER_HOST=inference_model_server |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duplicate environment variable definition for MODEL_SERVER_HOST; later entry overrides the earlier one. Remove the duplicate to avoid confusion.
Prompt for AI agents
Address the following comment on deployment/docker_compose/env.template at line 155:
<comment>Duplicate environment variable definition for MODEL_SERVER_HOST; later entry overrides the earlier one. Remove the duplicate to avoid confusion.</comment>
<file context>
@@ -0,0 +1,215 @@
+# USE_SEMANTIC_KEYWORD_EXPANSIONS_BASIC_SEARCH=
+
+## Model Configuration
+MODEL_SERVER_HOST=inference_model_server
+INDEXING_MODEL_SERVER_HOST=indexing_model_server
+# EMBEDDING_BATCH_SIZE=
</file context>
MODEL_SERVER_HOST=inference_model_server | |
# MODEL_SERVER_HOST=inference_model_server |
✅ Addressed in e44a636
1bb7402
to
11a0728
Compare
# Onyx Slack Bot Configs | ||
##### | ||
DANSWER_BOT_NUM_RETRIES = int(os.environ.get("DANSWER_BOT_NUM_RETRIES", "5")) | ||
ONYX_BOT_NUM_RETRIES = int(os.environ.get("ONYX_BOT_NUM_RETRIES", "5")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These slackbot changes are minimally risky imo. Since they are default off riskier behaviors like responding in every channel, it shouldn't cause any issues if they no longer apply. Also some of these I suspect don't work anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also implies that the new env variable is set properly and can be retrieved. I think it's a matter of making sure the new one is there not so much that the old was stays or not.
Do feel like this is quite risky.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ones that don't work (I removed some), don't feel super critical. It's outside the scope of this PR. My only concern is with variable renaming when some people might still be deploying with old vars and this becoming a "regression" even though it's not.
But reading through what they do, I'm not too concerned about that case.
HostPort: 3000 | ||
Protocol: tcp | ||
Environment: | ||
- Name: NEXT_PUBLIC_DISABLE_STREAMING |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not used I believe, might want to double check
# Log format to include request latency | ||
log_format custom_main '$remote_addr - $remote_user [$time_local] "$request" ' | ||
'$status $body_bytes_sent "$http_referer" ' | ||
'"$http_user_agent" "$http_x_forwarded_for" ' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following same convention as docker compose file naming, there's a default version and a prod version, just renamed the files, the diffs here are just from git not understanding the file renamings
'"$http_user_agent" "$http_x_forwarded_for" ' | ||
'rt=$request_time'; | ||
|
||
# Map X-Forwarded-Proto or fallback to $scheme |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May want to quickly sanity check that nothing is broken from the renaming. I did not test the prod setup
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall I think this PR is a bit too dangerous to just merge.
My opinion is to break this up into smaller PR's that are easier to digest and easier to read through.
Ideally the Slack bot changes are all one PR, then the compose unification is another. Helm can be it's own PR and lastly the script comes in once all of the individual workflows have been tested.
From personal experience, I do prefer Python Scripts for maintainability and ease of understanding but if this is truly a one line command approach to onyx then I can be convinced to do bash instead.
Maybe we can even have both options be available but again two things to maintain is not as easy as one.
The last thing for this is that many workflows will most likely break across the board. Many customers that have hardcoded workflows will definitely complain about this upgrade and thus we should think a bit more about the migration path and how we would introduce this to existing users of Onyx in a non-invasive way such that they are able to also upgrade easily to this new workflow.
Also if we are going to introduce the bash script or any script in general, we should have a integration or test suite to ensure that it never breaks and is the source of truth to upgrading or installing onyx.
- name: Start Docker containers | ||
run: | | ||
cd deployment/docker_compose | ||
cp env.template .env |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this required to get the test to pass? Or are there other dependencies that require this to be set.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me try to make it not required
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done!
run: | | ||
echo "Starting wait-for-service script..." | ||
docker logs -f danswer-stack-api_server-1 & |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as the onyx name is correctly populated I think this is fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ya it's the default now in the compose script, a lot of things are simplified now
#DANSWER_BOT_SLACK_APP_TOKEN=<REPLACE THIS> | ||
#DANSWER_BOT_SLACK_BOT_TOKEN=<REPLACE THIS> | ||
#ONYX_BOT_SLACK_APP_TOKEN=<REPLACE THIS> | ||
#ONYX_BOT_SLACK_BOT_TOKEN=<REPLACE THIS> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Beautiful
if cc_pair.access_type != AccessType.SYNC: | ||
task_logger.error( | ||
f"Recieved non-sync CC Pair {cc_pair.id} for external " | ||
f"Received non-sync CC Pair {cc_pair.id} for external " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lmao I wanted to fix a bot comment but I realized this same typo appeared a bunch of times so just fixed it across the board
# Onyx Slack Bot Configs | ||
##### | ||
DANSWER_BOT_NUM_RETRIES = int(os.environ.get("DANSWER_BOT_NUM_RETRIES", "5")) | ||
ONYX_BOT_NUM_RETRIES = int(os.environ.get("ONYX_BOT_NUM_RETRIES", "5")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also implies that the new env variable is set properly and can be retrieved. I think it's a matter of making sure the new one is there not so much that the old was stays or not.
Do feel like this is quite risky.
from slack_sdk.errors import SlackApiError | ||
|
||
from onyx.configs.onyxbot_configs import DANSWER_BOT_FEEDBACK_REMINDER | ||
from onyx.configs.onyxbot_configs import DANSWER_REACT_EMOJI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we not change this one as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah it's not even a thing I think people should change to be honest... ya we should change it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done!
from onyx.configs.onyxbot_configs import DANSWER_BOT_DISABLE_DOCS_ONLY_ANSWER | ||
from onyx.configs.onyxbot_configs import DANSWER_BOT_DISPLAY_ERROR_MSGS | ||
from onyx.configs.onyxbot_configs import DANSWER_BOT_NUM_RETRIES | ||
from onyx.configs.onyxbot_configs import DANSWER_REACT_EMOJI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above here.
backend/onyx/onyxbot/slack/utils.py
Outdated
the limit to be exceeded. | ||
""" | ||
if DANSWER_BOT_RESPONSE_LIMIT_PER_TIME_PERIOD == 0: | ||
if ONYX_BOT_RESPONSE_LIMIT_PER_TIME_PERIOD == 0: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a safe change that we should probably just do to be safe
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the understanding that everything should be routed through the .prod template instead of this one? Should we even keep this one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the default non SSL one. I think it's good to keep, the default deployment uses this. HTTPS setup is kind of a pain in the ass, I still don't fully understand it and we don't want to make the base deployment too painful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was the reasoning behind the Bash script? Was this more along the lines of with Bash you don't need to download anything but with Python you would need a working version of python?
I think as someone who is using docker, it probably can be assumed that they have python and so writing a python script would be nice to maintain and easier to debug in my opinion.
Open to hearing what your thoughts are
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pip is the python package manager, you can reuse it to manage other packages but it's less intuitive, it's like installing Onyx with npm, probably there's some way to get that working but it's not the right tool.
The other thing is with requiring a python installation which doesn't come out of the box with most operating systems whereas bash typically does. Maybe on Windows most people have pip instead of bash but meh...
a3f02f3
to
06ec9f8
Compare
Description
1 command install of Onyx
How Has This Been Tested?
Ran locally, some concerns:
Backporting (check the box to trigger backport action)
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.
Summary by cubic
Adds a one-command Docker Compose install with a guided script, a single compose file, and an env template to make setup fast and consistent. Also standardizes config names (DANSWER_BOT_* → ONYX_BOT_, LOG_DANSWER_ → LOG_ONYX_*) and cleans up nginx/web configs.
New Features
Migration