Skip to content

Conversation

rguan72
Copy link
Contributor

@rguan72 rguan72 commented Sep 17, 2025

Description

[Provide a brief description of the changes in this PR]

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

  • This PR should be backported (make sure to check that the backport attempt succeeds)
  • [Optional] Override Linear Check

Summary by cubic

Restructured TF Deep Research orchestration to use a single, chat-style message history across clarifier → orchestrator → closer, improving context continuity and reducing prompt duplication.

  • Refactors

    • Switched from concatenated prompt strings to structured System/Human/AI messages throughout the DR flow.
    • Clarifier now builds a base system message (tools, sources, KG types) and seeds orchestration_llm_messages with chat history and uploads.
    • Orchestrator appends the most recent answer summary (with/without docs based on mode) and uses messages directly for tool decision and purpose generation.
    • Closer generates the final answer from the accumulated message history (instead of rebuilding large prompt strings).
    • Added state fields: orchestration_llm_messages, all_entity_types, all_relationship_types; removed repeated KG/type lookups.
    • aggregate_context now supports most_recent to limit injected history to the latest iteration.
  • New Features

    • Introduced BASE_SYSTEM_MESSAGE_TEMPLATE, TOOL_CHOICE_WRAPPER_PROMPT, NEXT_TOOL_PURPOSE_PROMPT, and FINAL_ANSWER_DEEP_CITATION_PROMPT for clearer, modular prompting.
    • Added get_chat_history_messages to filter non-text parts and cap history length.
    • Only exposes KG entity/relationship types if KG is enabled, exposed, and the KG tool is actually available.
    • Search tool prompt now encourages parallel, distinct queries; decision prompts add explicit “what’s missing” guidance.

Copy link

vercel bot commented Sep 17, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
internal-search Ready Ready Preview Comment Sep 24, 2025 4:08am

Copy link

blacksmith-sh bot commented Sep 19, 2025

33 Jobs Failed:

Run Integration Tests v2 / integration-tests (connector_job_tests/google, connector-google) failed on "Pull Docker images"
[...]
86be50b42b67: Retrying in 1 second
4a22f1dd7d96: Download complete
unexpected EOF
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (connector_job_tests/jira, connector-jira) failed on "Pull Docker images"
[...]
47964e6bbe9b: Retrying in 1 second
a818a6f2ad3d: Retrying in 1 second
unexpected EOF
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (connector_job_tests/sharepoint, connector-sharepoint) failed on "Pull Docker images"
[...]
a818a6f2ad3d: Retrying in 2 seconds
961f082158e0: Retrying in 1 second
47964e6bbe9b: Retrying in 1 second
13f1b08d45fb: Retrying in 1 second
a818a6f2ad3d: Retrying in 1 second
unexpected EOF
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (connector_job_tests/slack, connector-slack) failed on "Pull Docker images"
[...]
13f1b08d45fb: Retrying in 1 second
961f082158e0: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/anonymous_user, tests-anonymous_user) failed on "Pull Docker images"
[...]
14bd20d89e4f: Retrying in 1 second
3e2a38691e82: Retrying in 1 second
86be50b42b67: Retrying in 1 second
ff5db6c3e62e: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/api_key, tests-api_key) failed on "Pull Docker images"
[...]
86be50b42b67: Retrying in 1 second
ff5db6c3e62e: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/auth, tests-auth) failed on "Pull Docker images"
[...]
3e2a38691e82: Retrying in 2 seconds
14bd20d89e4f: Retrying in 1 second
ff5db6c3e62e: Retrying in 1 second
86be50b42b67: Retrying in 1 second
3e2a38691e82: Retrying in 1 second
unexpected EOF
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/chat, tests-chat) failed on "Pull Docker images"
[...]
3e2a38691e82: Retrying in 1 second
ff5db6c3e62e: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/chat_retention, tests-chat_retention) failed on "Pull Docker images"
[...]
ff5db6c3e62e: Retrying in 1 second
14bd20d89e4f: Retrying in 1 second
unexpected EOF
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/connector, tests-connector) failed on "Pull Docker images"
[...]
47964e6bbe9b: Retrying in 1 second
a818a6f2ad3d: Retrying in 1 second
unexpected EOF
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/dev_apis, tests-dev_apis) failed on "Pull Docker images"
[...]
3e2a38691e82: Retrying in 1 second
86be50b42b67: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/document_set, tests-document_set) failed on "Pull Docker images"
[...]
47964e6bbe9b: Retrying in 1 second
961f082158e0: Retrying in 1 second
unexpected EOF
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/image_indexing, tests-image_indexing) failed on "Pull Docker images"
[...]
14bd20d89e4f: Retrying in 1 second
3e2a38691e82: Retrying in 1 second
86be50b42b67: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
f82acb7dbe45: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/index_attempt, tests-index_attempt) failed on "Pull Docker images"
[...]
3e2a38691e82: Retrying in 1 second
14bd20d89e4f: Retrying in 1 second
86be50b42b67: Retrying in 1 second
ff5db6c3e62e: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.
Run Integration Tests v2 / integration-tests (tests/indexing, tests-indexing) failed on "Pull Docker images"
[...]
f82acb7dbe45: Verifying Checksum
f82acb7dbe45: Download complete
961f082158e0: Retrying in 1 second
47964e6bbe9b: Retrying in 1 second
unexpected EOF
07d8bad72860: Verifying Checksum
07d8bad72860: Download complete
07d8bad72860: Pull complete
f82acb7dbe45: Pull complete
272fc460a5c8: Pull complete
db83d36ad2b5: Pull complete
febd2a84bb58: Pull complete
ab2b03209e6c: Pull complete
9f33a63fc3d0: Pull complete
Digest: sha256:b71218312c693bc06f119339164bdd0b7e7810c7cba78e180662141f88beadbd
Status: Downloaded newer image for experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
experimental-registry.blacksmith.sh:5000/integration-test-onyx-model-server:test-17965986191
All Docker images pulled successfully
Error response from daemon: No such image: experimental-registry.blacksmith.sh:5000/integration-test-onyx-backend:test-17965986191
Error: Process completed with exit code 1.

16 more jobs failed (See summary below for more details)

2 jobs failed running on non-Blacksmith runners.


Summary: 6 successful workflows, 3 failed workflows

Last updated: 2025-09-24 04:20:54 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants