Skip to content

feat(openai): support openai v5 #5816

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from

Conversation

sabrenner
Copy link
Collaborator

@sabrenner sabrenner commented Jun 3, 2025

What does this PR do?

With the release of OpenAI v5, the only thing we needed to account for was some del methods changing their name to delete. Otherwise, our instrumentation is largely unchanged.

This PR mostly fixes tests. There were a few things impacted:

  1. We removed our use of nock within OpenAI, as upgrading it caused a cascade of other issues. Instead, we simplify it into a custom server implementation to point OpenAI at (via baseURL/basePath). This also reduces line count/redundant nock calls in different test files 🙌
  2. The beta feature we supported is only for OpenAI v4 and does not need to be tested for v5 (as it does not exist anymore)
  3. There was a small File error, I think just on my machine which runs Node 18 by default - it likely won't impact CI as I think we run on latest but I wanted to handle it regardless.

Motivation

Support OpenAI v5 and unblock broken CI

MLOB-2927

Copy link

github-actions bot commented Jun 3, 2025

Overall package size

Self size: 9.56 MB
Deduped: 104.5 MB
No deduping: 105.02 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.6.0 | 30.47 MB | 30.47 MB | | @datadog/native-appsec | 8.5.2 | 19.33 MB | 19.34 MB | | @datadog/pprof | 5.8.0 | 12.55 MB | 12.92 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.14.0 | 120.58 kB | 841.68 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.9 | 25.11 kB | 25.11 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

Copy link

codecov bot commented Jun 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.18%. Comparing base (cc2b8a6) to head (d90cb85).
Report is 106 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5816      +/-   ##
==========================================
+ Coverage   79.15%   79.18%   +0.03%     
==========================================
  Files         522      522              
  Lines       24060    24060              
==========================================
+ Hits        19044    19052       +8     
+ Misses       5016     5008       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter
Copy link

pr-commenter bot commented Jun 3, 2025

Benchmarks

Benchmark execution time: 2025-06-04 13:52:57

Comparing candidate commit d90cb85 in PR branch sabrenner/change-openai-mocks with baseline commit cc2b8a6 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1269 metrics, 54 unstable metrics.

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Jun 3, 2025

Datadog Report

Branch report: sabrenner/change-openai-mocks
Commit report: c3f5f5d
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 1257 Passed, 0 Skipped, 26m 11.23s Total Time

@sabrenner sabrenner changed the title wip(openai): support openai v5 feat(openai): support openai v5 Jun 4, 2025
@sabrenner
Copy link
Collaborator Author

once again this work has gone out of date due to reworking mocking to a more shareable solution introduced in #5951, so I will reopen this PR separately

@sabrenner sabrenner closed this Jun 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant