Frequent fequest timeouts in OEmbedService's fetchPage

[hshiozawa]

Issue Summary

The request timeout (2000ms) for fetchPage in OEmbedService is too short. When trying to fetch large pages (such as Amazon or Rakuten which is one of largest e-commerce web site), the request often times out, preventing the creation of embed cards.

Steps to Reproduce

1. Paste amazon or rakuten link (https://www.amazon.com/Harry-Potter-Sorcerers-Stone-Book/dp/1338878921)
2. Timeout is occured.

Case 1 (Amazon)
https://github.com/user-attachments/assets/d414f138-73ca-41fc-b9fb-a83d69576cbf

Case 2 (Rakuten)
https://github.com/user-attachments/assets/3c030544-97a2-4bb8-8928-61049c0e3a5f

Ghost Version

main branch (be463e1)

Node.js Version

v20.18.1

How did you install Ghost?

yarn dev (from source code)

Database type

MySQL 8

Browser & OS version

macOS 15.3.2, Google Chrome 134.0.6998.118（Official Build）（arm64）

Relevant log / error output

[ghost] [2025-03-21 08:08:36] ERROR Encountered error when fetching oembed
[ghost] 
[ghost] Encountered error when fetching oembed
[ghost] Error ID:
[ghost]     b053d5d0-062b-11f0-8ddc-e9b316b17ab6
[ghost] Error Code: 
[ghost]     ETIMEDOUT
[ghost] ----------------------------------------
[ghost] RequestError: Timeout awaiting 'request' for 2000ms
[ghost]     at OEmbedService.fetchOembedDataFromUrl (/Users/hajime.shiozawa/sandbox/Ghost/ghost/core/core/server/services/oembed/OEmbedService.js:547:27)
[ghost]     at ClientRequest.<anonymous> (/Users/hajime.shiozawa/sandbox/Ghost/node_modules/got/dist/source/core/index.js:970:65)
[ghost]     at Object.onceWrapper (node:events:633:26)
[ghost]     at ClientRequest.emit (node:events:530:35)
[ghost]     at origin.emit (/Users/hajime.shiozawa/sandbox/Ghost/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)
[ghost]     at emitErrorEvent (node:_http_client:101:11)
[ghost]     at TLSSocket.socketErrorListener (node:_http_client:504:5)
[ghost]     at TLSSocket.emit (node:events:518:28)
[ghost]     at emitErrorNT (node:internal/streams/destroy:169:8)
[ghost]     at emitErrorCloseNT (node:internal/streams/destroy:128:3)
[ghost]     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
[ghost]     at Timeout.timeoutHandler [as _onTimeout] (/Users/hajime.shiozawa/sandbox/Ghost/node_modules/got/dist/source/core/utils/timed-out.js:36:25)
[ghost]     at listOnTimeout (node:internal/timers:583:11)
[ghost]     at process.processTimers (node:internal/timers:519:7)
[ghost] 
[ghost] [2025-03-21 08:08:36] ERROR "GET /ghost/api/admin/oembed/?url=https%3A%2F%2Fwww.amazon.com%2FHarry-Potter-Sorcerers-Stone-Book%2Fdp%2F1338878921" 422 9191ms
[ghost] 
[ghost] No provider found for supplied URL.
[ghost] "https://www.amazon.com/Harry-Potter-Sorcerers-Stone-Book/dp/1338878921"
[ghost] Error ID:
[ghost]     b053fce0-062b-11f0-8ddc-e9b316b17ab6
[ghost] ----------------------------------------
[ghost] ValidationError: No provider found for supplied URL.
[ghost]     at OEmbedService.unknownProvider (/Users/hajime.shiozawa/sandbox/Ghost/ghost/core/core/server/services/oembed/OEmbedService.js:101:15)
[ghost]     at OEmbedService.fetchOembedDataFromUrl (/Users/hajime.shiozawa/sandbox/Ghost/ghost/core/core/server/services/oembed/OEmbedService.js:553:25)
[ghost]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[ghost]     at async getResponse (/Users/hajime.shiozawa/sandbox/Ghost/ghost/api-framework/lib/pipeline.js:259:34)
[ghost]     at async ImplWrapper (/Users/hajime.shiozawa/sandbox/Ghost/ghost/api-framework/lib/pipeline.js:264:30)
[ghost]     at async Http (/Users/hajime.shiozawa/sandbox/Ghost/ghost/api-framework/lib/http.js:70:28)
[ghost]

Code of Conduct

• I agree to be friendly and polite to people in this repository

[hshiozawa]

Increasing the request timeout to 5 seconds at the following line resolved the issue:

Ghost/ghost/core/core/server/services/oembed/OEmbedService.js

Line 192 in be463e1

timeout: 2000,

amazon.mov

For some large websites, a 2-second request timeout feels too short. What do you think?