[RFE]: Possibility to configure URL to "Maven central"

[alixwar]

Is your feature request related to a problem? Please describe.
This is the issue that I'm facing: #5291

In my case (using version 12.1.0 of the gradle plugin):

Caused by: java.io.IOException: https://search.maven.org/solrsearch/select?q=1:8110ba5ec44be61aa96f8bd8d1c57eed0efdb7b9&wt=xml - Server status: 502 - Server reason: Bad Gateway
	at org.owasp.dependencycheck.utils.Downloader.fetchAndHandle(Downloader.java:724)
	at org.owasp.dependencycheck.utils.Downloader.fetchAndHandle(Downloader.java:685)
	at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:162)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:339)
	... 8 more

...

An error occurred while analyzing '/Users/xxxxx/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-scripting-compiler-embeddable/2.1.20/efd094224781affbc9aa8c60c5cae926517e028f/kotlin-scripting-compiler-embeddable-2.1.20.jar' (Central Analyzer).
Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.

Describe the solution you'd like
I'd like to be able to configure the plugin to use my own mirrored URL to Maven Central so that I don't have to rely on uptime for this public repository (bonus: less latency).

Describe alternatives you've considered
No alternatives

[jmsdc]

I am facing the same issue with the dependency check analysis in failure : read timeout

The search.maven.org server is often down
Maven server status: https://status.maven.org/

Checking for updates and analyzing dependencies for vulnerabilities
Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:362)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)

.......


An error occurred while analyzing '/home/ubuilder/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.9.20/6c69e7b26179394dafd3c90e630ef763b8c1a267/kotlin-stdlib-1.9.20-all.jar' (Central Analyzer): Could not connect to Central search. Analysis failed.
Could not connect to Central search. Analysis failed.
java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:362)
	at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
	at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
	at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.net.SocketTimeoutException: Read timed out

[HeliosDK]

I am also facing the same issue

13:09:46 [INFO] Analysis Started
13:10:42 [INFO] Finished Archive Analyzer (56 seconds)
13:10:42 [INFO] Finished File Name Analyzer (0 seconds)
13:10:45 [INFO] Finished Jar Analyzer (2 seconds)
13:18:33 [ERROR] Could not connect to Central search. Analysis failed.
13:18:33 java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries.
13:18:33 at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:362)
13:18:33 at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:228)
13:18:33 at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
13:18:33 at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
13:18:33 at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
13:18:33 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
13:18:33 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
13:18:33 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
13:18:33 at java.base/java.lang.Thread.run(Thread.java:1583)
13:18:33 Caused by: java.net.SocketTimeoutException: Read timed out
13:18:33 at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
13:18:33 at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
13:18:33 at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
13:18:33 at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
13:18:33 at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
13:18:33 at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
13:18:33 at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
13:18:33 at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
13:18:33 at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1461)
13:18:33 at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1066)
13:18:33 at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
13:18:33 at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
13:18:33 at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:250)
13:18:33 at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:56)
13:18:33 at org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:331)
13:18:33 at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:196)
13:18:33 at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.lambda$execute$0(InternalExecRuntime.java:236)
13:18:33 at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:791)
13:18:33 at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:233)
13:18:33 at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:120)
13:18:33 at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
13:18:33 at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:198)
13:18:33 at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
13:18:33 at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
13:18:33 at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
13:18:33 at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:150)
13:18:33 at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
13:18:33 at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
13:18:33 at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
13:18:33 at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:110)
13:18:33 at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
13:18:33 at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:183)
13:18:33 at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
13:18:33 at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
13:18:33 at org.owasp.dependencycheck.utils.Downloader.fetchAndHandle(Downloader.java:731)
13:18:33 at org.owasp.dependencycheck.utils.Downloader.fetchAndHandle(Downloader.java:701)
13:18:33 at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:162)
13:18:33 at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:339)
13:18:33 ... 8 common frames omitted
13:18:33 [WARN] An error occurred while analyzing '/tmp/dctemp46d5697b-9361-476f-9158-bd1494c16829/check17092004142145305218tmp/6/WEB-INF/lib/batch.jar' (Central Analyzer): Could not connect to Central search. Analysis failed.

[HeliosDK]

Possible that the solution is to transition to
https://central.sonatype.com/search
instead of
http://search.maven.org/#search

[alixwar]

Possible that the solution is to transition to https://central.sonatype.com/search instead of http://search.maven.org/#search

That would be a mitigation to the instability of http://search.maven.org/, not a solution

[HeliosDK]

Possible that the solution is to transition to https://central.sonatype.com/search instead of http://search.maven.org/#search

That would be a mitigation to the instability of http://search.maven.org/, not a solution

So as example:
http://search.maven.org/#search|ga|1|1%3A%222e88bd2818506573280599398f4198a04b2b2c8c%22
will lead to
https://central.sonatype.com/search?q=1:2e88bd2818506573280599398f4198a04b2b2c8c&smo=true
via redirect
but since your originate is http://search.maven.org if the site is down which it is alot se https://status.maven.org/
the plugin fails. So it makes sense to move to https://central.sonatype.com and skip the redirect.

Example code taken from
dependency-check-report1.xml line 192

[jeremylong]

Currently, the results of the two APIs differ. You can see this in other likely closed issues. Additionally, until there is published documentation about the non-legacy search API - I'm not going to migrate to it.

[HeliosDK]

So the solution is to use a solution that works if not down which seems to be the rule not exception.

[HeliosDK]

https://central.sonatype.org/faq/what-is-different-between-central-portal-and-legacy-ossrh/#self-service-migration

[clivejefferies]

I have this issue too. But for me it is slightly different. For a couple of months I have had this problem regardless of status. I am calling from CircleCI and if I use curl on one of the searches it works fine, but from DC, I always get this retry issue.

java.io.IOException: Finally failed connecting to Central search. Giving up after 7 tries. at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:370) at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:229) at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) Caused by: org.owasp.dependencycheck.utils.ForbiddenException: Forbidden access to MavenCentral https://search.maven.org/solrsearch/select?q=1:c7c5ac86a134a0f86bb6986d16ffabffb1b139ed&wt=xml - Server status: 403 - Server reason: Forbidden

Here are some of the the params I have tried to solve the issue:

./gradlew dependencyCheckAnalyze -Dhttp.agent="curl/7.81.0" -Dhttp.socket.timeout=30000 -Dhttp.connection.timeout=30000

However, nothing seems to work making this library unusable at this point.

[boardbloke]

Down again.

I've tried asking sonatype via their support desk to provide documentation to a set of APIs that achieve the same goal as the existing published APIs but which don't use seach.maven.org

I suggest anybody else wishing this was better did the same to see if that creates some traction.

[boardbloke]

Got this response from sonatype:

You have asked a question about the
search.maven.org search API.
search.maven.org is the legacy search service for Maven Central and is not receiving significant attention from the Central team. The team is currently focused on the sunset of OSSRH and supporting publishers on the new Central Publisher Portal. Once that work is complete, we intend to work to improve the search experience of https://central.sonatype.com/ and provide a dedicated API for that purpose. At that point, we will decommission
search.maven.org.

In the meantime, we have created a compatibility API https://central.sonatype.com/solrsearch/select that should fit many of the existing usecases of the legacy search API. Please try that out and let us know if it meets your needs or if there are gaps in your required functionality.

Thank you,
The Central Team

[keithboone]

I get the feeling that https://xkcd.com/2347/ is appropriate here.

What is the best place to put pressure on the central Sonatype team? Is there already an issue or feature request logged that we can upvote somewhere.

[boardbloke]

They don't have a support portal anymore as far as I know.

I emailed central-support@sonatype.com

Down again - pretty much all day today.