Add support for publishing maven-metadata.xml #1260
Conversation
|
@shs96c we added a flag to keep it backwards compatible, but we are of the opinion the default should be true. |
There was a problem hiding this comment.
We tested this locally using:
bazel run //tests/integration/pom_file:pom-example-with-runtime-dep.publish --define "maven_repo=file:///tmp/.m2"
(Note: we had to have set the publish_maven_metadata attr for the target)
Internally at Confluent, we tested the publishing via HTTP to AWS Code Artifactory as well.
| ByteArrayOutputStream os = new ByteArrayOutputStream(); | ||
| new MetadataXpp3Writer().write(os, metadata); | ||
| Files.write(newMavenMetadataXml, os.toByteArray()); | ||
| return upload(mavenMetadataUrl, credentials, newMavenMetadataXml); |
There was a problem hiding this comment.
nit: upload could probably take InputStream but I didn't want to change too much of the code;
Open to doing another pass after on this file if you are open to it.
| * Attempts to download the file at the given targetUrl. | ||
| * Valid protocols are: http(s) & file at the moment. | ||
| */ | ||
| private static CompletableFuture<Optional<String>> download( |
There was a problem hiding this comment.
nit: could move this to a separate file Downloaders to keep this file a bit smaller.
|
The test failures look genuine. I think the upload of the metadata should be disabled by default. Both Nexus and Artifactory can automatically calculate the maven metadata for you, and it's not always clear that every |
Maven repositories normally have a maven-metadata.xml file that indicate to the Maven
system what versions are available and which is to be considered the latest version.
```xml
<metadata modelVersion="1.1.0">
<groupId>com.mycompany.app</groupId>
<artifactId>my-app</artifactId>
<versioning>
<latest>1.0</latest>
<release>1.0</release>
<versions>
<version>1.0</version>
</versions>
<lastUpdated>20200731090423</lastUpdated>
</versioning>
</metadata>
```
At Confluent, we use AWS Code Artifactory which does not mark a Maven package as "published" unless
a new maven-metadata.xml is uploaded indicating so.
* Add support for reading existing maven-metadata.xml
* Add support for adding the new version to the metadata object
* Add support to upload the file for http & file protocols
* Add small test cases to validate SerDe for Maven metadata object from XML
Co-authored-by: Vince Rose <vrose@confluent.io>
Co-authored-by: Na Lou <nlou@confluent.io>
* Add s3 handling for maven-metadata Co-authored-by: jainruchir <rujain@confluent.io> * code review --------- Co-authored-by: jainruchir <rujain@confluent.io>
vinnybod
force-pushed
the
upload-maven-metadata
branch
from
d982843 to
81012bc
Compare
|
This also adds support for publishing to S3 which requires the client to update the maven-metadata.xml 🙂 @shs96c I addressed the broken test and it is disabled by default. |
| return CompletableFuture.supplyAsync( | ||
| () -> { | ||
| try { | ||
| HttpTransport httpTransport = new NetHttpTransport(); |
There was a problem hiding this comment.
All versions of the JDK that we support have the native HTTP handler. We already use this in the HttpDownloader class, which also handles things like using the .netrc file if there is one. Perhaps use that? We could do that in a follow-up PR, but what I don't want to happen is for us to have a proliferation of http clients in this repo.
| @@ -0,0 +1,74 @@ | |||
| package com.github.bazelbuild.rules_jvm_external.maven; | |||
|
|
|||
| import static org.junit.Assert.*; | |||
There was a problem hiding this comment.
Looks like you need to run bazel run scripts:format
|
Could you please run |
|
I removed the extra http client dependency. Tests are passing, but I need to do a bit more end to end testing since I just merged ~4 months of commits from master. |
Maven repositories normally have a maven-metadata.xml file that indicate to the Maven system what versions are available and which is to be considered the latest version.
At Confluent, we use AWS Code Artifactory which does not mark a Maven package as "published" unless a new maven-metadata.xml is uploaded indicating so.
Co-authored-by: Vince Rose vrose@confluent.io
Co-authored-by: Na Lou nlou@confluent.io