Skip to content

Adding support for Artifactory integration using Bearer token #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Adding support for Artifactory integration using Bearer token #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7fa1a20
Adding support for authentication using API/Bearer token
Rampai94 Oct 2, 2023
46ea0a5
Removed token getting exposed in logger
Rampai94 Oct 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,15 @@ export MVN_SETTINGS_REPO_USERNAME=username
export MVN_SETTINGS_REPO_PASSWORD=password
```

#### Remote https repo using API/Bearer token

The following environment variables can be set to have the Maven extension above automatically add both the repository (could be otherwise in `pom.xml` or in `settings.xml`) and the authentication for the server (would have to be configured in `settings.xml` without this extension):

```
export MVN_SETTINGS_REPO_URL=https://repo.myorg.com/path/to/repo
export MVN_SETTINGS_REPO_API_TOKEN=apiToken
```

#### Remote https repo without authentication

For the case no authentication is necessary, setting only one env variable is sufficient:
Expand All @@ -61,7 +70,7 @@ export MVN_SETTINGS_REPO_URL=https://repo.myorg.com/path/to/repo_no_auth

For this case, no virtual `server` entry is generated for this server.

#### Using multiple repositories
#### Using multiple repositories using credentials

It is also possible to use multiple repositories:

Expand All @@ -78,6 +87,21 @@ export MVN_SETTINGS_REPO_NAME2_PASSWORD=password2

For this case two repositories and two virtual server entries for are created. The order can be important for performance reasons, the repositories are added in natural order of their names (alphabetical).

#### Using multiple repositories using API/Bearer tokens

It is also possible to use multiple repositories using identity tokens:

```
# REPO_NAME1
export MVN_SETTINGS_REPO_NAME1_URL=https://repo.myorg.com/path/to/repo
export MVN_SETTINGS_REPO_NAME1_API_TOKEN=apiToken1
# REPO_NAME2
export MVN_SETTINGS_REPO_NAME2_URL=https://repo.myorg.com/path/to/repo
export MVN_SETTINGS_REPO_NAME2_API_TOKEN=apiToken2
```

For this case two repositories and two virtual server entries for are created. The order can be important for performance reasons, the repositories are added in natural order of their names (alphabetical).

#### Using file repositories

As generally true for Maven repositories, it is also possible to use file urls. To reference a file repository within the build repository itself, use the property `maven.multiModuleProjectDirectory` in the value of `MVN_SETTINGS_REPO_URL`, e.g. `MVN_SETTINGS_REPO_URL=file://${maven.multiModuleProjectDirectory}/vendor1/repository`. If the directory `.mvn/repository` exists, it is automatically added as file repository.
Expand Down Expand Up @@ -116,6 +140,12 @@ An example file might look like this
-DMVN_SETTINGS_REPO_NAME1_URL=https://repo.myorg.com/path/to/repo -DMVN_SETTINGS_REPO_NAME1_USERNAME=username1 -DMVN_SETTINGS_REPO_NAME1_PASSWORD=password1
```

or

```
-DMVN_SETTINGS_REPO_NAME1_URL=https://repo.myorg.com/path/to/repo -DMVN_SETTINGS_REPO_NAME1_API_TOKEN=apiToken1
```

Such a configuration can be distributed through the SCM along with the code (in case the values are not supposed to be treated as secrets).

## Usage with Adobe Experience Manager Cloud Manager
Expand Down Expand Up @@ -149,6 +179,18 @@ aio cloudmanager:set-pipeline-variables \
MVN_SETTINGS_REPO_PASSWORD <REPO_PASSWORD>
```

or

```
aio cloudmanager:set-pipeline-variables \
<PIPELINE_ID> \
--programId=<PROGRAM_ID> \
--variable \
MVN_SETTINGS_REPO_URL <REPO_URL> \
--secret \
MVN_SETTINGS_REPO_API_TOKEN <REPO_API_TOKEN>
```

The parameters `<PIPELINE_ID>` and `<PROGRAM_ID>` can be derived from URLs when browsing the Cloud Manager. The call needs to be made for each pipeline as set up in cloud manager (all non-prod and the prod pipeline).

See also official [Adobe documentation](https://docs.adobe.com/content/help/en/experience-manager-cloud-service/onboarding/getting-access/creating-aem-application-project.html#pipeline-variables) and [reference on GitHub](https://github.com/adobe/aio-cli-plugin-cloudmanager#aio-cloudmanagerset-pipeline-variables-pipelineid)
162 changes: 107 additions & 55 deletions ...n/java/biz/netcentric/maven/extension/repofromenv/FromEnvReposConfigurationProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,17 @@
import org.apache.maven.settings.Mirror;
import org.apache.maven.settings.Server;
import org.codehaus.plexus.logging.Logger;
import org.codehaus.plexus.util.xml.Xpp3Dom;

/**
* <p>
* Adds Maven repositories as found in environment variables or system properties to execution environment.
* Adds Maven repositories as found in environment variables or system
* properties to execution environment.
* </p>
*
* <p>
* <strong>This module is used by configuration only (not via Java API), see <a href=
* <strong>This module is used by configuration only (not via Java API), see
* <a href=
* "https://github.com/Netcentric/maven-ext-repos-from-env/blob/develop/README.md">https://github.com/Netcentric/maven-ext-repos-from-env/blob/develop/README.md</a>
* </strong>
* </p>
Expand All @@ -49,17 +52,18 @@ public class FromEnvReposConfigurationProcessor implements ConfigurationProcesso
static final String KEY_SUFFIX_URL = "_URL";
static final String KEY_SUFFIX_USERNAME = "_USERNAME";
static final String KEY_SUFFIX_PASSWORD = "_PASSWORD";
static final String KEY_SUFFIX_API_TOKEN = "_API_TOKEN";

static final String PROFILE_ID_REPOSITORIES_FROM_ENV = "repositoriesFromSysEnv";
static final String REPO_ID_PREFIX = "sysEnvRepo";

static final String KEY_MVN_SETTINGS_REPO_VERBOSE = "MVN_SETTINGS_REPO_LOG_VERBOSE";
static final String KEY_ENV_REPOS_FIRST = "MVN_SETTINGS_ENV_REPOS_FIRST";

static final String VAR_EXPR_MULTIMODULE_PROJECT_DIR = "${"+MavenCli.MULTIMODULE_PROJECT_DIRECTORY+"}";
static final String VAR_EXPR_MULTIMODULE_PROJECT_DIR = "${" + MavenCli.MULTIMODULE_PROJECT_DIRECTORY + "}";
static final String IMPLICIT_FILE_REPO_PATH = ".mvn/repository";
static final String IMPLICIT_FILE_REPO_ID = "repository-in-mvn-ext-folder";

static final String KEY_DISABLE_BYPASS_MIRRORS = "MVN_DISABLE_BYPASS_MIRRORS";

@Inject
Expand All @@ -69,27 +73,30 @@ public class FromEnvReposConfigurationProcessor implements ConfigurationProcesso

@Override
public void process(CliRequest cliRequest) throws Exception {
Map<String, String> systemProperties = System.getProperties().entrySet().stream().collect(Collectors.toMap(e -> (String) e.getKey(), e -> (String) e.getValue()));
Map<String, String> systemProperties = System.getProperties().entrySet().stream()
.collect(Collectors.toMap(e -> (String) e.getKey(), e -> (String) e.getValue()));
CompositeMap<String, String> configurationMap = new CompositeMap<>(systemProperties, System.getenv());
isVerbose = Boolean.parseBoolean(configurationMap.get(KEY_MVN_SETTINGS_REPO_VERBOSE));
boolean envReposFirst = Boolean.parseBoolean(configurationMap.get(KEY_ENV_REPOS_FIRST));
boolean disableBypassMirrors = Boolean.parseBoolean(configurationMap.get(KEY_DISABLE_BYPASS_MIRRORS));
List<RepoFromEnv> reposFromEnv = getReposFromConfiguration(configurationMap, cliRequest.getMultiModuleProjectDirectory());

List<RepoFromEnv> reposFromEnv = getReposFromConfiguration(configurationMap,
cliRequest.getMultiModuleProjectDirectory());

addImplicitFileRepo(reposFromEnv, cliRequest.getMultiModuleProjectDirectory());

configureMavenExecution(cliRequest.getRequest(), reposFromEnv, disableBypassMirrors, envReposFirst);

}

void configureMavenExecution(MavenExecutionRequest request, List<RepoFromEnv> reposFromEnv, boolean disableBypassMirrors, boolean envReposFirst) {
void configureMavenExecution(MavenExecutionRequest request, List<RepoFromEnv> reposFromEnv,
boolean disableBypassMirrors, boolean envReposFirst) {
if (!reposFromEnv.isEmpty()) {

logRepositoriesAndMirrors(request);

Profile repositoriesFromEnv = new Profile();

// add the maven repos
// add the maven repos
for (RepoFromEnv repoFromEnv : reposFromEnv) {

repositoriesFromEnv.addRepository(getRepository(repoFromEnv));
Expand All @@ -99,22 +106,28 @@ void configureMavenExecution(MavenExecutionRequest request, List<RepoFromEnv> re
if (repoFromEnv.getUsername() != null) {
request.addServer(getServer(repoFromEnv));
}
if (repoFromEnv.getApiToken() != null) {
request.addServer(getServerUsingApiToken(repoFromEnv));
}
}

// activate profile
repositoriesFromEnv.setId(PROFILE_ID_REPOSITORIES_FROM_ENV);
if(envReposFirst) {
logMessage("Repos from this extension are queried *before* default repositories from settings.xml (" + KEY_ENV_REPOS_FIRST + "=true)");
if (envReposFirst) {
logMessage("Repos from this extension are queried *before* default repositories from settings.xml ("
+ KEY_ENV_REPOS_FIRST + "=true)");
request.addProfile(repositoriesFromEnv);
} else {
logMessage("Repos from this extension are queried *after* default repositories from settings.xml (" + KEY_ENV_REPOS_FIRST + "=false)");
logMessage("Repos from this extension are queried *after* default repositories from settings.xml ("
+ KEY_ENV_REPOS_FIRST + "=false)");
request.getProfiles().add(0, repositoriesFromEnv);
}

request.addActiveProfile(PROFILE_ID_REPOSITORIES_FROM_ENV);

if (!disableBypassMirrors) {
bypassMirrorsForRepositoryIds(request, reposFromEnv.stream().map(RepoFromEnv::getId).collect(Collectors.toList()));
bypassMirrorsForRepositoryIds(request,
reposFromEnv.stream().map(RepoFromEnv::getId).collect(Collectors.toList()));
}
}
}
Expand All @@ -128,7 +141,7 @@ private void bypassMirrorsForRepositoryIds(MavenExecutionRequest request, List<S
logMessage("Reconfigured mirror " + mirror.getId() + " to only act as mirror of " + mirrorOf);
}
}

private void logMessage(String msg) {
if (isVerbose) {
logger.info(msg);
Expand All @@ -143,20 +156,21 @@ private void logRepositoriesAndMirrors(MavenExecutionRequest request) {
List<ArtifactRepository> pluginRepositories = request.getPluginArtifactRepositories();
List<Mirror> mirrors = request.getMirrors();
Function<? super ArtifactRepository, ? extends String> repoMapper = (ArtifactRepository repo) -> {
return repo.getId() + "(" + repo.getUrl() + ",releases:" + repo.getReleases() + ",snapshots:" + repo.getSnapshots() + ")";
return repo.getId() + "(" + repo.getUrl() + ",releases:" + repo.getReleases() + ",snapshots:"
+ repo.getSnapshots() + ")";
};
logMessage("Configured in settings.xml:\nrepositores:\n " +
repositories.stream().map(repoMapper).collect(Collectors.joining(", \n"))
+ (pluginRepositories != null && !pluginRepositories.isEmpty()
? "\nplugin repositories:\n "
+ pluginRepositories.stream().map(repoMapper).collect(Collectors.joining(", \n"))
: "")
+ (mirrors != null && !mirrors.isEmpty()
? "\nmirrors:\n "
+ mirrors.stream().map(
mirror -> mirror.getId() + "(mirrorOf:" + mirror.getMirrorOf() + ",url=" + mirror.getUrl() + ")")
.collect(Collectors.joining(", \n"))
: ""));
logMessage(
"Configured in settings.xml:\nrepositores:\n "
+ repositories.stream().map(repoMapper).collect(Collectors.joining(", \n"))
+ (pluginRepositories != null && !pluginRepositories.isEmpty() ? "\nplugin repositories:\n "
+ pluginRepositories.stream().map(repoMapper).collect(Collectors.joining(", \n")) : "")
+ (mirrors != null && !mirrors.isEmpty()
? "\nmirrors:\n "
+ mirrors.stream()
.map(mirror -> mirror.getId() + "(mirrorOf:" + mirror.getMirrorOf()
+ ",url=" + mirror.getUrl() + ")")
.collect(Collectors.joining(", \n"))
: ""));

}

Expand All @@ -182,59 +196,97 @@ private Server getServer(RepoFromEnv repoFromEnv) {
return server;
}

/**
* @author Rampai94
*
*/
private Server getServerUsingApiToken(RepoFromEnv repoFromEnv) {
Server server = new Server();
Xpp3Dom value = new Xpp3Dom("value");
value.setValue("Bearer " + repoFromEnv.getApiToken());
Xpp3Dom name = new Xpp3Dom("name");
name.setValue("Authorization");
Xpp3Dom property = new Xpp3Dom("property");
property.addChild(name);
property.addChild(value);
Xpp3Dom httpHeaders = new Xpp3Dom("httpHeaders");
httpHeaders.addChild(property);
Xpp3Dom serverConfiguration = new Xpp3Dom("configuration");
serverConfiguration.addChild(httpHeaders);
Xpp3Dom wagonProvider = new Xpp3Dom("wagonProvider");
wagonProvider.setValue("httpClient");
serverConfiguration.addChild(wagonProvider);
server.setId(repoFromEnv.getId());
server.setConfiguration(serverConfiguration);
return server;
}

List<RepoFromEnv> getReposFromConfiguration(Map<String, String> configMap, File reactorRootDir) {

List<RepoFromEnv> reposFromEnv = configMap.keySet().stream()
.filter(Objects::nonNull).sorted()
List<RepoFromEnv> reposFromEnv = configMap.keySet().stream().filter(Objects::nonNull).sorted()
.filter(key -> key.startsWith(KEY_PREFIX_MVN_SETTINGS_REPO) && key.endsWith(KEY_SUFFIX_URL))
.map(key -> key.substring(KEY_PREFIX_MVN_SETTINGS_REPO.length(), key.length() - KEY_SUFFIX_URL.length()))
.map(key -> key.substring(KEY_PREFIX_MVN_SETTINGS_REPO.length(),
key.length() - KEY_SUFFIX_URL.length()))
.map(repoEnvNameInKey -> {
// for the case the short form without like MVN_SETTINGS_REPO_URL is used, repoEnvNameInKey is and empty string
// for the case a key like MVN_SETTINGS_REPO_MYCOMP1_URL is used, the id is sysEnvRepoMYCOMP1
String id = REPO_ID_PREFIX + (repoEnvNameInKey.isEmpty() ? "" : repoEnvNameInKey.replaceFirst("_", ""));
// for the case the short form without like MVN_SETTINGS_REPO_URL is used,
// repoEnvNameInKey is and empty string
// for the case a key like MVN_SETTINGS_REPO_MYCOMP1_URL is used, the id is
// sysEnvRepoMYCOMP1
String id = REPO_ID_PREFIX
+ (repoEnvNameInKey.isEmpty() ? "" : repoEnvNameInKey.replaceFirst("_", ""));
String urlKey = KEY_PREFIX_MVN_SETTINGS_REPO + repoEnvNameInKey + KEY_SUFFIX_URL;
String url = configMap.get(urlKey);
String usernameKey = KEY_PREFIX_MVN_SETTINGS_REPO + repoEnvNameInKey + KEY_SUFFIX_USERNAME;
String username = configMap.get(usernameKey);
String passwordKey = KEY_PREFIX_MVN_SETTINGS_REPO + repoEnvNameInKey + KEY_SUFFIX_PASSWORD;
String password = configMap.get(passwordKey);
String apiTokenKey = KEY_PREFIX_MVN_SETTINGS_REPO + repoEnvNameInKey + KEY_SUFFIX_API_TOKEN;
String apiToken = configMap.get(apiTokenKey);
if (!isBlank(username) && isBlank(password)) {
throw new IllegalArgumentException("If property " + usernameKey + " is set, password property " + passwordKey
+ " also has to be set along with it");
throw new IllegalArgumentException("If property " + usernameKey + " is set, password property "
+ passwordKey + " also has to be set along with it");
}
if (!isBlank(url)) {
if (url.contains(VAR_EXPR_MULTIMODULE_PROJECT_DIR)) {
String reactorRootDirPath = reactorRootDir.toURI().getPath(); // only use forward slashes on
// all platforms
url = url.replace(VAR_EXPR_MULTIMODULE_PROJECT_DIR, reactorRootDirPath);
logMessage("Replaced " + VAR_EXPR_MULTIMODULE_PROJECT_DIR + " in url with "
+ reactorRootDirPath);
}
if (isBlank(username)) {
logMessage("Repository " + url + " has NOT configured credentials (env variables " + usernameKey + " and "
+ passwordKey + " are missing)");
logMessage("Repository " + url + " has NOT configured credentials (env variables "
+ usernameKey + " and " + passwordKey + " are missing)");
}
if(url.contains(VAR_EXPR_MULTIMODULE_PROJECT_DIR)) {
String reactorRootDirPath = reactorRootDir.toURI().getPath(); // only use forward slashes on all platforms
url = url.replace(VAR_EXPR_MULTIMODULE_PROJECT_DIR, reactorRootDirPath);
logMessage("Replaced "+VAR_EXPR_MULTIMODULE_PROJECT_DIR+" in url with "+reactorRootDirPath);
if (isBlank(apiToken)) {
logMessage("Repository " + url + " has NOT configured token (env variable " + apiTokenKey
+ " is missing)");
} else {
return new RepoFromEnv(id, url, apiToken);
}

return new RepoFromEnv(id, url, username, password);
} else {
logMessage("Property/Variable " + urlKey + " is configured but blank, not adding a repository");
return null;
}
})
.filter(Objects::nonNull)
.collect(Collectors.toList());

reposFromEnv.stream().forEach(repoFromEnv ->
// minimal line that we always log directly (regardless of MVN_SETTINGS_REPO_LOG_VERBOSE or -X parameter)
logger.info("Repository added from system properties or environment variables: " + repoFromEnv.getUrl() + " (id: " + repoFromEnv.getId() + (repoFromEnv.getUsername() != null ? " user: " + repoFromEnv.getUsername() : "") + ")")
);

}).filter(Objects::nonNull).collect(Collectors.toList());

reposFromEnv.stream().forEach(repoFromEnv ->
// minimal line that we always log directly (regardless of
// MVN_SETTINGS_REPO_LOG_VERBOSE or -X parameter)
logger.info(
"Repository added from system properties or environment variables: " + repoFromEnv.getUrl() + " (id: "
+ repoFromEnv.getId()
+ (repoFromEnv.getUsername() != null ? " user: " + repoFromEnv.getUsername() : "")
+ ")"));

return reposFromEnv;

}


void addImplicitFileRepo(List<RepoFromEnv> reposFromEnv, File multiModuleProjectDirectory) {
File implicitRepo = new File(multiModuleProjectDirectory, IMPLICIT_FILE_REPO_PATH);
if(implicitRepo.exists()) {
if (implicitRepo.exists()) {
reposFromEnv.add(0, new RepoFromEnv(IMPLICIT_FILE_REPO_ID, implicitRepo.toURI().toString(), null, null));
logger.info("Implicit file repository added for directory " + IMPLICIT_FILE_REPO_PATH);
}
Expand Down
Loading