-
Notifications
You must be signed in to change notification settings - Fork 331
Added View API and Rollback API for Versioned Security Configurations #5431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Divyanshi Chouksey <divvv@amazon.com>
private ValidationResult<SecurityConfiguration> viewAllVersions() throws IOException { | ||
SecurityConfigVersionDocument doc = versionsLoader.loadFullDocument(); | ||
|
||
return ValidationResult.error(OK, (builder, params) -> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why error? Should it be ValidationResult.success
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1. Should be ValidationResult.success here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ValidationResult.success only takes a SecurityConfiguration as content object, so in order to build a custom JSON I had to use this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.error() is not the correct usage. You can simply create response class that extends ToXContentObject and then return an instance of that object here.
You can refer this file: https://github.com/opensearch-project/security/pull/5389/files#diff-157c9f69cf6c49e618db3748a7803470432b527ce78fe5251228d093daa5483e
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We cannot use ValidationResult.error
, it is very misleading and wrong pattern
builder.field("security_configs"); | ||
Map<String, Object> plainConfigs = new LinkedHashMap<>(); | ||
for (Map.Entry<String, SecurityConfigVersionDocument.HistoricSecurityConfig<?>> entry : ver.getSecurity_configs().entrySet()) { | ||
Map<String, Object> scMap = new LinkedHashMap<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Update variable name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
var maybeVer = doc.getVersions().stream().filter(v -> versionId.equals(v.getVersion_id())).findFirst(); | ||
|
||
if (maybeVer.isEmpty()) { | ||
return ValidationResult.error(NOT_FOUND, payload(NOT_FOUND, "Version " + versionId + " not found")); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this here? We seem to be doing in rollbackCommon() as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack, doing it once.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This might make more sense to keep in rollbackCommon() since you would need to check this anyway for any caller of this function
throw new IOException("Rollback aborted: Could not fetch current config for cType '" + cTypeName + "'"); | ||
} | ||
|
||
SecurityDynamicConfiguration<?> sdc = SecurityDynamicConfiguration.empty(cType); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: rename or expand it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since everything is in context of security you can name the variables as updateConfig
or rollBackConfig
since this indicates a new entry to be created from existing entry.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack!
CType<?> cType = CType.fromString(cTypeName); | ||
if (cType == null) { | ||
throw new IOException("Rollback aborted: Unknown config type '" + cTypeName + "' found in version"); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: This check needs to be done before checking data for that cType?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also exception type IOException
doesn't seem appropriate. IllegalStateException
?
Check in other places as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should never be null since we are responsible for storing the information. A Ctype will be unknown only when security plugin moves away from current CTypes. Also this should throw NPE since we use cTypes as keys. Another option would be NoSuchFieldException.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack!
sdc.setPrimaryTerm(currentConfig.getPrimaryTerm()); | ||
|
||
for (Map.Entry<String, ?> configEntry : sc.getConfigData().entrySet()) { | ||
if ("_meta".equals(configEntry.getKey())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we skip _meta
? Good to add comments for these kinds for readers of code to understand
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1.
What is the expected behavior when config version changes? i.e. roles change from v7 to v8.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including _meta
creates a duplicate entry for the field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently we have rollback functionality for same versions of config types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including _meta creates a duplicate entry for the field
Why? which other code is adding _meta
field?
for (Map.Entry<CType<?>, SecurityDynamicConfiguration<?>> entry : configsToApply.entrySet()) { | ||
AbstractApiAction.saveAndUpdateConfigsAsync( | ||
securityApiDependencies, | ||
client, | ||
entry.getKey(), | ||
entry.getValue(), | ||
new ActionListener<IndexResponse>() { | ||
@Override | ||
public void onResponse(IndexResponse r) { | ||
latch.countDown(); | ||
} | ||
|
||
@Override | ||
public void onFailure(Exception e) { | ||
log.error("Rollback: failed to write config for cType={} : {}", entry.getKey().toLCString(), e.getMessage(), e); | ||
latch.countDown(); | ||
} | ||
} | ||
); | ||
log.info("Rollback: wrote config data for cType={}", entry.getKey().toLCString()); | ||
} | ||
|
||
if (!latch.await(20, java.util.concurrent.TimeUnit.SECONDS)) { | ||
throw new IOException("Timeout while writing rolled-back configs"); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Can we move this to another function?
- Timeout value can be constant
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Magic variables should be declared as class level constants.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
|
||
for (Map.Entry<CType<?>, SecurityDynamicConfiguration<?>> entry : backups.entrySet()) { | ||
try { | ||
AbstractApiAction.saveAndUpdateConfigs(securityApiDependencies, client, entry.getKey(), entry.getValue()).actionGet(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- High: This won't update cache
- Can we test for this as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have we tested the sync vs async behavior here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, async updates cache and triggers update for version index as well, sync doesn't.
return builder; | ||
} | ||
|
||
private boolean isConfigEqual(SecurityDynamicConfiguration<?> currentConfig, SecurityDynamicConfiguration<?> targetConfig) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: This can be util function I think. Its reusable by any other code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
} | ||
} | ||
|
||
private void rollbackConfigsToSecurityIndex(SecurityConfigVersionDocument.Version<?> versionData) throws IOException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Variable names are not clear. Reading is bit tough here. Need to go up and down to know what variable is.
Rename such that names give meaning of that it is intended for. For ex: currentConfig used here is a good naming, easy to understand
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
return ValidationResult.error(NOT_FOUND, payload(NOT_FOUND, "Version " + versionId + " not found")); | ||
} | ||
|
||
SecurityConfigVersionsLoader.sortVersionsById(doc.getVersions()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need sorting here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not required, removed.
In the context of the changes I understand why this operation will create a v3, but I do think its a bit confusing that a rollback creates a new "version". |
Is |
return PLUGINS_PREFIX + "/api"; | ||
} | ||
|
||
private String RollbackBase() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If these are all constants can we declare them as constant fields at the top of this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack!
private static final Logger log = LogManager.getLogger(RollbackVersionApiAction.class); | ||
|
||
private static final List<Route> routes = addRoutesPrefix( | ||
ImmutableList.of(new Route(POST, "/rollback"), new Route(POST, "/rollback/version/{versionID}")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By passing versionID
in does this effectively support moving forward as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it make sense to introduce a dryRun
url param that would return the patch applied as its output without actually applying the patch?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's essentially rollback, providing versionID
allows users to rollback to any particular version.
Yes, we are separately working on a dryRun API for previewing the patch applied and it's impact before actually applying it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for adding these APIs. I have added few comments from my initial review.
One outstanding question I have is, do admin's have an option to choose a particular config to rollback when rolling back to previous version? I'm not exactly aware of a scenario, but hypothetically an admin may have removed a user and also updated a role simultaneously, this would bring us to the scenario I mentioned.
SSL; | ||
SSL, | ||
VIEW_VERSION, | ||
ROLLBACK_VERSION; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These two should be grouped as version? A singular permission that allows rest-admin privileges to the view and rollback. Similar to other endpoints already present here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
View API is just GET request, while rollback API is a POST request, hence kept them separate. But will look into this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just version may not sound similar to for ex: say roles because GET /role/{}
, PUT /role/{}
is intuitive to understand but GET /version/{}
, PUT /version/{}
is not that intuitive. PUT/POST /version/{}
is not updating its entity, here it is doing rolling back which is not intuitive to understand
@@ -71,6 +71,8 @@ default String build() { | |||
.put(Endpoint.ROLESMAPPING, action -> buildEndpointPermission(Endpoint.ROLESMAPPING)) | |||
.put(Endpoint.TENANTS, action -> buildEndpointPermission(Endpoint.TENANTS)) | |||
.put(Endpoint.SSL, action -> buildEndpointActionPermission(Endpoint.SSL, action)) | |||
.put(Endpoint.VIEW_VERSION, action -> buildEndpointPermission(Endpoint.VIEW_VERSION)) | |||
.put(Endpoint.ROLLBACK_VERSION, action -> buildEndpointPermission(Endpoint.ROLLBACK_VERSION)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
similar to the comment above, this should be clubbed as one permission.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
private static final Logger LOGGER = LogManager.getLogger(ViewVersionApiAction.class); | ||
|
||
private static final List<Route> routes = addRoutesPrefix( | ||
ImmutableList.of(new Route(Method.GET, "/version"), new Route(Method.GET, "/version/{versionID}")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
get alll versions endpoint should be changed to /versions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
private ValidationResult<SecurityConfiguration> viewAllVersions() throws IOException { | ||
SecurityConfigVersionDocument doc = versionsLoader.loadFullDocument(); | ||
|
||
return ValidationResult.error(OK, (builder, params) -> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1. Should be ValidationResult.success here.
private static final Logger log = LogManager.getLogger(RollbackVersionApiAction.class); | ||
|
||
private static final List<Route> routes = addRoutesPrefix( | ||
ImmutableList.of(new Route(POST, "/rollback"), new Route(POST, "/rollback/version/{versionID}")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be renamed to /version/rollback
and /version/rollback/{versionId}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
src/main/java/org/opensearch/security/dlic/rest/api/ViewVersionApiAction.java
Outdated
Show resolved
Hide resolved
|
||
if (maybeVer.isEmpty()) { | ||
return ValidationResult.error(NOT_FOUND, payload(NOT_FOUND, "Version " + versionId + " not found")); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this method is pretty much identical with the one above, see if you can club it together to re-use code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack
|
||
return ValidationResult.error(OK, (builder, params) -> { | ||
XContentBuilder inner = buildVersionsJsonBuilder(List.of(maybeVer.get())); | ||
builder.copyCurrentStructure(JsonXContent.jsonXContent.createParser(null, null, BytesReference.bytes(inner).streamInput())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is see this pattern being repeated, 2 things:
- Can you use fromXContent and toXContent instead of buildVersionsJsonBuilder and copyCurrentStructure.
- See if you can over-load implementation of payload method in Responses.java that accepts JsonContent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ack will check this out.
src/test/java/org/opensearch/security/dlic/rest/api/RollbackVersionApiActionValidationTest.java
Outdated
Show resolved
Hide resolved
src/test/java/org/opensearch/security/dlic/rest/api/RollbackVersionApiTest.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Divyanshi Chouksey <divvv@amazon.com>
We are tracking all the changes in the version index, the rollback triggers an update to the security configurations hence creating a new version to track it. |
It's specific to individual config type |
Thanks for the review! |
Signed-off-by: Divyanshi Chouksey <77962346+divyanshi-0402@users.noreply.github.com>
SSL; | ||
SSL, | ||
VIEW_VERSION, | ||
ROLLBACK_VERSION; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just version may not sound similar to for ex: say roles because GET /role/{}
, PUT /role/{}
is intuitive to understand but GET /version/{}
, PUT /version/{}
is not that intuitive. PUT/POST /version/{}
is not updating its entity, here it is doing rolling back which is not intuitive to understand
private ValidationResult<SecurityConfiguration> viewAllVersions() throws IOException { | ||
SecurityConfigVersionDocument doc = versionsLoader.loadFullDocument(); | ||
|
||
return ValidationResult.error(OK, (builder, params) -> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We cannot use ValidationResult.error
, it is very misleading and wrong pattern
private ValidationResult<SecurityConfiguration> viewSpecificVersion(String versionId) throws IOException { | ||
SecurityConfigVersionDocument doc = versionsLoader.loadFullDocument(); | ||
|
||
var maybeVer = doc.getVersions().stream().filter(v -> versionId.equals(v.getVersion_id())).findFirst(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: what does maybeVer mean?
var maybeVer = doc.getVersions().stream().filter(v -> versionId.equals(v.getVersion_id())).findFirst(); | ||
|
||
if (maybeVer.isEmpty()) { | ||
return ValidationResult.error(NOT_FOUND, payload(NOT_FOUND, "Version " + versionId + " not found")); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This might make more sense to keep in rollbackCommon() since you would need to check this anyway for any caller of this function
sdc.setPrimaryTerm(currentConfig.getPrimaryTerm()); | ||
|
||
for (Map.Entry<String, ?> configEntry : sc.getConfigData().entrySet()) { | ||
if ("_meta".equals(configEntry.getKey())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including _meta creates a duplicate entry for the field
Why? which other code is adding _meta
field?
Description
Implementing View API and Rollback API for Versioned Security Configurations, build upon the changes from the PR : Versioned Security Configuration Management #5357.
API Design
View Security Configuration Versions
Purpose: View the complete history of security configuration changes, or any specific version of the security configurations.
Endpoint:
Access Control: Admin/Security Manager permissions required
Response Format for specific version:
Response Format for all versions :
Response format if the version doesn't exist :
Response if try to access with non admin role :
Rollback to desired Security Configuration Versions
Purpose : Allows rollback to immediate or any desired version of security
Endpoint:
Access Control: Admin/Security Manager permissions required
Response format to rollback to immediate previous version :
Response format to rollback to a specific version :
Response format if the version doesn't exist :
Response if try to access with non admin role :
Test details for multi-node testing:
Created two nodes :
opensearch-node1
andopensearch-node1
node 1 :
node 2 :
Rollback initiated from node-1:
Rollback initiated from node-2 :
Issues Resolved
Related to #5093
Testing
Unit Tests and Integration Tests included
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.