Cross-platform SqlColumnEncryptionCertificateStoreProvider support #3014
Conversation
* Only allow the use of the CurrentUser location for certificates. * Changed the PublishTestResults@2 step to ensure that failed tests are also published.
…t to fix problems importing these files on MacOS
…t to fix problems importing these files on MacOS
…' of https://github.com/edwardneal/SqlClient into feat/xplat-column-encryption-certificate-store-provider
Converting more unusually-embedded PFX files into embedded resources
edwardneal
force-pushed
the
feat/xplat-column-encryption-certificate-store-provider
branch
from
6966d54 to
c199ddd
Compare
Eliminating possibility of a certificate with a private key having been previously added to the user certificate store.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3014 +/- ##
==========================================
- Coverage 63.51% 60.55% -2.96%
==========================================
Files 293 286 -7
Lines 63810 63511 -299
==========================================
- Hits 40526 38461 -2065
- Misses 23284 25050 +1766
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
This has reached the point where it needs a rough review by the SqlClient team. We can make Secondly: around half of this PR's additions and a quarter of its file changes are because I moved the certificates which were previously static byte arrays in the code to .pfx files which are included within the test assembly as embedded resources. In the process, I also added a password to these .pfx files - macOS didn't seem to import them as certificates without one. Is this going to fall afoul of any security policies against checking private keys into the repo, or is it not a problem if they're only used for the tests? If all's well, I'll likely cherry-pick some of the test changes included in this PR into a new one; it should clean up the diff a little and make it easier to review. |
edwardneal
changed the title
|
I'm pretty sure the private keys and password won't pass credential scanners. Can the files be generated on the fly with random passwords? |
|
They can, and in some places they are - some of the tests launch a PowerShell script to do that. I'll work out a way to make this consistent across the tests... Edit: it looks like there are three or four different sets of hardcoded certificates; some have hardcoded passwords, some don't have passwords at all. I'm working on a common approach which does as you've suggested; that'll be a separate PR for ease of review. Second edit: I've decided to split this prerequisite into two PRs, the first of which is #3034. |
cheenamalhotra
added
the
Partner Approval Needed 🤝
| Debug.Assert(validLocations.Length == 2); | ||
| if (isSystemOp) | ||
| { | ||
| return ADP.ArgumentNull(TdsEnums.TCE_PARAM_MASTERKEY_PATH, StringsHelper.GetString(Strings.TCE_NullCertificatePathSysErr, validLocations[0], validLocations[1], @"/")); |
There was a problem hiding this comment.
The Assert() won't save us if validLocations has fewer than 2 elements. An existing problem, but FYI.
benrr101
removed
the
Partner Approval Needed 🤝
|
@edwardneal Sorry this one got lost, we had reached out to another team to see if this was something that aligns with their ideals. There doesn't seem to be any objections, so we are moving forward to try and take these changes (thank you for them, btw). However, there's some merge conflicts - if you can get those addressed, we can get the builds running :) |
|
@edwardneal I took a crack at resolving the merge conflicts myself, but it turns out most of the conflicts are due to your own efforts to remove hard-coded certificates from the tests. As such, it seems like you might be the best person to resolve those conflicts :) |
|
Hi @edwardneal - this PR has conflicts that need to be resolved. We are hoping to complete all 6.1-preview2 PRs this week. Will you have time to resovle the conflicts? |
edwardneal
force-pushed
the
feat/xplat-column-encryption-certificate-store-provider
branch
from
7903126 to
4c9e6d4
Compare
|
Thanks @paulmedynski, I've just resolved the conflicts - it looks like the initial certificate generation changes were sufficient to remove the hardcoded credentials. Would you mind running CI please, so we can make sure the Linux tests are working properly post-merge? |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
Thanks. It looks like the .netcore.Unix.cs file was retained during the merge. I've just removed it, can you please retrigger CI? |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
Test failures should hopefully be addressed, can you re-run CI please? It's not a problem if this falls into the next 6.1 preview. |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
Thanks. I'm happy with the Windows & Linux implementations, but there's clearly extra requirements when adding the certificates to the current user's certificate store in MacOS. I think the problem might be an absent Subject Alternate Name, but if it isn't then I'll need to test further. I've made the adjustment and would appreciate a final CI run. If this doesn't work, I think it'd be better to leave this out of the current preview; it'll take a week or two before I can start testing on MacOS in earnest. For reference, the failing stack trace is below. |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
@edwardneal it looks like the CI is still failing on mac, so I'm moving this out of 6.1-preview2. |
MacOS will throw an exception when adding a certificate to the user store if the PersistKeySet key storage flag is set. Instead, use the UserKeySet flag.
|
I've got local tests passing on a MacOS environment - it looks like the test certificates were being created with invalid KeyStorageFlags. I'd appreciate a CI run. |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Wider idea: #3048.
SqlColumnEncryptionCertificateStoreProvideris the only built-inSqlColumnEncryptionKeyStoreProviderimplementation which needs no significant code changes to enable cross-platform support out of the box. I've removed the Unix-specific guards on this and widened the test coverage.In the process, I've switched the Linux/macOS tests to use SqlColumnEncryptionCertificateStoreProvider rather than the Azure Key Vault provider. This doesn't seem to have had an impact on the code coverage statistics, but it's notable nonetheless.