Skip to content

Commit f2edda6

Browse files
authored
chore: audit logging (#468)
* chore: audit logs add audit logs for specific events * chore: test ci run ci pipeline * chore: ci set working dir for playwright * chore: ci set working dir for playwright * chore: ci set working dir for playwright * chore: ci set working dir for playwright * chore: ci server run server using playwright in ci * chore: ci server run server using playwright in ci * chore: db name update to use test db name * chore: db name update to use test db name * chore: cache playwright add caching for playwright browser install * chore: cache playwright add caching for playwright browser install * chore: cache playwright add caching for playwright browser install * chore: cache playwright run cached playwright tests * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: debug ci run ci pipeline for debugging * chore: test ci test ci pipeline * chore: test ci test ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: migrator migrate in ci pipeline * chore: event logging add event logging * chore: test-env time add shared var for test env delay * chore: event tests add client id to events and test cases * chore: ci reset ensure fresh test db in CI * chore: test utils use util function in all cases to fill otp * chore: documentation add docs for running the e2e suite
1 parent fe3e52e commit f2edda6

File tree

24 files changed

+936
-169
lines changed

24 files changed

+936
-169
lines changed

.github/workflows/otp-provider-tests.yml

Lines changed: 37 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,15 +31,13 @@ jobs:
3131
3232
- name: Install asdf
3333
uses: asdf-vm/actions/setup@v3
34+
3435
- name: Cache asdf tools
3536
uses: actions/cache@v4
3637
with:
3738
path: |
3839
/home/runner/.asdf
39-
key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}
40-
41-
- name: Install asdf
42-
uses: asdf-vm/actions/install@v3
40+
key: ${{ runner.os }}-${{ hashFiles('docker/otp-provider/.tool-versions') }}
4341

4442
- name: Install app specific asdf plugins
4543
run: |
@@ -60,10 +58,45 @@ jobs:
6058
pg_ctl start
6159
createdb runner || true
6260
chmod +x ./db-setup.sh
61+
psql -U postgres -c 'drop database if exists otp_test';
6362
./db-setup.sh otp_test
6463
working-directory: ./docker/otp-provider/.bin
6564

6665
- name: Run unit tests
6766
run: |
6867
yarn test
6968
working-directory: ./docker/otp-provider
69+
70+
- name: Cache Playwright browsers
71+
uses: actions/cache@v3
72+
with:
73+
path: ~/.cache/ms-playwright
74+
key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
75+
restore-keys: |
76+
${{ runner.os }}-playwright-
77+
78+
- name: Install Playwright Browsers
79+
run: yarn playwright install --with-deps
80+
working-directory: ./docker/otp-provider
81+
82+
- name: Migrate DB
83+
run: |
84+
yarn tailwind:build
85+
yarn build
86+
DB_NAME=otp_test node ./build/migrate.js
87+
working-directory: ./docker/otp-provider
88+
89+
- name: Seed DB
90+
run: psql -U postgres -d otp_test -f ./seed.sql
91+
working-directory: ./docker/otp-provider/e2e
92+
93+
- name: Run Playwright tests
94+
run: DB_NAME=otp_test yarn playwright test
95+
working-directory: ./docker/otp-provider
96+
97+
- uses: actions/upload-artifact@v4
98+
if: ${{ !cancelled() }}
99+
with:
100+
name: playwright-report
101+
path: ./docker/otp-provider/playwright-report/
102+
retention-days: 30

docker/otp-provider/.dockerignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
build
2+
node_modules
3+
e2e
4+
terraform
5+
/test-results/
6+
/playwright-report/
7+
/blob-report/
8+
/playwright/.cache/

docker/otp-provider/.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,8 @@
11
output.css
2+
3+
# Playwright
4+
node_modules/
5+
/test-results/
6+
/playwright-report/
7+
/blob-report/
8+
/playwright/.cache/

docker/otp-provider/README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,20 @@ The app runs locally using tsup to compile the server and client files into the
8383
'none');
8484
```
8585

86+
## End to End Tests
87+
88+
End to end testing is done with playwright. As prerequisite the end-to-end tests need a seeded db. Run the folowing from this directory:
89+
- `psql -c 'create database otp_test'`;
90+
- `yarn build && DB_NAME=otp_test node build/migrate.js`
91+
- `psql -d otp_test -f e2e/seed.sql`
92+
93+
This is only needed the first time to initialize the db. To run the tests run `yarn test:e2e`
94+
95+
For debugging, you can run `yarn playwright test --debug`. This is useful alongside adding `test.only` on the test to debug.
96+
For auto-generating tests, you can run `yarn playwright codegen` to click through the app and generate a test.
97+
98+
The test-version of the server has a few settings, using the env vars `NODE_ENV=test OTP_RESEND_INTERVAL_MINUTES=[2,3,3,4]`. When NODE_ENV is set to test, code resend intervals will be in seconds instead of minutes, useful for testing lockout functionality. You can adjust the `OTP_RESEND_INTERVAL_MINUTES` array to desired intervals in seconds then. It will also skip the CHES email callouts while in test mode.
99+
86100
## References
87101

88102
- https://github.com/panva/node-oidc-provider
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
import { test, expect } from '@playwright/test';
2+
import models from '../src/modules/sequelize/models';
3+
import { initURL } from './util';
4+
5+
const otpModel = models.get('Otp');
6+
7+
test.beforeEach(async () => {
8+
// Reset all OTPs between tests
9+
await otpModel.destroy({where: {}});
10+
});
11+
12+
test('Email Validations', async ({ page }, testInfo) => {
13+
await page.goto(initURL);
14+
await expect(page.locator('#signin-form')).toMatchAriaSnapshot(`- textbox "Email"`);
15+
16+
// Empty field validation
17+
await expect(page.getByRole('button', { name: 'Continue' })).toBeEnabled();
18+
await page.getByRole('button', { name: 'Continue' }).click();
19+
await expect(page.locator('#signin-form')).toMatchAriaSnapshot(`
20+
- textbox "Email"
21+
- text: Email is required.
22+
- button "Continue Caution" [disabled]:
23+
- img "Caution"
24+
`);
25+
26+
// Re-enables on input change
27+
await page.getByRole('textbox', { name: 'Email' }).fill('a@');
28+
await expect(page.getByRole('button', { name: 'Continue' })).toBeEnabled();
29+
30+
// Invalid email message
31+
await page.getByRole('button', { name: 'Continue' }).click();
32+
await expect(page.locator('#signin-form')).toMatchAriaSnapshot(`
33+
- textbox "Email": a@
34+
- text: Invalid email.
35+
- button "Continue Caution" [disabled]:
36+
- img "Caution"
37+
`);
38+
39+
await page.getByRole('textbox', { name: 'Email' }).fill('a@');
40+
await expect(page.getByRole('button', { name: 'Continue' })).toBeEnabled();
41+
await page.getByRole('button', { name: 'Continue' }).click();
42+
await expect(page.locator('#signin-form')).toMatchAriaSnapshot(`
43+
- textbox "Email": a@
44+
- text: Invalid email.
45+
- button "Continue Caution" [disabled]:
46+
- img "Caution"
47+
`);
48+
49+
// Proceeds with valid email. To prevent hitting cooldown for the parallel browsers, passing in the browser alias as part of email address.
50+
await page.getByRole('textbox', { name: 'Email' }).fill(`${testInfo.project.name}@b.com`);
51+
await page.getByRole('button', { name: 'Continue' }).click();
52+
53+
await page.waitForURL('**/otp')
54+
55+
await expect(page.getByRole('heading')).toMatchAriaSnapshot(`- heading "Enter your verification code" [level=2]`);
56+
});
57+
58+
test('Email Cooldown', async ({ page }, testInfo) => {
59+
// Fire a login to put the email into cooldown
60+
await page.goto(initURL);
61+
await page.getByRole('textbox', { name: 'Email' }).fill(`${testInfo.project.name}@b.com`);
62+
await page.getByRole('button', { name: 'Continue' }).click();
63+
64+
// Attempt another login with the same email and expect the cooldown error
65+
await page.goto(initURL);
66+
await page.getByRole('textbox', { name: 'Email' }).fill(`${testInfo.project.name}@b.com`);
67+
await page.getByRole('button', { name: 'Continue' }).click();
68+
69+
await expect(page.locator('#email-error')).toMatchAriaSnapshot(`- text: /Please wait \\d+ seconds before requesting a new code for this email address\\./`);
70+
});
Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
import { test, expect } from '@playwright/test';
2+
import models from '../src/modules/sequelize/models';
3+
import { changeOTP, fillOTP, initURL, clientId } from './util';
4+
import { config } from '../src/config';
5+
6+
const otpModel = models.get('Otp');
7+
const eventModel = models.get('Event');
8+
9+
test.beforeEach(async () => {
10+
// Reset all OTPs between tests
11+
await otpModel.destroy({ where: {} });
12+
await eventModel.destroy({ where: {} });
13+
});
14+
15+
16+
test('Send Code Event', async ({ page }, testInfo) => {
17+
await page.goto(initURL);
18+
const email = `${testInfo.project.name}@b.com`;
19+
20+
// Enter email and go to OTP page
21+
await page.getByRole('textbox', { name: 'Email' }).fill(email);
22+
await page.getByRole('button', { name: 'Continue' }).click();
23+
await page.waitForURL('**/otp');
24+
await expect(page.getByRole('heading')).toMatchAriaSnapshot(`- heading "Enter your verification code" [level=2]`);
25+
26+
let requestCodeEvents = await eventModel.findAll({ where: { eventType: 'REQUEST_OTP', email, clientId } });
27+
let resendCodeEvents = await eventModel.findAll({ where: { eventType: 'RESEND_OTP', email, clientId } });
28+
29+
expect(requestCodeEvents.length).toBe(1);
30+
expect(resendCodeEvents.length).toBe(0);
31+
32+
// The variable OTP_RESENDS_ALLOWED_PER_DAY is actually sends per day, including the initial. Hence the minus 1 here
33+
// Check each new resend creates an event
34+
for (let i = 0; i < Number(config.OTP_RESENDS_ALLOWED_PER_DAY) - 1; i++) {
35+
await expect(page.locator('#new-code-text')).toMatchAriaSnapshot(
36+
`
37+
- text: Can't find the code?
38+
- button "Resend code"
39+
`,
40+
);
41+
await page.getByRole('button', { name: 'Resend code' }).click();
42+
await page.waitForURL('**/otp');
43+
resendCodeEvents = await eventModel.findAll({ where: { eventType: 'RESEND_OTP', email, clientId } });
44+
expect(resendCodeEvents.length).toBe(i + 1);
45+
}
46+
47+
// Check max_resend event is not trigerred yet. Attempt a new login and expect event to be logged.
48+
let maxResendEvents = await eventModel.findAll({ where: { eventType: 'MAX_RESENDS', email, clientId } });
49+
expect(maxResendEvents.length).toBe(0);
50+
await page.goto(initURL);
51+
await page.getByRole('textbox', { name: 'Email' }).fill(email);
52+
await page.getByRole('button', { name: 'Continue' }).click();
53+
54+
await expect(page.locator('#signin-form')).toMatchAriaSnapshot(`
55+
- textbox "Email"
56+
- text: You have reached the maximum number of OTP requests for today. Please try again tomorrow.
57+
- button "Continue"
58+
`);
59+
60+
maxResendEvents = await eventModel.findAll({ where: { eventType: 'MAX_RESENDS', email, clientId } });
61+
expect(maxResendEvents.length).toBe(1);
62+
});
63+
64+
test('Retry code event', async ({page}, testInfo) => {
65+
await page.goto(initURL);
66+
const email = `${testInfo.project.name}@b.com`;
67+
68+
// Enter email and go to OTP page
69+
await page.getByRole('textbox', { name: 'Email' }).fill(email);
70+
await page.getByRole('button', { name: 'Continue' }).click();
71+
await page.waitForURL('**/otp');
72+
73+
const currentOtp = await otpModel
74+
.findOne({ where: { email: `${testInfo.project.name}@b.com`, active: true } })
75+
.then((res) => res.otp);
76+
const wrongOTP = changeOTP(String(currentOtp));
77+
78+
// Verify at zero to start
79+
let invalidOtpEvents = await eventModel.findAll({ where: { eventType: 'INVALID_OTP', email, clientId } });
80+
let maxAttemptsEvents = await eventModel.findAll({ where: { eventType: 'MAX_ATTEMPTS', email, clientId } });
81+
82+
expect(invalidOtpEvents.length).toBe(0);
83+
expect(maxAttemptsEvents.length).toBe(0);
84+
85+
for (let i = 0; i <= Number(config.OTP_ATTEMPTS_ALLOWED); i++) {
86+
await fillOTP(wrongOTP, false, page);
87+
await page.waitForSelector('#otp-error')
88+
invalidOtpEvents = await eventModel.findAll({ where: { eventType: 'INVALID_OTP', email, clientId } });
89+
expect(invalidOtpEvents.length).toBe(i + 1);
90+
91+
if (i === Number(config.OTP_ATTEMPTS_ALLOWED)) {
92+
maxAttemptsEvents = await eventModel.findAll({ where: { eventType: 'MAX_ATTEMPTS', email, clientId } });
93+
expect(maxAttemptsEvents.length).toBe(1);
94+
}
95+
}
96+
});
97+
98+
99+
test('Expired OTP Event', async ({page}, testInfo) => {
100+
await page.goto(initURL);
101+
const email = `${testInfo.project.name}@b.com`;
102+
103+
// Enter email and go to OTP page
104+
await page.getByRole('textbox', { name: 'Email' }).fill(email);
105+
await page.getByRole('button', { name: 'Continue' }).click();
106+
await page.waitForURL('**/otp');
107+
108+
const currentOtp = await otpModel
109+
.findOne({ where: { email: `${testInfo.project.name}@b.com`, active: true } });
110+
111+
// Set OTP timestamp to make it expired
112+
const fiveMinutesAgo = new Date(Date.now() - 5 * 60 * 1000);
113+
await otpModel.update({createdAt: fiveMinutesAgo}, {where: {id: currentOtp.id}} );
114+
115+
// Verify at zero to start
116+
let expiredOtpEvents = await eventModel.findAll({ where: { eventType: 'EXPIRED_OTP', email, clientId } });
117+
expect(expiredOtpEvents.length).toBe(0);
118+
119+
// assert filling in otp creates event
120+
await fillOTP(currentOtp.otp, false, page);
121+
expiredOtpEvents = await eventModel.findAll({ where: { eventType: 'EXPIRED_OTP', email, clientId } });
122+
expect(expiredOtpEvents.length).toBe(1);
123+
});

0 commit comments

Comments
 (0)