Skip to content

test(amazonq): add more tests for /transform #6183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Dec 9, 2024
12 changes: 11 additions & 1 deletion packages/amazonq/test/e2e/amazonq/framework/messenger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@ export class Messenger {
this.mynahUIProps.onFollowUpClicked(this.tabID, lastChatItem?.messageId ?? '', option[0])
}

clickCustomFormButton(action: { id: string; text?: string; formItemValues?: Record<string, string> }) {
if (!this.mynahUIProps.onCustomFormAction) {
assert.fail('onCustomFormAction must be defined to use it in the tests')
}

this.mynahUIProps.onCustomFormAction(this.tabID, action)
}

clickFileActionButton(filePath: string, actionName: string) {
if (!this.mynahUIProps.onFileActionClick) {
assert.fail('onFileActionClick must be defined to use it in the tests')
Expand Down Expand Up @@ -173,7 +181,9 @@ export class Messenger {

// Do another check just in case the waitUntil time'd out
if (!event()) {
assert.fail(`Event has not finished loading in: ${this.waitTimeoutInMs} ms`)
assert.fail(
`Event has not finished loading in: ${waitOverrides ? waitOverrides.waitTimeoutInMs : this.waitTimeoutInMs} ms`
)
}
}

Expand Down
347 changes: 347 additions & 0 deletions packages/amazonq/test/e2e/amazonq/transformByQ.test.ts
Copy link
Contributor Author

@dhasani23 dhasani23 Dec 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/runIntegrationTests to trigger them

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are also run automatically after merging.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,12 @@ import {
} from '../../../shared/telemetry/telemetry'
import { MetadataResult } from '../../../shared/telemetry/telemetryClient'
import { CodeTransformTelemetryState } from '../../telemetry/codeTransformTelemetryState'
import { getAuthType } from '../../../codewhisperer/service/transformByQ/transformApiHandler'
import DependencyVersions from '../../models/dependencies'
import { getStringHash } from '../../../shared/utilities/textUtilities'
import { getVersionData } from '../../../codewhisperer/service/transformByQ/transformMavenHandler'
import AdmZip from 'adm-zip'
import { AuthError } from '../../../auth/sso/server'
import { getAuthType } from '../../../auth/utils'

// These events can be interactions within the chat,
// or elsewhere in the IDE
Expand Down
3 changes: 3 additions & 0 deletions packages/core/src/amazonqGumby/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,7 @@
export { activate } from './activation'
export { default as DependencyVersions } from './models/dependencies'
export { default as MessengerUtils } from './chat/controller/messenger/messengerUtils'
export { GumbyController } from './chat/controller/controller'
export { TabsStorage } from '../amazonq/webview/ui/storages/tabsStorage'
export * as startTransformByQ from '../../src/codewhisperer/commands/startTransformByQ'
Copy link
Contributor Author

@dhasani23 dhasani23 Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unable to run the tests in packages/amazonq/test/e2e/amazonq/transformByQ.test.ts by importing these things from the core module directly where they're defined, as I get an import error when doing that. So instead I have to export them here and then (in the test file, line 11) import them from here.

export * from './errors'
14 changes: 12 additions & 2 deletions packages/core/src/auth/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import { formatError, ToolkitError } from '../shared/errors'
import { asString } from './providers/credentials'
import { TreeNode } from '../shared/treeview/resourceTreeDataProvider'
import { createInputBox } from '../shared/ui/inputPrompter'
import { telemetry } from '../shared/telemetry/telemetry'
import { CredentialSourceId, telemetry } from '../shared/telemetry/telemetry'
import { createCommonButtons, createExitButton, createHelpButton, createRefreshButton } from '../shared/ui/buttons'
import { getIdeProperties, isAmazonQ, isCloud9 } from '../shared/extensionUtilities'
import { addScopes, getDependentAuths } from './secondaryAuth'
Expand All @@ -45,7 +45,7 @@ import { Commands, placeholder } from '../shared/vscode/commands2'
import { Auth } from './auth'
import { validateIsNewSsoUrl, validateSsoUrlFormat } from './sso/validation'
import { getLogger } from '../shared/logger'
import { isValidAmazonQConnection, isValidCodeWhispererCoreConnection } from '../codewhisperer/util/authUtil'
import { AuthUtil, isValidAmazonQConnection, isValidCodeWhispererCoreConnection } from '../codewhisperer/util/authUtil'
import { AuthFormId } from '../login/webview/vue/types'
import { extensionVersion } from '../shared/vscode/env'
import { ExtStartUpSources } from '../shared/telemetry'
Expand Down Expand Up @@ -798,3 +798,13 @@ export function initializeCredentialsProviderManager() {
manager.addProviderFactory(new SharedCredentialsProviderFactory())
manager.addProviders(new Ec2CredentialsProvider(), new EcsCredentialsProvider(), new EnvVarsCredentialsProvider())
}

export async function getAuthType() {
let authType: CredentialSourceId | undefined = undefined
if (AuthUtil.instance.isEnterpriseSsoInUse() && AuthUtil.instance.isConnectionValid()) {
authType = 'iamIdentityCenter'
} else if (AuthUtil.instance.isBuilderIdInUse() && AuthUtil.instance.isConnectionValid()) {
authType = 'awsId'
}
return authType
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This above function was previously in transformApiHandler.ts, just moving it here so it's in a shared utils file

Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,9 @@ export async function parseBuildFile() {
export async function preTransformationUploadCode() {
await vscode.commands.executeCommand('aws.amazonq.transformationHub.focus')

void vscode.window.showInformationMessage(CodeWhispererConstants.jobStartedNotification)
void vscode.window.showInformationMessage(CodeWhispererConstants.jobStartedNotification, {
title: CodeWhispererConstants.jobStartedTitle,
})

let uploadId = ''
throwIfCancelled()
Expand Down Expand Up @@ -848,7 +850,9 @@ export async function postTransformationJob() {
}

if (transformByQState.isSucceeded()) {
void vscode.window.showInformationMessage(CodeWhispererConstants.jobCompletedNotification(diffMessage))
void vscode.window.showInformationMessage(CodeWhispererConstants.jobCompletedNotification(diffMessage), {
title: CodeWhispererConstants.transformationCompletedTitle,
})
} else if (transformByQState.isPartiallySucceeded()) {
void vscode.window
.showInformationMessage(
Expand Down
4 changes: 4 additions & 0 deletions packages/core/src/codewhisperer/models/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -554,6 +554,8 @@ export const noOngoingJobMessage = 'No ongoing job.'

export const nothingToShowMessage = 'Nothing to show'

export const jobStartedTitle = 'Transformation started'

export const jobStartedNotification =
'Amazon Q is transforming your code. It can take 10 to 30 minutes to upgrade your code, depending on the size of your project. To monitor progress, go to the Transformation Hub.'

Expand Down Expand Up @@ -636,6 +638,8 @@ export const jobCancelledChatMessage =

export const jobCancelledNotification = 'You cancelled the transformation.'

export const transformationCompletedTitle = 'Transformation complete'

export const diffMessage = (multipleDiffs: boolean) => {
return multipleDiffs
? 'You can review the diffs to see my proposed changes and accept or reject them. You will be able to accept changes from one diff at a time. If you reject changes in one diff, you will not be able to view or accept changes in the other diffs.'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,38 +34,28 @@ import {
import { sleep } from '../../../shared/utilities/timeoutUtils'
import AdmZip from 'adm-zip'
import globals from '../../../shared/extensionGlobals'
import { CredentialSourceId, telemetry } from '../../../shared/telemetry/telemetry'
import { telemetry } from '../../../shared/telemetry/telemetry'
import { CodeTransformTelemetryState } from '../../../amazonqGumby/telemetry/codeTransformTelemetryState'
import { calculateTotalLatency } from '../../../amazonqGumby/telemetry/codeTransformTelemetry'
import { MetadataResult } from '../../../shared/telemetry/telemetryClient'
import request from '../../../shared/request'
import { JobStoppedError, ZipExceedsSizeLimitError } from '../../../amazonqGumby/errors'
import { writeLogs } from './transformFileHandler'
import { AuthUtil } from '../../util/authUtil'
import { createCodeWhispererChatStreamingClient } from '../../../shared/clients/codewhispererChatClient'
import { downloadExportResultArchive } from '../../../shared/utilities/download'
import { ExportIntent, TransformationDownloadArtifactType } from '@amzn/codewhisperer-streaming'
import fs from '../../../shared/fs/fs'
import { ChatSessionManager } from '../../../amazonqGumby/chat/storages/chatSession'
import { encodeHTML } from '../../../shared/utilities/textUtilities'
import { convertToTimeString } from '../../../shared/datetime'
import { getAuthType } from '../../../auth/utils'

export function getSha256(buffer: Buffer) {
const hasher = crypto.createHash('sha256')
hasher.update(buffer)
return hasher.digest('base64')
}

export async function getAuthType() {
let authType: CredentialSourceId | undefined = undefined
if (AuthUtil.instance.isEnterpriseSsoInUse() && AuthUtil.instance.isConnectionValid()) {
authType = 'iamIdentityCenter'
} else if (AuthUtil.instance.isBuilderIdInUse() && AuthUtil.instance.isConnectionValid()) {
authType = 'awsId'
}
return authType
}

export function throwIfCancelled() {
if (transformByQState.isCancelled()) {
throw new TransformByQStoppedError()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@
import assert, { fail } from 'assert'
import * as vscode from 'vscode'
import * as sinon from 'sinon'
import { makeTemporaryToolkitFolder } from '../../../shared/filesystemUtilities'
import { DB, transformByQState, TransformByQStoppedError } from '../../../codewhisperer/models/model'
import {
finalizeTransformationJob,
parseBuildFile,
setMaven,
stopTransformByQ,
validateSQLMetadataFile,
} from '../../../codewhisperer/commands/startTransformByQ'
Expand Down Expand Up @@ -40,14 +41,14 @@ import {
} from '../../../codewhisperer/service/transformByQ/transformProjectValidationHandler'
import { TransformationCandidateProject, ZipManifest } from '../../../codewhisperer/models/model'
import globals from '../../../shared/extensionGlobals'
import { fs } from '../../../shared'
import { env, fs } from '../../../shared'
import { convertDateToTimestamp, convertToTimeString } from '../../../shared/datetime'

describe('transformByQ', function () {
let tempDir: string

beforeEach(async function () {
tempDir = await makeTemporaryToolkitFolder()
tempDir = (await TestFolder.create()).path
transformByQState.setToNotStarted()
})

Expand Down Expand Up @@ -149,6 +150,22 @@ describe('transformByQ', function () {
sinon.assert.calledWithExactly(stopJobStub, { transformationJobId: 'dummyId' })
})

it('WHEN stopTransformByQ called with job that has already terminated THEN stop API not called', async function () {
const stopJobStub = sinon.stub(codeWhisperer.codeWhispererClient, 'codeModernizerStopCodeTransformation')
transformByQState.setToSucceeded()
await stopTransformByQ('abc-123')
sinon.assert.notCalled(stopJobStub)
})

it('WHEN finalizeTransformationJob on failed job THEN error thrown and error message fields are set', async function () {
await assert.rejects(async () => {
await finalizeTransformationJob('FAILED')
})
assert.notStrictEqual(transformByQState.getJobFailureErrorChatMessage(), undefined)
assert.notStrictEqual(transformByQState.getJobFailureErrorNotification(), undefined)
transformByQState.setJobDefaults() // reset error messages to undefined
})

it('WHEN polling completed job THEN returns status as completed', async function () {
const mockJobResponse = {
$response: {
Expand Down Expand Up @@ -208,6 +225,16 @@ describe('transformByQ', function () {
assert.deepStrictEqual(actual, expected)
})

it(`WHEN transforming a project with a Windows Maven executable THEN mavenName set correctly`, async function () {
sinon.stub(env, 'isWin').returns(true)
const tempFileName = 'mvnw.cmd'
const tempFilePath = path.join(tempDir, tempFileName)
await toFile('', tempFilePath)
transformByQState.setProjectPath(tempDir)
await setMaven()
assert.strictEqual(transformByQState.getMavenName(), '.\\mvnw.cmd')
})

it(`WHEN zip created THEN manifest.json contains test-compile custom build command`, async function () {
const tempFileName = `testfile-${globals.clock.Date.now()}.zip`
transformByQState.setProjectPath(tempDir)
Expand All @@ -234,6 +261,19 @@ describe('transformByQ', function () {
})
})

it('WHEN zipCode THEN ZIP contains all expected files and no unexpected files', async function () {
const zipFilePath = path.join(tempDir, 'test.zip')
const zip = new AdmZip()
await fs.writeFile(path.join(tempDir, 'pom.xml'), 'dummy pom.xml')
zip.addLocalFile(path.join(tempDir, 'pom.xml'))
zip.addFile('manifest.json', Buffer.from(JSON.stringify({ version: '1.0' })))
zip.writeZip(zipFilePath)
const zipFiles = new AdmZip(zipFilePath).getEntries()
const zipFileNames = zipFiles.map((file) => file.name)
assert.strictEqual(zipFileNames.length, 2) // expecting only pom.xml and manifest.json
assert.strictEqual(zipFileNames.includes('pom.xml') && zipFileNames.includes('manifest.json'), true)
})

it(`WHEN zip created THEN dependencies contains no .sha1 or .repositories files`, async function () {
const m2Folders = [
'com/groupid1/artifactid1/version1',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,10 @@ import * as codeWhisperer from '../../codewhisperer/client/codewhisperer'
import assert from 'assert'
import { getSha256, uploadArtifactToS3, zipCode } from '../../codewhisperer/service/transformByQ/transformApiHandler'
import request from '../../shared/request'
import AdmZip from 'adm-zip'
import { setValidConnection } from '../util/connection'
import { transformByQState, ZipManifest } from '../../codewhisperer/models/model'
import globals from '../../shared/extensionGlobals'
import { fs } from '../../shared'
import { setValidConnection } from '../../testE2E/util/connection'

describe('transformByQ', async function () {
let tempDir = ''
Expand All @@ -35,10 +34,6 @@ describe('transformByQ', async function () {
await fs.writeFile(tempFilePath, 'sample content for the test file')
transformByQState.setProjectPath(tempDir)
const zipCodeResult = await zipCode({
dependenciesFolder: {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used for these tests

path: tempFilePath,
name: tempFileName,
},
projectPath: tempDir,
zipManifest: new ZipManifest(),
})
Expand Down Expand Up @@ -87,7 +82,7 @@ describe('transformByQ', async function () {
)
})

it('WHEN createUploadUrl THEN URL uses HTTPS and sets 60 second expiration', async function () {
it('WHEN createUploadUrl THEN URL uses HTTPS and sets 30 minute expiration', async function () {
const buffer = Buffer.from(await fs.readFileBytes(zippedCodePath))
const sha256 = getSha256(buffer)
const response = await codeWhisperer.codeWhispererClient.createUploadUrl({
Expand All @@ -96,17 +91,8 @@ describe('transformByQ', async function () {
uploadIntent: CodeWhispererConstants.uploadIntent,
})
const uploadUrl = response.uploadUrl
const usesHttpsAndExpiresAfter60Seconds = uploadUrl.includes('https') && uploadUrl.includes('X-Amz-Expires=60')
assert.strictEqual(usesHttpsAndExpiresAfter60Seconds, true)
})

it('WHEN zipCode THEN ZIP contains all expected files and no unexpected files', async function () {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this test to the file above with the other zipCode tests

const zipFiles = new AdmZip(zippedCodePath).getEntries()
const zipFileNames: string[] = []
zipFiles.forEach((file) => {
zipFileNames.push(file.name)
})
assert.strictEqual(zipFileNames.length, 2) // expecting only a dummy txt file and a manifest.json
assert.strictEqual(zipFileNames.includes(tempFileName) && zipFileNames.includes('manifest.json'), true)
const usesHttpsAndExpiresAfter30Minutes =
uploadUrl.includes('https') && uploadUrl.includes('X-Amz-Expires=1800')
assert.strictEqual(usesHttpsAndExpiresAfter30Minutes, true)
})
})
Loading