Skip to content

Add workflow_run api + webhook #33964

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 72 commits into
base: main
Choose a base branch
from

Conversation

ChristopherHX
Copy link
Contributor

@ChristopherHX ChristopherHX commented Mar 21, 2025

Implements

Fixes

  • workflow_job webhook url to no longer contain the runs/<run> part to align with api
  • workflow instance does now use it's name inside the file instead of filename if set

Refactoring

  • Moved a lot of logic from workflows/workflow_job into a shared module used by both webhook and api

TODO

  • Verify Keda Compatibility
  • Edit Webhook API bug is resolved

Closes #23670
Closes #23796
Closes #24898
Replaces #28047 and is much more complete

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Mar 21, 2025
@github-actions github-actions bot added modifies/translation modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code labels Mar 21, 2025
@ChristopherHX ChristopherHX marked this pull request as ready for review May 5, 2025 21:10
@ChristopherHX
Copy link
Contributor Author

Verification update of keda k8s autoscaler (current gitea 1.24-rc works only with modified garm).

Uses these apis

  • runs api
  • jobs of run api

I could change the existing keda github-runner e2e test to pass on my end by manually running gitea and setting up the test repository without changing non test content.

go test -v -tags e2e ./scalers/github_runner
=== RUN   TestScaler
    gitea_runner_test.go:219: --- setting up ---
    helper.go:247: deleting namespace github-runner-test-ns
    helper.go:301: waiting for namespace github-runner-test-ns deletion
    helper.go:301: waiting for namespace github-runner-test-ns deletion
    helper.go:301: waiting for namespace github-runner-test-ns deletion
    helper.go:301: waiting for namespace github-runner-test-ns deletion
    helper.go:234: Creating namespace - github-runner-test-ns
    helper.go:541: Applying template: secretTemplate
    helper.go:541: Applying template: authTemplate
    helper.go:541: Applying template: deploymentTemplate
    helper.go:541: Applying template: scaledObjectTemplate
    helper.go:541: Applying template: scaledJobTemplate
    helper.go:366: Waiting for pods in namespace to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 0
    helper.go:541: Applying template: scaledJobTemplate
    gitea_runner_test.go:326: --- testing scale out ---
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 1
    helper.go:326: Waiting for job count to hit target. Namespace - github-runner-test-ns, Current  - 1, Target - 1
    gitea_runner_test.go:333: --- testing scale in ---
    helper.go:291: all jobs ran successfully!
    helper.go:739: killing all pods in github-runner-test-ns namespace with selector app=github-runner-test-sj
    helper.go:541: Applying template: scaledObjectTemplate
    gitea_runner_test.go:306: --- testing none activation ---
    helper.go:494: Waiting for some time to ensure deployment replica count doesn't change from 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    helper.go:501: Deployment - github-runner-test-deployment, Current  - 0
    gitea_runner_test.go:311: --- testing scale out ---
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 0, Target - 1
    helper.go:437: Waiting for deployment replicas to hit target. Deployment - github-runner-test-deployment, Current  - 1, Target - 1
    gitea_runner_test.go:319: --- testing scale in ---
    helper.go:366: Waiting for pods in namespace to hit target. Namespace - github-runner-test-ns, Current  - 1, Target - 0
    helper.go:366: Waiting for pods in namespace to hit target. Namespace - github-runner-test-ns, Current  - 1, Target - 0
    helper.go:366: Waiting for pods in namespace to hit target. Namespace - github-runner-test-ns, Current  - 1, Target - 0
    helper.go:366: Waiting for pods in namespace to hit target. Namespace - github-runner-test-ns, Current  - 0, Target - 0
    helper.go:610: Deleting template: scaledJobTemplate
    helper.go:610: Deleting template: scaledObjectTemplate
    helper.go:610: Deleting template: deploymentTemplate
    helper.go:610: Deleting template: authTemplate
    helper.go:610: Deleting template: secretTemplate
    helper.go:247: deleting namespace github-runner-test-ns
    helper.go:301: waiting for namespace github-runner-test-ns deletion
    helper.go:301: waiting for namespace github-runner-test-ns deletion
    helper.go:301: waiting for namespace github-runner-test-ns deletion
--- PASS: TestScaler (133.36s)
PASS
ok      github.com/kedacore/keda/v2/tests/scalers/github_runner 133.750s

Maybe we could setup extended e2e test repositories to track external autoscaler compatibility

@ChristopherHX

This comment was marked as outdated.

@wxiaoguang
Copy link
Contributor

Ready for review?

@ChristopherHX
Copy link
Contributor Author

My time to respond or make changes is limited in the next seven days, due to exams.

Other than that, yes I think this is ready for review.

ListOptions: utils.GetListOptions(ctx),
}

if event := ctx.Req.URL.Query().Get("event"); event != "" {
Copy link
Contributor

@wxiaoguang wxiaoguang Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URL.Query will do ParseQuery again and agian.

Maybe we can just call FormString, or use a parsed query parsedQuery := ctx.Req.URL.Query()


runID := ctx.PathParamInt64("run")

shared.ListJobs(ctx, 0, repoID, runID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe ListJobs has done enough check to ensure runID belongs to repoID, while is it possible to explicitly check it here, or add comment to ListJobs to explain that no need extra check?

// Sync run status with db
job.Run = nil
if err := job.LoadAttributes(ctx); err != nil {
ctx.HTTPError(http.StatusInternalServerError, err.Error())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use ctx to report server internal error, then the err will also appear in logs

Comment on lines +90 to +95
// Sync run status with db
jobs[0].Run = nil
if err := jobs[0].LoadAttributes(ctx); err != nil {
return err
}
run := jobs[0].Run
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saw this code block many times. Maybe a common function:

func notifyWorkflowRunStatusUpdate(....) {
    job.Run = nil
    if err := job.LoadAttribute(); err != nil {
         log.Error()
    } else {
           notify_service.WorkflowRunStatusUpdate(
    }
}


status := convert.ToWorkflowRunAction(run.Status)

gitRepo, err := gitrepo.OpenRepository(context.Background(), repo)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use ctx?

Copy link
Contributor

@wxiaoguang wxiaoguang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No hurry, overall looks good. Thank you very much.

(Maybe will take a look again later)

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Jun 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm/need 1 This PR needs approval from one additional maintainer to be merged. modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code modifies/templates This PR modifies the template files modifies/translation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

workflow_run for action trigger event Webhook trigger for actions (CI) Implement workflow_* webhook event
5 participants