Skip to content

Commit bbcc814

Browse files
yangkb09tolzhabayevpapagian
authored
Chore: Add CI Pipeline to generate Grafana's OpenAPI specification (grafana#75393)
* chore: move over initial changes from sofia's pr grafana#58029 * chore: remove go_image * chore: begin removing edition, remove unused imports * chore: remove edition from swagger_gen.star and generate .drone.yml * chore: regen drone.yml * fix: fix order of load statements * fix: try #2 fix order of load statements * linter fixes * chore: add doc comment explaining purpose of new clone_pr_branch step * fix: add placeholder documentation for ver_mode arg * attempt #1 to import and use clone_enterprise_step_pr * Update scripts/drone/pipelines/swagger_gen.star Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com> * attempt #2 to import and use clone_enterprise_step_pr * fix: fix drone lint err invalid or unknown step dependency * fix: regen hmac to make drone run * fix: fix drone lint err * fix: update swagger-clean cmd in step * attempt to return non zero exit code * test to see if pipeline fails * fix: add git to clone pr branch step * fix: add git and make to swagger-gen step * try to rerun drone * debug: figure out why cannot find make swagger-clean * debug: see if cd grafana/grafana fixes things * debug: undo cd grafana/grafana * debug: add more logging statements * debug: try and remove cd grafana in swagger-gen * debug: removed grafana after clone statement; add debug before cloning * fix: remove disable clone * regen specs to see if swagger-gen step passes now * add descriptive error message to swagger-gen step * remove api-spec.json from .gitignore * revert backend change, regen spec * add back backend change, regen specs * Update scripts/drone/pipelines/swagger_gen.star Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com> * Update scripts/drone/pipelines/swagger_gen.star Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com> * revert gitignore change, set enterprise source to drone source branch * chore: remove unused variable, change committish var name to source * testing functionality: make a new BE change without gen spec, pipeline should fail * test functionality: does removing err msg cause step to fail properly * test functionality: add back err msg and && after * chore: remove debug statements from swagger gen step * chore: remove debug lines from clone_pr_branch step * test functionality: regen specs, swagger_gen step should pass * test funcionality: regen specs again ???? * chore: update swagger-gen step err msg * test functionality: make BE change dont regen spec, swagger gen should fail * test functionality: regen the specs, swagger-gen should pass * chore: revert test BE change, regen spec * chore: remove unused clone step * chore: regen drone.yml --------- Co-authored-by: Timur Olzhabayev <timur.olzhabayev@grafana.com> Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
1 parent be6b407 commit bbcc814

File tree

5 files changed

+219
-49
lines changed

5 files changed

+219
-49
lines changed

‎.drone.yml

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1132,6 +1132,66 @@ volumes:
11321132
clone:
11331133
retries: 3
11341134
depends_on: []
1135+
image_pull_secrets:
1136+
- dockerconfigjson
1137+
kind: pipeline
1138+
name: pr-swagger-gen
1139+
node:
1140+
type: no-parallel
1141+
platform:
1142+
arch: amd64
1143+
os: linux
1144+
services: []
1145+
steps:
1146+
- commands:
1147+
- apk add --update curl jq bash
1148+
- is_fork=$(curl "https://$GITHUB_TOKEN@api.github.com/repos/grafana/grafana/pulls/$DRONE_PULL_REQUEST"
1149+
| jq .head.repo.fork)
1150+
- if [ "$is_fork" != false ]; then return 1; fi
1151+
- git clone "https://$${GITHUB_TOKEN}@github.com/grafana/grafana-enterprise.git"
1152+
grafana-enterprise
1153+
- cd grafana-enterprise
1154+
- if git checkout ${DRONE_SOURCE_BRANCH}; then echo "checked out ${DRONE_SOURCE_BRANCH}";
1155+
elif git checkout main; then echo "git checkout main"; else git checkout main;
1156+
fi
1157+
environment:
1158+
GITHUB_TOKEN:
1159+
from_secret: github_token
1160+
image: alpine/git:2.40.1
1161+
name: clone-enterprise
1162+
- commands:
1163+
- apk add --update git make
1164+
- make swagger-clean && make openapi3-gen
1165+
- for f in public/api-merged.json public/openapi3.json; do git add $f; done
1166+
- if [ -z "$(git diff --name-only --cached)" ]; then echo "Everything seems up to
1167+
date!"; else echo "Please ensure the branch is up-to-date, then regenerate the
1168+
specification by running make swagger-clean && make openapi3-gen" && return 1;
1169+
fi
1170+
depends_on:
1171+
- clone-enterprise
1172+
environment:
1173+
GITHUB_TOKEN:
1174+
from_secret: github_token
1175+
image: golang:1.20.10-alpine
1176+
name: swagger-gen
1177+
trigger:
1178+
event:
1179+
- pull_request
1180+
paths:
1181+
exclude:
1182+
- docs/**
1183+
- '*.md'
1184+
include:
1185+
- pkg/**
1186+
type: docker
1187+
volumes:
1188+
- host:
1189+
path: /var/run/docker.sock
1190+
name: docker
1191+
---
1192+
clone:
1193+
retries: 3
1194+
depends_on: []
11351195
environment:
11361196
EDITION: oss
11371197
image_pull_secrets:
@@ -4607,6 +4667,6 @@ kind: secret
46074667
name: gcr_credentials
46084668
---
46094669
kind: signature
4610-
hmac: 93de8a710e23d3f1d31860f9eed34cd841b0a0eb48637971de4e8ce60a7c3df1
4670+
hmac: 29a933affceb9cc39b285d936de9e6327deedbb80f1285fa645d596f89ede442
46114671

46124672
...

‎public/api-merged.json

Lines changed: 47 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -639,30 +639,6 @@
639639
}
640640
}
641641
},
642-
"/admin/ldap-sync-status": {
643-
"get": {
644-
"description": "You need to have a permission with action `ldap.status:read`.",
645-
"tags": [
646-
"ldap_debug"
647-
],
648-
"summary": "Returns the current state of the LDAP background sync integration.",
649-
"operationId": "getSyncStatus",
650-
"responses": {
651-
"200": {
652-
"$ref": "#/responses/getSyncStatusResponse"
653-
},
654-
"401": {
655-
"$ref": "#/responses/unauthorisedError"
656-
},
657-
"403": {
658-
"$ref": "#/responses/forbiddenError"
659-
},
660-
"500": {
661-
"$ref": "#/responses/internalServerError"
662-
}
663-
}
664-
}
665-
},
666642
"/admin/ldap/reload": {
667643
"post": {
668644
"security": [
@@ -15166,6 +15142,53 @@
1516615142
}
1516715143
}
1516815144
},
15145+
"JSONWebKey": {
15146+
"type": "object",
15147+
"title": "JSONWebKey represents a public or private key in JWK format.",
15148+
"properties": {
15149+
"Algorithm": {
15150+
"description": "Key algorithm, parsed from `alg` header.",
15151+
"type": "string"
15152+
},
15153+
"CertificateThumbprintSHA1": {
15154+
"description": "X.509 certificate thumbprint (SHA-1), parsed from `x5t` header.",
15155+
"type": "array",
15156+
"items": {
15157+
"type": "integer",
15158+
"format": "uint8"
15159+
}
15160+
},
15161+
"CertificateThumbprintSHA256": {
15162+
"description": "X.509 certificate thumbprint (SHA-256), parsed from `x5t#S256` header.",
15163+
"type": "array",
15164+
"items": {
15165+
"type": "integer",
15166+
"format": "uint8"
15167+
}
15168+
},
15169+
"Certificates": {
15170+
"description": "X.509 certificate chain, parsed from `x5c` header.",
15171+
"type": "array",
15172+
"items": {
15173+
"$ref": "#/definitions/Certificate"
15174+
}
15175+
},
15176+
"CertificatesURL": {
15177+
"$ref": "#/definitions/URL"
15178+
},
15179+
"Key": {
15180+
"description": "Cryptographic key, can be a symmetric or asymmetric key."
15181+
},
15182+
"KeyID": {
15183+
"description": "Key identifier, parsed from `kid` header.",
15184+
"type": "string"
15185+
},
15186+
"Use": {
15187+
"description": "Key use, parsed from `use` header.",
15188+
"type": "string"
15189+
}
15190+
}
15191+
},
1516915192
"Json": {
1517015193
"type": "object"
1517115194
},

‎public/openapi3.json

Lines changed: 47 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6234,6 +6234,53 @@
62346234
"title": "JSONWebKey represents a public or private key in JWK format.",
62356235
"type": "object"
62366236
},
6237+
"JSONWebKey": {
6238+
"properties": {
6239+
"Algorithm": {
6240+
"description": "Key algorithm, parsed from `alg` header.",
6241+
"type": "string"
6242+
},
6243+
"CertificateThumbprintSHA1": {
6244+
"description": "X.509 certificate thumbprint (SHA-1), parsed from `x5t` header.",
6245+
"items": {
6246+
"format": "uint8",
6247+
"type": "integer"
6248+
},
6249+
"type": "array"
6250+
},
6251+
"CertificateThumbprintSHA256": {
6252+
"description": "X.509 certificate thumbprint (SHA-256), parsed from `x5t#S256` header.",
6253+
"items": {
6254+
"format": "uint8",
6255+
"type": "integer"
6256+
},
6257+
"type": "array"
6258+
},
6259+
"Certificates": {
6260+
"description": "X.509 certificate chain, parsed from `x5c` header.",
6261+
"items": {
6262+
"$ref": "#/components/schemas/Certificate"
6263+
},
6264+
"type": "array"
6265+
},
6266+
"CertificatesURL": {
6267+
"$ref": "#/components/schemas/URL"
6268+
},
6269+
"Key": {
6270+
"description": "Cryptographic key, can be a symmetric or asymmetric key."
6271+
},
6272+
"KeyID": {
6273+
"description": "Key identifier, parsed from `kid` header.",
6274+
"type": "string"
6275+
},
6276+
"Use": {
6277+
"description": "Key use, parsed from `use` header.",
6278+
"type": "string"
6279+
}
6280+
},
6281+
"title": "JSONWebKey represents a public or private key in JWK format.",
6282+
"type": "object"
6283+
},
62376284
"Json": {
62386285
"type": "object"
62396286
},
@@ -12481,30 +12528,6 @@
1248112528
]
1248212529
}
1248312530
},
12484-
"/admin/ldap-sync-status": {
12485-
"get": {
12486-
"description": "You need to have a permission with action `ldap.status:read`.",
12487-
"operationId": "getSyncStatus",
12488-
"responses": {
12489-
"200": {
12490-
"$ref": "#/components/responses/getSyncStatusResponse"
12491-
},
12492-
"401": {
12493-
"$ref": "#/components/responses/unauthorisedError"
12494-
},
12495-
"403": {
12496-
"$ref": "#/components/responses/forbiddenError"
12497-
},
12498-
"500": {
12499-
"$ref": "#/components/responses/internalServerError"
12500-
}
12501-
},
12502-
"summary": "Returns the current state of the LDAP background sync integration.",
12503-
"tags": [
12504-
"ldap_debug"
12505-
]
12506-
}
12507-
},
1250812531
"/admin/ldap/reload": {
1250912532
"post": {
1251012533
"description": "If you are running Grafana Enterprise and have Fine-grained access control enabled, you need to have a permission with action `ldap.config:reload`.",

‎scripts/drone/events/pr.star

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ load(
3232
"scripts/drone/pipelines/shellcheck.star",
3333
"shellcheck_pipeline",
3434
)
35+
load(
36+
"scripts/drone/pipelines/swagger_gen.star",
37+
"swagger_gen",
38+
)
3539
load(
3640
"scripts/drone/pipelines/test_backend.star",
3741
"test_backend",
@@ -137,6 +141,10 @@ def pr_pipelines():
137141
),
138142
docs_pipelines(ver_mode, trigger_docs_pr()),
139143
shellcheck_pipeline(),
144+
swagger_gen(
145+
get_pr_trigger(include_paths = ["pkg/**"]),
146+
ver_mode,
147+
),
140148
integration_benchmarks(
141149
prefix = ver_mode,
142150
),

‎scripts/drone/pipelines/swagger_gen.star

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
"""
2+
This module returns all pipelines used in OpenAPI specification generation of Grafana HTTP APIs
3+
"""
4+
5+
load(
6+
"scripts/drone/steps/lib.star",
7+
"clone_enterprise_step_pr",
8+
)
9+
load(
10+
"scripts/drone/utils/images.star",
11+
"images",
12+
)
13+
load(
14+
"scripts/drone/utils/utils.star",
15+
"pipeline",
16+
)
17+
load(
18+
"scripts/drone/vault.star",
19+
"from_secret",
20+
)
21+
22+
def swagger_gen_step(ver_mode):
23+
if ver_mode != "pr":
24+
return None
25+
26+
return {
27+
"name": "swagger-gen",
28+
"image": images["go"],
29+
"environment": {
30+
"GITHUB_TOKEN": from_secret("github_token"),
31+
},
32+
"commands": [
33+
"apk add --update git make",
34+
"make swagger-clean && make openapi3-gen",
35+
"for f in public/api-merged.json public/openapi3.json; do git add $f; done",
36+
'if [ -z "$(git diff --name-only --cached)" ]; then echo "Everything seems up to date!"; else echo "Please ensure the branch is up-to-date, then regenerate the specification by running make swagger-clean && make openapi3-gen" && return 1; fi',
37+
],
38+
"depends_on": [
39+
"clone-enterprise",
40+
],
41+
}
42+
43+
def swagger_gen(trigger, ver_mode, source = "${DRONE_SOURCE_BRANCH}"):
44+
test_steps = [
45+
clone_enterprise_step_pr(source = source),
46+
swagger_gen_step(ver_mode = ver_mode),
47+
]
48+
49+
p = pipeline(
50+
name = "{}-swagger-gen".format(ver_mode),
51+
trigger = trigger,
52+
services = [],
53+
steps = test_steps,
54+
)
55+
56+
return p

0 commit comments

Comments
 (0)