Skip to content

Commit 9d4a3f3

Browse files
author
ikethecoder
authored
Fix status results (#14)
1 parent 21fd1c4 commit 9d4a3f3

File tree

5 files changed

+232
-32
lines changed

5 files changed

+232
-32
lines changed

‎USER-JOURNEY.md‎

Lines changed: 30 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ services:
5555

5656
To view optional plugin examples go [here](/docs/samples/service-plugins).
5757

58+
> **Declarative Config** Behind the scenes, DecK is used to sync your configuration with Kong. For reference: https://docs.konghq.com/deck/overview/
59+
5860
> **Splitting Your Config:** A namespace `tag` with the format `ns.$NS` is mandatory for each service/route/plugin. But if you have separate pipelines for your environments (i.e./ dev, test and prod), you can split your configuration and update the `tags` with the qualifier. So for example, you can use a tag `ns.$NS.dev` to sync Kong configuration for `dev` Service and Routes only.
5961
6062
> **Upstream Services on OCP4:** If your service is running on OCP4, you should specify the Kubernetes Service in the `Service.host`. It must have the format: `<name>.<ocp-namespace>.svc` The Network Security Policies (NSP) will be setup automatically on the API Gateway side. You will need to create an NSP on your side looking something like this to allow the Gateway's test and prod environments to route traffic to your API:
@@ -69,13 +71,13 @@ spec:
6971
allow aps gateway to route traffic to your api
7072
source:
7173
- - $namespace=264e6f-test
72-
- app.kubernetes.io/name=kong
7374
- - $namespace=264e6f-prod
74-
- app.kubernetes.io/name=kong
7575
destination:
7676
- - app.kubernetes.io/name=my-upstream-api
7777
```
7878

79+
> **Migrating from OCP3 to OCP4?** Please review the [OCP4-Migration](docs/OCP4-MIGRATION.md) instructions to help with transitioning to OCP4 and the new APS Gateway.
80+
7981

8082
### gwa Command Line
8183

@@ -101,12 +103,13 @@ The Swagger console for the `gwa-api` can be used to publish Kong Gateway config
101103
**Install (for Linux)**
102104

103105
```
104-
curl -L -O https://bcgov.github.io/gwa-cli/gwa_v1.0.10_linux_x64.zip
105-
unzip gwa_v1.0.10_linux_x64.zip
106+
GWA_CLI_VERSION=v1.0.14; curl -L -O https://github.com/bcgov/gwa-cli/releases/download/$GWA_CLI_VERSION/gwa-cli-linux.zip
107+
unzip gwa-cli-linux.zip
108+
mv gwa-cli-linux gwa
106109
./gwa --version
107110
```
108111

109-
> MacOS or Windows? For Mac `gwa_v1.0.10_macos_x64.zip` and for Windows `gwa_v1.0.10_win_x64.zip`
112+
> MacOS or Windows? For Mac `gwa-cli-macos.zip` and for Windows `gwa-cli-win.exe.zip`
110113
111114
**Configure**
112115

@@ -166,6 +169,10 @@ ab -n 20 -c 2 https://${NAME}-api-gov-bc-ca.test.apsgw.xyz/headers
166169
167170
```
168171

172+
To help with troubleshooting, you can use the GWA API to get a health check for each of the upstream services to verify the Gateway is connecting OK.
173+
174+
Go to the <a href="https://gwa-api-264e6f-test.apps.silver.devops.gov.bc.ca/api/doc#/Service%20Status/get_namespaces__namespace__services">GWA API</a>, enter in the new credentials that were generated in step #2, click `Try it out`, enter your namespace and click `Execute`. The results are returned in a JSON object.
175+
169176
## 6. View metrics
170177

171178
The following metrics can be viewed in real-time for the Services that you configure on the Gateway:
@@ -183,7 +190,7 @@ You can also access the metrics from the `API Services Portal`.
183190

184191
## 7. Grant access to others
185192

186-
The `acl` command provides a way to update the access for the namespace. It expects an all-inclusive membership list, so the `--users` should have the full list of members. Any user that is a member but not in the `--users` list will be removed from the namespace.
193+
The `acl` command provides a way to update the access for the namespace. It expects an all-inclusive membership list, so if a user is not either part of the `--users` list or the `--managers` list, they will be removed from the namespace.
187194

188195
For elevated privileges (such as managing Service Accounts), add the usernames to the `--managers` argument.
189196

@@ -217,26 +224,32 @@ jobs:
217224
- uses: actions/checkout@v2
218225
with:
219226
fetch-depth: 0
227+
220228
- uses: actions/setup-node@v1
221229
with:
222230
node-version: 10
223231
TOKEN: ${{ secrets.GITHUB_TOKEN }}
224-
- env:
225-
GWA_NAMESPACE: global
232+
233+
- name: Get GWA Command Line
226234
run: |
227-
curl -L -O https://bcgov.github.io/gwa-cli/gwa_v1.0.10_linux_x64.zip
228-
unzip gwa_v1.0.10_linux_x64.zip
229-
export PATH=$PATH:`pwd`
235+
curl -L -O https://github.com/bcgov/gwa-cli/releases/download/v1.0.14/gwa-cli-linux.zip
236+
unzip gwa-cli-linux.zip
237+
mv gwa-cli-linux gwa
238+
export PATH=`pwd`:$PATH
230239
231-
cd ../.gwa/{$GWA_NAMESPACE}
240+
- name: Apply Namespace Configuration
241+
run: |
242+
export NS="platform"
243+
export PATH=`pwd`:$PATH
244+
cd .gwa/$NS
232245
233246
gwa init -T \
234-
--namespace=${GWA_NAMESPACE} \
235-
--client-id=${{ secrets.GWA_ACCT_ID }} \
236-
--client-secret=${{ secrets.GWA_ACCT_SECRET }}
247+
--namespace=$NS \
248+
--client-id=${{ secrets.TEST_GWA_ACCT_ID }} \
249+
--client-secret=${{ secrets.TEST_GWA_ACCT_SECRET }}
237250
238251
gwa pg
239252
240-
gwa acl --users acope@idir --managers acope@idir
241-
253+
gwa acl --managers acope@idir
254+
242255
```

‎docs/OCP4-MIGRATION.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
2+
# Upstream Services Migrating from OCP3 to OCP4
3+
4+
## What do I need to do to setup my Service on the new APS Gateway?
5+
6+
### Configure the APS Gateway
7+
8+
As part of migrating your service to OCP4, you will need to prepare the Gateway configuration for your Service. An example of what it would look like:
9+
10+
```
11+
services:
12+
- host: <my_ocp4_service>.<my_ocp4_namespace>.svc
13+
name: <my_ocp4_service>
14+
port: 80
15+
protocol: http
16+
```
17+
18+
The routes will have a minor change by no longer using the `https` protocol as SSL termination will be performed at the `OCP4 Edge` and the traffic to the `APS Gateway` will be unencrypted:
19+
20+
```
21+
services:
22+
- host: <my_ocp4_service>.<my_ocp4_namespace>.svc
23+
name: <my_ocp4_service>
24+
port: 80
25+
protocol: http
26+
routes:
27+
- name: <my_ocp4_service>-route-get
28+
hosts:
29+
- kirk.data.gov.bc.ca
30+
protocols:
31+
- http
32+
methods:
33+
- GET
34+
paths:
35+
- "/"
36+
```
37+
38+
The appropriate `tags` need to be added and optionally any `plugins` added.
39+
40+
Apply your changes to the `APS Gateway` by running the `gwa pg` command (see the [API Provider Flow](/USER-JOURNEY.md) for detailed instructions).
41+
42+
### Add the NetworkSecurityPolicy (NSP)
43+
44+
The `APS Gateway` needs to be able to route to your service from within the OCP4 cluster. To do this, add the following NSP to your OCP4 namespace (update the destination label matching based on your Pod labels):
45+
46+
```
47+
kind: NetworkSecurityPolicy
48+
apiVersion: security.devops.gov.bc.ca/v1alpha1
49+
metadata:
50+
name: aps-gateway-to-your-upstream-api
51+
spec:
52+
description: |
53+
allow the aps gateway to route traffic to your api
54+
source:
55+
- - $namespace=264e6f-test
56+
- - $namespace=264e6f-prod
57+
destination:
58+
- - app.kubernetes.io/name=my-upstream-api
59+
```
60+
61+
### Remove all Routes/Ingress
62+
63+
In `OCP3` you most likely had routes for `*.api`, `*.data` or `*.apps`. For `OCP4` your `Pod Service` does not need to have an external route because the `APS Gateway` is acting as the entry point for your service.
64+
65+
You should ensure that you do not have any routes that have the host as `*.api.gov.bc.ca`, `*.data.gov.bc.ca`, or `*.apps.gov.bc.ca`. Having routes with these hosts will conflict with the routes that the `APS Gateway` will create automatically based on your configuration. It is ok to create routes under the `.apps.silver.devops.gov.bc.ca` domain for dev and testing.
66+
67+
### Regression test your Service
68+
69+
Regression testing is being performed in the `APS Gateway` test environment. When you apply your changes in the test environment, your routing URLs are transformed into a custom vanity domain.
70+
71+
For example, a route for `kirk.data.gov.bc.ca` results in a public route:
72+
73+
`https://kirk-data-gov-bc-ca.test.apsgw.xyz`
74+
75+
This is what you will use to peform your regression testing. To find out the exact public route for your services, go to the `Services` tab on the `APS Services Portal`.
76+
77+
78+
### Switch production traffic to your OCP4 Service
79+
80+
**APS Gateway Production Setup**
81+
82+
When the Production instance of the `APS Gateway` is ready, communication will go out to API Owners as additional tasks will be required to enable the service on our Production environment:
83+
84+
| Task | Responsibility |
85+
| -------------------------------------------------------- | -------------- |
86+
| Create namespace and service account on the `APS Gateway` production instance | API Owner |
87+
| Apply your dev/test/prod Gateway configuration (either in your CI/CD pipeline or manually) | API Owner |
88+
| Update `Kong14` Service upstream host from `https://142.34.143.180` to `https://142.34.194.118` | APS Team
89+
| Decommission your OCP3 projects | API Owner |
90+
91+
### Final big-bang switchover
92+
93+
At the completion of regression testing, a big-bang switchover to the new APS Gateway will be performed by updating the DNS to point to the `OCP4 Edge`.
94+
95+
| Task | Responsibility |
96+
| -------------------------------------------------------- | -------------- |
97+
| Ownership of the Gateway configuration transitioned to Service teams | API Owners
98+
| Regression testing of services complete | API Owners
99+
| APS Gateway Production Setup completed | API Owners
100+
| **Switchover** Update DNS for *.api.gov.bc.ca, *.data.gov.bc.ca to point to 142.34.194.118 | APS Team |
101+
102+
103+
At the time of switchover, we expect that there will be a mix of upstream services running on OCP3, running on OCP4, running on DataBC Servers and running on external cloud. But the expectation is that all the teams will have ownership of the Gateway configuration for their respective Services, and empowered to do ongoing changes.
104+
105+
106+
# Appendix
107+
108+
## Terminology
109+
110+
* `WAM` : Web Application Management
111+
* `Kong14` : The current Kong 0.14 implementation run on Data BC infrastructure (142.34.140.8, 142.34.5.17)
112+
* `APS Gateway` : The new Kong 2.1.x implementation run on Openshift Container Platform V4 (OCP4)
113+
* `OCP3 Edge` : The OCP3 Edge server (142.34.208.209, more known as *.pathfinder.gov.bc.ca, or 142.34.143.180, more known as *.pathfinder.bcgov)
114+
* `OCP4 Edge` : The OCP4 Edge server (142.34.194.118), more known as *.apps.silver.devops.gov.bc.ca)
115+
* `Pod Service` : The actual Kubernetes Service that is run within your Pod (`oc get services`)
116+
* `OCP3 Route` : A Route on the OCP3 Edge that is the relationship between your Pod Service and the OCP3 Edge
117+
118+
## How it works currently
119+
120+
Current (Oct 2020) flow for `*.api.gov.bc.ca` and `*.data.gov.bc.ca` domains where the upstream is on OCP3, is:
121+
122+
`Internet` -> `Kong14` -> `OCP3 Edge` -> `Pod Service`
123+
124+
On `Kong14`, a Service example looks something like this and is administered via email to David or Craig, and they use Konga:
125+
126+
```
127+
services:
128+
- host: 142.34.143.180
129+
name: apidata-ocp-https-kirk-jobs
130+
port: 443
131+
protocol: https
132+
```
133+
134+
With the route:
135+
136+
```
137+
routes:
138+
- hosts:
139+
- kirk.data.gov.bc.ca
140+
protocols:
141+
- http
142+
- https
143+
```
144+
145+
The `kirk` team is then responsible for creating an `OCP3 Route` and the `Pod Service`.
146+

‎docs/samples/service-plugins/service-jwt-keycloak.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
services:
2+
- name: MY_REST_API
3+
tags: [ _NS_ ]
4+
plugins:
5+
- name: jwt-keycloak
6+
tags: [ _NS_ ]
7+
enabled: true
8+
config:
9+
client_roles: null
10+
allowed_iss:
11+
- https://authz-264e6f-dev.apps.silver.devops.gov.bc.ca/auth/realms/aps
12+
run_on_preflight: true
13+
iss_key_grace_period: 10
14+
maximum_expiration: 0
15+
claims_to_verify:
16+
- exp
17+
consumer_match_claim_custom_id: false
18+
cookie_names: []
19+
scope: null
20+
uri_param_names:
21+
- jwt
22+
roles: null
23+
consumer_match: false
24+
well_known_template: https://authz-264e6f-dev.apps.silver.devops.gov.bc.ca/auth/realms/aps/.well-known/openid-configuration
25+
consumer_match_ignore_not_found: false
26+
anonymous: null
27+
algorithm: RS256
28+
realm_roles: null
29+
consumer_match_claim: azp

‎microservices/gatewayApi/clients/kong.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,5 +25,5 @@ def recurse_get_records (result, url):
2525
result.extend(data)
2626

2727
if json['next'] is not None:
28-
recurse_get_routes (result, json['next'])
28+
recurse_get_records (result, json['next'])
2929
return result

‎microservices/gatewayApi/v1/routes/gw_status.py‎

Lines changed: 26 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
import urllib3
55
import certifi
66
import socket
7+
from urllib.parse import urlparse
78
from flask import Blueprint, jsonify, request, Response, make_response, abort, g, current_app as app
89

910
from v1.auth.auth import admin_jwt, enforce_authorization
@@ -32,11 +33,12 @@ def get_statuses(namespace: str) -> object:
3233
status = "UP"
3334
reason = ""
3435

35-
36+
actual_host = None
3637
host = None
3738
for route in routes:
3839
if route['service']['id'] == service['id'] and 'hosts' in route:
39-
host = clean_host(route['hosts'][0])
40+
actual_host = route['hosts'][0]
41+
host = clean_host(actual_host)
4042

4143
try:
4244
addr = socket.gethostbyname(service['host'])
@@ -49,23 +51,30 @@ def get_statuses(namespace: str) -> object:
4951
try:
5052
headers = {}
5153
if host is None or service['host'].endswith('.svc'):
52-
r = requests.get(url, headers=headers, timeout=1.0)
54+
r = requests.get(url, headers=headers, timeout=3.0)
5355
status_code = r.status_code
5456
else:
57+
u = urlparse(url)
58+
5559
headers['Host'] = host
56-
log.info("GET %-30s %s" % (url, headers))
60+
log.info("GET %-30s %s" % ("%s://%s" % (u.scheme, u.netloc), headers))
5761

5862
urllib3.disable_warnings()
59-
pool = urllib3.HTTPSConnectionPool(
60-
service['host'],
61-
assert_hostname=host,
62-
server_hostname=host,
63-
cert_reqs='CERT_NONE',
64-
ca_certs=certifi.where()
65-
)
63+
if u.scheme == "https":
64+
pool = urllib3.HTTPSConnectionPool(
65+
"%s" % (u.netloc),
66+
assert_hostname=host,
67+
server_hostname=host,
68+
cert_reqs='CERT_NONE',
69+
ca_certs=certifi.where()
70+
)
71+
else:
72+
pool = urllib3.HTTPConnectionPool(
73+
"%s" % (u.netloc)
74+
)
6675
req = pool.urlopen(
6776
"GET",
68-
"/",
77+
u.path,
6978
headers={"Host": host},
7079
assert_same_host=False,
7180
timeout=1.0,
@@ -112,14 +121,17 @@ def get_statuses(namespace: str) -> object:
112121
reason = "UNKNOWN"
113122

114123
log.info("GET %-30s %s" % (url,reason))
115-
response.append({"name": service['name'], "upstream": url, "status": status, "reason": reason, "host": host})
124+
response.append({"name": service['name'], "upstream": url, "status": status, "reason": reason, "host": host, "test_host": actual_host})
116125

117126
return make_response(jsonify(response))
118127

119128
def build_url (s):
120129
schema = default(s, "protocol", "http")
130+
defaultPort = 80
131+
if schema == "https":
132+
defaultPort = 443
121133
host = s['host']
122-
port = default(s, "port", 80)
134+
port = default(s, "port", defaultPort)
123135
path = default(s, "path", "/")
124136
if 'url' in s:
125137
return s['url']

0 commit comments

Comments
 (0)