Skip to content

Commit 3a6f14e

Browse files
committed
Update docs
Signed-off-by: Ching Yi, Chan <qrtt1@infuseai.io>
1 parent e97d749 commit 3a6f14e

File tree

12 files changed

+1058
-727
lines changed

12 files changed

+1058
-727
lines changed

docs/CLI/admin/datasets.md

Lines changed: 446 additions & 0 deletions
Large diffs are not rendered by default.

docs/CLI/apps.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,4 +214,4 @@ stop: False
214214
status: Ready
215215
message: Deployment is ready
216216
pods: [{'logEndpoint': 'http://primehub-python-sdk.primehub.io/api/logs/pods/app-code-server-26fcc-765bf579c5-srcft'}]
217-
```
217+
```

docs/CLI/datasets.md

Lines changed: 11 additions & 277 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,6 @@ primehub datasets list
5555

5656
## Examples
5757

58-
### Query datasets
59-
6058
The `datasets` command is a group specific resource. It only works after the `group` assigned.
6159

6260
Using `list` to find all datasets in your group:
@@ -66,286 +64,22 @@ $ primehub datasets list
6664
```
6765

6866
```
69-
id name displayName description type
70-
----------- ----------- -------------------------- ------------------------------- ------
71-
pv-dataset pv-dataset the dataset created by SDK It is a PV dataset pv
72-
env-dataset env-dataset env-dataset make changes to the description env
67+
id name displayName description type
68+
------ ------ ------------- ------------- ------
69+
kaggle kaggle kaggle pv
7370
```
7471

7572
If you already know the name of a dataset, use the `get` to get a single entry:
7673

7774
```
78-
$ primehub datasets get dataset
75+
$ primehub datasets get kaggle
7976
```
8077

8178
```
82-
id: pv-dataset
83-
name: pv-dataset
84-
displayName: the dataset created by SDK
85-
description: It is a PV dataset
86-
type: pv
87-
pvProvisioning: auto
88-
volumeSize: 1
89-
enableUploadServer: True
90-
uploadServerLink: http://primehub-python-sdk.primehub.io/dataset/hub/pv-dataset/browse
91-
global: False
92-
groups: [{'id': 'a962305b-c884-4413-9358-ef56373b287c', 'name': 'foobarbar', 'displayName': '', 'writable': False}, {'id': 'a7a283b5-c0e2-4b79-a78c-39c630324762', 'name': 'phusers', 'displayName': 'primehub users', 'writable': False}]
93-
```
94-
95-
### Admin actions for datasets
96-
97-
These actions only can be used by administrators:
98-
99-
* create
100-
* update
101-
* delete
102-
103-
For `create` and `update` require a dataset configuration, please see above examples.
104-
105-
### Fields for creating or updating
106-
107-
| field | required | type | description |
108-
| --- | --- | --- | --- |
109-
| name | required | string | it should be a valid resource name for kubernetes |
110-
| displayName | optional | string | display name for this dataset |
111-
| description | optional | string | |
112-
| global | optional | boolean | when a dataset is global, it could be seen for each group |
113-
| type | required | string | one of ['pv', 'nfs', 'hostPath', 'git', 'env'] |
114-
| url | conditional | string | **MUST** use with `git` type |
115-
| pvProvisioning | conditional | string | onf of ['auto', 'manual'], **MUST** use with `pv` type. This field only uses in `CREATE` action |
116-
| nfsServer | conditional | string | **MUST** use with `nfs` type |
117-
| nfsPath | conditional | string | **MUST** use with `nfs` type |
118-
| hostPath | conditional | string | **MUST** use with `hostPath` type |
119-
| variables | optional | dict | **MAY** use with `env` type. It is key value pairs. All values have to a string value. For example: `{"key1":"value1","key2":"value2"}`. |
120-
| groups | optional | list of connected groups (dict) | please see the `connect` examples |
121-
| secret | optional | dict | **MAY** use with `git` type | bind a `secret` to the `git` dataset |
122-
| volumeSize | conditional | integer | **MUST** use with `pv` type. The unit is `GB`.|
123-
| enableUploadServer | optional | boolean | it only works with one of ['pv', 'nfs', 'hostPath'] writable types |
124-
125-
> There is a simple rule to use fields for `UPDATE`. All required fields should not be in the payload.
126-
127-
For example, there is a configuration for creating env dataset:
128-
129-
```bash
130-
primehub datasets create <<EOF
131-
{
132-
"name": "env-dataset",
133-
"description": "",
134-
"type": "env",
135-
"variables": {
136-
"ENV": "prod",
137-
"LUCKY_NUMBER": "7"
138-
}
139-
}
140-
EOF
141-
```
142-
143-
After removing required `name` and `type` fields, it could be used with updating:
144-
145-
```bash
146-
primehub datasets update env-dataset <<EOF
147-
{
148-
"description": "make changes to the description",
149-
"variables": {
150-
"ENV": "prod",
151-
"LUCKY_NUMBER": "8"
152-
}
153-
}
154-
EOF
155-
```
156-
157-
For updating, giving things that you want to make different:
158-
159-
```bash
160-
primehub datasets update env-dataset <<EOF
161-
{
162-
"groups": {
163-
"connect": [
164-
{
165-
"id": "a7a283b5-c0e2-4b79-a78c-39c630324762",
166-
"writable": false
167-
}
168-
]
169-
}
170-
}
171-
EOF
172-
```
173-
174-
175-
176-
177-
178-
### PV type
179-
180-
```json
181-
{
182-
"name": "pv-dataset",
183-
"displayName": "the dataset created by SDK",
184-
"description": "It is a PV dataset",
185-
"type": "pv",
186-
"global": false,
187-
"groups": {
188-
"connect": [
189-
{
190-
"id": "a7a283b5-c0e2-4b79-a78c-39c630324762",
191-
"writable": true
192-
},
193-
{
194-
"id": "a962305b-c884-4413-9358-ef56373b287c",
195-
"writable": false
196-
}
197-
]
198-
},
199-
"pvProvisioning": "auto",
200-
"volumeSize": 1
201-
}
202-
```
203-
204-
Save the configuration to `create-dataset.json` and run `create`:
205-
206-
```
207-
primehub datasets create --file create-dataset.json
208-
```
209-
210-
The example creates a PV dataset. According to the type `pv`, these fields become `required`:
211-
* pvProvisioning: how does the PV create? `auto` means PV will create automatically, `manual` means the system administrator should create it.
212-
* volumeSize: the capacity in GB when `auto` creates it.
213-
214-
The `group.connect` will bind two groups to the dataset. One is a writable group and another is readonly group.
215-
216-
217-
### NFS type
218-
219-
```json
220-
{
221-
"name": "nfs-dataset",
222-
"type": "nfs",
223-
"groups": {
224-
"connect": [
225-
{
226-
"id": "a7a283b5-c0e2-4b79-a78c-39c630324762",
227-
"writable": true
228-
}
229-
]
230-
},
231-
"nfsServer": "1.2.3.4",
232-
"nfsPath": "/data"
233-
}
234-
```
235-
236-
Save the configuration to `create-dataset.json` and run `create`:
237-
238-
```
239-
primehub datasets create --file create-dataset.json
240-
```
241-
242-
The example creates a NFS dataset. According to the type `nfs`, these fields become `required`:
243-
* nfsServer: the address of a NFS server
244-
* nfsPath: the mount path of a NFS server
245-
246-
### HostPath type
247-
248-
```json
249-
{
250-
"name": "host-path-dataset",
251-
"description": "",
252-
"type": "hostPath",
253-
"groups": {
254-
"connect": [
255-
{
256-
"id": "a7a283b5-c0e2-4b79-a78c-39c630324762",
257-
"writable": true
258-
}
259-
]
260-
},
261-
"hostPath": "/opt/data"
262-
}
263-
```
264-
265-
Save the configuration to `create-dataset.json` and run `create`:
266-
267-
```
268-
primehub datasets create --file create-dataset.json
269-
```
270-
271-
The example creates a hostPath dataset. According to the type `hostPath`, the `hostPath` field becomes `required`. You should put an absolute path that available in the node.
272-
273-
### Git type
274-
275-
```json
276-
{
277-
"name": "git-dataset",
278-
"type": "git",
279-
"url": "https://github.com/datasets/covid-19"
280-
}
281-
```
282-
283-
or with a `secret`
284-
285-
```json
286-
{
287-
"name": "git-dataset",
288-
"type": "git",
289-
"url": "https://github.com/datasets/covid-19",
290-
"secret": {
291-
"connect": {
292-
"id": "gitsync-secret-public-key-for-git-repo"
293-
}
294-
}
295-
}
296-
```
297-
298-
Save the configuration to `create-dataset.json` and run `create`:
299-
300-
```
301-
primehub datasets create --file create-dataset.json
302-
```
303-
304-
The example creates a git dataset. According to the type `git`, `url` field becomes `required`. You should put a git repository url.
305-
306-
If the url needs a credential, you could use `secret` to connect the pre-set secret (an SSH public key).
307-
308-
### ENV type
309-
310-
```json
311-
{
312-
"name": "env-dataset",
313-
"description": "",
314-
"type": "env",
315-
"variables": {
316-
"ENV": "prod",
317-
"LUCKY_NUMBER": "7"
318-
}
319-
}
320-
```
321-
322-
Save the configuration to `create-dataset.json` and run `create`:
323-
324-
```
325-
primehub datasets create --file create-dataset.json
326-
```
327-
328-
The example creates an ENV dataset. According to the type `env`, `variables` field becomes `required`. You could put many key-value pairs. Be careful, the key and value should be string values.
329-
330-
### Group connect/disconnect
331-
332-
All dataset types could connect or disconnect to groups, but there is subtle difference between `CREATE` and `UPDATE`.
333-
334-
```json
335-
{
336-
"connect": [
337-
{
338-
"id": "a7a283b5-c0e2-4b79-a78c-39c630324762",
339-
"writable": true
340-
}
341-
],
342-
"disconnect": [
343-
{
344-
"id": "a7a283b5-c0e2-4b79-a78c-39c630324762"
345-
}
346-
]
347-
}
348-
```
349-
350-
* `disconnect` is only available for `UPDATE`
351-
* `connect` are both available (`CREATE` `UPDATE`)
79+
primehub datasets get kaggle
80+
id: kaggle
81+
name: kaggle
82+
displayName: kaggle
83+
description:
84+
type: pv
85+
```

docs/CLI/deployments.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -364,4 +364,4 @@ endpoint: http://primehub-python-sdk.primehub.io/deployment/quickstar
364364
availableReplicas: 1
365365
message: Deployment is being deployed and not available now
366366
pods: [{'name': 'deploy-quickstart-iris-cfmnh-68889b97cc-8qgvf'}, {'name': 'deploy-quickstart-iris-cfmnh-68889b97cc-wxzs8'}]
367-
```
367+
```

0 commit comments

Comments
 (0)