Skip to content

Commit bd2aa64

Browse files
Omarabdul3zizOmarabdul3ziz
authored
Merge pull request #1155 from threefoldtech/development_compose_init_dependency_resolution
implementation of a docker-compose-like cli for managing deployment of services simultaneously
2 parents b0f3b22 + 124fe56 commit bd2aa64

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+1635
-828
lines changed

‎grid-compose/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,5 @@ bin/*
22
.pre-commit-config.yaml
33
out
44
full_example.yml
5+
invalid
6+
grid-compose.yml

‎grid-compose/README.md

Lines changed: 79 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Grid-Compose
22

3-
is a tool for running multi-vm applications on TFGrid defined using a Yaml formatted file.
3+
is a tool similar to docker-compose created for running multi-vm applications on TFGrid defined using a Yaml formatted file.
4+
5+
The yaml file's structure is defined in [docs/config](docs/config.md).
46

57
## Usage
68

@@ -13,7 +15,7 @@ is a tool for running multi-vm applications on TFGrid defined using a Yaml forma
1315
grid-compose [OPTIONS] [COMMAND]
1416

1517
OPTIONS:
16-
-f, --file: path to yaml file, default is ./grid-compose.yaml
18+
-f, --file: path to yaml file, default is ./grid-compose.yml
1719

1820
COMMANDS:
1921
- version: shows the project version
@@ -22,7 +24,6 @@ COMMANDS:
2224
- ps: list deployments on the grid
2325
OPTIONS:
2426
- -v, --verbose: show full details of each deployment
25-
- -o, --output: redirects the output to a file given its path
2627
```
2728

2829
Export env vars using:
@@ -66,26 +67,97 @@ Refer to examples in the [examples](examples) directory to have a look at differ
6667

6768
OPTIONS:
6869

69-
- `-f, --file`: path to the yaml file, default is `./grid-compose.yaml`
70+
- `-f, --file`: path to the yaml file, default is `./grid-compose.yml`
71+
72+
### Example
73+
74+
```bash
75+
./bin/grid-compose up
76+
```
77+
78+
output:
79+
80+
```bash
81+
3:40AM INF starting peer session=tf-848216 twin=8658
82+
3:40AM INF deploying network... name=miaminet node_id=14
83+
3:41AM INF deployed successfully
84+
3:41AM INF deploying vm... name=database node_id=14
85+
3:41AM INF deployed successfully
86+
3:41AM INF deploying network... name=miaminet node_id=14
87+
3:41AM INF deployed successfully
88+
3:41AM INF deploying vm... name=server node_id=14
89+
3:41AM INF deployed successfully
90+
3:41AM INF all deployments deployed successfully
91+
```
7092

7193
### down
7294

95+
The down command cancels all deployments on the grid.
96+
7397
```bash
7498
./bin/grid-compose down [OPTIONS]
7599
```
76100

77101
OPTIONS:
78102

79-
- `-f, --file`: path to the yaml file, default is `./grid-compose.yaml`
103+
- `-f, --file`: path to the yaml file, default is `./grid-compose.yml`
104+
105+
### Example
106+
107+
```bash
108+
./bin/grid-compose down
109+
```
110+
111+
output:
112+
113+
```bash
114+
3:45AM INF starting peer session=tf-854215 twin=8658
115+
3:45AM INF canceling deployments projectName=compose/8658/net1
116+
3:45AM INF canceling contracts project name=compose/8658/net1
117+
3:45AM INF project is canceled project name=compose/8658/net1
118+
```
80119

81120
### ps
82121

122+
The ps command lists all deployments on the grid.
123+
83124
```bash
84125
./bin/grid-compose ps [FLAGS] [OPTIONS]
85126
```
86127

87128
OPTIONS:
88129

89-
- `-f, --file`: path to the yaml file, default is `./grid-compose.yaml`
130+
- `-f, --file`: path to the yaml file, default is `./grid-compose.yml`
131+
132+
### Example
133+
134+
```bash
135+
./bin/grid-compose ps
136+
```
137+
138+
output:
139+
140+
```bash
141+
3:43AM INF starting peer session=tf-851312 twin=8658
142+
143+
Deployment Name | Node ID | Network | Services | Storage | State | IP Address
144+
------------------------------------------------------------------------------------------------------------------------------------------------------
145+
dl_database | 14 | miaminet | database | dbdata | ok | wireguard: 10.20.2.2
146+
dl_server | 14 | miaminet | server | webdata | ok | wireguard: 10.20.2.3
147+
```
148+
149+
FLAGS:
150+
90151
- `-v, --verbose`: show full details of each deployment
91-
- `-o, --output`: redirects the output to a file given its path(in json format)
152+
153+
### version
154+
155+
The version command shows the project's current version.
156+
157+
```bash
158+
./bin/grid-compose version
159+
```
160+
161+
## Future Work
162+
163+
Refer to [docs/future_work.md](docs/future_work.md) for more information on the future work that is to be done on the grid-compose project.

‎grid-compose/cmd/down.go

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,20 @@ package cmd
33
import (
44
"github.com/rs/zerolog/log"
55
"github.com/spf13/cobra"
6+
"github.com/threefoldtech/tfgrid-sdk-go/grid-compose/internal/app"
67
)
78

89
var downCmd = &cobra.Command{
910
Use: "down",
1011
Short: "cancel your project on the grid",
1112
Run: func(cmd *cobra.Command, args []string) {
13+
app, ok := cmd.Context().Value("app").(*app.App)
14+
if !ok {
15+
log.Fatal().Msg("app not found in context")
16+
}
17+
1218
if err := app.Down(); err != nil {
1319
log.Fatal().Err(err).Send()
1420
}
1521
},
1622
}
17-
18-
func init() {
19-
rootCmd.AddCommand(downCmd)
20-
}

‎grid-compose/cmd/ps.go

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,22 +3,25 @@ package cmd
33
import (
44
"github.com/rs/zerolog/log"
55
"github.com/spf13/cobra"
6+
"github.com/threefoldtech/tfgrid-sdk-go/grid-compose/internal/app"
67
)
78

89
var psCmd = &cobra.Command{
910
Use: "ps",
1011
Short: "list deployments on the grid",
1112
Run: func(cmd *cobra.Command, args []string) {
12-
flags := cmd.Flags()
13+
verbose, err := cmd.Flags().GetBool("verbose")
14+
if err != nil {
15+
log.Fatal().Err(err).Send()
16+
}
17+
18+
app, ok := cmd.Context().Value("app").(*app.App)
19+
if !ok {
20+
log.Fatal().Msg("app not found in context")
21+
}
1322

14-
if err := app.Ps(cmd.Context(), flags); err != nil {
23+
if err := app.Ps(cmd.Context(), verbose); err != nil {
1524
log.Fatal().Err(err).Send()
1625
}
1726
},
1827
}
19-
20-
func init() {
21-
psCmd.PersistentFlags().BoolP("verbose", "v", false, "all information about deployed services")
22-
psCmd.PersistentFlags().StringP("output", "o", "", "output result to a file")
23-
rootCmd.AddCommand(psCmd)
24-
}

‎grid-compose/cmd/root.go

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,13 @@
11
package cmd
22

33
import (
4+
"context"
45
"os"
56

67
"github.com/rs/zerolog"
78
"github.com/rs/zerolog/log"
89
"github.com/spf13/cobra"
9-
"github.com/threefoldtech/tfgrid-sdk-go/grid-compose/internal"
10-
)
11-
12-
var (
13-
app *internal.App
14-
configPath string
15-
network string
16-
mnemonic string
10+
"github.com/threefoldtech/tfgrid-sdk-go/grid-compose/internal/app"
1711
)
1812

1913
func Execute() {
@@ -22,23 +16,35 @@ func Execute() {
2216
}
2317
}
2418

25-
// TODO: Validate command line arguments
19+
// TODO: validate command line arguments
2620
var rootCmd = &cobra.Command{
2721
Use: "grid-compose",
2822
Short: "Grid-Compose is a tool for running multi-vm applications on TFGrid defined using a Yaml formatted file.",
2923
PersistentPreRun: func(cmd *cobra.Command, args []string) {
30-
var err error
31-
app, err = internal.NewApp(network, mnemonic, configPath)
24+
network := os.Getenv("NETWORK")
25+
mnemonic := os.Getenv("MNEMONIC")
26+
configPath, _ := cmd.Flags().GetString("file")
27+
28+
app, err := app.NewApp(network, mnemonic, configPath)
29+
3230
if err != nil {
3331
log.Fatal().Err(err).Send()
3432
}
33+
34+
ctx := context.WithValue(cmd.Context(), "app", app)
35+
cmd.SetContext(ctx)
3536
},
3637
}
3738

3839
func init() {
39-
network = os.Getenv("NETWORK")
40-
mnemonic = os.Getenv("MNEMONIC")
41-
rootCmd.PersistentFlags().StringVarP(&configPath, "file", "f", "./grid-compose.yaml", "the grid-compose configuration file")
40+
rootCmd.PersistentFlags().StringP("file", "f", "./grid-compose.yml", "the grid-compose configuration file")
41+
42+
rootCmd.AddCommand(downCmd)
43+
rootCmd.AddCommand(upCmd)
44+
rootCmd.AddCommand(versionCmd)
45+
46+
psCmd.PersistentFlags().BoolP("verbose", "v", false, "all information about deployed services")
47+
rootCmd.AddCommand(psCmd)
4248

4349
log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})
4450
}

‎grid-compose/cmd/up.go

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,20 @@ package cmd
33
import (
44
"github.com/rs/zerolog/log"
55
"github.com/spf13/cobra"
6+
"github.com/threefoldtech/tfgrid-sdk-go/grid-compose/internal/app"
67
)
78

89
var upCmd = &cobra.Command{
910
Use: "up",
1011
Short: "deploy application on the grid",
1112
Run: func(cmd *cobra.Command, args []string) {
13+
app, ok := cmd.Context().Value("app").(*app.App)
14+
if !ok {
15+
log.Fatal().Msg("app not found in context")
16+
}
17+
1218
if err := app.Up(cmd.Context()); err != nil {
1319
log.Fatal().Err(err).Send()
1420
}
1521
},
1622
}
17-
18-
func init() {
19-
rootCmd.AddCommand(upCmd)
20-
}

‎grid-compose/cmd/version.go

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
package cmd
33

44
import (
5-
"fmt"
5+
"log"
66

77
"github.com/spf13/cobra"
88
)
@@ -16,11 +16,7 @@ var versionCmd = &cobra.Command{
1616
Use: "version",
1717
Short: "Get latest build tag",
1818
Run: func(cmd *cobra.Command, args []string) {
19-
fmt.Println(version)
20-
fmt.Println(commit)
19+
log.Println(version)
20+
log.Println(commit)
2121
},
2222
}
23-
24-
func init() {
25-
rootCmd.AddCommand(versionCmd)
26-
}

‎grid-compose/docs/cases.md

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
These are most if not all the cases supported by the grid compose cli when deploying one or more service to the grid.
1+
These are most if not all the cases supported by the grid compose tool when deploying one or more services on the grid.
22

33
## Single Service
44

@@ -9,7 +9,7 @@ This is probably the simplest case there is.
99
- Filter the nodes based on the resources given to the service and choose a random one.
1010
- Generate a default network and assign it to the deployment.
1111

12-
Refer to example [single_service_1.yml](../examples/single_service_1.yml)
12+
Refer to example [single_service_1.yml](/examples/single-service/single_service_1.yml)
1313

1414
### Case 2 - Node ID Given + No Assigned Network
1515

@@ -19,14 +19,14 @@ Refer to example [single_service_1.yml](../examples/single_service_1.yml)
1919
- If there is a node available, prompt the user if they would like to use it instead.
2020
- Generate a default network and assign it to the deployment.
2121

22-
Refer to example [single_service_2.yml](../examples/single_service_2.yml)
22+
Refer to example [single_service_2.yml](/examples/single-service/single_service_2.yml)
2323

2424
### Case 3 - Assigned Network
2525

2626
- Either use the assigned node id or filter the nodes for an available node if no node id given.
2727
- Use the network assigned to the service when deploying.
2828

29-
Refer to example [single_service_3.yml](../examples/single_service_3.yml)
29+
Refer to example [single_service_3.yml](/examples/single-service/single_service_3.yml)
3030

3131
## Multiple Services
3232

@@ -55,10 +55,29 @@ If no networks are defined, then all the services will use the **default generat
5555

5656
Refer to examples
5757

58-
- [two_services_same_network_1.yml](../examples/two_services_same_network_1.yml)
59-
- [two_services_same_network_2.yml](../examples/two_services_same_network_2.yml)
60-
- [two_services_same_network_3.yml](../examples/two_services_same_network_3.yml)
58+
- [two_services_same_network_1.yml](/examples/multiple-services/two_services_same_network_1.yml)
59+
- [two_services_same_network_2.yml](/examples/multiple-services/two_services_same_network_2.yml)
60+
- [two_services_same_network_3.yml](/examples/multiple-services/two_services_same_network_3.yml)
6161

6262
### Different Networks
6363

6464
Simple divide the services into groups having the same network(given or generated) and deal with each group using the approached described in the previous [section](#same-networkno-network).
65+
66+
Refer to examples
67+
68+
- [multiple_services_diff_network_1.yml](/examples/multiple-services/multiple_services_diff_network_1.yml)
69+
- [multiple_services_diff_network_2.yml](/examples/multiple-services/multiple_services_diff_network_2.yml)
70+
- [multiple_services_diff_network_3.yml](/examples/multiple-services/multiple_services_diff_network_3.yml)
71+
72+
## Dependencies
73+
74+
The tool supports deploying services that depend on each other. You can define dependencies in the yaml file by using the `depends_on` key, just like in docker-compose.
75+
76+
Refer to examples:
77+
78+
- deploying services that depend on each other on different networks:
79+
- [diff_networks.yml](/examples/dependency/diff_networks.yml)
80+
- deploying services that depend on each other on the same network:
81+
- [same_network.yml](/examples/dependency/same_network.yml)
82+
- a service that would depend on multiple services:
83+
- [multiple_dependencies.yml](/examples/dependency/multiple_dependencies.yml)

0 commit comments

Comments
 (0)