Skip to content

Commit 7c0fe8c

Browse files
fubuloubufubuloubu
authored
Merge branch 'main' into feat/set-autosign
2 parents 0cb9ad8 + 282c7ec commit 7c0fe8c

File tree

16 files changed

+1788
-64
lines changed

16 files changed

+1788
-64
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ python3 setup.py install
4747

4848
## Quick Usage
4949

50-
Checkout [the example](./example.py) to see how to use the library.
50+
Checkout [the example](https://github.com/ApeWorX/silverback/blob/main/example.py) to see how to use the library.
5151

5252
```{note}
5353
The example makes use of the [Ape Tokens](https://github.com/ApeWorX/ape-tokens) plugin.

‎docs/commands/cluster.rst‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Cloud Platform
2+
**************
3+
4+
.. click:: silverback._cli:login
5+
:prog: silverback login
6+
:nested: none
7+
8+
.. click:: silverback._cli:cluster
9+
:prog: silverback cluster
10+
:nested: full
11+
:commands: workspaces, new, list, info, health
12+
13+
.. click:: silverback._cli:vars
14+
:prog: silverback cluster vars
15+
:nested: full
16+
:commands: new, list, info, update, remove
17+
18+
.. click:: silverback._cli:bots
19+
:prog: silverback cluster bots
20+
:nested: full
21+
:commands: new, list, info, update, remove, health, start, stop, logs, errors

‎docs/commands/run.rst‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
1-
run
2-
***
1+
Local Development
2+
*****************
33

44
.. click:: silverback._cli:run
5-
:prog: run
5+
:prog: silverback run
6+
:nested: none
7+
8+
.. click:: silverback._cli:worker
9+
:prog: silverback worker
610
:nested: none

‎docs/index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
88
userguides/quickstart
99
userguides/development
10+
userguides/platform
1011
```
1112

1213
```{eval-rst}
@@ -15,6 +16,7 @@
1516
:maxdepth: 1
1617
1718
commands/run.rst
19+
commands/cluster.rst
1820
```
1921

2022
```{eval-rst}

‎docs/userguides/development.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Developing a Silverback Application
1+
# Developing Applications
22

33
In this guide, we are going to show you more details on how to build an application with Silverback.
44

@@ -197,12 +197,12 @@ export SILVERBACK_RESULT_BACKEND_URI="redis://127.0.0.1:6379"
197197
silverback worker -w 2 "example:app"
198198
```
199199

200-
This will run one client and 2 workers and all queue data will be go through Redis.
200+
The client will send tasks to the 2 worker subprocesses, and all task queue and results data will be go through Redis.
201201

202202
## Testing your Application
203203

204204
TODO: Add backtesting mode w/ `silverback test`
205205

206-
## Deploying to the Silverback Platform
206+
## Deploying your Application
207207

208-
TODO: Add packaging and deployment to the Silverback platform, once available.
208+
Check out the [Platform Deployment Userguide](./platform.html) for more information on how to deploy your application to the [Silverback Platform](https://silverback.apeworx.io).

‎docs/userguides/platform.md‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# Deploying Applications
2+
3+
In this guide, we are going to show you more details on how to deploy your application to the [Silverback Platform](https://silverback.apeworx.io).
4+
5+
## Creating a Cluster
6+
7+
The Silverback Platform runs your Applications (or "Bots") on dedicated managed application Clusters.
8+
These Clusters will take care to orchestrate infrastructure, monitor, run your triggers, and collect metrics for your applications.
9+
Each Cluster is bespoke for an individual or organization, and isolates your applications from others on different infrastructure.
10+
11+
Before we deploy our Application, we have to create a Cluster.
12+
If you haven't yet, please sign up for Silverback at [https://silverback.apeworx.io](https://silverback.apeworx.io).
13+
14+
Once you have signed up, you can actually create (and pay for) your Clusters from the Silverback CLI utility by first
15+
logging in to the Platform using [`silverback login`][silverback-login],
16+
and then using [`silverback cluster new`][silverback-cluster-new] to follow the steps necessary to deploy it.
17+
18+
```{note}
19+
The Platform UI will let you create and manage Clusters using a graphical experience, which may be preferred.
20+
The CLI experience is for those working locally who don't want to visit the website, or are locally developing their applications.
21+
```
22+
23+
## Connecting to your Cluster
24+
25+
To connect to a cluster, you can use commands from the [`silverback cluster`][silverback-cluster] subcommand group.
26+
For instance, to list all your available bots on your cluster, use [`silverback cluster bots list`][silverback-cluster-bots-list].
27+
To obtain general information about your cluster, just use [`silverback cluster info`][silverback-cluster-info],
28+
or [`silverback cluster health`][silverback-cluster-health] to see the current status of your Cluster.
29+
30+
If you have no bots, we will first have to containerize our Applications and upload them to a container registry that our Cluster is configured to access.
31+
32+
```{note}
33+
Building a container for your application can be an advanced topic, we have included the `silverback build` subcommand to help assist in generating Dockerfiles.
34+
```
35+
36+
## Building your Bot
37+
38+
TODO: Add build process and describe `silverback build --autogen` and `silverback build --upgrade`
39+
40+
TODO: Add how to debug containers using `silverback run` w/ `taskiq-redis` broker
41+
42+
## Adding Environment Variables
43+
44+
Once you have created your bot application container image, you might know of some environment variables the image requires to run properly.
45+
Thanks to it's flexible plugin system, ape plugins may also require specific environment variables to load as well.
46+
Silverback Clusters include an environment variable management system for exactly this purpose,
47+
which you can manage using [`silverback cluster vars`][silverback-cluster-vars] subcommand.
48+
49+
The environment variable management system makes use of a concept called "Variable Groups" which are distinct collections of environment variables meant to be used together.
50+
These variable groups will help in managing the runtime environment of your Applications by allowing you to segregate different variables depending on each bot's needs.
51+
52+
To create an environment group, use the [`silverback cluster vars new`][silverback-cluster-vars-new] command and give it a name and a set of related variables.
53+
For instance, it may make sense to make a group of variables for your favorite Ape plugins or services, such as RPC Providers, Blockchain Data Indexers, Etherscan, etc.
54+
You might have a database connection that you want all your bots to access.
55+
56+
```{warning}
57+
All environment variables in Silverback Clusters are private, meaning they cannot be viewed after they are uploaded.
58+
However, your Bots will have full access to their values from within their runtime environment, so be careful that you fully understand what you are sharing with your bots.
59+
60+
Also, understand your build dependencies within your container and make sure you are not using any vulnerable or malicious packages.
61+
62+
**NEVER** upload your private key in a plaintext format!
63+
64+
Use _Ape Account Plugins_ such as [`ape-aws`](https://github.com/ApeWorX/ape-aws) to safely manage access to your hosted keys.
65+
```
66+
67+
```{note}
68+
The Etherscan plugin _will not function_ without an API key in the cloud environment.
69+
This will likely create errors running your applications if you use Ape's `Contract` class.
70+
```
71+
72+
To list your Variable Groups, use [`silverback cluster vars list`][silverback-cluster-vars-list].
73+
To see information about a specific Variable Group, including the Environment Variables it includes, use [`silverback cluster vars info`][silverback-cluster-vars-info]
74+
To remove a variable group, use [`silverback cluster vars remove`][silverback-cluster-vars-remove],
75+
76+
```{note}
77+
You can only remove a Variable Group if it is not referenced by any existing Bot.
78+
```
79+
80+
Once you have created all the Variable Group(s) that you need to operate your Bot, you can reference these groups by name when adding your Bot to the cluster.
81+
82+
## Deploying your Bot
83+
84+
You are finally ready to deploy your bot on the Cluster and get it running!
85+
86+
To deploy your Bot, use the [`silverback cluster bots new`][silverback-cluster-bots-new] command and give your bot a name,
87+
container image, network to run on, an account alias (if you want to sign transactions w/ `app.signer`),
88+
and any environment Variable Group(s) the bot needs.
89+
If everything validates successfully, the Cluster will begin orchestrating your deployment for you.
90+
91+
You should monitor the deployment and startup of your bot to make sure it enters the RUNNING state successfully.
92+
You can do this using the [`silverback cluster bots health`][silverback-cluster-bots-health] command.
93+
94+
```{note}
95+
It usually takes a minute or so for your bot to transition from PROVISIONING to STARTUP to the RUNNING state.
96+
If there are any difficulties in downloading your container image, provisioning your desired infrastructure, or if your application encounters an error during the STARTUP phase,
97+
the Bot will not enter into the RUNNING state and will be shut down gracefully into the STOPPED state.
98+
99+
Once in the STOPPED state, you can make any adjustments to the environment Variable Group(s) or other runtime parameters in the Bot config;
100+
or, you can make code changes and deploy a new image for the Bot to use.
101+
Once ready, you can use the `silverback cluster bots start` command to re-start your Bot.
102+
```
103+
104+
If at any time you want to view the configuration of your bot, you can do so using the [`silverback cluster bots info`][silverback-cluster-bots-info] command.
105+
You can also update metadata or configuration of your bot using the [`silverback cluster bots update`][silverback-cluster-bots-update] command.
106+
Lastly, if you want to shutdown and delete your bot, you can do so using the [`silverback cluster bots remove`][silverback-cluster-bots-remove] command.
107+
108+
```{note}
109+
Configuration updates do not redeploy your Bots automatically, you must manually stop and restart your bots for changes to take effect.
110+
```
111+
112+
```{warning}
113+
Removing a Bot will immediately trigger a SHUTDOWN if the Bot is not already STOPPED.
114+
```
115+
116+
## Monitoring your Bot
117+
118+
Once your bot is successfully running in the RUNNING state, you can monitor your bot with a series of commands
119+
under the [`silverback cluster bots`][silverback-cluster-bots] subcommand group.
120+
We already saw how you can use the [`silverback cluster bots list`][silverback-cluster-bots-list] command to see all bots managed by your Cluster (running or not).
121+
122+
To see runtime health information about a specific bot, again use the [`silverback cluster bots health`][silverback-cluster-bots-health] command.
123+
You can view the logs that a specific bot is generating using the [`silverback cluster bots logs`][silverback-cluster-bots-logs] command.
124+
Lastly, you can view unacknowledged errors that your bot has experienced while in the RUNNING state
125+
using the [`silverback cluster bots errors`][silverback-cluster-bots-errors] command.
126+
127+
```{warning}
128+
Once in the RUNNING state, your Bot will not stop running unless it experiences a certain amount of errors in quick succession.
129+
Any task execution that experiences an error will abort execution (and therefore not produce any metrics) but the Bot **will not** shutdown.
130+
131+
All errors encountered during task exeuction are reported to the Cluster for later review by any users with appriopiate access.
132+
Tasks do not retry (by default), but updates to `app.state` are maintained up until the point an error occurs.
133+
134+
It is important to keep track of these errors and ensure that none of them are in fact critical to the operation of your Application,
135+
and to take corrective or preventative action if it is determined that it should be treated as a more critical failure condition.
136+
```
137+
138+
```{note}
139+
Your Bots can also be monitored from the Platform UI at [https://silverback.apeworx.io](https://silverback.apeworx.io).
140+
```
141+
142+
## Controlling your Bot
143+
144+
As we already saw, once a Bot is configured in a Cluster, we can control it using commands from the [`silverback cluster bots`][silverback-cluster-bots] subcommand group.
145+
For example, we can attempt to start a Bot that is not currently running (after making configuration or code changes)
146+
using the [`silverback cluster bots start`][silverback-cluster-bots-start] command.
147+
We can also stop a bot using [`silverback cluster bots stop`][silverback-cluster-bots-stop] that is currently in the RUNNING state if we desire.
148+
149+
```{note}
150+
Controlling your bots can be done from the Platform UI at [https://silverback.apeworx.io](https://silverback.apeworx.io), if you have the right permissions to do so.
151+
```
152+
153+
TODO: Updating runtime parameters
154+
155+
## Viewing Measured Metrics
156+
157+
TODO: Downloading metrics from your Bot
158+
159+
[silverback-cluster]: ../commands/cluster.html#silverback-cluster
160+
[silverback-cluster-bots]: ../commands/cluster.html#silverback-cluster-bots
161+
[silverback-cluster-bots-errors]: ../commands/cluster.html#silverback-cluster-bots-errors
162+
[silverback-cluster-bots-health]: ../commands/cluster.html#silverback-cluster-bots-health
163+
[silverback-cluster-bots-info]: ../commands/cluster.html#silverback-cluster-bots-info
164+
[silverback-cluster-bots-list]: ../commands/cluster.html#silverback-cluster-bots-list
165+
[silverback-cluster-bots-logs]: ../commands/cluster.html#silverback-cluster-bots-logs
166+
[silverback-cluster-bots-new]: ../commands/cluster.html#silverback-cluster-bots-new
167+
[silverback-cluster-bots-remove]: ../commands/cluster.html#silverback-cluster-bots-remove
168+
[silverback-cluster-bots-start]: ../commands/cluster.html#silverback-cluster-bots-start
169+
[silverback-cluster-bots-stop]: ../commands/cluster.html#silverback-cluster-bots-stop
170+
[silverback-cluster-bots-update]: ../commands/cluster.html#silverback-cluster-bots-update
171+
[silverback-cluster-health]: ../commands/cluster.html#silverback-cluster-health
172+
[silverback-cluster-info]: ../commands/cluster.html#silverback-cluster-info
173+
[silverback-cluster-new]: ../commands/cluster.html#silverback-cluster-new
174+
[silverback-cluster-vars]: ../commands/cluster.html#silverback-cluster-vars
175+
[silverback-cluster-vars-info]: ../commands/cluster.html#silverback-cluster-vars-info
176+
[silverback-cluster-vars-list]: ../commands/cluster.html#silverback-cluster-vars-list
177+
[silverback-cluster-vars-new]: ../commands/cluster.html#silverback-cluster-vars-new
178+
[silverback-cluster-vars-remove]: ../commands/cluster.html#silverback-cluster-vars-remove
179+
[silverback-login]: ../commands/cluster.html#silverback-login

‎setup.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@
7474
"packaging", # Use same version as eth-ape
7575
"pydantic_settings", # Use same version as eth-ape
7676
"taskiq[metrics]>=0.11.3,<0.12",
77+
"tomlkit>=0.12,<1", # For reading/writing global platform profile
78+
"fief-client[cli]>=0.19,<1", # for platform auth/cluster login
7779
],
7880
entry_points={
7981
"console_scripts": ["silverback=silverback._cli:cli"],

0 commit comments

Comments
 (0)