enhancement#93 : Implemented TF code for firestore creation.

terraform/cloud-functions/distributed/README.md

@@ -161,33 +161,20 @@ Autoscaler infrastructure, with the exception of Cloud Scheduler, lives.
         cloudresourcemanager.googleapis.com
     ```
 
-6.  Create a Google App Engine app, to enable the APIs for Cloud Scheduler and Firestore.
+6.  The Autoscaler state can be stored in either Firestore or Cloud Spanner.
 
-    ```sh
-    gcloud app create --region="${AUTOSCALER_APP_ENGINE_LOCATION}"
-    ```
-
-7.  The Autoscaler state can be stored in either Firestore or Cloud Spanner.
-
-    In case you want to use Firestore, update the database created with the
-    Google App Engine app to use [Firestore native mode][firestore-native].
-
-    ```sh
-    gcloud firestore databases update --type=firestore-native
-    ```
-
-    In case you want to use Cloud Spanner, no action is needed at this point.
+    Firestore database will be created by default, In case you want to use Cloud
+    Spanner, no action is needed at this point.
 
 ### Deploying the Autoscaler
 
-1.  Set the project ID, region, zone and App Engine location in the
+1.  Set the project ID, region and zone in the
     corresponding Terraform environment variables
 
     ```sh
     export TF_VAR_project_id="${AUTOSCALER_PROJECT_ID}"
     export TF_VAR_region="${AUTOSCALER_REGION}"
     export TF_VAR_zone="${AUTOSCALER_ZONE}"
-    export TF_VAR_location="${AUTOSCALER_APP_ENGINE_LOCATION}"
     ```
 
 2.  Change directory into the Terraform scaler-project directory and initialize
@@ -229,14 +216,12 @@ topic and function in the project where the Spanner instances live.
     gcloud config set project "${APP_PROJECT_ID}"
     ```
 
-4.  Choose the [region and zone][region-and-zone] and
-    [App Engine Location][app-engine-location] where the Application project
+4.  Choose the [region and zone][region-and-zone] where the Application project
     will be located.
 
     ```sh
     export APP_REGION=us-central1
     export APP_ZONE=us-central1-c
-    export APP_APP_ENGINE_LOCATION=us-central
     ```
 
 5.  Use the following command to enable the Cloud APIs:
@@ -252,13 +237,6 @@ topic and function in the project where the Spanner instances live.
         cloudbuild.googleapis.com
     ```
 
-6.  Create an App to enable Cloud Scheduler, but do not create a Firestore
-    database:
-
-    ```sh
-    gcloud app create --region="${APP_APP_ENGINE_LOCATION}"
-    ```
-
 ### Deploy the Application infrastructure
 
 1.  Set the project ID, region, zone and App Engine location in the
@@ -268,7 +246,6 @@ topic and function in the project where the Spanner instances live.
     export TF_VAR_project_id="${APP_PROJECT_ID}"
     export TF_VAR_region="${APP_REGION}"
     export TF_VAR_zone="${APP_ZONE}"
-    export TF_VAR_location="${APP_APP_ENGINE_LOCATION}"
     ```
 
 2.  If you want to create a new Spanner instance for testing the Autoscaler, set

terraform/cloud-functions/per-project/README.md

@@ -130,14 +130,12 @@ In this section you prepare your project for deployment.
     gcloud config set project "${PROJECT_ID}"
     ```
 
-4.  Choose the [region and zone][region-and-zone] and
-    [App Engine Location][app-engine-location] where the Autoscaler
+4.  Choose the [region and zone][region-and-zone] where the Autoscaler
     infrastructure will be located.
 
     ```sh
     export REGION=us-central1
     export ZONE=us-central1-c
-    export APP_ENGINE_LOCATION=us-central
     ```
 
 5.  Enable the required Cloud APIs
@@ -155,24 +153,6 @@ In this section you prepare your project for deployment.
       cloudresourcemanager.googleapis.com
     ```
 
-6.  Create a Google App Engine app, to enable the APIs
-    for Cloud Scheduler and Firestore
-
-    ```sh
-    gcloud app create --region="${APP_ENGINE_LOCATION}"
-    ```
-
-7.  The Autoscaler state can be stored in either Firestore or Cloud Spanner.
-
-    In case you want to use Firestore, update the database created with the
-    Google App Engine app to use [Firestore native mode][firestore-native].
-
-    ```sh
-    gcloud firestore databases update --type=firestore-native
-    ```
-
-    In case you want to use Cloud Spanner, no action is needed at this point.
-
 ## Deploying the Autoscaler
 
 1.  Set the project ID, region and zone in the corresponding Terraform
@@ -203,10 +183,16 @@ In this section you prepare your project for deployment.
     For more information on how to make your Spanner instance to be managed by
     Terraform, see [Importing your Spanner instances](#importing-your-spanner-instances)
 
-3.  If you chose to store the state in Firestore, skip this step. If you want
-    to store the state in Cloud Spanner and you don't have a Spanner
-    instance yet for that, then set the following variable so that Terraform
-    creates an instance for you named `autoscale-test-state`:
+3.  If you chose to store the state in Firestore, set the value of the following
+    variable to true and continue to step 5.
+
+    ```sh
+    export TF_VAR_terraform_firestore_create=true
+    ```
+
+4.  Alternatively, if you want to store the state in Cloud Spanner and you don't
+    have a Spanner instance yet for that, then set the following variable so that
+    Terraform creates an instance for you named `autoscale-test-state`.
 
     ```sh
     export TF_VAR_terraform_spanner_state=true
@@ -223,7 +209,7 @@ In this section you prepare your project for deployment.
     ```
 
     If you already have a Spanner instance where state must be stored,
-    only set the the name of your instance:
+    only set the the name of your instance.
 
     ```sh
     export TF_VAR_spanner_state_name=<INSERT_YOUR_STATE_SPANNER_INSTANCE_NAME>
@@ -244,20 +230,20 @@ In this section you prepare your project for deployment.
     For more information on how to make your existing Spanner instance to be
     managed by Terraform, see [Importing your Spanner instances](../per-project/README.md#importing-your-spanner-instances)
 
-4.  Change directory into the Terraform per-project directory and initialize it.
+5.  Change directory into the Terraform per-project directory and initialize it.
 
     ```sh
     cd "${AUTOSCALER_DIR}"
     terraform init
     ```
 
-5.  Import the existing AppEngine application into Terraform state
+6.  Import the existing AppEngine application into Terraform state
 
     ```sh
     terraform import module.scheduler.google_app_engine_application.app "${PROJECT_ID}"
     ```
 
-6.  Create the Autoscaler infrastructure. Answer `yes` when prompted, after
+7.  Create the Autoscaler infrastructure. Answer `yes` when prompted, after
     reviewing the resources that Terraform intends to create.
 
     ```sh

terraform/gke/README.md

@@ -184,9 +184,9 @@ In this section you prepare your project for deployment.
     export TF_VAR_terraform_spanner_test=true
     ```
 
-    On the other hand, if you do not want to create a new Spanner instance
-    because you already have an instance for the Autoscaler to monitor, set the
-    name name of your instance in the following variable
+    On the other hand, if you do not want to create a new Spanner instance because
+    you already have an instance for the Autoscaler to monitor, set the name of your
+    instance in the following variable
 
     ```sh
     export TF_VAR_spanner_name=<INSERT_YOUR_SPANNER_INSTANCE_NAME>
@@ -207,38 +207,23 @@ In this section you prepare your project for deployment.
 
 ## Using Firestore for Autoscaler state
 
-1.  To use Firestore for the Autoscaler state, choose the
-    [App Engine Location][app-engine-location] where the Autoscaler
-    infrastructure will be created, for example:
+### Setting up Firestore for storing Autoscaler state using terraform
 
-    ```sh
-    export APP_ENGINE_LOCATION=us-central
-    ```
+1.  A Firestore database will be created by default for storing the state of the
+    Autoscaler in the [Firestore native mode][firestore-native].
 
-2.  Enable the additional APIs:
+2.  Enable the additional APIs necessary for using Firestore:
 
     ```sh
-    gcloud services enable \
+      gcloud services enable \
       appengine.googleapis.com \
       firestore.googleapis.com
     ```
 
-3.  Create a Google App Engine app to enable the API for Firestore:
-
-    ```sh
-    gcloud app create --region="${APP_ENGINE_LOCATION}"
-    ```
-
-4.  To store the state of the Autoscaler, update the database created with the
-    Google App Engine app to use [Firestore native mode][firestore-native].
-
-    ```sh
-    gcloud firestore databases update --type=firestore-native
-    ```
+3.  Next, continue to [Deploying the Autoscaler](#deploying-the-autoscaler)
 
-    You will also need to make a minor modification to the Autoscaler
-    configuration. The required steps to do this are later in these
-    instructions.
+4.  You will also need to make a minor modification to the Autoscaler
+    configuration. The required steps to do this are later in these instructions.
 
 5.  Next, continue to [Deploying the Autoscaler](#deploying-the-autoscaler)
 
@@ -441,7 +426,7 @@ similar process.
     the schema of the configuration, see the [Poller configuration]
     [autoscaler-config-params] section.
 
-9.  If you have chosen to use Firestore to hold the Autoscaler state as described
+9.  If you wish to use Firestore to hold the Autoscaler state as described
     above, edit the above files, and remove the following lines:
 
     ```yaml
@@ -501,8 +486,8 @@ following the instructions above.
 
 ### If the Poller fails to run successfully
 
-1.  If you have chosen to use Firestore for Autoscaler state and you see the
-    following error in the logs:
+1.  If you have chosen to use Firestore (default) for storing Autoscaler state
+    and you see the following error in the logs:
 
     ```sh
      Error: 5 NOT_FOUND: Database not found: projects/<YOUR_PROJECT>/instances/autoscale-test/databases/spanner-autoscaler-state

terraform/gke/main.tf

@@ -61,9 +61,10 @@ module "autoscaler-cluster" {
 module "firestore" {
   source = "../modules/firestore"
 
-  project_id      = var.project_id
-  poller_sa_email = module.autoscaler-base.poller_sa_email
-  scaler_sa_email = module.autoscaler-base.scaler_sa_email
+  project_id                  = var.project_id
+  terraform_firestore_create  = var.terraform_firestore_create
+  poller_sa_email             = module.autoscaler-base.poller_sa_email
+  scaler_sa_email             = module.autoscaler-base.scaler_sa_email
 }
 
 module "spanner" {

terraform/gke/variables.tf

@@ -71,3 +71,8 @@ variable "terraform_dashboard" {
   type        = bool
   default     = true
 }
+
+variable "terraform_firestore_create" {
+  type    = bool
+  default = false
+}

terraform/modules/firestore/main.tf

@@ -15,8 +15,8 @@
  */
 
 /*
- * While the Firestore database is created using the gcloud CLI, the
- * Terraform-created service account used by the Scaler needs read/write
+ * The Firestore database is created using this terraform module and the
+ * terraform-created service account used by the Scaler needs read/write
  * permissions to the instance in the appropriate project if Spanner
  * is not being used to hold state.
  */
@@ -27,3 +27,16 @@ resource "google_project_iam_member" "scaler_sa_firestore" {
   role    = "roles/datastore.user"
   member  = "serviceAccount:${var.scaler_sa_email}"
 }
+
+resource "google_project_service" "firestore" {
+  project = var.project_id
+  service = "firestore.googleapis.com"
+}
+
+resource "google_firestore_database" "database" {
+  project     = var.project_id
+  name        = "(default)"
+  location_id = var.location
+  type        = "FIRESTORE_NATIVE"
+  depends_on = [google_project_service.firestore]
+}

terraform/modules/firestore/variables.tf

@@ -18,6 +18,18 @@ variable "project_id" {
   type = string
 }
 
+variable "region" {
+  description = "The region where the infrastructure will be deployed."
+  type    = string
+  default = "us-central1"
+}
+
+variable "location" {
+  description = "The location within the region where the infrastructure will be deployed."
+  type = string
+  default = "nam5"
+}
+
 variable "poller_sa_email" {
   type = string
 }
@@ -26,7 +38,7 @@ variable "scaler_sa_email" {
   type = string
 }
 
-variable "region" {
-  type    = string
-  default = "us-central1"
-}
+variable "terraform_firestore_create" {
+  type = bool
+  default = false
+}