Azure Storage Configuration

Azure Storage Configuration

You can provide Azure-related settings to Cloudfuse (blob or datalake) in two ways (listed in order of precedence):

1. Configuration file
2. Environment variables

Note: Environment variables are intended primarily for authentication. Use the YAML configuration file for persistent settings.

Using a configuration file

Create a YAML config and pass it to the mount command:

cloudfuse mount ./mount --config-file=./config.yaml

Helpful references in this repository:

• Base configuration with all options: https://github.com/Seagate/cloudfuse/blob/main/setup/baseConfig.yaml
• Example file-cache config: https://github.com/Seagate/cloudfuse/blob/main/sampleFileCacheConfig.yaml
• Example streaming config: https://github.com/Seagate/cloudfuse/blob/main/sampleStreamingConfig.yaml

Example azstorage section (fill with your values):

azstorage:
  # Required
  type: block|adls <type of storage account to be connected. Default - block>
  account-name: <name of the storage account>
  container: <name of the storage container to be mounted>
  endpoint: <storage account endpoint (example - https://account-name.blob.core.windows.net)>
  mode: key|sas|spn|msi <kind of authentication to be used>

  account-key: <storage account key>
  # OR
  sas: <storage account sas>
  # OR
  appid: <storage account app id / client id for MSI>
  resid: <storage account resource id for MSI>
  objid: <object id for MSI>
  # OR
  tenantid: <storage account tenant id for SPN>
  clientid: <storage account client id for SPN>
  clientsecret: <storage account client secret for SPN>
  oauth-token-path: <path to file containing the OAuth token>

Environment variables

Cloudfuse supports a set of Azure environment variables useful for authentication and selecting the target container. These are handy when you prefer not to store secrets in a config file:

• General options
  • AZURE_STORAGE_ACCOUNT — storage account name
  • AZURE_STORAGE_ACCOUNT_TYPE — 'block' or 'adls'
  • AZURE_STORAGE_ACCOUNT_CONTAINER — container name to mount
  • AZURE_STORAGE_BLOB_ENDPOINT — blob endpoint (defaults to *.blob.core.windows.net)
  • AZURE_STORAGE_AUTH_TYPE — auth type override (Key, SAS, MSI, SPN; case-insensitive)
• Account key auth
  • AZURE_STORAGE_ACCESS_KEY
• SAS token auth
  • AZURE_STORAGE_SAS_TOKEN
• Managed Identity (MSI)
  • AZURE_STORAGE_IDENTITY_CLIENT_ID
  • AZURE_STORAGE_IDENTITY_OBJECT_ID
  • AZURE_STORAGE_IDENTITY_RESOURCE_ID
  • MSI_ENDPOINT — custom MSI endpoint (uses MSI_SECRET as Secret header)
  • MSI_SECRET — secret for custom MSI endpoint
• Service Principal (SPN)
  • AZURE_STORAGE_SPN_CLIENT_ID
  • AZURE_STORAGE_SPN_TENANT_ID
  • AZURE_STORAGE_AAD_ENDPOINT
  • AZURE_STORAGE_SPN_CLIENT_SECRET
  • AZURE_STORAGE_AUTH_RESOURCE — scope used when requesting a token
• Proxy
  • http_proxy, https_proxy