Setting an external stage on source snowflake stream with destination bigquery fails

[mayurbarge]

Sling version:

1.5.11

What is the Operating System?

Mac

Do you have a CLI Pro/Platform subscription?

No

Description of the issue

Hello,

We have an external provider for data, an external read-only share is created by the provider for us.
The database is APPSTORE with schema DIMENSIONS.
As this is an external share we are unable to create an internal stage on this database.

So we have set up our own database called MYDB with schema PUBLIC.

We tried with below configuration for resource

SlingConnectionResource(
name="APPSTORE_SNOWFLAKE",
type="snowflake",
host="somehost",
database="APPSTORE",
schema="DIMENSIONS",
internal_stage="MYDB.PUBLIC.APPSTORE_STAGE",
role="SOME_ROLE",
user=...,
password=...,
)

When replication begins I could see sling starts writing data to the GCS location referred by MYDB.PUBLIC.APPSTORE_STAGE. However the process fails between with error.

~ SQL Error for:
GET @MYDB.PUBLIC.APPSTORE_STAGE"/sling_temp/1773048223761 'file:///var/folders/nm/lg9v9c_n3pd02jz9rrxzn9v40000gp/T/snowflake/get/2026-03-09T145343.761' overwrite=true parallel=12
091003 (22000): Failure using stage area. Cause: [APPSTORE_STAGE GET and PUT commands are not supported with external stage]

We have already raised this issue earlier. I think at that time setting internal_stage was not even documented on the page. #630

Replication Configuration

source: APPSTORE_SNOWFLAKE
target: APPSTORE_BIGQUERY

defaults:
  object: "{target_schema}.{stream_table_lower}"
  mode: full-refresh

  source_options:
  #  format: parquet
    empty_as_null: false
  target_options:
    merge_strategy: delete_insert # Update existing rows, insert new rows for incremental mode

streams:
  dimensions.sample_table:
    mode: incremental
    update_key: last_updated_time
env:
  SLING_LOADED_AT_COLUMN: timestamp
  SLING_KEEP_TEMP: true

Log Output

Running Sling replication with command: /Users/some-user/.sling/bin/sling/v1.5.11/sling run -r /var/folders/nm/lg9v9c_n3pd02jz9rrxzn9v40000gp/T/sling-replication-8e56212c-393a-4853-b14c-9cb4f0fc58d5.json
config file for debugging: /var/folders/nm/lg9v9c_n3pd02jz9rrxzn9v40000gp/T/sling-replication-8e56212c-393a-4853-b14c-9cb4f0fc58d5.json
2026-03-09 16:15:45 +0530 - dagster - ERROR - appstore_sling_dbt_cloud_job - a51754b8-6d88-42e6-8c48-10ee965e2a55 - 29920 - appstore_snowflake_assets - STEP_FAILURE - Execution of step "appstore_snowflake_assets" failed.

dagster._core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "appstore_snowflake_assets"::

Exception: 4:15PM INF Sling CLI | https://slingdata.io
4:15PM INF Sling Replication | appstore_SNOWFLAKE -> appstore_BIGQUERY | dimensions.sample_table
4:15PM INF connecting to source database (snowflake)
4:15PM INF connecting to target database (bigquery)
4:15PM INF getting checkpoint value (last_updated_time)
4:15PM INF reading from source database
4:15PM INF execution failed

fatal:
~ failure running replication (see docs @ https://docs.slingdata.io/sling-cli)
--------------------------- dim.sample_table ---------------------------
~ SQL Error for:
GET @MYDB.PUBLIC.APPSTORE_STAGE/sling_temp/1773053139075 'file:///var/folders/nm/lg9v9c_n3pd02jz9rrxzn9v40000gp/T/snowflake/get/2026-03-09T161539.075' overwrite=true parallel=12
091003 (22000): Failure using stage area. Cause: [APPSTORE_STAGE GET and PUT commands are not supported with external stage]
Sling command failed:
4:15PM INF Sling CLI | https://slingdata.io
4:15PM INF Sling Replication | appstore_SNOWFLAKE -> appstore_BIGQUERY | dimensions.sample_table
4:15PM INF connecting to source database (snowflake)
4:15PM INF connecting to target database (bigquery)
4:15PM INF getting checkpoint value (last_updated_time)
4:15PM INF reading from source database
4:15PM INF execution failed

fatal:
~ failure running replication (see docs @ https://docs.slingdata.io/sling-cli)
--------------------------- dim.sample_table ---------------------------
~ SQL Error for:
GET @MYDB.PUBLIC.APPSTORE_STAGE/sling_temp/1773053139075 'file:///var/folders/nm/lg9v9c_n3pd02jz9rrxzn9v40000gp/T/snowflake/get/2026-03-09T161539.075' overwrite=true parallel=12
091003 (22000): Failure using stage area. Cause: [APPSTORE_STAGE GET and PUT commands are not supported with external stage]

[mayurbarge]

Also there seems no way to tell sling to unload files with PARQUET format for snowflake external stage.

[flarco]

@mayurbarge can you try providing an internal (not external) stage for internal_stage?
I am assuming that APPSTORE is the shared account database correct? I don't think you need to use the external share as well, just use an internal share for sling to UNLOAD from external/shared table into internal share. I haven't tested this, so I'm not sure if that will work.

Also, can you try target_options.use_bulk: false to see if that works? it won't unload to stage but do a straight SELECT from table by using cursor, so it will be slow for big datasets.

[mayur-peakxv]

@flarco APPSTORE is a read only share from our provider. As we are using external share from the provider, we will not be able to create internal_stage. Doing straight SELECT is very slow for us, hence we were trying to leverage sling for unloading snowflake data directly to GCS.

[mayur-peakxv]

External stages fail with error
091003 (22000): Failure using stage area. Cause: [MY_EXTERNAL_STAGE GET and PUT commands are not supported with external stage]
The code makes use of GET/PUT snowflake commands which are specifically meant for internal stages.
The UnloadViaStage method specifically use GET snowflake command.

database_snowflake.go

[mayurbarge]

The sling replication follows 5 steps in sequence.
snowflake COPY INTO -> Download files locally with GET → Transform to internal file -> Load to GCS (when stage enabled) -> BigQuery
The local download and then upload to GCS is an additional overhead, supporting external stage would eliminate the need to download files locally and would in turn use Snowflake's capability to unload data into GCS with external stage configuration.

[mayur-peakxv]

Our ETL pipelines are using dagster with sling cli integration and deployed inside GKE. We are able to setup an internal_stage for snowflake source, however the local downloads result into container resources (storage) getting exhausted.
When sling cli is used inside a container like GKE or Cloud run the local download approach is inefficient, what we need is cloud-to-cloud transfer directly from Snowflake to GCS skipping local downloads.