Skip to content

Declarative api config for resource should have table_name property #2698

Open
@matipos2

Description

@matipos2

Feature description

As of today in api config (declarative) there is name param for the resources:

source = rest_api_source(
...
"resources": [
                {
                    "name": f"financials_{symbol}",  # this will be used as a table name and resource name
                    "endpoint": {
...)

There is no possibility to configure desired table name. Normally we may it is possible, but not if we want to make it in declarative way. So, my proposition is:

source = rest_api_source(
...
"resources": [
                {
                    "name": f"financials_{symbol}",  # this will be used as resource name and table name if table_name param is not set
                    "table_name": "raw_table",       # table name where the data should be ingested
                    "endpoint": {
...)

Are you a dlt user?

Yes, community version

Use case

I have incremental load from the api per some filter. The data is identical and the source is the same, so it should go to the same table, but the incremental is different per symbol. That's why resource name should be controlled separately from the table name.

Proposed solution

Add new param table_name

Related issues

No response

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentation

Type

No type

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions