Skip to content

fix: override is_string/is_number/is_float type classification for Databricks types#1390

Open
psaikaushik wants to merge 3 commits intodatabricks:mainfrom
psaikaushik:fix/1380-column-type-classification
Open

fix: override is_string/is_number/is_float type classification for Databricks types#1390
psaikaushik wants to merge 3 commits intodatabricks:mainfrom
psaikaushik:fix/1380-column-type-classification

Conversation

@psaikaushik
Copy link
Copy Markdown
Contributor

Summary

col.is_string(), col.is_number(), col.is_float(), and col.is_numeric() all return False for Databricks/Spark column types because the base Column class only recognizes PostgreSQL-style type names.

Closes #1380
Related: dbt-labs/dbt-spark#658

Root Cause

The base Column class from dbt-adapters defines is_string() etc. using type name sets that include PostgreSQL types (text, character varying, etc.) but not Spark/Databricks types (string, int, bigint, double, etc.).

DatabricksColumn inherits from SparkColumn which inherits from the base Column, but neither overrides these classification methods.

Fix

Override is_string(), is_number(), is_float(), is_integer(), and is_numeric() on DatabricksColumn with the complete set of Spark/Databricks type names:

  • String types: string, varchar, char, text, etc.
  • Numeric types: tinyint, smallint, int, integer, bigint, long, float, double, decimal, numeric, real (plus decimal(p,s) variants)
  • Float types: float, double, real
  • Integer types: tinyint, smallint, int, integer, bigint, long

Verification

Column: name03 | Data Type: string | Is String: True  (was: False)
Column: age    | Data Type: int    | Is Number: True  (was: False)

Checklist

  • No breaking changes — only adds overrides, doesn't change any existing behavior for correctly-typed columns
  • Covers all Spark/Databricks primitive types

@sd-db
Copy link
Copy Markdown
Collaborator

sd-db commented Apr 21, 2026

HI @psaikaushik can you select Allow edits from maintainers or do a rebase from main. The tests are failing because the branch is in an inconsistent state w.r.t CI and can 't review without it.

Copy link
Copy Markdown
Collaborator

@sd-db sd-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us look to add tests + update CHANGELOG

…lumn

The base Column class type classification methods don't recognize
Spark/Databricks type names like 'string', 'int', 'bigint', 'double'.
This causes col.is_string() to return False for string columns and
col.is_number() to return False for numeric columns.

Override these methods on DatabricksColumn with the complete set of
Databricks/Spark type names.

Closes databricks#1380
@psaikaushik psaikaushik force-pushed the fix/1380-column-type-classification branch from b317432 to e491e09 Compare April 21, 2026 22:07
@psaikaushik
Copy link
Copy Markdown
Contributor Author

HI @psaikaushik can you select Allow edits from maintainers or do a rebase from main. The tests are failing because the branch is in an inconsistent state w.r.t CI and can 't review without it.

Rebased and also allowed edits for maintainers. thanks for pointing it out! @sd-db

@psaikaushik
Copy link
Copy Markdown
Contributor Author

Let us look to add tests + update CHANGELOG

Added tests and also updated changelog

@psaikaushik psaikaushik force-pushed the fix/1380-column-type-classification branch from e83556c to 2e6f1b6 Compare April 21, 2026 22:14
…ricks#1380)

Address review feedback: add parametrized unit tests covering all five
overridden methods (is_string, is_integer, is_float, is_number,
is_numeric) with true/false cases and case-insensitive checks, and
add a CHANGELOG entry under 1.11.7.

https://claude.ai/code/session_017ZAXMwLSnqz4FqvTt5H7D1
@psaikaushik psaikaushik force-pushed the fix/1380-column-type-classification branch from 2e6f1b6 to 8948b0c Compare April 21, 2026 22:17
@sd-db sd-db self-assigned this Apr 22, 2026
@psaikaushik psaikaushik force-pushed the fix/1380-column-type-classification branch from 261e768 to dbed48b Compare April 22, 2026 16:35
@psaikaushik
Copy link
Copy Markdown
Contributor Author

@sd-db : Fixed the broken formatting test. Please take a look now. thanks!

@psaikaushik psaikaushik requested a review from sd-db April 22, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dbt Classes Instance Methods is_string, is_numeric, is_number, is_float don't seem to work

2 participants