Skip to content

Commit 24511dd

Browse files
cloud-fanyaooqinn
authored andcommitted
[SPARK-47791][SQL][FOLLOWUP] Avoid invalid JDBC decimal scale
### What changes were proposed in this pull request? This is a follow-up of #45976 to fix a regression. Some JDBC dialects may have decimal scale larger than precision. Before #45976 , we used `DecimalType.bounded` which limits the scale to 38 and avoids this issue in some cases. But `DecimalPrecisionTypeCoercion.bounded` does not limit decimal scale and is more likely to hit the issue. This PR proposes to make JDBC decimal scale no larger than precision. Note: we can also fix `DecimalPrecisionTypeCoercion.bounded` but I'd like to reduce the blast radius here. ### Why are the changes needed? fix a regression that some JDBC queries start to fail after #45976 ### Does this PR introduce _any_ user-facing change? no, the issue is not release yet. ### How was this patch tested? The change is very obvious, but testing needs to install other dialects so skip it ### Was this patch authored or co-authored using generative AI tooling? no Closes #50673 from cloud-fan/decimal. Lead-authored-by: Wenchen Fan <wenchen@databricks.com> Co-authored-by: Wenchen Fan <cloud0fan@gmail.com> Signed-off-by: Kent Yao <yao@apache.org>
1 parent 124a17f commit 24511dd

File tree

1 file changed

+5
-1
lines changed
  • sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc

1 file changed

+5
-1
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,11 @@ object JdbcUtils extends Logging with SQLConfHelper {
203203
case java.sql.Types.DECIMAL | java.sql.Types.NUMERIC if scale < 0 =>
204204
DecimalType.bounded(precision - scale, 0)
205205
case java.sql.Types.DECIMAL | java.sql.Types.NUMERIC =>
206-
DecimalPrecisionTypeCoercion.bounded(precision, scale)
206+
DecimalPrecisionTypeCoercion.bounded(
207+
// A safeguard in case the JDBC scale is larger than the precision that is not supported
208+
// by Spark.
209+
math.max(precision, scale),
210+
scale)
207211
case java.sql.Types.DOUBLE => DoubleType
208212
case java.sql.Types.FLOAT => FloatType
209213
case java.sql.Types.INTEGER => if (signed) IntegerType else LongType

0 commit comments

Comments
 (0)