Find closest convex function for piecewise linear data by jd-lara · Pull Request #513 · NREL-Sienna/InfrastructureSystems.jl

jd-lara · 2025-11-17T22:02:26Z

Implement make_convex and make_concave functions using the Pool Adjacent Violators Algorithm (PAVA) for isotonic regression. This provides an optimal O(n) solution for finding the closest convex/concave approximation to non-convex PiecewiseLinearData and PiecewiseStepData.

New functions:

isotonic_regression / antitonic_regression: Core PAVA implementation
make_convex / make_concave: Convert piecewise data to convex/concave form
convexity_violations: Find indices where convexity is violated
convexity_gap: Measure the maximum convexity violation
approximation_error: Compute error between original and approximated data

Features:

Multiple weighting schemes (:uniform, :length, custom)
Multiple anchor options for PiecewiseLinearData (:first, :last, :centroid)
Support for L1, L2, and Linf error metrics

Implement make_convex and make_concave functions using the Pool Adjacent Violators Algorithm (PAVA) for isotonic regression. This provides an optimal O(n) solution for finding the closest convex/concave approximation to non-convex PiecewiseLinearData and PiecewiseStepData. New functions: - isotonic_regression / antitonic_regression: Core PAVA implementation - make_convex / make_concave: Convert piecewise data to convex/concave form - convexity_violations: Find indices where convexity is violated - convexity_gap: Measure the maximum convexity violation - approximation_error: Compute error between original and approximated data Features: - Multiple weighting schemes (:uniform, :length, custom) - Multiple anchor options for PiecewiseLinearData (:first, :last, :centroid) - Support for L1, L2, and Linf error metrics

Remove antitonic_regression, make_concave for PiecewiseStepData and PiecewiseLinearData, and related tests. The library only needs conversion from concave to convex (make_convex), not the reverse.

github-actions · 2025-11-17T22:15:13Z

src/function_data.jl

+        return _get_x_lengths(x_coords)
+    elseif weights isa Vector{Float64}
+        length(weights) == n_segments ||
+            throw(ArgumentError("Custom weights must have length $n_segments, got $(length(weights))"))


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

throw(ArgumentError("Custom weights must have length $n_segments, got $(length(weights))"))

throw(

ArgumentError(

"Custom weights must have length $n_segments, got $(length(weights))",

),

)

github-actions · 2025-11-17T22:15:13Z

test/test_function_data.jl

+    approximated = IS.PiecewiseStepData([0.0, 1.0, 2.0, 3.0], [7.5, 7.5, 15.0])
+
+    # L2 error with uniform weights
+    err_l2 = IS.approximation_error(original, approximated; metric = :L2, weights = :uniform)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

err_l2 = IS.approximation_error(original, approximated; metric = :L2, weights = :uniform)

err_l2 =

IS.approximation_error(original, approximated; metric = :L2, weights = :uniform)

github-actions · 2025-11-17T22:15:13Z

test/test_function_data.jl

+    @test err_l2 ≈ expected_l2
+
+    # L1 error
+    err_l1 = IS.approximation_error(original, approximated; metric = :L1, weights = :uniform)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

err_l1 = IS.approximation_error(original, approximated; metric = :L1, weights = :uniform)

err_l1 =

IS.approximation_error(original, approximated; metric = :L1, weights = :uniform)

github-actions · 2025-11-17T22:15:14Z

test/test_function_data.jl

+    @test err_l1 ≈ expected_l1
+
+    # Linf error
+    err_linf = IS.approximation_error(original, approximated; metric = :Linf, weights = :uniform)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

err_linf = IS.approximation_error(original, approximated; metric = :Linf, weights = :uniform)

err_linf =

IS.approximation_error(original, approximated; metric = :Linf, weights = :uniform)

github-actions · 2025-11-17T22:15:14Z

test/test_function_data.jl

+    @test err ≈ expected
+
+    # Test invalid metric
+    @test_throws ArgumentError IS.approximation_error(original, approximated; metric = :invalid)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

@test_throws ArgumentError IS.approximation_error(original, approximated; metric = :invalid)

@test_throws ArgumentError IS.approximation_error(

original,

approximated;

metric = :invalid,

)

Refactor approximation_error functions to use Julia's standard LinearAlgebra functions instead of manually implementing norms: - L2 norm: Use norm() with weighted scaling - L1 norm: Use dot() for weighted sum - L∞ norm: Use norm(diff, Inf) This makes the code more idiomatic and leverages optimized implementations from Julia's standard library.

PabloBotinGP · 2025-12-09T00:06:23Z

Using these implementations on EasterInterConnection.jl. Once proved to work properly, I will include review the functions and contribute to this PR.

…#526) * Modified slope convexity and concavity checks to return False when curve is not convex/concave. * Uploading script that uses newly defined methods in function_data.jl for convexifying non-convex functions * Added method check_nonconvex, which will be very useful. Recently added scripts are examples of how to use them. * Added scripts that serve as example of how to use the is_nonconvex, PAVA, approximation_error and other methods. These scripts worked succesfully wit pwl curves. They are designed to work with all type of curves but have not been tested for other curves atm. --------- Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov>

…od-01NXcPZ4Xpj2YsW4FUurN8X9

* Add convex approximation methods for piecewise data Implement make_convex and make_concave functions using the Pool Adjacent Violators Algorithm (PAVA) for isotonic regression. This provides an optimal O(n) solution for finding the closest convex/concave approximation to non-convex PiecewiseLinearData and PiecewiseStepData. New functions: - isotonic_regression / antitonic_regression: Core PAVA implementation - make_convex / make_concave: Convert piecewise data to convex/concave form - convexity_violations: Find indices where convexity is violated - convexity_gap: Measure the maximum convexity violation - approximation_error: Compute error between original and approximated data Features: - Multiple weighting schemes (:uniform, :length, custom) - Multiple anchor options for PiecewiseLinearData (:first, :last, :centroid) - Support for L1, L2, and Linf error metrics * Remove make_concave functions - only convex conversion needed Remove antitonic_regression, make_concave for PiecewiseStepData and PiecewiseLinearData, and related tests. The library only needs conversion from concave to convex (make_convex), not the reverse. * Use LinearAlgebra library for norm calculations Refactor approximation_error functions to use Julia's standard LinearAlgebra functions instead of manually implementing norms: - L2 norm: Use norm() with weighted scaling - L1 norm: Use dot() for weighted sum - L∞ norm: Use norm(diff, Inf) This makes the code more idiomatic and leverages optimized implementations from Julia's standard library. * Claude/convex approximation method 01 n xc pz4 xpj2 ys w4 f uur n8 x9 (#526) * Modified slope convexity and concavity checks to return False when curve is not convex/concave. * Uploading script that uses newly defined methods in function_data.jl for convexifying non-convex functions * Added method check_nonconvex, which will be very useful. Recently added scripts are examples of how to use them. * Added scripts that serve as example of how to use the is_nonconvex, PAVA, approximation_error and other methods. These scripts worked succesfully wit pwl curves. They are designed to work with all type of curves but have not been tested for other curves atm. --------- Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: PBGP <pbotin@nrel.gov> Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov>

* Find closest convex function for piecewise linear data (#513) * Add convex approximation methods for piecewise data Implement make_convex and make_concave functions using the Pool Adjacent Violators Algorithm (PAVA) for isotonic regression. This provides an optimal O(n) solution for finding the closest convex/concave approximation to non-convex PiecewiseLinearData and PiecewiseStepData. New functions: - isotonic_regression / antitonic_regression: Core PAVA implementation - make_convex / make_concave: Convert piecewise data to convex/concave form - convexity_violations: Find indices where convexity is violated - convexity_gap: Measure the maximum convexity violation - approximation_error: Compute error between original and approximated data Features: - Multiple weighting schemes (:uniform, :length, custom) - Multiple anchor options for PiecewiseLinearData (:first, :last, :centroid) - Support for L1, L2, and Linf error metrics * Remove make_concave functions - only convex conversion needed Remove antitonic_regression, make_concave for PiecewiseStepData and PiecewiseLinearData, and related tests. The library only needs conversion from concave to convex (make_convex), not the reverse. * Use LinearAlgebra library for norm calculations Refactor approximation_error functions to use Julia's standard LinearAlgebra functions instead of manually implementing norms: - L2 norm: Use norm() with weighted scaling - L1 norm: Use dot() for weighted sum - L∞ norm: Use norm(diff, Inf) This makes the code more idiomatic and leverages optimized implementations from Julia's standard library. * Claude/convex approximation method 01 n xc pz4 xpj2 ys w4 f uur n8 x9 (#526) * Modified slope convexity and concavity checks to return False when curve is not convex/concave. * Uploading script that uses newly defined methods in function_data.jl for convexifying non-convex functions * Added method check_nonconvex, which will be very useful. Recently added scripts are examples of how to use them. * Added scripts that serve as example of how to use the is_nonconvex, PAVA, approximation_error and other methods. These scripts worked succesfully wit pwl curves. They are designed to work with all type of curves but have not been tested for other curves atm. --------- Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: PBGP <pbotin@nrel.gov> Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> * update file structure * Find closest convex function for piecewise linear data (#513) * Add convex approximation methods for piecewise data Implement make_convex and make_concave functions using the Pool Adjacent Violators Algorithm (PAVA) for isotonic regression. This provides an optimal O(n) solution for finding the closest convex/concave approximation to non-convex PiecewiseLinearData and PiecewiseStepData. New functions: - isotonic_regression / antitonic_regression: Core PAVA implementation - make_convex / make_concave: Convert piecewise data to convex/concave form - convexity_violations: Find indices where convexity is violated - convexity_gap: Measure the maximum convexity violation - approximation_error: Compute error between original and approximated data Features: - Multiple weighting schemes (:uniform, :length, custom) - Multiple anchor options for PiecewiseLinearData (:first, :last, :centroid) - Support for L1, L2, and Linf error metrics * Remove make_concave functions - only convex conversion needed Remove antitonic_regression, make_concave for PiecewiseStepData and PiecewiseLinearData, and related tests. The library only needs conversion from concave to convex (make_convex), not the reverse. * Use LinearAlgebra library for norm calculations Refactor approximation_error functions to use Julia's standard LinearAlgebra functions instead of manually implementing norms: - L2 norm: Use norm() with weighted scaling - L1 norm: Use dot() for weighted sum - L∞ norm: Use norm(diff, Inf) This makes the code more idiomatic and leverages optimized implementations from Julia's standard library. * Claude/convex approximation method 01 n xc pz4 xpj2 ys w4 f uur n8 x9 (#526) * Modified slope convexity and concavity checks to return False when curve is not convex/concave. * Uploading script that uses newly defined methods in function_data.jl for convexifying non-convex functions * Added method check_nonconvex, which will be very useful. Recently added scripts are examples of how to use them. * Added scripts that serve as example of how to use the is_nonconvex, PAVA, approximation_error and other methods. These scripts worked succesfully wit pwl curves. They are designed to work with all type of curves but have not been tested for other curves atm. --------- Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: PBGP <pbotin@nrel.gov> Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> * update file structure * remove scripts * add some additional testing * Convexification of cost curves (#545) * Moved convexity checks from function_data to convexity_checks (already existing) and reviewed their logic, getting rid of redundant is_nonconvex() and minor changes on the others to ensure linear curves are considered convex * Modified function_data test according to latest changes * Created test for convexity checks, passed * Aliased is_concave to not is_convex and deprecated the function * Restructuration, added new dispatch type for is_convex and defined multiple d ispatch for make_convex, making sure heat rate curves (incremental curves) are treated as they need to, performing an integration and der ivating back * Merged convexity_checks into make_convex * merge continutation * restructre make_convex sctiope * removed deprecations because is_concave definitions were previously defined and I dont wand to mess with them * Removed make_convex methods for IncrementalCurve{LinearFunctionData} and AverageRateCurve{LinearFunctionData} because it did not make sense given how quadratic curves are being convexified * reorganized code into convexity checks and make convex * Check ValueCurve convexity directly via slopes * Added make_convex methods for fuel and cost curves * Modified is_convex to delegate to function data * Created multiple dispatch for ValueCurves, corresponding tests and modified IS.jl order to include cost alias correctly * Added colinear segment merging to make_function, which required creating a new method, added/modified tests accordingly * Modified convexification logic in InputOutputCurve{QuadraticFunctionData} curves * Removed quadratic from multiple dispatch. Added comments with reasons. * Removed make_convex dispatch for nputOutputCurve{LinearFunctionData} * Added NotImplementedError for make_convex, is_convex and approximation_error() * Added info statements to make_convex() * Removed unnecessary merge_colinear_segments() dispatch for linear and quadratic curves * Removed deleted method cyrve type definitions tests --------- Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> Co-authored-by: Abril-Guevara, Sara <Sara.AbrilGuevara@nrel.gov> * fix file * update the md file for claude to stop looking for master * fix testing * simplify the implementation for internally defined weights * remove custom weights * some improvements to function data * Removed is_concave() deprecated method definitions * Define is_concave() and slope_copncavity_checks() back because it is used for a different purpose by PSY * Rename make_convex() into make_convex_approximation() * format update Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Added is_concave(::CostCurve{PiecewiseIncrementalCurve}) dispatch * Simplified curve multiple dispatch definition * Defined is_valid_data() and integrated into make_convex_approximation() * Reviewed warn and error logs * Update src/function_data/convexity_checks.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_convexity_checks.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Removed NotImplementedError() calls * Added is_strictly_decreasing and is_strictly_increasing() methods and modified is_valid_data() to be generic. Also, added generator_name argument to make_convex() in order to identify the generator in the log * style review * Modified make_convex_approximation(): 1) Added is_strictly_increasing() check, 2) Got rid of internal make_convex_approximation(), 3) Modified loggin to only print the error and the name of the generator. * Remove unused convexity_violation and convexity_gap() methods * Created dedicated increasing_curve_convex_approximation() that applies sppecifically to increasing curves. * Added info log message to merge_colinear_segments * Update src/function_data/convexity_checks.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_merge_colinear.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add missing docstring * remove unsued piece of code * throw errors instead of log erros * minor update to claude.md * update to convexity check tolerance * update make_convext to throw erros and use tolerances * update testing * docstring fix * Modified the logic of the increasing_curve_convex_approximation() AverageRate and Increment al curve definitions for simplicity, removing the skip_validation parameters. Reviewed docstrings * Added an additional colinear segments merge after the convexification is applied * Update src/function_data/make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_function_data.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update test/test_make_convex.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: PBGP <pbotin@nrel.gov> Co-authored-by: PabloBotinGP <Pablo.Botin@nrel.gov> Co-authored-by: Abril-Guevara, Sara <Sara.AbrilGuevara@nrel.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

claude added 2 commits November 17, 2025 22:00

Remove make_concave functions - only convex conversion needed

2fbd04b

Remove antitonic_regression, make_concave for PiecewiseStepData and PiecewiseLinearData, and related tests. The library only needs conversion from concave to convex (make_convex), not the reverse.

github-actions bot reviewed Nov 17, 2025

View reviewed changes

jd-lara changed the base branch from main to jd/convex_checks January 15, 2026 01:37

jd-lara marked this pull request as ready for review January 15, 2026 01:37

Merge branch 'jd/convex_checks' into claude/convex-approximation-meth…

a9cb4d8

…od-01NXcPZ4Xpj2YsW4FUurN8X9

jd-lara merged commit 5fcd1e3 into jd/convex_checks Jan 15, 2026
0 of 7 checks passed

jd-lara deleted the claude/convex-approximation-method-01NXcPZ4Xpj2YsW4FUurN8X9 branch January 22, 2026 04:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Find closest convex function for piecewise linear data#513

Find closest convex function for piecewise linear data#513
jd-lara merged 5 commits intojd/convex_checksfrom
claude/convex-approximation-method-01NXcPZ4Xpj2YsW4FUurN8X9

jd-lara commented Nov 17, 2025

Uh oh!

github-actions bot Nov 17, 2025

Uh oh!

github-actions bot Nov 17, 2025

Uh oh!

github-actions bot Nov 17, 2025

Uh oh!

github-actions bot Nov 17, 2025

Uh oh!

github-actions bot Nov 17, 2025

Uh oh!

PabloBotinGP commented Dec 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	err_l2 = IS.approximation_error(original, approximated; metric = :L2, weights = :uniform)
	err_l2 =
	IS.approximation_error(original, approximated; metric = :L2, weights = :uniform)

	err_l1 = IS.approximation_error(original, approximated; metric = :L1, weights = :uniform)
	err_l1 =
	IS.approximation_error(original, approximated; metric = :L1, weights = :uniform)

	err_linf = IS.approximation_error(original, approximated; metric = :Linf, weights = :uniform)
	err_linf =
	IS.approximation_error(original, approximated; metric = :Linf, weights = :uniform)

Conversation

jd-lara commented Nov 17, 2025

Uh oh!

github-actions bot Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

PabloBotinGP commented Dec 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants