Skip to content

Spectral mismatch calculation code #1524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Sep 13, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/sphinx/source/whatsnew/v0.9.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ Enhancements
* Improve error message about uneven time intervals for
:py:func:`~pvlib.clearsky.detect_clearsky` and :py:func:`~pvlib.temperature.prilliman`
(:issue:`1476`, :pull:`1490`)
* New module and function to calculate spectral mismatch
:py:func:`~pvlib.spetrum.mismatch.calc_spectral_mismatch`
(:issue:`1523`, :pull:`1524`)

Bug fixes
~~~~~~~~~
Expand Down Expand Up @@ -59,9 +62,10 @@ Contributors
* Adam R. Jensen (:ghuser:`AdamRJensen`)
* Naman Priyadarshi (:ghuser:`Naman-Priyadarshi`)
* Chencheng Luo (:ghuser:`roger-lcc`)
* Prajwal Borkar (:ghuser:`PrajwalBorkar`)
* Prajwal Borkar (:ghuser:`PrajwalBorkar`)
* Kevin Anderson (:ghuser:`kanderso-nrel`)
* Cliff Hansen (:ghuser:`cwhanse`)
* Jules Chéron (:ghuser:`jules-ch`)
* Kurt Rhee (:ghuser:`kurt-rhee`)
* Will Hobbs (:ghuser:`williamhobbs`)
* Anton Driesse (:ghuser:`adriesse`)
Binary file added pvlib/data/astmg173.xls
Binary file not shown.
164 changes: 164 additions & 0 deletions pvlib/spectrum/mismatch.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
"""
The ``mismatch`` module provides functions for spectral mismatch calculations.
"""

import pvlib
import numpy as np
import pandas as pd
from scipy.interpolate import interp1d
import os


def get_sample_sr(wavelength=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to future-proof this function such that it will be able to return other spectral responses in the future? For example:

get_spectral_response(kind='c-si', wavelength=None)

^I would also prefer to write out "sr" as it is not immediately clear to me that it stands for "spectral response"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that calling out the material is important. For the name, I'd prefer something like get_example_sr or get_example_spectral_response, because I think sample could be more easily confused with a test device, as opposed to a reference device, and here it could be either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed get_example_spectral_response would be good.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no problems making a name and parameter change to pass the review but I think it would be inappropriate to put additional technology options into this function in the future. I believe SR data should be in data files in general.

'''
Generate a generic smooth c-si spectral response for tests and experiments.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If data was found, could this function be extended to return spectral response for other cell types? It appears capable of doing so, and if that's correct, I'd suggest adding a kwarg cell_type="csi" or similar to switch between default data sets.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spectral response data should generally be in data files I think. (Lots coming.) This is really just something for people who have nothing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still vote for adding a kwarg now, rather than later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you thinking of a NotImplemented exception for values other thane "csi" this time around?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like that, yes


Parameters
----------
wavelength: 1-D sequence of numeric, optional
The wavelengths at which the sample sr should be interpolated. [nm]
By default the wavelengths are from 280 to 1200 in 5 nm intervals.

Notes
-----
This sr is based on measurements taken on a reference cell.
The measured data points have been adjusted by PV Performance Labs so that
standard cubic spline interpolation produces a curve without oscillations.
'''
SR_DATA = np.array([[ 290, 0.00],
[ 350, 0.27],
[ 400, 0.37],
[ 500, 0.52],
[ 650, 0.71],
[ 800, 0.88],
[ 900, 0.97],
[ 957, 1.00],
[1000, 0.93],
[1050, 0.58],
[1100, 0.21],
[1150, 0.05],
[1190, 0.00]]).transpose()

if wavelength is None:
resolution = 5.0
wavelength = np.arange(280, 1200 + resolution, resolution)

interpolator = interp1d(SR_DATA[0], SR_DATA[1],
kind='cubic',
bounds_error=False,
fill_value=0.0,
copy=False,
assume_sorted=True)

sr = pd.Series(data=interpolator(wavelength), index=wavelength)

sr.index.name = 'wavelength'
sr.name = 'sr'

return sr


def get_am15g(wavelength=None):
'''
Read the ASTM G173-03 AM1.5 global tilted spectrum, optionally interpolated
to the specified wavelength(s).

Parameters
----------
wavelength: 1-D sequence of numeric, optional
The wavelengths at which the spectrum should be interpolated. [nm]
By default the 2002 wavelengths of the standard are returned.

Notes
-----
This function uses linear interpolation. If the reference spectrum is too
coarsely interpolated, its integral may deviate from 1000.37 W/m2.

More information about reference spectra is found here:
https://www.nrel.gov/grid/solar-resource/spectra-am1.5.html

The original file containing the reference spectra can be found here:
https://www.nrel.gov/grid/solar-resource/assets/data/astmg173.xls
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
More information about reference spectra is found here:
https://www.nrel.gov/grid/solar-resource/spectra-am1.5.html
The original file containing the reference spectra can be found here:
https://www.nrel.gov/grid/solar-resource/assets/data/astmg173.xls
More information about reference spectra can be found in [1]_.
The original file containing the reference spectra can be found in [2]_.
References
----------
.. [1] `Reference Air Mass 1.5 Spectra
<https://www.nrel.gov/grid/solar-resource/spectra-am1.5.html>`_
.. [2] `ASTM G-173-03 spreadsheet
<https://www.nrel.gov/grid/solar-resource/assets/data/astmg173.xls>`_

'''

pvlib_path = pvlib.__path__[0]
filepath = os.path.join(pvlib_path, 'data', 'astmg173.xls')

g173 = pd.read_excel(filepath, index_col=0, skiprows=1)
am15g = g173['Global tilt W*m-2*nm-1']

if wavelength is not None:
interpolator = interp1d(am15g.index, am15g,
kind='linear',
bounds_error=False,
fill_value=0.0,
copy=False,
assume_sorted=True)

am15g = pd.Series(data=interpolator(wavelength), index=wavelength)

am15g.index.name = 'wavelength'
am15g.name = 'am15g'

return am15g


def calc_spectral_mismatch(sr, e_sun, e_ref=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the name of this function should somehow reflect that it is specific to the case of a broadband reference device.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about: calc_spectral_mismatch_to_broadband()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I am going to update my suggestion to: calc_field_spectral_mismatch()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer calc_spectral_mismatch_field to make space for _lab, someday.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea: keep the function general (not specifically relative to broadband reference) add an optional sr_ref parameter that defaults to 1.0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a very reasonable suggestion, but perhaps it could be done in a future PR with some more debate about the range of interesting options and different ways of implementing them?

"""
Calculate the spectral mismatch under a given measured spectrum with
respect to a reference spectrum.

Parameters
----------
sr: pandas.Series
The spectral response of one (photovoltaic) device.
The index of the Series must contain wavelength values in nm.
e_sun: pandas.DataFrame or pandase.Series
One or more measured irradiance spectra in a pandas.DataFrame
having wavelength in nm as column index. A single spectrum may be
be given as a pandas.Series having wavelength in nm as index.
e_ref: pandas.Series, optional
The reference spectrum to use for the mismatch calculation.
The index of the Series must contain wavelength values in nm.
The default is the ASTM G173-03 global tilted spectrum.

Returns
-------
smm: pandas.Series or float if a single measured spectrum is provided.

Notes
-----
If the default reference spectrum is used it is linearly interpolated
to the wavelengths of the measured spectrum. To achieve alternate
behavior e_ref can be transformed before calling this function and
provided as an argument.

The spectral response is linearly interpolated to the wavelengths of the
spectrum with which is it multiplied internally (e_sun and e_ref). To
achieve alternate behavior the sr can be transformed before calling
this function.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about IEC 61853-3 (ed. 1.0), specifically, equation (6) in section 7.3?

This is where the implementation here does not cover that formula precisely.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate a little bit on where you see the problem?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, the equation shows the pre-integrated values of the reference and sun/field spectra in a similar manner to your proposed function signature. Do the pre-integrated values need to be different from the values that would be obtained from an actual integration?

I have some comments about the problems with this calculation of the standard in the docstring of my implementation in pvpltools.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adriesse Sorry that I have not been able to look at this closely again this week.

Do the pre-integrated values need to be different from the values that would be obtained from an actual integration?

This is perhaps the key question, assuming that the IEC 61853-3 use case is an important one for this code addition. (I have some issues with this formula, but it is what it is.) At this point I am assuming that this use case is important, and it requires some consideration in addition to the "primary reference cell calibration with a cavity radiometer" use case that I previously examined here. I intend to look closer at this this weekend.

There is also one last use case, which I am personally interested in. This is for PVfit's "irradiance" calculation of F=Isc/Isc0. The spectral-mismatch approach here provides an alternative to the approach that I presented at the recent PVPMC workshop. While I would like to see this code cover this case to my satisfaction, please know that I am not trying to hold up this PR based on this criterion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might prefer to see future functions for specific use cases such as calibration or energy rating. A single generic function could be fewer lines of code, but may not be as inviting for users who are not as broadly informed on the subject.

I have been involved in many discussions about IEC 61853-3. The next version will look different, probably skipping the mismatch or spectral correction factor in this form altogether. Anyway, it would be hard to cover the current version in this function due to the use of banded solar spectral irradiance data.

Copy link
Contributor

@markcampanelli markcampanelli Sep 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I derived equation (6) in section 7.3 in IEC 61853-3 (ed. 1.0) from first principles. Recall that this is used in IEC 61853-3 to compute the equivalent irradiance under the standard spectrum that would generate the same short-circuit current in the device under the given operating condition, denoted here as E_equiv. One issue I found is that technically the spectral response of the device should depend on temperature. A second issue to note is that the upper limit of integration on the integrals, 'lambda_e", is fixed at 4 microns, which corresponds to the assumed spectral-response cutoff of total POA irradiance as measured by a pyranometer and (presumably) reported in the profiles in IEC 61853-4. In the study below, a cutoff change to 1.7 microns requires scaling the total POA irradiance accordingly.

I then computed the SSM and E_equiv that would be computed by code under review vs. an implementation that covers equation (6) in IEC 61853-3. The error appears small for the 4 micron cutoff, but becomes very significant for the 1.7 micron cutoff. What is quite interesting, and what I did not expect, is that the SMM changes significantly based on the wavelength cutoff, and this change is necessary to compute a consistent E_equiv! Of course, these results are contingent upon 3rd-party review :).

This is the modified source code to run the comparison. It can be used to compute both algorithms.

def get_am15g(wavelength=None):
    # Contributed by Anton Driesse (@adriesse), PV Performance Labs. Aug. 2022
    # Modified by Mark Campanelli for testing purposes.

    pvlib_path = pvlib.__path__[0]
    filepath = os.path.join(pvlib_path, 'data', 'astm_g173_am15g.csv')

    am15g = pd.read_csv(filepath, index_col=0).squeeze()  # 0.28 to 4 microns
    am15g_integral_to_4_micron = np.trapz(am15g, x=am15g.index, axis=-1)  # W/m^2
    iec_integral_to_4_micron = 997.47  # W/m^2, from IEC 60904-3 Ed. 2 (2008)
    am15g_scaled_to_iec = iec_integral_to_4_micron / am15g_integral_to_4_micron * am15g

    if wavelength is not None:
        interpolator = interp1d(am15g.index, am15g,
                                kind='linear',
                                bounds_error=False,
                                fill_value=0.0,
                                copy=False,
                                assume_sorted=True)

        am15g = pd.Series(data=interpolator(wavelength), index=wavelength)

        interpolator = interp1d(am15g_scaled_to_iec.index, am15g_scaled_to_iec,
                                kind='linear',
                                bounds_error=False,
                                fill_value=0.0,
                                copy=False,
                                assume_sorted=True)

        am15g_scaled_to_iec = pd.Series(data=interpolator(wavelength), index=wavelength)

    am15g.index.name = 'wavelength'
    am15g.name = 'am15g'

    am15g_scaled_to_iec.index.name = 'wavelength'
    am15g_scaled_to_iec.name = 'am15g'

    return am15g, am15g_scaled_to_iec


def calc_spectral_mismatch_field(sr, e_sun, e_sun_tot, e_ref=None, e_ref_tot=None):
    # Contributed by Anton Driesse (@adriesse), PV Performance Labs. Aug. 2022
    # Modified by Mark Campanelli for testing purposes.

    # get the reference spectrum at wavelengths matching the measured spectra
    if e_ref is None:
        e_ref, _ = get_am15g(wavelength=e_sun.T.index)

    # a helper function to make usable fraction calculations more readable
    def integrate(e):
        return np.trapz(e, x=e.T.index, axis=-1)

    if e_sun_tot is None:
        # Use integrated spectral irradiance, typically an approximation
        # that misses "tail" of distribution.
        e_sun_tot = integrate(e_sun)

    if e_ref_tot is None:
        # Use integrated spectral irradiance, typically an approximation
        # that misses "tail" of distribution.
        e_ref_tot = integrate(e_ref)

    # interpolate the sr at the wavelengths of the spectra
    # reference spectrum wavelengths may differ if e_ref is from caller
    sr_sun = np.interp(e_sun.T.index, sr.index, sr, left=0.0, right=0.0)
    sr_ref = np.interp(e_ref.T.index, sr.index, sr, left=0.0, right=0.0)

    # calculate usable fractions
    uf_sun = integrate(e_sun * sr_sun) / e_sun_tot
    uf_ref = integrate(e_ref * sr_ref) / e_ref_tot

    # mismatch is the ratio or quotient of the usable fractions
    smm = uf_sun / uf_ref

    if isinstance(e_sun, pd.DataFrame):
        smm = pd.Series(smm, index=e_sun.index)

    return smm

This is the Jupyter notebook code that runs the comparison:

import numpy as np
from pvlib import spectrum
from pvlib.tests import test_spectrum

# Standard spectra from 0.28 to 4 micron, e_ref_iec is am15g (from ASTM G173-03) scaled to "match" IEC 60904-3 Ed. 2 (2008).
e_ref_am15g_astm, e_ref_am15g_iec = spectrum.get_am15g()

# IEC 60904-3 Ed. 2 (2008), cumulatives are provided in standard.
e_ref_am15g_iec_idx_to_1p7_micron = e_ref_am15g_iec.T.index <= 1700
e_ref_am15g_iec_integral_to_1p7_micron = 942.88  # W/m^2
e_ref_am15g_iec_integral_to_4_micron = 997.47  # W/m^2
e_ref_am15g_iec_integral_to_infinity = 1000.  # W/m^2

# ASTM G173-03, calculate cumulatives not provided in standard.
e_ref_am15g_astm_idx_to_1p7_micron = e_ref_am15g_astm.T.index <= 1700
e_ref_am15g_astm_integral_to_1p7_micron = np.trapz(e_ref_am15g_astm[e_ref_am15g_astm_idx_to_1p7_micron], x=e_ref_am15g_astm.index[e_ref_am15g_astm_idx_to_1p7_micron], axis=-1)  # W/m^2
e_ref_am15g_astm_integral_to_4_micron = np.trapz(e_ref_am15g_astm, x=e_ref_am15g_astm.index, axis=-1)  # W/m^2
e_ref_am15g_astm_integral_to_infinity = None  # W/m^2, undefined in standard

# Get example solar spectrum from test fixture.
_, e_sun = test_spectrum.spectrl2_data.__pytest_wrapped__.obj()  # 0.3 to 4 micron
e_sun = e_sun.set_index('wavelength')
e_sun = e_sun.transpose().loc['specglo']
e_sun_idx_to_1p7_micron = e_sun.T.index <= 1700
e_sun_integral_to_1p7_micron = np.trapz(e_sun[e_sun_idx_to_1p7_micron], x=e_sun.T.index[e_sun_idx_to_1p7_micron], axis=-1)
e_sun_integral_to_4_micron = np.trapz(e_sun, x=e_sun.T.index, axis=-1)  # 708.4855034837434 W/m^2
scale_factor_estimate = e_ref_am15g_iec_integral_to_infinity / e_ref_am15g_iec_integral_to_4_micron
e_sun_integral_to_infinity = scale_factor_estimate * e_sun_integral_to_4_micron

# Relative spectral response for Si, 0.29 to 1.19 micron.
sr = spectrum.get_example_spectral_response()

# SMM computations IEC 61853-3 imply reference temperature in all situations, which is technically incorrect.

# Calculate SMM using formula in IEC 61853-3, using IEC 60904-3 Ed. 2 (2008) spectrum.
# Different cutoff wavelengths used for solar total irradiance integral according to the spectraradiometer cutoff.
iec_smm_inf = spectrum.calc_spectral_mismatch_field(
    sr, e_sun, e_sun_integral_to_infinity, e_ref=e_ref_am15g_iec, e_ref_tot=e_ref_am15g_iec_integral_to_infinity)
iec_smm_4_micron = spectrum.calc_spectral_mismatch_field(
    sr, e_sun, e_sun_integral_to_4_micron, e_ref=e_ref_am15g_iec, e_ref_tot=e_ref_am15g_iec_integral_to_infinity)
iec_smm_1p7_micron = spectrum.calc_spectral_mismatch_field(
    sr, e_sun[e_sun_idx_to_1p7_micron], e_sun_integral_to_1p7_micron, e_ref=e_ref_am15g_iec, e_ref_tot=e_ref_am15g_iec_integral_to_infinity)

# Calculate SMM using proposed formula double truncation of integrals with ASTM G173-03 spectrum.
astm_smm_4_micron = spectrum.calc_spectral_mismatch_field(sr, e_sun, None)
astm_smm_1p7_micron = spectrum.calc_spectral_mismatch_field(sr, e_sun[e_sun_idx_to_1p7_micron], None)

print("IEC am15g totals to 1.7 um, 4 um, and infinity (W/m^2):", e_ref_am15g_iec_integral_to_1p7_micron, e_ref_am15g_iec_integral_to_4_micron, e_ref_am15g_iec_integral_to_infinity)
print("ASTM am15g totals to 1.7 um, 4 um, and infinity (W/m^2):", e_ref_am15g_astm_integral_to_1p7_micron, e_ref_am15g_astm_integral_to_4_micron, e_ref_am15g_astm_integral_to_infinity)
print("e_sun totals to 1.7 um, 4 um, and infinity (W/m^2):", e_sun_integral_to_1p7_micron, e_sun_integral_to_4_micron, e_sun_integral_to_infinity)

print("IEC smm to infinity:", iec_smm_inf)
print("IEC smm to 4 um:", iec_smm_4_micron)
print("IEC smm to 1.7 um:", iec_smm_1p7_micron)
print("IEC am15g-equivalent E to infinity:", iec_smm_inf*e_sun_integral_to_infinity)
print("IEC am15g-equivalent E to 4 um:", iec_smm_4_micron*e_sun_integral_to_4_micron)
print("IEC am15g-equivalent E to 1.7 um:", iec_smm_1p7_micron*e_sun_integral_to_1p7_micron)

print("ASTM smm to 4 um:", astm_smm_4_micron)
print("ASTM smm to 1.7 um:", astm_smm_1p7_micron)
print("ASTM am15g-equivalent E to 4 um:", astm_smm_4_micron*e_sun_integral_to_4_micron)
print("ASTM am15g-equivalent E to 1.7 um:", astm_smm_1p7_micron*e_sun_integral_to_1p7_micron)

Here is the output for the comparison. Note the consistency of E_equiv in the "IEC version" of the computation.

IEC am15g totals to 1.7 um, 4 um, and infinity (W/m^2): 942.88 997.47 1000.0
ASTM am15g totals to 1.7 um, 4 um, and infinity (W/m^2): 945.6188921247738 1000.3706555734423 None
e_sun totals to 1.7 um, 4 um, and infinity (W/m^2): 668.028852782929 708.4855034837434 710.2825182549283
IEC smm to infinity: 0.9913868433999364
IEC smm to 4 um: 0.9939014139772988
IEC smm to 1.7 um: 1.054093308636961
IEC am15g-equivalent E to infinity: 704.1647436949111
IEC am15g-equivalent E to 4 um: 704.1647436949111
IEC am15g-equivalent E to 1.7 um: 704.164743694911
ASTM smm to 4 um: 0.9923965546943565
ASTM smm to 1.7 um: 0.9902678450884945
ASTM am15g-equivalent E to 4 um: 703.0985727081635
ASTM am15g-equivalent E to 1.7 um: 661.5274925022902

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for your detailed analysis, Mark. I agree that there is potential for small differences with the tails, and large differences due to the cut-off at 1700 nm. In this field application the cutoff at 1700 nm is only made if there is no information available about the spectrum beyond. In that situation there are not a lot of choices available, and cutting off the reference spectrum at the same wavelength appears to be a reasonable solution.

# get the reference spectrum at wavelengths matching the measured spectra
if e_ref is None:
e_ref = get_am15g(wavelength=e_sun.T.index)

# interpolate the sr at the wavelengths of the spectra
# reference spectrum wavelengths may differ if e_ref is from caller
sr_sun = np.interp(e_sun.T.index, sr.index, sr, left=0.0, right=0.0)
sr_ref = np.interp(e_ref.T.index, sr.index, sr, left=0.0, right=0.0)

# a helper function to make usable fraction calculations more readable
def integrate(e):
return np.trapz(e, x=e.T.index, axis=-1)

# calculate usable fractions
uf_sun = integrate(e_sun * sr_sun) / integrate(e_sun)
uf_ref = integrate(e_ref * sr_ref) / integrate(e_ref)
Comment on lines +228 to +229
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concerning the validity of these two computations, consider these words of caution from IEC 60904-7 (Ed. 4, 2009)---

In Formula (6) the two integrals in the first fraction have to be taken over the entire
wavelength range of sensitivity of the thermopile detector. This in general poses a problem,
as the sensitivity range of such detectors is larger than the range readily measurable with
spectroradiometers.

I'm trying to put together a clear example where this leads to error in the computation, as well as some idea of the magnitude. (IIRC, this is also where the energy beyond 4 microns in the standard spectrum matters, which is specified differently between ASTM G173-03 and 60904-3 (2008).)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The words of caution are certainly applicable, but I think an error in the result can only arise from a deficiency in the inputs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly suggest that this algorithm be updated to allow more precise calculations when the total broadband "sun" irradiance is known. This covers, for example, the case for primary reference cell calibrations at NREL where a cavity radiometer provides the total irradiance alongside a spectral radiometer with a smaller response window. (This is where the discrepancy was previously discovered as a significant source of error. Admittedly, this level of error may not be important for other types of workflows.)

Here is a suggestion on how to address this using two changes:

  1. In the function get_am15g, scale this spectrum from ASTM G173-03 so that it matches the broadband spectrum in IEC 60904-3 (2008), in which the tail irradiance is fully specified and the full curve (to lambda=infinity) integrates to precisely 1000 W/m^2. Namely, change this line in get_am15g
am15g = pd.read_csv(filepath, index_col=0, squeeze=True)

to

    am15g_raw = pd.read_csv(filepath, index_col=0, squeeze=True)
    am15g_raw_integral_to_4_micron = np.trapz(am15g_raw, x=am15g_raw.index, axis=-1)  # W/m^2
    iec_integral_to_4_micron = 997.47  # W/m^2, from IEC 60904-3 (2008)
    am15g = iec_integral_to_4_micron / am15g_raw_integral_to_4_micron * am15g_raw
  1. Pass optional total irradiance parameters to the function calc_spectral_mismatch, namely
def calc_spectral_mismatch(sr, e_sun, e_sun_tot=None, e_ref=None, e_ref_tot=1000):

and then add the following logic to calc_spectral_mismatch before the usable fraction calculations:

    if e_sun_tot is None:
        # Use integrated spectral irradiance, typically an approximation
        # that misses "tail" of distribution.
        e_sun_tot = integrate(e_sun)

    if e_ref_tot is None:
        # Use integrated spectral irradiance, typically an approximation
        # that misses "tail" of distribution.
        e_ref_tot = integrate(e_ref)

    # calculate usable fractions
    uf_sun = integrate(e_sun * sr_sun) / e_sun_tot
    uf_ref = integrate(e_ref * sr_ref) / e_ref_tot

I have an example that shows an error on the order of 0.25% in the spectral correction factor from ignoring the integral of the tail. This well approximates the degree of error that I recall discovering at NREL. I am happy to share this out separately but am omitting here for brevity.

I have also reached out to Carl Osterwald about the scaling and "tail" discrepancies between ASTM G173-03 and IEC 60904-3 (2008), which seems to persist.

Finally, I still have some questions about the numerical effect of partitioning on the integrations for coarse SR functions, but I would like to settle the above first before further investigation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One change to the above: I recommend that no default value be provided for e_sun_tot in calc_spectral_mismatch, so that the user must explicitly provide a None value in order to use the approximation via the integral of the spectral irradiance over a partial interval, i.e.,

def calc_spectral_mismatch(sr, e_sun, e_sun_tot, e_ref=None, e_ref_tot=1000):

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand what you're talking about and had multiple exchanges with Carl on the subject some years ago. If the sun produced something close to the reference spectrum we would expect about 2.5 or 2.9 W/m^2 beyond 4000 nm.

The main focus in this work is on modules and systems in the field, where someone is lucky if they have a spectro-radiometer that goes to 1700 nm. What is beyond the measurement range is essentially unknown, therefore, by default this function also truncates the reference spectrum to 1700 nm (in this case). This is not the true mismatch to full broadband, but the portion of the mismatch that is calculable based on the available measurements. The tail beyond 4000 nm does not make any difference here.

Actually, an advanced user can already achieve what you propose simply by providing a suitable reference spectrum of their own. They can take the far IR portion from E1125 and append it, for example, and scale it as they like, so this does not require changes in the code.

But if you do that, you also have to extend the measured sun spectra from 1100 or 1700 to 10000 nm (or infinity) or at least estimate its integral that far out. NREL measures to 2400 nm and makes this estimate under optimal clear sky conditions, which is far easier than for all-sky field applications.

I think the functions work correctly for the intended scope of this feature, which should perhaps have been more clearly defined.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without supplying total irradiances separately, I don't agree that I could use the code as is to confidently compute a primary reference cell calibration using the IEC 60904-3 (2008) reference spectrum and a cavity radiometer for the measured broadband irradiance. I think users should be fairly warned that this usage is out of scope.

I will run some calculations for a spectro-radiometer cutoff at 1700nm. I anticipate that the error in the spectral correction could go significantly higher than 0.25% in this case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should add that adding extra parameters for integrated values is not a bad idea in itself. They're just not needed for the field application and would create more opportunities for mistakes by non-expert users.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do appreciate this point about usability. I think I come from the viewpoint that the algorithm should ask by default for the inputs to get the theoretically correct computation and then make the user "opt in" to any approximations. That said, I see that this approach could lead to more confusion, especially if the approximation is typically (1) good and/or (2) the only possible solution given the available measurements.

In any case, here I think we have the outstanding question of when the truncation of two of the four integrals in the "field" spectral correction creates "cancelling errors". Hopefully I'll soon have some estimates to share on this.


# mismatch is the ratio or quotient of the usable fractions
smm = uf_sun / uf_ref

if isinstance(e_sun, pd.DataFrame):
smm = pd.Series(smm, index=e_sun.index)

return smm
73 changes: 69 additions & 4 deletions pvlib/tests/test_spectrum.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
import pytest
from numpy.testing import assert_allclose
from numpy.testing import assert_allclose, assert_approx_equal, assert_equal
import pandas as pd
import numpy as np
from pvlib import spectrum
from pvlib.spectrum import mismatch
from .conftest import DATA_DIR

SPECTRL2_TEST_DATA = DATA_DIR / 'spectrl2_example_spectra.csv'


@pytest.fixture
def spectrl2_data():
# reference spectra generated with solar_utils==0.3
Expand All @@ -34,9 +34,9 @@ def spectrl2_data():
'aerosol_turbidity_500nm': 0.1,
'dayofyear': 75
}
df = pd.read_csv(SPECTRL2_TEST_DATA)
df = pd.read_csv(SPECTRL2_TEST_DATA, index_col=0)
# convert um to nm
df['wavelength'] *= 1000
df['wavelength'] = np.round(df['wavelength'] * 1000, 1)
df[['specdif', 'specdir', 'specetr', 'specglo']] /= 1000
return kwargs, df

Expand Down Expand Up @@ -106,3 +106,68 @@ def test_aoi_gt_90(spectrl2_data):
for key in ['poa_direct', 'poa_global']:
message = f'{key} contains negative values for aoi>90'
assert np.all(spectra[key] >= 0), message


def test_get_sample_sr():
# test that the sample sr is read and interpolated correctly
sr = mismatch.get_sample_sr()
assert_equal(len(sr), 185)
assert_equal(np.sum(sr.index), 136900)
assert_approx_equal(np.sum(sr), 107.6027)

wavelength = [270, 850, 957, 1200, 4001]
expected = [0.0, 0.92732, 1.0, 0.0, 0.0]

sr = mismatch.get_sample_sr(wavelength)
assert_equal(len(sr), len(wavelength))
assert_allclose(sr, expected, rtol=1e-5)


def test_get_am15g():
# test that the reference spectrum is read and interpolated correctly
e = mismatch.get_am15g()
assert_equal(len(e), 2002)
assert_equal(np.sum(e.index), 2761442)
assert_approx_equal(np.sum(e), 1002.88, significant=6)

wavelength = [270, 850, 957, 1200, 4001]
expected = [0.0, 0.893720, 0.270670, 0.448250, 0.0]

e = mismatch.get_am15g(wavelength)
assert_equal(len(e), len(wavelength))
assert_allclose(e, expected, rtol=1e-6)


def test_calc_mismatch(spectrl2_data):
# test that the mismatch is calculated correctly with
# - default and custom reference sepctrum
# - single or multiple sun spectra

# sample data
_, e_sun = spectrl2_data
e_sun = e_sun.set_index('wavelength')
e_sun = e_sun.transpose()

e_ref = mismatch.get_am15g()
sr = mismatch.get_sample_sr()

# test with single sun spectrum, same as ref spectrum
mm = mismatch.calc_spectral_mismatch(sr, e_sun=e_ref)
assert_approx_equal(mm, 1.0, significant=6)

# test with single sun spectrum
mm = mismatch.calc_spectral_mismatch(sr, e_sun=e_sun.loc['specglo'])
assert_approx_equal(mm, 0.992393, significant=6)

# test with single sun spectrum, also used as reference spectrum
mm = mismatch.calc_spectral_mismatch(sr,
e_sun=e_sun.loc['specglo'],
e_ref=e_sun.loc['specglo'])
assert_approx_equal(mm, 1.0, significant=6)

# test with multiple sun spectra
expected = [0.973016, 0.995571, 0.899784, 0.992393]

mm = mismatch.calc_spectral_mismatch(sr, e_sun=e_sun)
assert mm.index is e_sun.index
assert_allclose(mm, expected, rtol=1e-6)