-
Notifications
You must be signed in to change notification settings - Fork 32
AEMO Tables
This page summarise the different tables available through the NEMOSIS tool. It gives both information on the contents of the tables, where the raw data can be found online and how you should expect the tables to behave when filtered by NEMOSIS. The behaviour described here should be taken as the idealised behaviour, or how NEMOSIS is intended to behave, if you observe something different congratulations! This is your opportunity to contribute, raise an issue and help make NEMOSIS better.
Most tables available through NEMOSIS are dynamic tables, this means they are tables that when queried will return different results depending on the start and end time provided i.e the contents of the tables change with respect to time.
Many of the dynamic tables are derived from tables in AEMO's Market Management System (MMS) dataset, the full summary of these tables is available in this pdf. The summary that follows is based partial on this document, but is also interpretive in nature as a precise definition of each table is not always provided. The names of the following sections aim to be as descriptive as possible, with the actual AEMO MMSDM name provided in parenthesis, note NEMOSIS uses the MMS name.
The data contained in each table is briefly summarised by referring to columns they contain, for more detail on each column see the column summary.
Often the columns in these tables are a subset of the MMS table's columns, the full set can be accessed using lower level NEMOSIS functions.
The following tables all contain the column SETTLEMENTDATE in the context of these tables this column defines the particular 5 min dispatch interval that a data record applies to, therefore when NEMOSIS queries these tables only records when the SETTLEMENDATE timestamp is between the start (exclusive) and end time (inclusive) will be returned. Some tables do contain this column but do not conform to this generalisation and details of exceptions are given on a per table basis.
This table reports the results of the central dispatch process by 5 min interval and by dispatch unit.
Columns: SETTLEMENTDATE, DUID, INTERVENTION, DISPATCHMODE, AGCSTATUS, INITIALMW, TOTALCLEARED, RAMPDOWNRATE, RAMPUPRATE, LOWER5MIN, LOWER60SEC, LOWER6SEC, RAISE5MIN, RAISE60SEC, RAISE6SEC, LOWERREG, RAISEREG, SEMIDISPATCHCAP, AVAILABILITY.
This table provides a summary of the dispatch results (as reported in DISPATCHLOAD), aggregated to the 30 min trading interval level by averaging. The SETTLEMENTDATE reported is for the 30 min trading interval ending at the given time. When querying this table ensure the SETTLEMENTDATEs for the trading intervals of interest are included between the start and end time.
Columns: As per the table DISPATCHLOAD
This table reports the results of the central dispatch process by 5 min interval and by interconnector.
Columns: SETTLEMENTDATE, INTERCONNECTORID, INTERVENTION, METEREDMWFLOW, MWFLOW, MWLOSSES.
This table provides a summary of the interconnector dispatch results (as reported in DISPATCHINTERCONNECTORRES), aggregated to the 30 min trading interval level by averaging. The SETTLEMENTDATE reported is for the 30 min trading interval ending at the given time. When querying this table ensure the SETTLEMENTDATEs for the trading intervals of interest are included between the start and end time.
Columns: As per the table TRADINGTRADINGINTERCONNECT
This table reports the results of the central dispatch process by 5 min interval and by market region.
Columns: SETTLEMENTDATE, REGIONID, INTERVENTION, TOTALDEMAND, AVAILABLEGENERATION, AVAILABLELOAD, DEMANDFORECAST, DISPATCHABLEGENERATION, DISPATCHABLELOAD, NETINTERCHANGE, EXCESSGENERATION, LOWER5MINLOCALDISPATCH, LOWER60SECLOCALDISPATCH, LOWER6SECLOCALDISPATCH, RAISE5MINLOCALDISPATCH, RAISE60SECLOCALDISPATCH, RAISE6SECLOCALDISPATCH, LOWERREGLOCALDISPATCH, RAISEREGLOCALDISPATCH, INITIALSUPPLY, CLEAREDSUPPLY, TOTALINTERMITTENTGENERATION, DEMAND_AND_NONSCHEDGEN, UIGF, SEMISCHEDULE_CLEAREDMW, SEMISCHEDULE_COMPLIANCEMW.
This table provides a summary of the region dispatch results (as reported in DISPATCHREGIONSUM), aggregated to the 30 min trading interval level by averaging. The SETTLEMENTDATE reported is for the 30 min trading interval ending at the given time. When querying this table ensure the SETTLEMENTDATEs for the trading intervals of interest are included between the start and end time.
Columns: As per the table DISPATCHREGIONSUM
This table reports the price of various market services by 5 min interval and by market region.
Columns: SETTLEMENTDATE, REGIONID, INTERVENTION, RRP, RAISE6SECRRP, RAISE60SECRRP, RAISE5MINRRP, RAISEREGRRP, LOWER6SECRRP, LOWER60SECRRP, LOWERREGRRP, PRICE_STATUS.
This table provides a summary of the region dispatch results (as reported in DISPATCHPRICE), aggregated to the 30 min trading interval level by averaging. The SETTLEMENTDATE reported is for the 30 min trading interval ending at the given time. When querying this table ensure the SETTLEMENTDATEs for the trading intervals of interest are included between the start and end time.
Columns: As per the table DISPATCHPRICE
This table reports the scada values of scheduled, semi-scheduled and non-scheduled (where available) units where available.
Columns: SETTLEMENTDATE, DUID, SCADAVALUE.
This table reports generic constraints imposed on the dispatch process by 5 min interval, to the best of the authors knowledge it is the definitive information on which constraints were used in each dispatch interval. The definition of the constraints set out in this table can be found in the tables GENCONDATA, SPDREGIONCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and SPDINTERCONNECTORCONSTRAINT. The mapping between tables is from CONSTRAINTID to GENCONID_EFFECTIVEDATE, GENCONID_VERSIONNO to GECONID and EFFECTIVEDATE and VERSIONNO.
Columns: SETTLEMENTDATE, RUNNO, CONSTRAINTID, INTERVENTION, RHS, MARGINALVALUE, VIOLATIONDEGREE, LASTCHANGED, GENCONID_EFFECTIVEDATE, GENCONID_VERSIONNO, LHS.
This table set out the capacity bids used in the central dispatch process. The records are by 5 min interval, by dispatch unit ID and by bid type. It also contains important constraining variables that generators submit along side there bids.
Columns: DUID, BANDAVAIL1, BANDAVAIL2, BANDAVAIL3, BANDAVAIL4, BANDAVAIL5,BANDAVAIL6, BANDAVAIL7, BANDAVAIL8, BANDAVAIL9, BANDAVAIL10, MAXAVAIL, RAMPUPRATE, RAMPDOWNRATE, BIDTYPE, SETTLEMENTDATE, ENABLEMENTMIN, ENABLEMENTMAX, LOWBREAKPOINT, HIGHBREAKPOINT, INTERVAL_DATETIME.
This table set out the price bids used in the central dispatch process. The records are by day, but apply to all dispatch intervals on that market day. Note the definition of a market day in the NEM, as starting at 04:00 AM, means that a record with SETTLEMENTDATE "2017/01/01 00:00:00" applies to all intervals between (inclusive) "2017/01/01 04:05:00" and "2017/01/02 04:00:00".
Columns: SETTLEMENTDATE, DUID, BIDTYPE, OFFERDATE, VERSIONNO, PRICEBAND1, PRICEBAND2, PRICEBAND3, PRICEBAND4, PRICEBAND5, PRICEBAND6, PRICEBAND7, PRICEBAND8, PRICEBAND9, PRICEBAND10, T1, T2, T3, T4
The filtering by date for these tables occurs on the EFFECTIVEDATE column. The most recent effective dates applicable are returned, including multiple versions if they exist. For longer query windows (i.e greater time between start and end time), the multiple effective dates for the same record maybe returned, as different records will apply to different times within the window.
This table defines constraint matrix coefficients for regions, the absence of factor implies a zero coefficient, Note the table just defines constraints as they relate to regions, a full definition of any one constrain may require combining data from GENCONDATA, SPDREGIONCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and DISPATCHCONSTRAINT.
Columns: REGIONID, EFFECTIVEDATE, VERSIONNO, GENCONID, FACTOR, LASTCHANGED, BIDTYPE.
This table defines constraint matrix coefficients for connection points, the absence of factor implies a zero coefficient. Note the table just defines constraints as they relate to regions, a full definition of any one constrain may require combining data from GENCONDATA, SPDREGIONCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and DISPATCHCONSTRAINT.
Columns: CONNECTIONPOINTID, EFFECTIVEDATE, VERSIONNO, GENCONID, FACTOR, BIDTYPE, LASTCHANGED.
This table defines constraint matrix coefficients for interconnectors, the absence of factor implies a zero coefficient. Note the table just defines constraints as they relate to regions, a full definition of any one constrain may require combining data from GENCONDATA, SPDREGIONCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and DISPATCHCONSTRAINT.
Columns: INTERCONNECTORID, EFFECTIVEDATE, VERSIONNO, GENCONID, FACTOR, LASTCHANGED
This table defines general information about constraints applied to the central dispatch process. It maps to SPDREGIONCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINTS and SPDINTERCONNECTORCONSTRAINT with the columns GECONID, EFFECTIVEDATE and VERSIONNO. Note the RHS values given in this table may not always be the ones used in dispatch, some RHS values are dynamically determined, for actual RHS values used in dispatch see the table DISPATCHCONSTRAINT.
Columns: GENCONID, EFFECTIVEDATE, VERSIONNO, CONSTRAINTTYPE, CONSTRAINTVALUE, DESCRIPTION, GENERICCONSTRAINTWEIGHT, LASTCHANGED, DISPATCH, PREDISPATCH, STPASA, MTPASA, LIMITTYPE, REASON
This table provides source of information on dispatch units that is used in the central dispatch process, other dispatch unit information can be found in the table DUDETAILSUMMARY. Other useful information on the units that is not directly relevant to dispatch is more likely to be found in the Registration Master List.
Columns: DUID, EFFECTIVEDATE, VERSIONNO, CONNECTIONPOINTID, REGISTEREDCAPACITY, AGCCAPABILITY, DISPATCHTYPE, MAXCAPACITY,LASTCHANGED,STARTTYPE. NORMALLYONFLAG.
This table provides a second source of information on dispatch units that is used in the central dispatch process. Other useful information on the units that is not directly relevant to dispatch is more likely to be found in the Registration Master List. This table is by START_DATE and END_DATE rather than EFFECTIVEDATE, but the behaviour is similar. The difference in behaviour is that records with an END_DATE before the start time specified are excluded from the results.
Columns: DUID, START_DATE, END_DATE, DISPATCHTYPE, CONNECTIONPOINTID, REGIONID, STATIONID, PARTICIPANTID, LASTCHANGED, TRANSMISSIONLOSSFACTOR, STARTTYPE, DISTRIBUTIONLOSSFACTOR, SCHEDULE_TYPE, MAX_RAMP_RATE_UP, MAX_RAMP_RATE_DOWN.
This table is used in the determination of recovery payments for regulation services, more information is available from AEMO here. This data maps to Elements_FCAS table by ELEMENTNUMBER column and the Variables_FCAS by VARAIBLENUMBER column. Note NEMOSIS complies this table from current and historical data published by AEMO, however there is a 13 month gap in data availability between the current and historical data sets and this may result in NEMOSIS failing to compile data from this table. If your local data cache is regularly update with AEMO's current data set then this would allow you to close the availability over a 13 month period.
Columns: TIMESTAMP, ELEMENTNUMBER, VARIABLENUMBER, VALUE, VALUEQUALITY
While these tables may be periodically updated by AEMO, they do not contain a time based column and hence will not be filtered by any start and end time supplied. If you require the most up to date tables from AEMO delete or rename the corresponding tables from your local data cache and NEMOSIS will download the most recent versions. Note the names provided in parenthesis below are NEMOSIS names not AEMO names.
This table defines the element numbers used in the 4 second FCAS data. Note it appears AEMO has deleted the market names from this data set and no publicly available mapping of element numbers to market names appears to exist. More information is available from AEMO here
Columns: ELEMENTNUMBER, ELEMENTNAME, ELEMENTTYPE, NAME.
This table defines the variable numbers used in the 4 second FCAS data. More information is available from AEMO here
Columns: VARIABLENUMBER, VARIABLETYPE.
This table provides registration data for generators and loads in the NEM. It is derived from excel sheet provide by AEMO here. Note NEMOSIS uses the Generators and Scheduled Loads tab, but drops duplicate entries for the same DUID values, this removes unit level information such as capacity, but enables the data set to more easily combined with MMS datasets. AEMO also removes retired generators from this data set which can be problematic for historical analysis, we are looking to overcome this issue by hosting historical versions of the Registration and Exemption file on GitHub but this has not been implemented yet.
Columns: Participant, Station Name, Region, Dispatch Type, Category, Classification, Fuel Source - Primary, Fuel Source - Descriptor, Technology Type - Primary, Technology Type - Descriptor, Aggregation, DUID.