Explainer: What is the NEM, anyway, and how is it tracking in the shift to renewables?

The national electricity market is not really national, as it covers only the east coast of Australia. It’s also not really just a market. It’s better described as a system of systems, including a wholesale and ancillary services market laid across multiple other systems to maintain generator technical operation and system stability. All of which is intended to deliver security of supply in the short and long term, at the lowest cost to consumers.

That’s the plan, anyway. It doesn’t always work that way. There have been many changes made in the last decade to keep up with the rapid transformation of the power system, as we move away from coal and to a renewable, zero carbon fleet.

I’d like to discuss through some of the challenges that have emerged in the NEM, driven by the physics of the transition, and then explore some of the reform work currently underway to address these challenges.

First, a few facts about the NEM.

As mentioned, the ‘National Electricity Market’ is the term that is used to describe something that is not really national, and not really (or only) just a ‘market’.

The NEM is increasingly being described as a system of systems – that is, a wholesale market system that is overlaid on many other technical, financial and social systems and institutions. The overall goal of this ‘system of systems’ is to deliver a reliable, secure and low cost supply of energy for Australian consumers.

Starting with the NEM as a market, some key elements of the market design include:

– The NEM is an ‘energy only market’. All energy is traded through this wholesale spot market, with various secondary contracting markets (including OTC, exchange traded and PPA) existing to hedge risk in the spot market. There is currently no separate mechanism to pay for investment in capacity, with all capital / capacity investment costs recovered through trading in the single energy only market, although this has been the subject of recent debate.

– There are no ‘spinning reserve’ markets, however multiple co-optimised frequency control ancillary service markets exist to manage power system frequency deviations in both system normal and contingency conditions [with thanks to enelX].

– In order to enable this capital recovery, the NEM wholesale market has a very high market price cap of $15,500/MWh.

– The NEM energy only market runs at a 5 minute security constrained dispatch, and also settles on a 5 minute timeframe. There is no formal ahead market for energy dispatch, although information is provided on a rolling basis prior to actual dispatch to enable efficient forecasting by AEMO as to system needs, and to enable efficient decision making by the market.

– Prices are determined on a zonal basis, with each zone corresponding to each of the eastern states of Australia. There is one pricing node in each state.

– Congestion risk is borne by generators. Generators do not pay any charges for connecting to the power system beyond those associated with the costs of connection assets – this is known as ‘open access’. However this also means there are no ‘firm transmission rights’, or ways to hedge volume risk against the zonal node price.

– Connection to the system is based around a negotiation framework, where generators negotiate the level of the specific technical standards their plant must meet in order to connect to the system.

– Key NEM system performance outcomes are described on the basis of ‘reliability’ and ‘security’. These terms have specific meanings in the NEM, with security referring to the ability to maintain the system with key technical operating parameters such as voltage and frequency stability, and reliability referring to the aggregate supply capability of the NEM.

The physical network is best described as ‘long and stringy’. With approximately 40,000 kilometers of transmission lines and cables, its one of the longest power systems in the world. The system is interconnected with several high voltage (typically in the range of 225-500kv) lines connecting each state. There are also three DC interconnectors.

Figure 1: The NEM power system. Source: AEMO, 2022 ISP

There are over 504 registered participants in the NEM, including market generators, transmission network service providers, distribution network service providers, and market customers. It supplies approximately 10.7 million customers, with an annual throughput of 204 TWh. AU$11.5 billion was traded in the NEM in FY 2020-21.

There is a total electricity generating capacity of 65,252 MW (as at December 2021). Of interest is the fact that there is approximately 14 GW of distributed solar (as at Dec 2021). Collectively this distributed solar is the largest generator in the NEM.[1]

The bulk of energy supply in the NEM has historically been from thermal coal generation. This includes a mix of lignite and anthracite coal generators. As discussed below, renewables are rapidly encroaching on the role of coal as the dominant generator. Massive new investment in renewables, storage and transmission is needed to replace coal in coming years.

This physical transition is having a number of major implications across the entirety of the ‘system of systems’ that makes up the NEM.

NEM physical transition

The NEM transition can be broadly described by reference to the following key physical elements:

Centralised to dispersed: Historically, the bulk generation fleet was based around access to coal, so power stations were either located near mines, or close to major rail infrastructure to enable coal delivery, and/or close to water sources to allow for cooling. Typically, this meant that multiple coal generators have been located in close proximity, in areas including the Hunter valley and Central coast in NSW and the Latrobe Valley in Victoria.

Transmission infrastructure was developed on the basis of these coal generation locations, and large generation centres emerged. In NSW and Victoria, these were often within a few hundred kilometres of the major load centres. Electrically however, these generating regions were ‘close’ to load centres, being connected via high voltage, low impedance transmission corridors that enabled high energy throughput to meet demand.

From 2007 onwards, increasing volumes of renewables have entered, with wind and then solar predominating. These generators often locate where renewable resources are optimal, but also where some transmission capacity was available. Typically this has resulted in many renewable generators connecting away from the traditional coal generation centres. As described below, this has led to issues related to losses and curtailment.

Synchronous to non-synchronous: The historic thermal coal and gas generating fleet was synchronous in nature, while most renewables have been connected utilised power system electronics – usually described in the NEM as ‘non-synchronous’ (or asynchronous).

This key trend has given rise to a large number of issues, firstly based on frequency stability, but also increasingly related to more complex physical phenomena such as ‘system strength’ and general power system stability. A recent move toward grid forming inverter technology offers some way forward to manage these issues.

Dispatchable to variable: Another trend is the move away from a thermal ‘dispatchable’ generation fleet, towards more variable renewable energy (VRE) generation. As VRE penetration has increased, the underlying basis of the dispatch mix has shifted to reflect the energy resource availability of wind and solar. The zero short run marginal cost of these generators has resulted in them often displacing coal generation as the cheapest form of generation and has forced some coal units to increasingly ‘ramp’ their output up and down on a daily basis.

This trend is having a number of implications across the various NEM systems. In the wholesale market it has resulted in price variability and volatility, with prices depressed during high VRE periods, but higher prices during other periods. Physically, ‘residual demand’ (demand minus VRE energy availability) also creates challenged in the system, as per the above mention of the need for ‘ramping’ response.

Capacity to energy constrained: The NEM has historically been capacity constrained, meaning that during traditional reliability at risk periods (hot summer afternoons with peak demand driven by residential cooling load), reliability was maintained by focussing on investment in megawatts of installed capacity. As the NEM generation mix changes to an increasingly VRE dominated fleet, reliability at risk periods may be increasingly based on energy constraints, with an emphasis on driving investment in capacity that can deliver sustained supply of megawatt hours (energy).

For example, elevated and sustained residential winter heating demand, during extended periods of low VRE availability, may shift the nature of NEM reliability at risk periods. As discussed below, this may necessitate a change in the key market settings of the NEM. This is an area that I am particularly keen to explore in the German context, given the extensive investment in renewables and retirement of coal and nuclear generation underway.

Most of the key challenges in the NEM can be traced to the physical transitions described above

While the NEM has been generally successful in terms of delivering energy to reliably meet consumer demand, a number of challenges are emerging.

System stability: System stability is typically broken down into frequency, voltage and rotor angle stability. Other sources of instability have recently been discussed, particularly so called ‘converter driven’ instability. These are set out in the diagram below.

In the NEM, frequency stability arose as a key issue from around 2016 onwards. A number of factors, including a lack of available primary frequency control from both non-synchronous VRE and synchronous generators, combined with falling system inertia, resulted in a general degradation of frequency control in the NEM. This contributed to a particularly catastrophic outcome in 2016, where a severe storm resulted in electrical separation of the South Australian region from the rest of the NEM, with the resultant rapid frequency deviation causing a cascading failure of generation in the now islanded region.

More recently, so called ‘system strength’ stability issues have emerged in the NEM. System strength is a general term that is often interpreted to mean different things. The generally accepted interpretation is that it refers to a distorted voltage waveform. This is equivalent to a ‘weak’ power system, with declining levels of fault current and a power system increasingly subject to converter driven instability during both system normal and under disturbance conditions.

In order to prevent these weak grid conditions from resulting in propagating failure of the grid (potentially triggering blackouts), AEMO has been required to severely curtail of a lot of renewable, non-synchronous generation across the NEM, with major commercial implications for developers.

A related issue is how to best manage a grid with increasing penetrations of ‘grid following’ and ‘grid forming’ inverters. This requires rethinking how to standardise the performance of power electronic connected generators, and whether a ‘replicate’ or ‘replace’ philosophy should be adopted when considering how to operate a power system that is predominantly based on this form of generation.

Changing reliability concerns: As mentioned above, a general trend in the NEM is the shift from a capacity constrained to an energy constrained system. While the current reliability at risk periods remain based around traditional ‘super peak’ demand periods, this may change in future and energy constraints may become more predominant.

This trend is demonstrated in the figure below, which shows some of the different ‘system stressors’, or reliability risk drivers that will need to be managed under a high VRE fleet with less reliable thermal coal generation. Energy constraints are described on the right, under the term ‘Availability duration gap’.

Generation connection issues: Actually connecting new renewable generators has become increasingly difficult in the NEM. This is due to a number of factors. For example, the system stability issues identified above, coupled with the sheer volume of new generators seeking to connect in often remote parts of the power system, has meant that connections are often extensively delayed. This is necessary to undertake complex modelling, to ensure that the system will remain stable, once generators are connected.

Perhaps paradoxically, these delays in connection caused by modelling to ensure system security, could inadvertently result in longer term reliability concerns. These extensive delays can mean that entry of new renewable capacity is delayed, at precisely the same time as coal generation exit is accelerating. A failure to bring forward this new renewable investment may therefore result in MW capacity shortfalls, driving reliability problems in the medium to long term.

What’s being done about it

The renewables industry, working with the NEM regulatory bodies, is leading the way in reforming the NEM, across multiple fronts. Some of these are discussed below, however there are many other areas where huge amounts of work are being undertaken to facilitate the transition across the multiple systems of the NEM.

New system strength frameworks: The NEM rule maker, the Australian Energy Market Commission, recently introduced a new framework to address the system strength issues identified above. Industry has worked closely with the AEMC, and now the system operator AEMO, to implement these changes.

This new framework introduces a new technical standard, which requires the Australian Energy Market Operator (AEMO) and network businesses to plan to deliver services to maintain system strength. To do so, AEMO must project minimum levels of fault current needed to maintain underlying system stability, particularly effective operation of network protection equipment. AEMO must also project volumes of expected renewable generation that will connect.

Network businesses are then responsible for planning and building assets and/or procuring services to meet the requirements defined by AEMO.

These system strength frameworks also impose new financial obligations on the generators who make use of this system strength service provided by networks – specifically, those generators who connect using ‘grid following’ inverters close to nodes where the network business provides the system strength service. Generators must also meet specific technical standards when they connect, to maintain stable operation down to a given level of short circuit ratio (a common metric used as a proxy for system strength).

Next stages of work on this area will involve developing new frameworks to recognise how generators connecting using grid forming inverters can contribute to maintaining the ‘strength’ of the system. This requires work to be completed by AEMO to define the technical characteristics of grid forming inverters. Work will also need to be undertaken to understand the full range of system stability services that can be provided by inverter connected generation, such as system restart as well as network support for voltage or transient stability.

New frequency control requirements and services: A range of new obligations and system services have been introduced to address the frequency stability issues identified above.

On the obligation front, recent changes made by the AEMC are exploring how to mandate the provision of some primary frequency control capability. Work is ongoing as to how exactly this is remunerated, given that it may impose operating costs on generators. Other changes made to the generator technical performance requirements will also improve both the active and passive frequency stability capability of individual generators.

AEMO and networks also face requirements to maintain a given level of system inertia. If AEMO identifies a likely shortfall in system inertia within a given region, networks must procure services to manage this shortfall. This may include providing traditional synchronous machine inertia (from the rotating mass of a synchronous generator or synchronous condenser), or ‘inertia support services’ that may include the virtual or ‘synthetic’ inertia that can be provided by batteries.

Finally, new ancillary service markets have recently been introduced to reflect the capability of assets like batteries to provide very rapid active power injection to manage frequency deviations.

Reforms to the connection process: Given its importance to maintaining reliability of supply, the Australian clean energy industry has led the reform of the connection process. Working collaboratively with the system operator AEMO, the CEC has led the Connections Reform Initiative, which has explored and developed solutions to speed up the connection process.

 

Currently, the CEC is focused on speeding up the process for getting generators registered and able to export power to the market. Other key reform areas are focussed on how to coordinate the modelling and assessment of multiple connections.

Maintaining reliability in a high renewables generating system: Finally, extensive work is currently underway to rethink how we deliver the investment necessary to maintain a reliable supply of energy for customers.

While a traditional form of capacity mechanism had been proposed, this has now been rejected by governments and new models are being explored. There are many models that might be utilised, such as a renewable energy storage target to drive investment in dispatchable storage assets, and other approaches to bring forward investment in ‘bulk energy’ provision from renewables. The CEC has advocated for mechanisms to bring forward the critical investment needed in new storage, transmission and generation, before coal generators exit the market.

Individual States are also leading in the buildout of ‘Renewable Energy Zones’, which combine new transmission investment, storage assets and large volumes of renewables, to enable a coordinated provision of reliable energy.

Finally, work is being undertaken to explore how reliability is best defined and maintained through the energy market in the long term. This work involves looking at the reliability standard and considering how it might be refined to reflect the changing nature of reliability at risk periods in the NEM.

Christiaan Zuur is Director of Energy Transformation at the Clean Energy Council. This article is a lightly edited version of a presentation made in Berlin, Germany.