New rules flag big switch in energy markets to cheaper, smarter grid

Two new rules flagged by the Australian Energy Market Commission overnight herald the start of fundamental changes to the way Australia’s energy markets operate, and a belated switch away from the centralised fossil-fuel network to a smarter, cleaner, cheaper and more reliable grid of the future.

In a sign that the penny may finally have dropped for the conservative energy market rule maker – under intense pressure from state energy ministers, consumers and analysts – the AEMC’s new rules address the two biggest cost centres for electricity consumers: networks and wholesale markets.

And they recognise the increasing role that new technologies such as battery storage, localised renewable energy and demand response will play in the market, and in the transition away from highly polluting, and increasingly expensive, fossil fuel generation.

With networks, which account for nearly half the cost of household bills, the AEMC has finally recognised that building more poles and wires should no longer be the default position of network companies, so it has moved to make it easier for them to consider distributed energy, which means localised renewables and storage, as a cheaper alternative.

“Developments in battery storage, demand response and other technologies may provide alternatives to ‘poles and wires’ when parts of the network need to be replaced,” it said in its draft ruling.

This will likely help pave the way for network owners such as Western Power to break down their “centralised” grid and break it into a modular network of micro grids, mostly renewables based and either stand alone or connected with a “thin wire” to the main network. Operators in Queensland, NSW, Victoria and South Australia are expected to follow suit.

Even so, there is some skepticism amongst energy analysts about whether these changes go far enough, and introduce enough competition to the market. They say their impact may demand on decisions by the Australian Energy Regulator. But it is considered to be a step in the right direction.

In wholesale markets, which account for around one third of the costs, the change is more profound – the adoption of a “5-minute” settlement period that will encourage faster, smarter and cleaner alternatives such as battery storage – paired with renewables – to compete with the relatively slow and clunky gas-fired generators in responding to changes in demand, supply and voltage.

The AEMC has been mulling this proposed rule change for nearly 18 months, and even though it couched the changes in terms of needing a rapid response to the “intermittency of wind and solar”, it appears it has finally accepted that the fossil fuel oligopoly has been rorting the wholesale markets, and pushing up costs to consumers through their bidding practices within the current 30 minute period.

The rule change had been proposed by Queensland zinc producer Sun Metals, which had become sick of the price gyrations (mostly up) caused by the bidding practices of the fossil fuel generators.

Sun Metals has also decided to build its own 116MW solar plant to supply electricity to its expanding smelter because it is cheaper than the coal and gas fired generation it gets from the grid.

However, unlike the dramatic change to forex markets, where the Australian currency was floated overnight, the AEMC proposes in a “directions” paper that the 5-minute rule change be introduced over a three year period.

This is both a recognition by the AEMC of the potential complexities and costs of the changes to metering and IT systems, and possibly of the fossil fuel generators’ threat that the lights may go out if any change is too abrupt.

Such threats have been the not-always-unspoken default position of the fossil fuel generators on virtually every change in policy and rules – from the carbon price, through to the renewable energy targets, energy efficiency, demand response and emission limits.

In this case, the oligopoly suggested the costs would be so great that it would force generators out of the market. Such threats should normally be treated with absolute contempt, but as the South Australia government found out to its cost in February, some generators can simply not provide power, leading to blackouts.

Or they could withhold capacity, forcing up prices, or go light on maintenance, leading to reliability issues – as one of the main network owners, Spark Infrastructure, highlighted in its recent submission to the Finkel Review.

In the case of the settlement period, despite huge objections from the main generating companies, the AEMC has decided that the benefits of the 5-minute rule change outweighs the costs.

A move to five minute settlement would align the physical electricity system – which matches demand and supply every five minutes – with the price signal provided by the spot market for that five minute period. The only reason it ever had a 30 minute settlement period was because of the limitations in metering and data processing in the 1990s.

But this 30-minute settlement period has opened itself up to rorting by the big generators, who have been shown to withdraw capacity and push up prices in one five minute bidding period, usually the first or second in a period, knowing such tactics would guarantee a high price for the entire 30 minute period.

Capacity then floods the market as generators “pile in” – as the AEMC described it – “to share the benefit of the high price event.”

It provided as an example this event just a few weeks ago, on March 21, in South Australia, where over five dispatch periods the price was inflated in the first bidding period (and one second bidding period) of each 30-minute interval.

Screen Shot 2017-04-12 at 10.42.24 AM

“This bidding behaviour has the potential to significantly distort operational, usage and investment incentives, creating productive, allocative, and investment inefficiency,” the AEMC said. “It is difficult to reconcile a framework that contributes to such behaviour as being consistent with the NEO.”

(This refers to the National Electricity Objective which still, outrageously, does not include the environment)

Indeed, the costs of the bidding practice is illustrated in this graph below. In the first interval, the average cost of generation over those five periods is nearly $10,000/MWh, while in the second period is is $1,538/MWh. As the slow gas generators finally get their machines into gear to get a “slice of the action”, the prices slide into negative territory.

Screen Shot 2017-04-12 at 10.49.13 AM

In short, it recognises that most of the price spikes witnessed in the electricity market in recent years are not the cause of a supply shortfall, or of renewable energy impacts as many would have you believe – but of deliberate acts by generators to artificially inflate prices. Consumers are now paying the price of this market sabotage.

It is difficult to imagine a more blatant sort. But there have been plenty of them.

AGL even admitted to the ability of generators to force “random” price spikes, while arguing that many peaking gas plants would be too slow to take advantage in a 5-minute settlement. “If such spikes were sufficiently random, this may force fast start generators to exit the market – as they may no longer be capable of generating a sufficient revenue to remain viable.”

The AEMC acknowledges that a 5-minute settlement would provide a more accurate signal that would lower prices and make the market more efficient, and encourage new technologies such as battery storage and demand response.

“(The 30-minute settlement) can also lead to bidding behaviour and operational decisions that result in responses occurring up to 25 minutes after they are needed by the power system,” the AEMC wrote.

“This could increase the cost of supplying electricity in the short and long term. There is some evidence of this occurring in the market today …. a more efficiently functioning wholesale market will in turn provides the benefits of lower supply costs and lower retail prices for consumers.”

Analysts and promoters of new technologies expressed their support for the new rules, albeit with caveats. With the 5-minute rule, they wondered how the AEMC arrived at a 3-year transition period, although they recognised the costs involved. It is understood some of the major fossil fuel generators argued for a 5 or 10 year transition.

Ross Garnaut, chairman of Zen Energy, a battery storage provider that has pushed for these changes, said there is no need for such a long transition.

“Markets adjust quickly to changes of greater dimension than would be required by the introduction of 5 minute settlement,” he said in an emailed statement to RenewEconomy.

“The proposed  long transition would slow the introduction of new technologies that are already making large contributions to grid stability in other developed countries. To allow three years to pass before introducing 5 minute settlement, would unnecessarily expose Australians to insecurity in power supply over a challenging period. ”

The Greens also argued that the change could happen immediately.  “This change is long overdue. The process has been glacially slow and it has been holding back more storage coming online. Another 3 year wait is not acceptable,” energy spokesman Adam Bandt said.

“We must guard against complacency. The incumbent fossil fuel generators will fight this rule because they want to keeping gaming a system that is set up to allow them to profit at the public’s expense. They will want to stretch a transition for as long as possible.”

Reaction to the network ruling was more varied, with some accepting it as welcome move forward, though dependent on other initiatives, and others saying it did little or nothing to address the basic regulatory issues around network spending.

Hugh Grant, the executive director of  ResponseAbility, and a long-time critic of excessive network spending, said he welcomed the 5-minute rule, but described the network adjudication from the AEMC as “far too little too late”.

“It is highly unlikely that any real positive outcomes will arise from such incremental changes to the current regulatory arrangements,” he said in emailed comments.

Grant said the governance arrangements of the regulatory test (known as RIT-D) and the AER’s anticipated demand management framework are “fundamentally flawed” as they continue to provide the “networks with the power to decide whether non network solutions are deployed, and will continue to provide the networks with windfall profits for ‘business as usual’ activities.”

“In the absence of meaningful reform, Australia’s energy consumers should place more faith in technology and market forces, rather than regulation, to address the systemic bias in the current regulatory framework.”

The AEMC is inviting submissions to its directions paper on the 5-minute rule by May 24. It then proposes to issue a “draft determination” in July, 2017, and a final determination in September, which means it could be late 2020 before the changes are actually put in place.

Comments

36 responses to “New rules flag big switch in energy markets to cheaper, smarter grid”

  1. George Michaelson Avatar
    George Michaelson

    If not a three year transition, then what? 18 months? Seriously. FCAS capable systems don’t grow on trees, they have to be planted and watered and nurtured into life. People have to make things. Planning cycles have to be met. So, could an entire national market move to 5minutes in 6 months? If you want to float “why three” then address the underlying question: not “oh, this 50Mw said they could be ready next week” but “systemically, nationwide, the entire supply chain is tooled up to bid on 5”

    control systems and financial models presumably have to be changed too. I doubt if it is, but you would hope the software has a knob to adjust the bid/duration interval, but even that demands a planned downtime, reload, run, test cycle.

    legal contracts, you know, that paperwork somebody signed? it has a lifetime. maybe somebody holds paper which doesn’t age out for two years. maybe the AEMO knows that lawsuits are lying with the sharks at any change under three years?

    this stuff doesn’t “just magically happen”

    1. Darren Avatar
      Darren

      I read through this decision paper this morning. As much as I dont like a long transition, the logic behind it made sense with regards to a number of upgrades that must take place and changes that have to be enacted along with contracts currently active.

      1. George Michaelson Avatar
        George Michaelson

        Thank you. thats pretty much what I expected. I’d love less than three, but I do not see this as a hostile act by AEMC but realism in the face of the world as it is.

    2. Ray Miller Avatar
      Ray Miller

      If the generators stopped today gaming the system to increase their profits 3 years is reasonable, but since there has not been any sign of changed behavior it should be implemented yesterday. The amount of money being ripped of is so great it should be cost effective to have it up and running within 12 months; the target should be 1st July 2018.
      What this nonsense about contracts? The real world sees changes all the time ask any worker about how secure any job is, get real.

      If the only reason for the 30min rule was lack of metering technology and processing power.
      My question is; When was the line crossed which made moving to 5 minutes technically possible?
      If the answer is larger than a couple of years ago the whole staff at the AEMC should be sacked today!

      1. neroden Avatar
        neroden

        This is unfortunately correct. Allowing 3 more years of theft is no good. It would be appropriate to require full implementation within 12 months for those networks which are capable of doing the upgrades within that time, with extensions being possible by application. In the meantime, any generator caught manipulating the 30-minute market should have their licenses revoked and their assets auctioned off.

  2. David Pethick Avatar

    An interesting read. They have asked some very broad questions of respondents, including this beauty: “What kinds of generator bidding behaviours would emerge under five minute settlement as compared with 30 minute settlement?”. Good luck getting a useful answer on that one from anyone running spot trading!

    I expect the transition period will be accepted (grudgingly). Somewhat surprising that it actually turns out to be a 5 year period for Type 4 metering, which makes up the bulk of the meters and a pretty large chunk of the demand by (say) 2022.

    Optional demand-side participation is a joke. It’s laughable that Sun Metals proposed this, and yet they want to retain the option to participate in a 5 minute or 30 minute settlement market (and shift the administrative and risk burden onto others). If it’s good enough for coal and gas power plants to be given an “incentive” to invest in fast start capability, big energy users can also have the same incentive to invest in fast demand-response capability.

    The AEMC is right to raise concerns over disruptions to the (important) cap market. I think they underplay it – cap volumes will be greatly reduced in the early quarters of 5 minute settlements. Sellers will be wary and will want to see the market in operation through at least one high demand season. It’s hard to understand their certainty later in the paper that 5 minute volatility will be less than 30 minute volatility.

    On recent form, I doubt much changes between this directions paper and the draft rule changes. They’ll shave off some of the hairy bits (e.g. scada, opt-in) and try and push it through.

    Just my $0.02/MWh…

    Cheers.

    Dave P.

    1. Mungo M. Avatar
      Mungo M.

      Sun Metals has very fast demand response capability. What irks them is late spikes: unforecast spikes late in a trading period. Even if they bail out with say 10 minutes to go, they’ll still pay a price for the preceding 20 minutes of usage that reflects the spike. Not so if they can pay in 5-minute increments.

      1. David Pethick Avatar

        I understand.

        Perhaps I wasn’t clear in my criticism. Sun Metals wants 5-minute settlements to be an opt-in scheme for large energy consumers, but mandatory for generators. This adds enormous complexity (and cost) and is hypocritical.

        Cheers.

        Dave P.

  3. Tom Avatar
    Tom

    That top graph of March 21st from 10:30-13:00 in SA is fantastic. Should be published in every newspaper tomorrow.

  4. Ray Miller Avatar
    Ray Miller

    The AEMC solution is a discussion paper? What does this mean? Further delays?
    The AEMC if they had any credibility and not beholden to the business would rule the change will happen 1st July 2018 giving more than enough notice considering how everyone has know for some time this was inevitable.

  5. CaresAboutHealth Avatar
    CaresAboutHealth

    Anyone care to comment on whether the pattern shown in Fig 3.4 is consistent with the bidding in good faith rule, and if not what should be done about it?

    1. David Pethick Avatar

      I don’t believe it is. There is some commentary on this in section 3.3.4 of the discussion paper.

      I’ve commented in the past on this topic and believe strict enforcement of the “genuine intention” clause would lead to a dramatic reduction in late rebids.

      I encourage anyone interested in this topic to read one of AER’s reports (e.g. https://www.aer.gov.au/wholesale-markets/market-performance/prices-above-5000-mwh-2-february-2017-qld). Appendix A always makes for fascinating reading, as some participants shift generation from low to high prices, and others from high to low prices.

      Cheers.

      Dave P.

    2. Ron Horgan Avatar
      Ron Horgan

      Is this evidence of manipulative misleading fraudulent behaviour resulting in profiteering which should be declared illegal?

    3. AllanO Avatar
      AllanO

      The way that peaking plant bids were initially set up on that day was with most capacity offered at very high prices throughout the day. Once a 5 minute price spike eventuated near the start of a half hour, those generators rebid capacity to low prices (“piled in”) but only for the balance of that half hour, most leaving their original high price offers in place for the later half hour intervals. So once the half hour ticked over, the situation replayed itself – likelihood of a price spike in the first five minutes, followed by rebids down but just for the balance of that half hour, and so on.

      So the rebidding that did happen was from high prices to low prices. The previous attention on “late rebidding”, leading to the good faith rule change was about sudden rebids from low prices to high prices. It’s possible to view the behaviour above as a way of achieving a similar outcome without explicit late rebids from low to high (ie the high price bids at the start of each half hour were always in place, it’s only the “piling in” which involves rebids). And peaking generators might argue that the rebids down are good faith because until there is a price spike to react to (and capture benefits from via the 5/30 effect), they would risk losing money to place bids at or near running cost and face the chance of being started up for only a couple of five minute intervals, given that start costs are quite high for some peakers.

      Whatever, going to 5 minute settlement would eliminate the incentive for this kind of behaviour, and give the peakers more of a headache about how to offer their capacity.

      You can see more analysis of the day in question at http://www.wattclarity.com.au/2017/03/a-sudden-series-of-sa-spikes/

  6. Chris Fraser Avatar
    Chris Fraser

    Thanks to RE for alerting us, as always. IF we may assume that the withdrawal of energy is a cynical attempt at price manipulation every 30 minutes, I think it would be reasonable to assume that a successful bid from a generator would come with reasonable reliability of service. That is, if they withdraw generation and force the price up, there should be a plausible excuse for that withdrawal – and a minimum time out of the market for technical staff to address the ‘problem’.Otherwise, that bidder should find a stand in generator for the next 29 minutes at their own cost. From my understanding of reliability of different technology types, it won’t be the clean generators that have to do much backing up.

  7. Cooma Doug Avatar
    Cooma Doug

    If the rules stayed as they are now, grid scale storage could be used as a tool to ramp up spot prices. Its not an obvious thing but a reality.

    The rules can be such, that the cheapest, most efficient and modern technologies would dominate with the weight being down on prices.
    Ultimately the grid load profile would be an optimum efficient flat line 24/7.

  8. humanitarian solar Avatar
    humanitarian solar

    Demand response won’t plat a role because inverters have a fast response to changes in the load. Anyone who invests in demand response is waisting money on technology which will soon be obsolete.

    1. baseload renewables Avatar
      baseload renewables

      That’s an interesting observation. Could you please provide further information, or examples as to why this is the case?

      1. humanitarian solar Avatar
        humanitarian solar

        Because when wind and solar are put into a battery and the inverter gets it back out, the inverter matches the load accurately. The only role demand response could possibly have in a renewables grid, is if there was ever a problem with some part of the grid or some microgrid, that didn’t have a high enough peak power rating of it’s inverters. Otherwise the inverters will match the load perfectly, in milliseconds, as long as the storage keeps getting topped up sufficiently to keep up.

        1. Chris Fraser Avatar
          Chris Fraser

          The quick reaction time of the storage system is generally a good, but really works well when its storage capacity is much higher than the difference of clean inputs (including their intermittencies) and normal loadings. To provide say 99% grid uptime, storage volume has to be calculated and account for wind and solar volumes which can be forecast days ahead of the loads. When storage is expensive (such as now), demand-side incentives could still allow a saving in storage volumes.

          1. humanitarian solar Avatar
            humanitarian solar

            All the drama about demand tripping out the SA grid is air conditioning in summer. If storage is an issue it would only need to cover a peak from sunset to early evening I reckon. So solar with minimal storage will have it covered.

          2. Chris Fraser Avatar
            Chris Fraser

            Yes, on hot sunny days in summer when the PV is going strongly the storage requirement isn’t much (unless the houses with the aircon are disadvantaged with no shading and heavy thermal masses). I was thinking more about winter storms over several days and grid still working.

          3. humanitarian solar Avatar
            humanitarian solar

            I was thinking purely for the present now moment of the challenges they are experiencing with their present situation and nothing more. Yes I agree more storage will also be needed in the future for high levels of renewables.

        2. Ren Stimpy Avatar
          Ren Stimpy

          Kenso the main role that wind, distributed solar and batteries play is in reducing demand for fossil powered dreadnoughts like mega watt polluting, inefficient coal chewing power stations.

  9. Charles Hunter Avatar
    Charles Hunter

    Every time I read a story about this problem (legacy generators rigging the system and the long slow march to the supposed promised land of 5-minute settlement), I am astonished by how a far simpler fix is completely overlooked.

    The basic issue with the current 30-minute settlement is that it uses the arithmetic mean (the “average”) to calculate what everyone is paid for the settlement period. As any statistician will tell you, the arithmetic mean is sensitive to the presence of extreme values. In this case, those extreme values are the high bids.

    What statisticians do when the arithmetic mean is not a useful measure of central tendency is to use the median instead. The median is far less sensitive to extreme values than the mean.

    If the market was behaving as intended (all generators always bid at their long-term cost of production plus a reasonable profit) then the mean and median would both be close in numerical value so no generator behaving reasonably would lose money. Using the median would remove the incentive to game the system.

    Switching to the median from the arithmetic mean is a rule change that could be implemented overnight. No metering systems would need to be recalibrated.

    My guess is that changing, now, to “30-minute settlement based on median” would also smooth the path to “5-minute settlement based on median”.

    1. Chris Fraser Avatar
      Chris Fraser

      Even better, let’s just ignore the outliers and then use arithmetic median, especially if outliers appear regularly spaced in a conspicuous 30 minute pattern, and don’t come with a realistic explanation.

      1. Charles Hunter Avatar
        Charles Hunter

        The difficulty with ignoring outliers is going to be in defining what constitutes an outlier. Let’s say you pick something like “more than three standard deviations from the mean.” The problem is that the standard deviation itself is based on the mean so it will also be influenced if someone is gaming the system. Alternatively, let’s say you pick three times the inter-quartile range below Q1 and above Q3. I think that could probably be gamed by two generators acting in concert (perish the thought). More to the point, if you use any outlier-based rule I think you’d effectively be telling any would-be gamers of the system exactly how far they could stretch the rubber before getting busted. My guess is that whatever definition you pick for an outlier is either going to be game-able, or the arguments about why that definition is inappropriate and likely to lead to every coal-fired generator closing before noon tomorrow will bog everything down resulting in no change. Something in the back of my mind is also telling me that the notion of “outlier” goes hand in hand with “random data”. Whatever we’re seeing here, I don’t think it qualifies as random.

        1. Greg Hudson Avatar
          Greg Hudson

          The rubber stretching is already in place. It is $5000/MW. Which IMO is the maximum price that should be allowed, not $14000/MW.

    2. ozmq Avatar
      ozmq

      Charles, that’s too simple and quick. It couldn’t possibly be implemented. We need to think about this for 3 years, then ask for an extension because the software isn’t ready yet.

      1. Chris Fraser Avatar
        Chris Fraser

        ???? Yeah we need go-getters with ‘musk-like’ thinking now, not after the energy barons retire …

    3. Ray Miller Avatar
      Ray Miller

      Great idea Charles. Could you apply to the AEMC for a rule change that would apply both to the current 30 minute and all future settlement periods?

    4. neroden Avatar
      neroden

      If you use the median, they’ll bid the price up for exactly 4 out of every 6 5-minute periods within the 30-minute period and then glut it for the other two. 🙁

      1. Charles Hunter Avatar
        Charles Hunter

        I might be missing something about how the process works. What I thought was happening now was one generator bidding very high in each 30-minute period, then other generators pile in with lower bids. They all know the single very high bid will distort the mean so even generators making negative bids to “get a slice of the action” know they will be paid much more than they bid. In Figure 3.5 in this article, the mean is $1441 so my understanding is that that is what all six generators would have been paid for that 30-minute settlement period. The median of the same data is minus $567. The very real risk of too many negative bids resulting in negative earnings for everyone for a settlement period would probably operate to keep unrealistically low bids from being made in the first place. On this data, and assuming the two highest bids stayed the same ($9791 & $1538), the two next-highest bids would have to average $1441 for the median to be the same as the mean of the original data. Basically, in a six-generator scenario, four independent generators would have to make high-ish bids for the outcome to be worse than it is now. I don’t rule out the possibility that four supposedly independent generators might collude to rig the price in a settlement period but each time it happened it would add a lot of evidence that collusion was occurring, possibly even enough to satisfy an Australian court.

  10. Ian Avatar
    Ian

    ‘Scuse the ignorance, but why can’t battery or other types of storage or generation not take advantage of these 30 minute bonanzas. After all batteries can emulate any generation profile in the short term. $10 000/MWH is good money in anyone’s book. They could beat the slow poke gas generators at any bidding event, surely, because they are so much more nimble.

    1. Mike Shackleton Avatar
      Mike Shackleton

      The gas generators would work out how to game the batteries – possibly create more high price events in order to encourage the batteries to discharge – then create other high price events that they know the batteries can’t meet.

    2. Mungo M. Avatar
      Mungo M.

      The rule change is a precondition to making it worthwhile for much of, say, battery storage to participate. A spike of $10,000/MWh for 5-minutes may be worthwhile if the operator is paid for that period, but right now payment (spot price) is the average of all six dispatch interval in a trading interval, and five of those might be at a much, much lower price. So there’s a real risk of committing to uneconomic dispatch.

Get up to 3 quotes from pre-vetted solar (and battery) installers.