OSM RIP

After three years of work on the Operational Security Mechanism (OSM) rule change, reaching Draft Determination last October, the Australian Energy Market Commission (AEMC) changed course last week with a guidance note that appears to call the whole thing off.  

The OSM was to be a mechanism by which the market operator (AEMO) could start assets that provide Essential System Services (ESS) to secure the power system. An automated tool that made it simple, for example, to start a standby generator when it was needed to stabilise the grid.

However, after a great deal of work on the tool, the AEMC decided that it was looking decidedly  unsimple, and that the market would be simpler, for the time being at least, without it.

The guidance also puts on ice an Australian Energy Council (AEC) proposal for an inertia spot market.

We look at what was the OSM, how it came to be, and where the AEMC’s note is guiding us.

Essential System Services

Essential System Services (ESS) refers to electrical engineering phenomena, outside of energy, that are necessary to maintain the power system’s security. The National Electricity Market (NEM) has explicit spot markets for energy and Frequency Control Ancillary Services (FCAS), but has less clear mechanisms for purchasing other ESSs such as voltage control, inertia and the rather broad concept of “system strength”.

This was of no great consequence for the first 15 years of the NEM, because these ESS were naturally provided in abundance by the conventional synchronous machines used in coal, gas and hydro power stations. The challenge is how to obtain them with the decline of these historical sources, since the new sources of energy, being wind, solar and batteries do not naturally produce them. They are Inverter Based Resources (IBR).

South Australia faced these issues first. From as early as 2017, periods of surplus renewable energy were creating low and negative spot prices, during which there was no market reason for any conventional synchronous units to stay online.

But to avoid an unacceptably “weak” grid, several conventional generators were required on-line for system strength. This has led to the common situation, when low prices are forecast, of AEMO directing some gas-fired units to remain or come online. This is an “intervention”; AEMO has stepped outside the market to restore system security. Generators are obliged to comply and receive some compensation which is charged to customers.

AEMO repeatedly calling upon their intervention powers is clearly a failure in the market design.  A term is “missing market”, i.e., AEMO has no existing competitive mechanism, such as it does with FCAS, to procure this service from these generators. A large chapter of the Energy Security Board’s (ESB) Post-2025 project was dedicated to filling these missing markets. This was the genesis of the AEC’s proposal for an FCAS-style spot market in one ESS, inertia.

There are some more things to be aware of:

• There is overlap between the responsibilities of the transmission companies and AEMO in managing power system security. Although it is a “missing market” as far as AEMO is concerned, transmission companies do have existing powers to procure ESS from the generators themselves. But, at least in the South Australian case, they were not used.
• All of the ESS that are declining from conventional power stations can be provided from a form of network equipment, synchronous condensers, which are basically conventional generators connected to the grid but without a power station attached. They are usually built by the monopoly transmission company and paid as part of customers’ network charges. This was the approach preferred by the South Australian transmission provider.
• A great hope for power systems is that IBRs can be designed, at only a modest additional cost, to provide the same services as synchronous machines. These are termed “grid forming” inverters/batteries. Whilst there are promising developments, engineers are yet to be fully convinced and they are not yet considered equivalent.

In the meantime, AEMO have at times had to give so many directions to gas-fired plants that they can find it overwhelming, and, if this is to continue, quite reasonably want a tool to automate it.

System Strength Reforms

In 2021 the AEMC made a Rule Change, proposed by TransGrid, to efficiently procure one ESS, system strength. This Rule places the responsibility of procuring it with the transmission provider, not AEMO. It includes regulatory oversight that it does not favour its own monopoly assets over those that can be procured cheaper, say, from a standby synchronous generator.

These reforms have gained widespread industry support and promise efficiency in procuring the service in the planning timeframe. However, transmission providers do not operate in the market, and to the extent that operating system strength contracts require day-to-day decision making, it was hoped AEMO would perform this task through the OSM.

A self-committed market

The NEM operates with self-commitment and real-time pricing which mean that generators themselves make day-to-day decisions on when to operate in response to energy and FCAS market signals. They use pre-dispatch forecasts to look ahead several hours to decide whether it is worth starting or shutting down a generator. Commitment is one of a spot trader’s critical tasks, with some units taking hours and tens of thousands of dollars to start. And, when they get it wrong, say because real-time prices turned out lower than forecasts, the generator wears the loss.

In contrast, North American markets mostly use central-commitment where such decisions are performed by the market operator. Traders bid start-up costs and times, and it is up to the market operator to decide when to start and stop the units. The startup costs, and the costs of running during low energy prices, are always paid by the market operator to the generator. It is combined with a day-ahead market where prices are quoted from a day-ahead forecast, with on-the-day variations resolved through a balancing market. The resulting costs are then socialised and charged to customers.

AEMO’s former CEO, Audrey Zibelman, preferred the central-commitment/day-ahead approach and AEMO vigorously pursued it during her tenure.

However, the AEC, amongst others, was unconvinced, as real-time pricing and self-commitment has served the NEM well, and, most importantly, decentralises risk. In 2020, the AEC engaged a detailed consultancy into the concepts. The consultants described the existing self-commitment/real-time pricing environment as a “mega algorithm” in the following diagram which mostly converges to an efficient commitment, dispatch and pricing solution, and leaves costs where they fall, rather than socialising.

Figure 1 The Mega-Algorithm

Source: Creative Energy Consulting Report

AEMO’s key argument for change was the repeated need to direct units on-line in South Australia, outside of this mega-algorithm. The AEC and its consultants agreed this was a major failing but saw it as a symptom of a “missing market” rather than a fault of self-commitment. It is unsurprising that generators do not self-commit units to provide an unpaid service.

The AEC’s self-commitment/real-time pricing view ultimately won the day and support for a full day-ahead/central commitment NEM has disappeared. Nevertheless, interest in a more limited “ahead mechanism” remained with respect to committing units to provide ESS that, for whatever reason, do not have an existing spot market. Indeed, the AEC’s consultancy recommended a rather minimalist Unit-Commitment for Security (UCS) tool for exactly this purpose.

The OSM also seemed to be pursuing this more minimalist objective and might be described as the remnants of the day-ahead/central commitment push.

What was the Operational Security Mechanism?

From late 2020 the AEMC began work on introducing “aheadness” in the scheduling of ESS. The AEMC does not have the power to progress its own Rule Change but happened to have two vaguely related proposals on the books, from Hydro Tasmania and Delta Electricity. The AEMC has the power to create an “improved” rule from these, which it exercised to develop the OSM with AEMO, who provided most of the design support.

The team developed a system where ESS providers who expected to be on standby for some period could present a price at which they would be prepared to come on-line. If AEMO’s tool decided they would be useful, it would call them on-line and pay them the price as bid. If it worked, it was hoped to:

• Perform the scheduling of any system strength or other contracts that transmission providers had procured from market-facing assets.
• Replace the need for AEMO to direct plants when required for system security.
• Allow some additional ESS to be procured beyond the minimum for system security if it saves money through a more efficient dispatch, e.g., by raising network limits.

These goals seem reasonable and uncontroversial, but once the team started building, they ran into great complexity.

Figure 2 OSM Processes

Source: AEMC Draft Determination

The biggest problem is that the OSM wouldn’t live in isolation. The units that it might call on are also available to self-commit into the energy and FCAS markets, and as their traders will participate in all three, the existence of any mechanism will affect the others. Thus, there had to be iterative “gate-closures” where generators’ self-commitment plans are fed into the OSM, which then runs a very complex look-ahead optimisation of potential additional commitments. And this in turn would influence traders’ subsequent decisions.

What was wrong with the Operational Security Mechanism?

As seen in Figure 2, the OSM was going to be a complex beast. This is the principal concern cited in the guidance note. But, even if it could be built, there were also philosophical concerns raised in submissions:

• The OSM recruits assets rather than the actual services that the grid requires. This breaks from the NEM’s technology-neutral philosophy, which, for example, buys and pays for FCAS equally from each of generators, batteries and demand-side-response, according only to the quantity of FCAS that they can actually deliver. The technology-neutral philosophy is also that if an asset can sell more than one service, then that is fine, and it is up to the owner to stack up the different value streams. By contrast, when operating the OSM, AEMO pays an asset once, and then draws upon all the services it happens to have.
• As it purchases assets rather than services, the OSM is necessarily a “pay as bid” market rather than one that pays at a marginal clearing price. “Pay as bid” arrangements are problematic in repeated auctions and give poor investment signals.
• The interactions with the self-committing energy and FCAS markets are very complex and potentially problematic. Because the objective function of the OSM was to lower the total cost of operating the system, it could not be prohibited from making decisions for energy and FCAS purposes, effectively over-ruling traders’ self-commitments.
• By providing a path to commit physical assets rather than defining and paying for services, it detracts from the ESB’s recommended path of developing competitive spot markets in those services, such as the AEC’s proposed inertia market.

Scheduling System Strength Contracts

One of the intended functions of the OSM was to handle the day-to-day scheduling of system strength contracts that transmission providers are procuring to protect weak parts of their networks. There is now an understandable concern from the networks as to how this can happen without the OSM, and whether the networks will have to become involved in operations and make decisions that they are not well placed to make. Examples include:

• how to schedule and dispatch system strength services at least overall cost to customers, given these decisions impact energy and FCAS market outcomes, and
• how to achieve this in adjoining regions given system strength support crosses regional borders.

AEMO does have manual processes that could potentially schedule these assets, but if the contracts become numerous, it is doubtful that those could do the job.

Transmission businesses also have concerns for their ability to manage the settlement of the activation or enablement costs of system strength. The OSM was designed for the market to pay these costs. In the absence of the OSM, transmission businesses will face cash-flow issues if they have to fund and recover volatile operating costs.

What does the AEMC propose instead of the OSM?

The guidance note speaks of using existing mechanisms in the planning timeframe as a “simple yet effective approach”. This seems to mean arrangements that look a little like the System Strength frameworks, with transmission providers deciding what to contract. The remainder of the OSM rule change will be dedicated to considering and refining these, a pathway of no relevance at all to the original rule proposals which the AEMC is using.

Inertia Spot Market

The guidance note also appears to put on ice the AEC’s proposal for an inertia spot market. The note reminds us that transmission providers can already enter contracts to acquire inertia, and the AEMC appears to be doubling down on this pathway rather than having a national competitive spot market in the service, run by AEMO.

This is a disappointment. Unlike system strength and voltage control, inertia is a genuinely global ESS, i.e., it can be purchased from anywhere. And as engineers have long commoditised it with the “Megawatt-second” unit, it is ideally suited for a spot market. Indeed, delegating the task from AEMO to transmission providers introduces difficulty simply because there are multiple transmission providers. How will they efficiently co-ordinate to buy what is a national product?

It is ironic that the AEC feared development of an OSM might detract focus from its proposed inertia spot market. Yet the extinction of the OSM appears far worse.

Conclusion

The end of the OSM is something of an anti-climactic conclusion of a process that began five years ago with powerful voices promoting a revolutionary change of the NEM into a day-ahead/central commitment market. By 2021 that maximalist change had already shrunk into a proposal for a mechanism to commit only some units some of the time. But trying to implement a half-pregnant central commitment mechanism proved much harder than it sounds.

Institutions often find it hard to step off a path in which they have invested great effort, so the AEMC’s ability to make this big strategic change of direction is impressive. However, the suggested new path is not necessarily one which the AEC would embrace.

Firstly, there is the practical question of what tool, in the absence of the OSM, is going to schedule the ESS contracts that transmission providers are signing up right now. But more significantly, the new direction anticipates local monopolists purchasing services that could be better purchased by one competitive national market.