If this is a common way of operating a grid, it makes sense to define open standards so that software and devices are interoperable across any grid.

As a VPP provider/aggregator, I want to be able to deploy my software to support other grids without needing to interface with a whole different set of protocols or APIs. As a grid operator, I don’t want to have multiple demand response implementations for multiple VPPs/aggregators. If a VPP/aggregator goes out of business, I want another provider to be able to easily take over its program. As a citizen, I want a fast roll-out of energy flexibility solutions to reduce fossil fuel usage and keep costs low. Open standards facilitate all of these requirements.

Open Automated Demand Response (OpenADR)2 is one such standard.

Subscribe now

OpenADR Overview

As implied by its name, this standard focuses on device load shifting. It describes how an entity should publish demand response (DR) events and how participants should respond on receipt.

The OpenADR Alliance has over 220 members worldwide and over 319 certified products. California, the UK, Germany and the Netherlands all mandate OpenADR for use cases such as device connectivity in buildings, energy networks and EV charging3.

Technical Rundown

In OpenADR a Virtual Top Node (VTN) is a server that defines DR programs and events. A Virtual End Node (VEN) is a server that represents a DER or set of DERs. It receives programs and events from the VTN and then decides how to execute the action using the DERs it represents. Many VENs can connect to one VTN but one VEN cannot connect to another VEN. A very basic setup would be a grid operator-owned VTN and a VPP-owned VEN that represents a set of heat pumps. The VTN can request the heat pumps lower their consumption through events that are processed by the VEN.

OpenADR allows a VEN to also be a VTN. This enables hierarchical relationships within a system. For example, an aggregator may control DERs within multiple buildings. Each building could be a VTN and also a VEN that connects to a grid-wide VTN. Groups of DERs are represented as VENs within the buildings. When the grid-wide VTN publishes an event, the building VENs can delegate to its VTN to plan a course of action. This results in another event that is processed by the DER VENs.

OpenADR is an API-based standard. It describes a set of HTTP REST endpoints that must be implemented by a VTN so that VENs can be made aware of demand response events, and enable them to report the performance or status of the DERs under their control.

The standard has a number of different versions. OpenADR 3 is the latest version with the specification being finalised in 20244. OpenADR 2 was finalised in 2015 and has more widespread adoption. v3 incorporates more modern software development methodologies and data exchange formats including5:

• JSON instead of XML

• REST API instead of SOAP

• Oauth2 security model

The rest of this post focuses on OpenADR 3.

Understanding OpenADR 3: Core Components

The key concepts in OpenADR 3 are:

• Programs

• Events

• Reports

• Subscriptions

Programs

A program represents a demand response offering from an energy provider, where the provider can run multiple programs simultaneously (such as dynamic pricing and load shedding). Each program can specify particular events and reporting requirements needed to achieve its business goals. A VTN provides a /programs HTTP endpoint so that VENs can understand what programs are available.

Events

An event indicates a DR scenario is available to VENs. It contains a start time, duration and a payload to indicate the type of demand response. This could be a very simple DR such as setting a DER’s electricity consumption to a certain level, or day-ahead electricity prices so DERs can adjust their usage when prices are low or high. Events can also contain intervals to indicate different DR levels at specific times during the event. In the day-ahead electricity price example, this could mean 24 intervals to indicate the price of electricity at each hour of the day. Events can also target specific DERs represented by the VEN.

OpenADR defines a set of event types that address common DR scenarios including price indicators, load shedding, dispatching and alerts. The /events HTTP endpoint of a VTN enables VENs to discover DR events available.

Reports

Programs and events can require VENs to submit reports. The VTN owner may want to know DERs’ electricity usage during an event or their status. This is especially important for correct billing or for a VEN to indicate its capacity for load shedding. The /reports HTTP endpoint of the VTN is available for VENs to send their reports.

Subscriptions

VENs can subscribe to changes in programs or new events by making a request to the VTN with a callback URL. The VTN will call this URL when the conditions of the subscription have been met. Typically VENs will only subscribe to infrequent events such as alerts and poll for things that change more often such as new events. VTNs offer a /subscriptions HTTP endpoint so VENs can subscribe to different operations.

Use Cases

I’ve already described simpler use cases of OpenADR for DR scenarios such as load shedding and dynamic pricing. OpenADR 3 also enables more advanced use cases including dynamic operating envelopes and dynamic capacity management.

Dynamic Operating Envelopes

Popularised in Australia due to its widespread deployments of solar and batteries, dynamic operating envelopes enable grid operators to set upper and lower bounds for the import and export of electricity6. The growth of DERs means the grid is having to cope with bidirectional flows of energy. Dynamic operating envelopes improve system stability through constraint setting. The limits can be varied in short time intervals to help respond to customer usage patterns and the weather. They also provide better visibility into how the network operates in real-time.

OpenADR supports dynamic operating envelopes through its IMPORT_CAPACITY_LIMIT and EXPORT_CAPACITY_LIMIT event types. VTNs can set these limits on a minute, hourly, daily or monthly basis. There are also IMPORT/EXPORT_CAPACITY_SUBSCRIPTION event types typically with a years-long interval. If there is an extended communications interruption between the VTN and VEN such that the event schedule has been exhausted, VENs can use these event values as the default limits.

Dynamic Capacity Management

A DER may have a capacity level for importing or exporting electricity that it typically never exceeds. When it needs more it can request the amount required and the fee it’s willing to pay. OpenADR supports this dynamic capacity management through a number of event and report types.

A VEN will know its capacity level through the IMPORT_CAPACITY_SUBSCRIPTION and EXPORT_CAPACITY_SUBSCRIPTION event types. VENs can submit reports to request more capacity with IMPORT/EXPORT_CAPACITY_RESERVATION and IMPORT/EXPORT_CAPACITY_RESERVATION_FEE and the time periods for which the extra capacity is needed. The request is accepted when the VTN creates equivalent events. VTNs can also indicate what additional capacity is available through the IMPORT/EXPORT_CAPACITY_AVAILABLE event types.

Enabling Tomorrow's Smart Grid

With OpenADR 3, the standard has evolved to incorporate modern software development practices while addressing increasingly complex energy management scenarios. The hierarchical VTN/VEN model provides the flexibility needed to accommodate various stakeholders, from grid operators to individual DER owners, within a single interoperable framework.

The future of energy flexibility depends on open standards that allow seamless communication between diverse systems and devices. OpenADR's continued development and adoption will be essential in creating more resilient, efficient, and sustainable electrical grids capable of meeting the challenges of the energy transition.

Grid orchestration fills this gap. By providing real-time and high-fidelity data of local grid conditions, grid orchestration enables operators to understand how specific DERs impact grid reliability.

Core to the idea of grid orchestration is the Distribution System Operator (DSO) concept. A DSO manages the grid at a local level through high quality telemetry of assets and forecasting loads at every node. It dispatches assets to maintain a reliable flow of electricity and can enable local resources to participate in grid-services markets such as peak load reduction.

Bridging the Grid Visibility Gap

Why can’t an ADMS and DERMS provide high-fidelity visibility into local grid conditions? An ADMS doesn’t have a view of all meters and has limited telemetry across the network. It typically only sees a static load profile for each transformer, based on the type of meters it serves (residential, commercial, industrial). This makes the power flow equation3 easier to solve but with the growth of DERs, an ADMS lacks insight into accurate electricity demand across parts of the grid.

Grid orchestration providers such as Camus4 gather all meter data to give a view into what’s happening at the edge. This is aggregated back to where the ADMS ends, providing utilities and grid operators with a more data-driven view of the grid.

DERMS focuses on monitoring and control of DERs. They typically don’t know about local grid conditions near DERs. This makes dispatching assets tricky as operators need to switch between multiple systems to understand what can be done safely5.

An orchestration platform integrates data from multiple systems to give operators a comprehensive view of grid conditions. It maximises the value of DERMS investments “by enabling precise, grid-aware dispatch across all resource types”6.

Subscribe now

Case studies

Whilst each of the organisations below offer grid orchestration capabilities, they also provide DERMS and, in the case of GE Vernova, an ADMS. Kraken leans much more into the DERMS space than Camus. In fact some of the features each company offers can be complementary. For example, Kraken could use the grid context Camus provides to plan and dispatch the right assets to maintain grid stability.

Camus Energy

Camus’s vision is to enable US utilities to take on the DSO role:

Since we've gone to the trouble of describing a DSO and what it does, our goal is really to create a software platform that will enable a utility to take on that role. So a much more real time and local operations model that can include local resources as part of the supply and demand landscape and ultimately include them into capacity management and network management for the grid and let them get paid for it7.

Their grid orchestration platform integrates with multiple systems to provide a live view of what’s happening at the grid edge8. An advanced meter forecasting capability means the platform can predict loads at every point on the grid helping to inform “how the ADMS executes outage restoration, fault location, automated switching, and other distribution optimization activities”9. The platform can also dispatch all types of DERs including those managed by non-utility aggregators and helps utilities plan their systems by surfacing DER profiles.

Software platform

Camus deploys their software on Google Cloud Platform10. Services primarily written in Python ingest data from systems like ADMS and SCADA and are fed into BigQuery11. Camus uses NOAA weather forecasts12 for their meter forecasting that is also captured in BigQuery.

So, BigQuery sits at the centre of the software platform with many downstream consumers. Camus’s forecasting system reads and writes to BigQuery. It feeds time series data into BigTable13 for latency sensitive applications. A Postgres database is used for everything else - grid models, geodatabase deliveries and utility-specific data models.

A control server written in Go enables dispatching of DERs through vendor-specific code. A Vue web app overlays a utility’s grid on Google Maps. Users can see all substations, transformers, meters and DERs, and view real-time telemetry and forecasted load for each device14.

AI

Using data from utilities’ Geographic Information Systems (GIS), smart meters and NOAA weather forecasts, Camus’s supervised machine learning system with XGBoost can provide accurate forecasts for millions of meters15. XGBoost models also have explainable results and tools such as SHAP16 enable users to understand how much an input to the model contributed to the output.

ML can also inform power flow models. The models include solvers that attempt to solve the model’s equations by starting with an initial guess and refining until it achieves a desired accuracy. This is a very computationally intensive process. ML forecasts can make the process much faster:

initial guesses provided to the solver are much closer to reality which in turn give the power flow solver a head-start in solving its physics calculations. This framework significantly decreases the computational effort and at the same time, provides visibility at the midpoints of the distribution grid with high fidelity.17

Kraken

Kraken18 provides an operating system for energy. Their platform, deployed on AWS, combines advanced grid management with DER optimisation. It provides utilities and operators with comprehensive monitoring and control of distribution grids, offering real-time visibility into network performance, power quality, and fault detection.

Their system excels in integrating and managing a diverse range of behind-the-meter assets, including electric vehicles, heat pumps, and home batteries, while enabling the creation of smart tariffs and products that help balance grid demand. Through its data analytics capabilities, Kraken helps predict demand, prevent outages, and optimise distribution, ultimately improving grid resilience while delivering cost savings to consumers and enabling the smooth integration of renewable energy sources into existing infrastructure.

In collaboration with a Scottish Distribution Network Operator (DNO), Kraken developed a model of how electricity load is dispatched across the DNO’s network19. This helped inform the launch of a dynamic distribution grid tariff so that DERs could be orchestrated to operate at the “precise times and locations where distribution constraints arise”.

GE Vernova Grid OS

GE Vernova also aggregate data from multiple systems as part of their Grid Orchestration platform20. As well as utility GIS and asset topology data, they incorporate weather forecasts, satellite imagery, wildfire monitoring and vegetation data. This enables operators to understand risks to the grid and how DER management could overcome peaks in demand. Like Camus and Kraken, GE Vernova aim to provide real-time telemetry and forecasted load across the network so operators can more effectively manage the grid.

Grid Orchestration: The Missing Link in the Modern Grid

The emergence of grid orchestration platforms marks a crucial evolution in grid management, filling gaps that ADMS and DERMS cannot address alone. Companies like Camus and Kraken are leading this transformation by leveraging machine learning and cloud computing to create platforms that can predict and respond to grid conditions with unprecedented accuracy. By integrating multiple data streams with real-time telemetry, these solutions not only solve immediate grid management challenges but enable the transition to a decentralised energy future.

7

Volts

Managing a distributed grid

In this episode, Astrid Atkinson, co-founder of Camus Energy, talks about her company’s “grid orchestration” work of helping utilities see, track, and coordinate the distributed energy resources in their territories…

Listen now

2 years ago · 69 likes · 7 comments · David Roberts

Without any improvements to the efficiency of the grid, we would need to build much more capacity such as larger numbers of electricity generation sources (which could rely on fossil fuels) and transmission infrastructure. Solutions like DERMS reduce the need for costly system upgrades. A 2023 transmission study by New England’s Independent System Operator found that reducing 2050’s predicted peak load through demand response and energy efficiency programs (which leverage solutions like DERMS and Virtual Power Plants2), could reduce transmission costs by $9 billion3. A Wood Mackenzie analysis discovered that DER project management costs such as DERMS was 100x cheaper than physical infrastructure costs4.

Whilst ADMS tends to be dominated by more established companies and conglomerates, there’s been an explosion in the number of startups offering DERMS5. Unlike ADMS, which has to be deployed in utility control rooms, DERMS can be standalone SaaS products hosted in the cloud with a web app and API. They are able to integrate with behind-the-meter DERs (e.g. smart thermostats, home batteries, EVs) thanks to OEMs providing APIs to enable third party access to these devices.

What I’ve described so far is known as a “Grid-Edge” DERMS6. There are also centralised DERMS that are often managed alongside an ADMS and enable control of front-of-meter DERs (e.g. large scale fleets of batteries controlled by the utility).

DERMS architecture

These are the main components of a DERMS:

• DER command and control system - often requires adapters to integrate with devices from different OEMs

• Utility IT integration system - so the DERMS can understand grid conditions through systems like ADMS and metering infrastructure

• Customer notification system - alerts customers that their device will be used for demand response for example

• Web application/API - for administrators and grid operators to get an overview of the DERs managed by the DERMS and trigger demand response events

• Forecasting system - often uses machine learning to predict future DER capacity

Generac Concerto Case Study

The Generac Concerto7 DERMS system integrates with over 60 third party DER manufacturers. It provides a single platform for managing these DERs and can optimise the grid whilst “accounting for the constraints of every asset in the system”8.

The platform is built using AWS and uses ECS, Lambda and EC2 to integrate with DERs using REST APIs and standardised protocols such as OpenADR, IEC 104 and DNP3. Postgres and DynamoDB store telemetry, configuration and grid events.

Concerto provides a browser-based web app for utility administrators and grid operators. AWS’s Cognito service enables customer identity and access management.

Grid service events are handled using API Gateway. For example, an ADMS could request emergency load shed on a distribution circuit:

Concerto determines the optimal target group of assets to solve the grid need with a goal of cost minimization for utilities or third-party owners, and it initiates dispatch with corresponding set points for each asset. It performs near-real-time, closed-loop control throughout the event, making sure that the desired capacity and energy are delivered.

AWS has a more detailed write-up about Concerto.

DERMS and VPPs

Both DERMS and VPPs control and aggregate DERs, and forecast their available capacity. It’s reasonable to ask what is the difference between them?

A VPP aggregates DERs to provide grid services for monetary gain. For example, VPP providers will receive payments when they lower energy consumption to keep the grid balanced.

A DERMS is simply used to monitor and analyse DERs’ status and performance. It can also help grid operators manage the grid through DER control. There is no monetisation of assets in a DERMS.

Many DERMS providers offer VPP modules given they have already implemented DER command and control that VPPs require.

Unlocking greater value from DERMS

A lack of standardisation makes DERMS development slower and inhibits their growth. Generac Concerto has over 60 different integrations with DERs because each of them use different protocols and data formats for communication. It’s not possible to have a viable DERMS platform without dozens of integrations, each of which takes time to develop. Implementing standards such as OpenADR9 and IEEE 2030.510 could accelerate the move to a smart grid that takes better advantage of renewable energy.

Conclusion

As we decarbonise our societies by electrifying everything11, we need better control over new producers and consumers of energy. DERMS have an important role to play in managing and optimising the electricity grid. The latest cloud software is enabling the development and deployment of these systems such as Generac Concerto. DERMS will be crucial in ensuring grid stability, efficiency, and reliability as we transition to a more distributed and clean energy landscape.

In 2023 440 GW of additional renewables capacity was added worldwide1. This is more than the combined power capacity of Germany and Spain. The US DER market will double between 2022 and 20272. Consumer electricity demand will grow nearly 4x mainly due to EV charging3. The US energy storage market will install almost 66 GW of capacity between 2023 and 20274.

The software systems that grid operators and utilities have relied on are no longer sufficient for meeting these new challenges. The US Department of Energy (DoE) estimates that deploying new grid technologies could support an additional 20-100 GW of incremental peak demand5. This is up to 13% of current US peak demand. The DoE also reckons these technologies could “defer an estimated $5-35B in transmission and distribution infrastructure costs over the next five years”.

Software will play an important part in achieving this additional capacity and cost savings. In particular grid management software including Advanced Distribution Management Systems (ADMS), Distributed Energy Resource Management Systems (DERMS) and grid orchestration tech are vital in this transition.

My next series of posts will explore these technologies beginning with ADMS. In researching this space you very quickly realise this industry loves their alphabet soup6. Many of these concepts overlap with one another and it can be hard to disentangle the differences between DERMS, Virtual Power Plants, Demand Response Management Systems, SCADA etc. I am sympathetic to the argument that these are legacy problems and many overlapping systems makes grid management more complex than it needs to be7.

Nevertheless the market has coalesced around product offerings in the ADMS and DERMS space. So let’s start by digging into ADMS.

What is an ADMS?

An ADMS is a comprehensive software platform that serves as the backbone of modern electric utility control rooms. It provides utilities with essential tools for monitoring, controlling, and optimising the distribution grid, ensuring reliability, stability, and efficiency in an increasingly complex energy landscape.

ADMS builds on traditional DMS by integrating additional systems such as Outage Management and SCADA (Supervisory Control and Data Acquisition). The latter provides control and monitoring of power plants.

Key Functions and Capabilities

1. Real-time Monitoring and Control

   1. Captures real-time data from across the distribution network, including voltage, current, equipment status, and smart meter information.

   2. Provides direct control over utility assets such as transformers, capacitor banks, and switches.

   3. Offers comprehensive situational awareness of the grid's operational status.

2. Grid Modelling and Optimisation

   1. Runs power flow models that are important to the functioning of the electricity system. The models analyse how electricity flows through the grid and help identify potential issues like overloads, voltage violations, or stability problems.

   2. Conducts forecasting and look-ahead operations to predict future load patterns.

   3. Implements grid optimisation techniques like Volt/VAR control to minimise losses and improve efficiency.

3. Outage Management and Reliability

   1. Utilises Fault Location, Isolation, and Service Restoration (FLISR) capabilities that reroute power in the event of a fault to restore power to as many customers as possible

   2. Enables faster fault detection and outage management

   3. Implements strategies to withstand disturbances and disasters

4. DER Integration and Management

   1. Models various DER types, including solar PV, battery storage, EV charging stations, and co-generation facilities

   2. Visualises DERs within the system, similar to other network assets

   3. Incorporates DERs into network energisation status calculations and fault current contributions

   4. Identifies grid-forming resources for localised power restoration during outages

5. System Integration

   1. Acts as an integration framework for multiple utility IT systems

   2. Interfaces with Geographic Information Systems (GIS), Customer Information Systems (CIS), Meter Data Management (MDM) systems, and Energy Management Systems (EMS)

By improving situational awareness and enabling automatic system reconfiguration, ADMS boosts grid reliability and reduces outage duration. It efficiently integrates DERs, supporting the transition to a more decentralised, two-way power flow system. As a foundational technology, ADMS paves the way for solutions like DERMS and Virtual Power Plants8.

Examples

Using Schneider Electric’s ADMS solution9, Arizona Public Service (APS) implemented an ADMS to address wildfire risks and increased DER penetration10. This enabled better management of rooftop solar integration and wildfire prevention.

Sacramento Municipal Utility District (SMUD) launched an ADMS alongside a DERMS to support their Zero Carbon Plan11. Using software provided by Open Systems International12, this facilitated the shift to a two-way decentralised distribution system and optimised DER management.

Vector NZ’s ADMS

Since 2020 Vector Limited, New Zealand's largest electricity and gas distributor, has leveraged AWS Cloud capabilities to innovate and improve its services. To manage its growing and complex network, Vector implemented GE's ADMS13 on AWS Outposts14.

AWS Outposts enables AWS infrastructure and services to be run on premises. This meant Vector could deploy their ADMS using EC2 and EBS. Future evolution of their infrastructure may see usage of EKS and S3.

Vector's ADMS deployment demonstrates a robust, highly available architecture designed to ensure continuous operation of critical infrastructure. The system utilises two physically isolated AWS Outposts racks, each connected to a different Availability Zone in the AWS Sydney Region, creating an active-active setup with multiple ADMS instances and near real-time data replication. This redundancy extends across network, infrastructure, and application layers, safeguarding against various failure scenarios. Connectivity between the Outposts and the AWS Region is secured through redundant AWS Direct Connect links, enabling AWS to perform maintenance and monitoring while ensuring that sensitive data remains within Vector's network, accessible to on-premises systems via local gateways.

The AWS Reference Architecture for ADMS15 demonstrates this setup:

Conclusion

ADMS is proving to be a crucial technology in the evolving landscape of electricity distribution. As grids become increasingly decentralised due to the proliferation of renewable energy sources and DERs, ADMS offers utilities the comprehensive tools needed to monitor, control, and optimise their networks effectively.

The case studies of Arizona Public Service, Sacramento Municipal Utility District, and particularly Vector Limited in New Zealand demonstrate the real-world applications and benefits of ADMS. Vector's implementation, leveraging AWS Outposts, showcases how cloud technologies can be adapted to meet the stringent requirements of critical infrastructure, providing high availability, resilience, and security.

Looking ahead, the integration of ADMS with other emerging technologies like DERMS and VPPs promises to further revolutionise grid management, supporting the global shift towards cleaner, more efficient, and more resilient energy systems.

Virtual Power Plants (VPPs) reduce the need for these “peaker” plants by aggregating distributed energy resources (DERs). DERs such as rooftop solar, home/office batteries, EVs and chargers, smart home devices and electric water heaters, are being installed at greater and greater rates. In the US the DER market is expected to double by 2027 with 262 gigawatts of additional capacity installed1 (enough to power 750,000 American homes). These DERs can help the grid by reducing or meeting demand through remote control. This avoids having to burn dirty fuels during peak loads.

DERs can also reduce the need for additional electricity grid transmission and distribution. We need to “electrify everything” to meet our climate goals. This will require the grid to bring on more and more sources of generation and increase the transmission and distribution infrastructure (pylons, overhead lines and substations). DERs require fewer infrastructure upgrades through greater efficiency - devices can be smart about when to consume electricity and when to provide it to the grid.

By aggregating DERs, VPPs provide a more predictable and easily managed way of helping the grid stay balanced. In the US around 60 gigawatts of VPPs could add $20 billion in societal benefits over a 10-year period. This includes carbon emission reductions, energy resilience, and transmission and distribution infrastructure deferral2.

Software is at the heart of a VPP. A distributed energy resource management system (DERMS) is needed to control the DERs (often in the cloud). Other systems calculate the capacity of the VPP and bid into electricity markets. Before getting into the software we need to understand the basic principles of a VPP.

How do VPPs work?

VPPs aggregate DERs by enrolling customers in a program. There are currently three models3:

1. Utilities directly enrol customers by using existing DERs or offering DER purchase enticements e.g. Green Mountain Power’s battery VPP and Duke Energy’s managed EV charging VPP.

2. Manufacturer or retailers of DERs enrol customers in a VPP e.g. Tesla, Ford, GM, Sunrun, Sunnova

3. VPP Platform company enrols customers from a variety of different DER products e.g. Voltus, AutoGrid, and Leap, for example, recruit participants, directly or via partnerships, with a variety of DERs in residential and non-residential settings.

By aggregating these DERs the VPP can offer services like a normal power plant such as delivering electricity to the grid, demand reduction and ancillary services (which help maintain grid health). These services are sold to the grid and customers enrolled in a VPP make money from having their devices controlled by it. The VPP will often bid into an electricity market to offer demand reduction at a particular price. It knows its available capacity by using historical data from its DERs and forecasting algorithms.

VPP Software Platform

These are the main components of a VPP software platform:

1. A system to enrol customers in the VPP

2. A system that receives telemetry from the DERs and can control them (DERMS)

3. A system that predicts the capacity of the VPP and bids into the grid operator’s electricity market

4. A system that can receive requests to reduce load etc

This diagram gives an overview of a hypothetical microservices architecture for a VPP:

Case studies

Tesla

Tesla’s Powerwall batteries are wifi-connected devices that can power homes. In 2022 Tesla’s VPP had enrolled over 4500 of these devices and was able to contribute 33 MW of electricity during peak load events in California4 (enough to power hundreds of homes for a year). Tesla uses a software stack including Scala, Akka, Kafka and gRPC APIs to provide its VPP. In 2020 Tesla engineers gave a talk at QCon London that described their VPP architecture (all images are taken from the presentation)5:

DERMS

Tesla’s Powerwall devices connect to a cloud platform via websockets. Customers voluntarily enrol their devices in the VPP program, and the system represents the state of each device using Akka actors. These actors are organised into parent actors that aggregate multiple devices. The data flow within the system is as follows:

• Websocket message telemetry is sent to Kafka

• Akka Streams services read these messages from Kafka. 

• The Akka actors, which reside in Kubernetes pods, receive these messages. 

• If a pod fails or scales, actor rebalancing occurs. 

• The actors receive state refreshes from new messages sent by the devices. 

• gRPC APIs enable querying the cluster to retrieve individual or aggregated telemetry data. 

The control service, implemented using Akka Streams, resolves all devices through the assets service, reads relevant battery telemetry, and plans the discharge of batteries to meet the required capacity. This planning ensures efficient battery usage by avoiding discharging batteries that are already low, and it communicates these discharge requirements and timeframes to each battery.

Autobidder

Tesla's Autobidder is a workflow orchestration system integrated into the VPP that uses market data to forecast VPP energy capacity and make bids into electricity markets based on price predictions. The forecasting and optimization services are primarily written in Python, while the rest of the system is implemented in Scala. The Autobidder system employs co-optimization strategies to meet both global and local objectives. For example, the grid may require battery discharge due to a peak event, while customers may want their batteries fully charged due to an incoming storm. Using the same Akka architecture as the DERMS, Autobidder pushes price forecasts to devices through the control service. The devices then develop a plan for battery balancing that considers both local and global objectives. This plan is pushed back up the telemetry architecture, with aggregations at each node. Autobidder utilizes this aggregate information to determine its bidding strategy. If a device loses connectivity, it continues to make plans based on the best estimate of the global objective. After a certain period offline, the device switches to local optimization only to maintain efficient operation.

Lunar Gridshare

Lunar Gridshare provides a VPP by connecting third party devices to its platform. It is “connected to 77,000 residential devices across Europe, Japan, and the United States, representing 290 megawatts of power and 725 megawatt-hours of energy capacity. This is enough to power 578,000 average US homes for one hour.”6

The platform is built using a number of AWS services including:

• AWS IoT - manages the third party devices enrolled in the VPP and sends data to the cloud via MQTT (a lightweight messaging protocol)

• AWS Kinesis - device telemetry is pushed onto Kinesis message queues to be consumed downstream

• AWS Step Functions - coordinates AWS Lambdas to make forecasts and control the device fleet

AWS have a more detailed write-up of the Gridshare platform and architecture.

Virtual Peaker

Virtual Peaker has partnered with various utilities in the US to provide a VPP. Their platform aggregates third party smart thermostats, smart water heaters, EV chargers and batteries. In a partnership with the Pacific Gas & Electric Company in California, they aggregated 525 batteries into a VPP from various providers including Tesla and Sonnen7: 

The future for VPPs

The capacity of VPPs is still very small compared with the rest of the electricity grid. To reduce our reliance on fossil fuel peaker plants and help the grid cope with an increase in electricity demand, we need to unblock the development of VPPs in the following ways:

Standardisation

Communication protocols across DERs and VPP platforms are fragmented. This makes it complex and costly to manage DERs for utilities and platform providers. Regulators could mandate that DER manufacturers use open source APIs or standards that make it easier to integrate them into third-party platforms. The underlying technologies of VPPs also lack consistent standards. Providers all implement telemetry collection and DER interconnection in different ways. 

A number of working groups have been established to solve this fragmentation. OpenADR8 standardises the message format used when the grid needs to reduce demand automatically and for managing DERs. This simplifies data exchange between grid entities (utilities, grid operators and energy control systems). The Open Vehicle Grid Interface Protocol (OVGIP)9 provides a common communication protocol between EVs and the grid. Matter10 is an open source connectivity standard for smart home devices.

There is also a lack of standardisation around DER and VPP modelling and forecasting tools. Utilities often perceive VPPs as less reliable than traditional assets because of wide variation in these methods. Open source tooling would help increase VPP adoption. 

Virtualisation innovation

As we electrify everything and grid infrastructure is upgraded, increasing generation, transmission and substation capacity is only one way of achieving these upgrades. An alternative is to allow the market to bid on these enhancements which could involve virtual upgrades. In a conversation between Duncan Campbell and David Roberts on the Volts podcast11, an example is given of insulating houses rather than upgrading a transformer:

If I go to houses connected to that particular transformer and install insulation on all of them, thereby reducing their demand for all the houses in that particular area, I have obviated the need for that transformer upgrade. I have basically provided a virtual transformer upgrade. - private providers could bid on this virtual stuff when new need arises.

Bill payers could also “pay private companies to do what amounts to infrastructure upgrades”:

I'm talking about ratepayers paying the private company that gives them the best price. Right? Versus just paying one private company offering one solution. Right. Let's make all of them offer various solutions at various prices with various costs of capital and see what happens. And you don't have to go with the DERs, by the way. Like, if the system operator doesn't think they're going to be reliable, doesn't think they're cost-effective, don't do it. Right. But that is the only way for us to actually develop the knowledge of what do these things cost, who will finance them, what are their capabilities? Is to start holding the auctions.

This framework would likely increase the adoption of VPPs by enabling these platforms to offer demand reduction and electricity generation as an alternative to building fossil fuel power plants or upgrading the grid infrastructure.

By addressing these areas and fostering collaboration among policymakers, industry, and technology providers, we can pave the way for the broader adoption and successful integration of VPPs into the future energy landscape.

11

Volts

Now is the time for distributed energy

In this episode, Duncan Campbell of Scale Microgrid Solutions makes the case that distributed energy resources (DERs) — solar panels, EVs, home batteries, etc. — are, thanks to rising electricity demand and constraints on grid expansion, poised for a tsunami of deployment…

Listen now

2 years ago · 49 likes · 18 comments · David Roberts

As a software engineer I’ve wondered how I can use my skills and knowledge to tackle this challenge. Climate tech is often of the harder variety than the soft. The biggest source of carbon emissions is burning fossil fuels for energy. We need solar, wind, hydro, nuclear and cutting-edge tech like geothermal power to displace fossil fuels.

But software has an important role to play in the energy transition. As we “electrify everything” the grid will come under increasing load and we need software to orchestrate the grid in smarter ways. The growth in smart home appliances is increasing and software can make the grid more reliable by controlling these devices to reduce demand or provide electricity. Software is important for forecasting renewable energy capacity and modelling tools can help grid operators manage the grid more effectively.

This newsletter will go in-depth on these topics and detail the types of software systems used to assist the energy transition. My software engineering background is in cloud technologies so expect a lot of discussion around cloud system design and application protocols.

Subscribe now

Subscribe now