Wednesday, March 24, 2010

Smartgrid Skunkworks

Dearest engineers and hackers, and also their management,

Recent vulnerabilities found in smart meters and HAN devices have shown a number of weaknesses in the engineering practices used to build these devices and their constituent components. A vulnerability in a chip or library is fixed slowly, and it is a very rare event that the meter and thermostat vendors affected by the vulnerability are notified by their suppliers. Because of this, vulnerabilities are spreading downward through the supply chain, and the engineers of smart grid devices are left uninformed.

While those utilities that actively investigate security have a considerable amount of bargaining power with their immediate suppliers, the rest of the supply chain has no similar leverage to compel security notifications. Chip and library vendors are failing to notify the meter vendors that depend upon their components. Even when the meter vendors are notified directly of vulnerabilities, thermostat and other HAN vendors can have no realistic expectation of such a privilege.

Despite having found many vulnerabilities in microcontrollers and LPAN radio chips, I have never seen one single security issue mentioned in the errata sheets of these devices. It has been a year since I first reported to Texas Instruments that the RAM of their Chipcon 8051 core is exposed to an attacker, but there's not one scrap of documentation from the firm to its customers suggesting that they make the simple patch of moving the key variables to Flash memory. The example ZigBee stack for the chip is still vulnerable to this attack, even after recent patches! A year later, exactly two debugger commands are all that are required to extract keys from nearly every ZigBee SEP device with a Chipcon radio, and no one knows to patch their code! (Do not be smug if you are an Ember customer. The EM2xx chips are unpatchably vulnerable to debugger key extraction, and there is no mention of this in the chip's errata sheet either.)

As chip and library vendors have failed to document the publicly known vulnerabilities in their products, and as they have often been unable or unwilling to repair them, the most expedient remedy to this problem is a separate line of communication. At least one point of reference must exist for the engineers trying to build these products.

For these reasons, I have created a skunkworks mailing list for the announcement and discussion of smart grid vulnerabilities, particularly but not exclusively those in AMI equipment. This is to be a list for engineering discussion, by engineers and security researchers. Anonymous posts and lurking are welcome, but politics and committee items are not.

For this reason, I especially request that those firms which care about security ask--or perhaps even require--their engineering staff to subscribe. This list is the appropriate place to post questions concerning the secure use of a particular radio chip, fragment of code, or anything else which is too low level or vendor-specific to be mentioned in standards.

If your firm is unwilling to allow its engineers to post, please at least compel them to follow the posts of others. In saying nothing, they will still learn how to make more secure products along with all sorts of fascinating gossip about your competitors. Your firm has every right to keep its mouth shut, but keeping its ears shut is a betrayal of each and every one of your customers.

To kickstart this mailing list, I will make it my first site of public disclosure for smart grid vulnerabilities over the coming months. The subscription link is below, and I invite you to join me in preventing smart grid vulnerabilities before they are created.

Thank you kindly,
--Travis Goodspeed
Belt Buckle Engineer
Security Hobbyist


伯臻采男 said...
This comment has been removed by a blog administrator.
少菁 said...
This comment has been removed by a blog administrator.
黃郁順 said...

great msg for me, thanks a lot dude˙﹏˙

BennieS_0123Godina said...
This comment has been removed by a blog administrator.
Matt said...

Would it not be a simple matter to disable the debug interface as part of manufacturing? Such as shown on pg 54 of the CC2530 User's Guide (SWRU191A).

Matt Farley

Travis Goodspeed said...


The DBGLOCK bit of the Chipcon 8051 devices, including the CC2530, can be cleared by erasing the device. Such an erasure clears the Flash regions of memory, but RAM (XDATA), allowing keys and similar things to be recovered. There is no way to permanently disable debugging on a Chipcon 8051 device other than to physically break the pins.


Matt said...

I don't see a good place to hook in a patch on the TI-MAC, but in the ZStack, AesLoadKey() and AESLoadIV() look like good functions to patch.


IT.Luddite said...

Followed the rabbit hole and and ended up over here. Amazing posts, absolutely got my mind going crazy considering the possible avenues of playing. I started going through your older posts and came across this one (Smartgrid Skunkworks). Much to my dismay, I find myself as an intended target of the post and went to join the Google Groups you linked to and found that Google has killed it off due to TOS crap.

Have you had the opportunity to resurrect the group elsewhere? I would welcome the opportunity to learn and apply the lessons to the AMI and DA networks I have responsibility for.

Bogdan Tomoiaga said...

Interesting indeed. What about a Pareto based approach? ... like in this paper:

Burkjard said...

Recent vulnerabilities found in smart meters and HAN devices have ...

Blogger said...

Did you know you can create short links with Shortest and get money for every click on your shortened links.