Who Will Build the Smart Grid?

Story reprinted with permission of MuniWireless.

There should be some interesting conversation at the Smart Grid 2010 Summer V-Summit this month, following announcements of $200 million in funds to improve the way we create, connect and use power; plus the federal government's entry into critical infrastructure security dubbed the “Perfect Citizen Program.” It is clear that smart-grid momentum has started and will continue to move forward. Finding ways to securely and efficiently improve global power production and distribution is an ongoing process.

We do need a little creative imagination while recognizing the realities of current smart-grid infrastructure and security. For example, power company engineers have sometimes said that a smart meter design under regulatory cost guidelines needs to last at least 15 to 20 years while software engineers warn that home area network application software will change every six months. R&D engineers have said that it will be 10 to 15 years before we have the technologies and infrastructure to deploy smart grids and yet a few small cities have already deployed smart power and utility systems and are already using the same infrastructure for other city services. I have never seen such a swing in views. However, there is a common point of agreement: Smart-grid efficiencies will be achieved with or without stimulus funding.
To understand why there is such a disparity in thinking when it comes to the smart grid, I would suggest looking at the diagram on the Office of Electricity Delivery and Energy Reliability site. When participating in professional smart grid discussions I found, in the same group conversation, one person talking about technology and another talking about policy and regulation. Power companies look at things one way, consumers and environmental groups another, with policymakers sometimes bringing everything to a halt. There are specific solutions that support the progress of smart grid, and to employ them, it may be time for a fresh look.
The smart grid and the solutions to its successes are not simple. So it's essential to find someone who has already done what you are looking for and also to find a consulting and real-world integration group that has practical experience in the subject.

You Can’t Just Switch on the Smart Grid

I have done a lot of upgrading from legacy communication networks to digital IP networks and it’s a lot easier to build a brand new IP network than to interface dissimilar legacy technologies into one homogeneous network.
In a recent Smart Grid Alliance webinar Doug Preece, utilities industry specialist from Capgemini, summed up just how big and complex the opportunity is. He referenced the SmartGrid.gov site as a way to track the billions of grant awards already released by the Smart Grid Investment Grant (SGIG) program. In fact there is so much work to be done Preece stated that there would be “a tremendous draw on resources across the industry, vendors and service providers,” and that “there in no lack of opportunity.”
Dave Malkin the policy manager for GE Energy has been following a variety of issues as they relate to the smart grid. One of the top policy issues he is tracking is cyber security policy. "Cyber security has been a hot button in Congress for months now” said Malkin. He addressed his concern on how current cyber policy passed by Congress could affect the smart grid. He agrees that we need to, “effectively address real-world smart-grid cyber security threats but not in a way that might stifle smart-grid innovation investment.”
As smart grid policy and technology discussions continue, the small city of Rock Hill, S.C., already decided to start a municipal wireless network a few years back. With the network technology designed by Tropos Networks, the city of Rock Hill is an example of a municipally owned and operated utility that initially deployed the network for smart meters -- power and water. Today, the citywide wireless broadband mesh network is helping other city departments improve efficiencies, customer satisfaction, public safety and billing accuracy, while significantly reducing operational costs.
”The city of Rock Hill incorporated wireless broadband as part of the city’s strategy to build a multi-use communications foundation. This is a great example of a community that started out with a much smaller plan focused on modernizing and improving utility services and expanded their vision to one which has produced long-term benefits for the community, ” said Denise Barton, marketing director, Tropos Networks. Building the Smart Grid is a major undertaking but the answer may be "just do it." In previous articles I disclosed security concerns from industry professionals in today’s power grids with solutions in place developed in military and DoD applications that have direct and immediate application to the current smart-grid security problems. What we did in security is a good model of what we need to do now when transforming our global power grids.

With billions in stimulus grant released and GE Energy offering $200 million in venture capital to small businesses, there is a clear message. We are dong this and we are doing this now. We need to use what we have today in immediate deployable technologies while finding migration paths from the old power grid infrastructure to the new smart grid. Big government, big business and small business will be working together addressing critical global energy needs while creating jobs. Count me in.

Larry Karisny is the director of Project Safety.org and a consultant supporting local wireless broadband, smart grid, transportation and security platforms.

The Smart Grid Needs to Get Smart About Security

"Hacking" a smart meter or an entire grid requires no physical access -- just access to the same Internet connections used to manage the network. Reprinted with permission of MuniWireless.

With all due respect to the power companies, why should they even know how to spell IP? Their history in communications was to build stand-alone power facilities and substations connected with point-to-point microwave communication links (many times upgraded to their own dark fiber point-to-points). With this kind of money and private network capabilities, why would you ever worry about security? You lived on you own island with your own power and communications grid and everything was just fine.

Then came the smart grid. By definition, the smart grid requires a two-way digital technology to control appliances at consumers' homes to save energy, reduce cost and increase reliability and transparency. A big change for power companies and admittedly a whole new learning curve with many power companies like PG&E setting up their own test labs to begin learning this. (See Inside PGE's Smart Grid Lab -- Chris Knudsen, director of the technology innovation center at PG&E, shows what they're tinkering with).

Utility Meter

It didn't take long for problems to occur. Again, you need to understand that even smart meters were just dusted off 20-year-old designs that were lying around waiting for someone to push the power companies into the 21st century. These designs were never meant to securely send and store data real time. It wasn't long before serious security issues were found and were reported by respected security firms like InGuardian and IOactive. And we are not talking about someone hacking your PC. When it comes to the power grid, the costs of remote hack attacks are potentially more dramatic. "The cost factor here is what's turned on its head. We lose control of our grid, that's far worse than a botnet taking over my home PC," said Matthew Carpenter, senior security analyst of InGuardian, speaking at a panel at the RSA Security Conference in San Francisco. So now with little knowledge of the Internet and security the power companies have billions of dollars of grants in hand with one big problem. The grants mandate an iron-clad security platform.

To add to the smart-grid security problems some people think the power grid is the main target in the new battle in cyber wars. Richard Clarke, the former federal anti-terrorism czar, has now turned his attention to a new national security threat, putting an attack of the power grid on the front lines. In a recent Newsweek article, Clarke was quoted as saying, "The U.S. government, [National Security Administration], and military have tried to access the power grid's control systems from the public Internet. They've been able to do it every time they have tried. They have even tried to issue commands to see if they could get generators to explode. That's the famous Aurora experiment in Idaho. Well, it worked. And we know there are other real cases, like the power grid taken out in Brazil as part of a blackmail scheme. So the government knows it can be done, the government admits it can be done, the government intends to do it to other countries. Even the Chinese military has talked publicly about how they would attack the U.S. power grid in a war and cause cascading failures."

So what can we do to secure the grid now while upgrading it to smart-grid capabilities?

Ed Smith, CEO of WirelessWall has one word, "Attack." Having a military background he understands that you begin an attack by crippling an enemy's communication and critical infrastructure. His civilian background has a long history of situational crisis management, using rapid response teams to facilitate the successful conclusion to crisis situations. Armed with security that exceeds the DoD 8100.2 (DoD Directive on wireless security) and FIPS 140-2 End-to-End Security that was developed for the U.S. Navy to provide secure, mobile shipboard networks, Smith knows he has an immediately implementable data security solution that is simply not being recognized.

"People in the civilian sector are not upgrading their security for business reasons, basically to save money, not for security reasons," said Smith. "That can be tolerated if you are protecting data that involves a loss of money, but it is inexcusable when the lack of protection of data involves the loss of life. Let there be no doubt that an attack on critical infrastructure is an act of war and it is absolutely appropriate to use an available military solution to protect civilian lives.

"We can't afford not to put good enough security in our power grids," continued Smith. "My company has offered our platform of higher security to VISA and others in the financial industry and made it clear that the retail industry POS terminals Data Security Standard (PCI DSS) has already been hacked, but nothing will be changed unless there are more attacks that cause greater losses. The PCI DSS standard will have to be raised, and ultimately will, but the smart power grid protection has to be implemented now."

Like the old David and Goliath story, the power companies need to start embracing smaller company expertise and leverage their learning curve. Like the security story of WirelessWall, the expertise of how to build these wireless network platforms resides in the companies that have had their products tested in real-world municipal, public safety and military environments. Companies like Tropos Networks, Trillium (SkyPilot), Mesh Dynamics, Strix Systems and Proxim, just to name a few, they were the trail blazers that learned along the way and can now bring tested wireless network expertise to the smart grid.

With secure wireless solutions out there, power companies need to leverage the expertise of these wireless pioneers that have been there, done that and are ready to support a secure a wireless smart grid network with their tested solutions.

Larry Karisny is the director of Project Safety.org and a consultant supporting local wireless broadband, smart grid, transportation and security platforms.

Smart Grid Security, Ground Zero for Cyber Security

"One of things incumbent on all of us is to introduce strong authentication into the fabric of the smart grid. We did not do that with the Internet." Vint Cerf (pictured) Photo by Terence Brown. Story reprinted with permission of MuniWireless.

It was pretty amazing to see the amount of people involved in Conductivity Week in Santa Clara California last week. They were all there positioning their expertise on how to build and secure the smart grid. With NIST, WiFi Alliance, Zigbee Alliance, and the IEEE and hundreds of vendors and speakers attending, it was like a wireless IP Mecca of intellectuals all contributing to this global energy network requirement.
The Godfather of the Internet, Vint Cerf, opened the meeting and ended his keynote speech with a daunting announcement, "One of things incumbent on all of us is to introduce strong authentication into the fabric of the smart grid," Cerf said. "We did not do that with the Internet.
"My excuse is public key cryptography [was] not even publically written about until 1977 which is just about when TCP/IP was getting standardized, Cerf said.”But today we don't want devices to respond to control from something that’s not authenticated."
So what is the smart grid anyway? Wikipedia defines it rather well: "A smart grid delivers electricity from suppliers to consumers using two-way digital technology to control appliances at consumers' homes to save energy, reduce cost and increase reliability and transparency. It overlays the electricity distribution grid with an information and net metering system." With this definition, why is the smart grid such a security issue?
We need to first look at how our power grid operates today. Power distribution and monitoring today is in its initial stages of becoming a smart grid with some substation network intelligence often connected by microwave, power line and/or fiber-optic point-to-points. Although these core network infrastructures are very basic, they may prove useful in operating the needed private IP backbone of the smart grid. These network backbones were not meant to securely connect two-way digital connections from every home, every building every factory and every energy-using appliance throughout the power companies service area. In fact, adding millions of these connections to the power grid distribution system is no easy task in network or network security.
Power companies are in the precarious position of having to do something now while preparing for the future. In my earlier article, grid security firms and even past cyber-security czars clearly explained today’s power grid vulnerabilities. With $3.375 billion kicked in by the federal government and even more funds added by power and utility companies, they need to produce now but only if a smart grid security plan can be demonstrated. This leaves the power companies in a tough position of needing to do something today while being prepared to migrate smart grid security platforms to newer standards. The bottom line is there is no hurry up and wait when it comes to deploying and securing smart grid networks. There is only hurry up and be prepared to hurry up again.
So what were are the security problems and how can they be immediately addressed? The smart grid network, smart grid operating center and back office are pretty much secure. The problem is when you start connecting smart-grid devices to homes, business building and factories. You now have opened up the potential of accessing the smart-grid distribution network though millions of smart-grid end user access points. This network edge connection is called layer 2 which in the past had limited concern for security. It now has to lock out smart-grid end users while also having the capability of running independently and interperably throughout the smart grid.

Lessons From Foxholes

In trying to explain the layer 2 security and its importance, this Memorial Day I remembered some old WW II stories from my father. His Army job was to make field communications work in sometimes impossible situations. These WWII networks were very basic, running wires from foxhole to foxhole (layer 3) then connecting to the wireless field radios (layer 2). Even if the wireline connections were cut or if the radio battery died, the battle continued. If the wires were cut (layer 3) the pre-intelligence was already given from the commanders in the foxhole (layer 2) with the special code commands being conveyed man-to-man (today’s machine to machine, M2M). When I asked my dad what the most important network connection was, surprisingly, the answer was the radios in the foxholes and the man-to-man communications. They were able to continue the battle operating in layer 2 and M2M without command layer 3 connectivity. This is how we need to build and secure smart grids networks today.

Micro-Grids

In a response in my earlier article, The Smart Grid Needs to Get Smart About Security, Niall McShane, a smart-grid consultant commented: "We need to start the process of re-architecting the grid into smaller, localized micro-grids that are loosely coupled in a federation to help balance supply and demand across wider geographic areas which can also island from the macro-grid to prevent the propagation of faults. In this way we move from a single large target that can be attacked and that will then propagate the fault throughout the network to a large number of much smaller targets."
While micro-grids offer physical network security, a new awareness of the importance of layer 2 security is becoming recognized. Strong security encryption must reside at the layer 2 network level when you are collecting two-way digital information from the smart-grid network edge. Whether you are protecting personal information from the home or stopping potential grid-network access from the network home gateway, layer 2 is where this security must reside. There are also additional advantages of security mobility and scalability that can only be offered at the layer 2 level.
These important layer 2 features have also been documented in a recent white paper Portland: A Scalable Fault-Tolerant Layer 2. Just like the WWII soldiers were sometimes connected and sometimes not connected to the communication command center, smart-grid edge applications will need to securely and independently migrate applications, computation and storage into local data centers spread across the smart-grid edge. With all this intelligence being gathered, encryption latency also becomes an issue. In a recent white paper from the Rochester Institute of Technology addressing latency testing, layer 2 encryption out-performed the layer 3 latency adding more advantages to targeting security at the layer 2 level.
The 802.1AE standard was designed to protect data in transit on a hop-by-hop basis (see Information Week article, New Protocols Secure Layer 2), ensuring that the frames are not altered between Layer 2 devices such as switches, routers and hosts. 802.1AE isn't a replacement for Layer 3 security but does ensure that frames are protected from eavesdropping and manipulation at Layer 2 between peers. All traffic passing between two switches is protected using the same security parameters.

There are companies recognizing the importance of layer 2 security with impressive orders starting to come in. Marvell shipped more than 1 million ports of 1GE and 10GE link processors powered by first-generation Marvell LinkCrypt technology. Designed to merge Media Access Control (MAC) layer security functions into the Ethernet physical layer, LinkCrypt plays a key role in the integration of standards-compliant security solutions to expand the security perimeter in enterprise, data center, metropolitan networks and 3G/4G cellular infrastructures.
Cities, counties and even atomic plants are working with companies like TLC-Chamonix adding end-to-end security to their networks. Their premier FIPS 140-2 validated software-based solution is being used to protect wireless networks at the Layer 2 level while also offering the only Mesh-certified software solution. With DOD installations in place, TLC-Chamonix is finding that DOE and Government FIPS 140-2 mandates are requiring higher security levels over many enterprise, local, county and critical network infrastructures. Rather than embedded chip sets, TLC-Chamonix offers a vendor-agnostic software solution across a variety of network equipment and platforms.
From today’s smart-grid breaches to tomorrow’s smart-grid needs, security requirements all seem to be pointing to layer 2 and micro-grids network topologies. Maybe today’s smart-grid security problems can open the door to a complete new set of cyber-security platforms. With billions released to build the smart grid, we should immediately focus the funds and expertise to securing the critical infrastructure of our nation's power grid. Let’s make Smart Grid security the ground zero of cyber security.

Larry Karisny is the director of Project Safety.org and a consultant supporting local wireless broadband, smart grid, transportation and security platforms.