QUICK! You are on duty in a secret control room in a nondescript, windowless building. The sign out front is so small that people driving by cannot read it, and it may give no clue what goes on inside, anyway. But your task is crucial: you are matching the ever-changing power needs of tens of millions of electricity customers with supply coming from hundreds of electricity generators, deciding which units will run and which ones will be idle, and making quick adjustments for the generators you can’t schedule, like the wind machines and solar panels.
Hardly anybody will ever know you are here, unless you mess up.
All is going smoothly until you get a message from a neighboring electrical entity requesting emergency assistance. A quick glance at your computer screen tells you that you have sufficient spare capacity to help.
Should you:
A. Call your contracts department and ask what price you will charge?
B. Go ahead and raise the generation in your area by the required amount?
C. Review the computer system used by all the generators to see what transmission is available?
D. Set up an emergency schedule with your neighbor?
OVERVIEW The California I.S.O. is one of 100 such facilities that balance supply and demand in the North American grid. Credit Max Whittaker for The New York Times
If you answered D, and you also gave the correct answers to several much more complicated questions, you are on your way to a job in an increasingly tough and essential field: managing the North American power grid.
“The bar is being raised,” said Lourdes Estrada-Salinero, the director of operations compliance and control at the California Independent System Operator, one of the more than 100 “balancing authorities” that are responsible for coordinating supply with demand in some portion of the North American grid. About 40 percent of all the energy used in the United States — all the oil, gas and coal, uranium, wind and falling water — is turned into electricity before it is consumed, and that fraction seems destined to rise, as more air-conditioners, electric cars and yet-to-be invented hand-held gizmos are added to customers’ inventory. But the tolerance for failure is getting lower, and the power mix is getting more complicated. States, led by California, are demanding that an increasing fraction of the electricity come from sources that can turn themselves on or off with very little warning, as the weather changes and the wind and sunlight vary.
Utility companies used to hire and train their own people to operate their systems. As the grid became more interconnected, some utilities organized themselves into power pools, with one control room handling the supply for multiple utilities. The earliest was the entity now known as PJM, which used to stand for Pennsylvania-Jersey-Maryland but now stretches into all or part of Delaware, Virginia, West Virginia, Ohio, Illinois and scattered parts of other states. New York had a power pool and so did the six New England states.
The job got exponentially more complicated when the federal government pressed the pools to convert into power markets, where the utilities would sell off their generating stations and third parties would be allowed to build generators. The hour-by-hour decisions about who would generate to serve what load were made mostly by an auction process and turned the pools into “independent system operators.”
The human operator’s job, though, was mostly still unregulated until August 2003, when a series of errors in a control room in Carmel, Ind., at the Midwest Independent System Operator, created the biggest blackout in history. (For future reference, when disabling vital computer systems to install upgrades, kindly do not neglect to warn the system operators and do not leave the systems shut off when you go to lunch.) The lights went out as far away as New York.
After that blackout, a utility that operates in the central United States hired Najmedin Meshkati, an engineering professor and “human factors” specialist at the University of Southern California, to review operations in some of its control centers. (Dr. Meshkati asked that the utility not be named because it did not make the report public.) The report stressed that operators needed “total systems comprehension” to understand what was happening on the grid. But sometimes the job is set up in a way that will overload the operator, he found. And the operations centers can have an “unspoken ‘macho’ culture” in which operators think that asking for help will jeopardize their job performance rating, he discovered.
To improve operations in the control centers and reduce the frequency of blackouts, Congress gave the Federal Energy Regulatory Commission the authority to enforce detailed new rules. But that government agency lacked the expertise to write them; for that, it turned to a voluntary organization, now called the North American Electric Reliability Corporation, which it designated as a standards-setting and enforcement agency. It is still setting rules, almost a decade after the blackout, and it is requiring licenses for people who hold various jobs in the control centers.
Holding such a certification is a key part of the résumé for a class of workers who are neither white-collar nor blue-collar. They might be called plastic collar, the people whose necks may or may not be girded in a necktie, but are sure to have a lanyard for an ID card that incorporates a computer chip that will get them into the windowless, label-less control rooms. They sit through an exam of approximately three hours, sometimes after sessions on online schools that have sprung up to help applicants cram. They end up with a certificate to frame on the wall.
There are about 6,000 such professionals, although some are management employees and do not work regular shifts. And some staff duplicate control rooms, located miles from the main control room, ready to take over if there is a fire or mechanical failure, or if the villains decipher the sign out front.
They get 200 hours of training every three years, and their continuing education resembles what the airlines give their pilots: extended sessions in full-scale simulators, with a computer playing the role of real hardware so the trainers can set up dire problems and see if the trainees can diagnose the situation and respond fast enough to prevent catastrophe.
The increased training did not come without difficulty. Daniel E. Frank, a Washington-based lawyer who specializes in utility issues, said, “It takes time to do it and nobody has a lot of time. The utility industry is no different from anybody else.”
“Any time you have to go in to training means somebody is away from the actual control room operations,” he said. Some centers maintain six teams to keep operations going around the clock; at any given time, one team is in training. On the job, the operators typically work four 12-hour days a week, alternating between days and nights.
And they are paid well. With overtime, they commonly earn six-figure salaries, and they work in rural, low-cost areas.
Recruiting is a challenge, though. Grid entities look for candidates with some background in engineering, but they also need certain personality traits, like the ability to work collaboratively but not to debate endlessly. People with military backgrounds are favored, because they often have appropriate organizational and technical skills.
One aspect that makes the job complicated is that on the grid these days, there is a market not just for electricity, but also for “ancillary services.” These include the ability to ramp up and down quickly, which will be required as the wind and sun vary in intensity; the ability to add or subtract very large amounts of power in tiny fractions of a second, to keep the alternating current system working as closely as possible to 60 alternations per second; the ability to step in to control voltage; the ability to stand by for hours or days at a time, poised to start up if something goes wrong; and the ability, if everything goes wrong, to begin generating with no outside power to help.
Operations are also governed by rules about how much air pollution a generating station is allowed to emit; how much it is permitted to raise the temperature of the lake or river it uses for cooling water; and how much power must be generated within a geographic area, regardless of capacity elsewhere, to ensure reliability. Some of that is built into computer programming, and some of it is drilled into the operators’ heads.
John T. McCain is a former captain in the Marines who recently completed his training at the California I.S.O. as a “real-time scheduler,” coordinating what generation will run from hour to hour. “It reminds me of training that I’ve had in the military,” he said. “It’s fast-paced, and there’s a lot to learn.” In some respects, it is more difficult than military training, he said.
His boss, Stephen Berberich, the president and chief executive of the I.S.O., said, “It’s an interesting mix between physics and policy and economics.”
