The Launch Control Center and Firing Rooms

Abstract

This chapter explores the LCC and how it was organized. We will touch on the innovative computer technologies and the intricate network of electrical support equipment that enabled several hundred engineers to control and test all operations of the Saturn V and its supporting facilities.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

All launch facilities at Cape Canaveral featured a launch pad located near a hardened blockhouse that contained a launch control room. As launch vehicles grew larger and became more sophisticated, and as their explosive potential increased, the Cape’s launch complexes incorporated larger blockhouses that were increasingly farther from the launch pad.

The blockhouse at LC-5/6 (site of Alan Shepard and Gus Grissom’s Mercury/Redstone launches) was only about 300 ft (91 m) from the center of the launch pads. A large window in the blockhouse faced the pad. In contrast, the huge blockhouse at LC-34 (site of the Saturn IB launches up through Apollo 7) was more 1,800 ft (550 m) from the center of the launch pad, and there were no windows. The explosive potential of a Saturn V mandated that the launch control center be at least 3 miles (4.8 km) from the launch pad.

To meet the aggressive launch schedule necessary to put men on the Moon by the end of the 1960s, NASA needed a novel approach to assembling, testing, and launching multiple rockets. Kurt Debus’ insight into addressing the issue was a Launch Control Center (LCC) with four separate firing rooms, located at a safe distance from the launch pad. Each firing room could be dedicated to the test and checkout of a single mission, regardless of whether the rocket was in the VAB or at the launch pad. This enabled NASA to have up to four Saturn V missions in flow at any given time.

This chapter explores the LCC and how it was organized. We will touch on the innovative computer technologies and the intricate network of electrical support equipment that enabled several hundred engineers to control and test all operations of the Saturn V and its supporting facilities.

The Launch Control Center

The LCC is the command center for the checkout and launch of rockets from Launch Complex 39. The LCC is adjacent to the VAB and connected to it by an enclosed walkway bridge (Fig. 6.1).

6.1

The Launch Control Center (foreground) and the VAB during the Apollo 11 rollout. Source: NASA

The first floor of the LCC contained an entry lobby and support functions including a cafeteria, the communications control room (CCR), and the complex control center (CCC). The CCR housed the equipment supporting the operational intercom system (OIS) and the operational TV system (OTV).

The CCC was in room 1P9 of the LCC, immediately to the left after one entered the front door of the building. The CCC had consoles similar to those in firing rooms, but they controlled all the ground systems (high-pressure gas distribution systems, air conditioning and pressurization, pad catacombs, pad terminal connection room, power distribution controls, LUT and VAB elevators, etc.).

Telemetry, tracking equipment, and instrumentation equipment occupied the second floor of the LCC during the Apollo era. IBM and the stage contractors operated 45 racks of telemetry checkout equipment in room 2P10.

The third floor of the LCC was devoted to the firing rooms, computer rooms, and conference rooms. The fourth floor contained offices and equipment for the large Eidophor projector displays in the firing rooms. The firing rooms were the heart of the LCC, and we will spend the bulk of this chapter discussing their layout and operation (Fig. 6.2).

6.2

Cross-section of the Launch Control Center illustrating how the firing rooms fit into the overall building plan. Source: Author’s adaptation of NASA diagram

Firing Rooms

All of the familiar Apollo launch film footage showing people sitting in a cavernous room in front of rows and rows of control panels was taken in the firing rooms. The firing rooms were active for much more than launches, however. They were the nerve centers for launch vehicle checkout activities at LC-39. They contained control, monitoring, and display equipment for automated testing and launch of the Saturn V. Personnel stationed in the firing rooms ran all of a stage’s tests from the time it was stacked on the LUT through launch.

Many people are under the impression that the firing room was the same room as the mission operations control room (MOCR) in Mission Control. This misperception has been repeatedly reinforced, even in otherwise excellent documentaries about Apollo. Nothing will make a KSC worker more upset than someone asking him or her about working in Mission Control!

Notwithstanding the fact that the firing rooms were in Florida and the MOCR was in Houston as of the early Gemini program, the two facilities were of vastly different size (each of the four firing rooms was about ten times the size of the MOCR), and they served very different purposes. The firing rooms controlled all the testing, preparation, fueling, and launch of the vehicle, up to the moment of liftoff. The moment the umbilicals pulled away from the vehicle, the firing room’s role in a mission was essentially complete, except for the range safety function and safing pad equipment. The firing room had no control over the vehicle once the umbilicals disconnected. Within 10 s of liftoff, the vehicle had cleared the launch tower; Mission Control in Houston took complete control from that point forward in the mission (Figs. 6.3 and 6.4).

6.3

Ike Rigell (foreground) monitors the final half-hour of the Apollo 16 countdown in firing room 1, April 16, 1972. Source: NASA/Jerome Bascom-Pipp

6.4

The mission operations control room in Houston’s Mission Control Center during an Apollo 16 EVA. Do not confuse this room with the KSC firing rooms! Source: NASA/J. L. Pickering

The firing rooms were analogous to the control rooms in the blockhouses at LC-34 or LC-37, built on a much larger scale. There were about 450 consoles in an Apollo-era firing room at the Launch Control Center. Astronaut Dave Scott told the author: “Comparing it with Gemini, the difference was astounding. It never ceases to amaze me. When I was on Gemini VIII, we spent a lot of time at the Cape. One weekend, Neil [Armstrong] says to me, ‘Hey, they’re building the Apollo launch center and the firing room—let’s go have a look.’ The Gemini launch control center had about 20–24 consoles, and that was pretty big compared to Mercury. So we drive over and they’re building the VAB, and we go up into the VIP room and look down at the firing room. 450 consoles. We said, ‘We’d better go back! It ain’t gonna work! No WAY it’s gonna work!’”

The four firing rooms in the LCC corresponded to the four high bays in the VAB. When the complex was designed, Debus’ idea was that each firing room could oversee the checkout of a Saturn V in one of the VAB’s high bays, with up to four in process at once. Strictly speaking, each firing room was connected to a single LUT during processing for a mission, and that LUT could either be in the VAB or at the launch pad. The key concept was that each firing room would be dedicated to a single Saturn V throughout that vehicle’s processing flow at KSC, through stacking, checkout, and launch. After a mission launched, the LUT returned to the VAB for refurbishment, and it and firing room were then assigned to another mission whose processing flow was about to begin.

After LC-39 was designed and while construction was underway, NASA determined that only three launch vehicles would be in flow at a time. Therefore, only three LUTs were built, and only three high bays in the VAB were activated for launch processing. Firing room 4 was re-purposed to serve as a management information and control room where the status of LC-39 activation and mission processing flows were tracked and managed.

Firing room 1 was the first to be activated. It controlled the test of the AS-500F facility vehicle and the checkout and launch of the AS-501 (Apollo 4) mission. Firing room 2 was activated for the next Saturn V launch, AS-502 (Apollo 6). Firing room 3 came online to process and launch Apollo 10, but was not used again as a primary firing room until the processing of the Saturn IBs for Skylab and ASTP (Fig. 6.5).

6.5

Four firing rooms filled the third floor of the LCC. Firing room 1 was most frequently used one during the Apollo program. Source: NASA/Ward

Each firing room looked out to the LC-39 launch pads to the northeast through a 2-in. (5 cm) thick, laminated and tinted window that ran the entire width of the firing room. Electrically-controlled louvers could be closed over the window as a sunshade and for protection in case of an explosion on the launch pad. With its wide windows, the firing room also provided a convenient, climate-controlled place for VIPs to watch a Saturn V roll out to the launch pad.

One would think that a room filled with 450 electrical consoles, many of them with dozens of 28 V indicator lamps, would be very warm from the heat generated by all the electrical equipment. Engineer Bob Pound considered the opposite to be true; it was at times uncomfortably chilly in the firing room: “Keeping the area at the right temperature actually worked by adding heat into the cooled air, rather than cooling it down more. Of course, those computers needed a lot of cooling; they ran awful hot. To save energy, we’d let it run as cold as possible, because the chilled water came from the cooling towers out there by the VAB. They pumped as much cold as they could, and then to get it to the right temperature, they’d warm it up with heat from the consoles. We had to wear sweaters in there most of the time.”

Looking out from the fourth floor over the firing rooms were glassed-in rooms that were originally intended as VIP observation areas. Instead, they became offices for the NASA test supervisors. Lead test supervisor Don Phillips occupied the office overlooking firing room 4; the other test supervisors shared the offices overlooking firing rooms 1 and 2 (Fig. 6.6).

6.6

Original plan for firing room 1 layout, May 21, 1965. Some positions in area A were reassigned prior to the start of Saturn V flights, but the overall layout remained consistent throughout Apollo. Detailed console assignment information is shown in Appendix E. Source: NASA/Ward

Firing rooms 1, 2, and 3 had essentially identical layouts. The firing rooms were divided into separate areas that had specific operational significance. Each area was comprised of racks of consoles and displays. Two letters and a number designated a console’s location in the firing room. The first letter was the area, the second letter was the row, and the number was the position along the row, numbered from left to right. For example, the S-II engine 201 console was at position BA16: area B, row A, 16th console from the left.

Personnel on console in the firing room were identified by their operational intercom system (OIS) call sign. For example, the call sign for the test engineer manning the S-II engine console was C2EC.

The layout of the firing room was a physical manifestation of the management hierarchy. The following sections describe the function of each area and the roles of the personnel who occupied consoles.

Area A

Area A was populated by the managers of the test and launch processes. Four rows of consoles stair-stepped down from the window. The higher the row in which a person sat in area A, the more comprehensive his or her job function and responsibility. Management personnel could look out over the entire firing room and monitor the activity.

Row AA (the uppermost row) was reserved for KSC and program management. Seated in this row were Ike Rigell, Hans Gruene, Kurt Debus, Rocco Petrone, Walt Kapryan, Robert Gray, John Williams, the public affairs officer, the Saturn program manager from MSFC, and the Apollo program manager from MSC.

Row AB was for the personnel who managed test operations. These included the space vehicle test supervisor, launch vehicle test conductor, and spacecraft managers from NASA, Grumman, and Rockwell. It is important to remember that the chief NASA spacecraft test conductors were stationed in the ACE rooms in the MSOB—not the firing room—during space vehicle tests and launches. Spacecraft operations had representatives in the firing room, but control of spacecraft testing resided solely in the MSOB ACE rooms.

Row AC was for the test conductors and systems engineers for the various launch vehicle stages, spacecraft operations managers, the astronaut representative and communicator (OIS call sign Stoney), and KSC chief of medical services.

Row AD seated NASA managers for networks, engineering, guidance and control, mechanical and propulsion systems, instrumentation, and quality assurance; operations managers from the stage contractors and Bendix; instrumentation and communications controllers; and range safety, pad safety, and security.

The management personnel in area A monitored and directed tests and launches. OTV feeds and summary-level event displays on their consoles provided information, and they could monitor multiple channels on the operational intercom system during a test.

Off to the sides of area A were two triangular, glass-fronted rooms. With one’s back to the main window in firing room 1, the room on the left was an observation room for VIPs and guests attending a launch. On the right was the operations management room (OMR, also sometimes referred to as the management support room or mission support room). Executives from NASA headquarters, other NASA centers, and the military observed and monitored launch activities here. George Mueller, Sam Phillips, and Wernher von Braun were among those who sat in the OMR during a launch.

Each contractor stage test conductor in area A monitored a stage operations panel with indicator lights showing the status of major systems on his stage. These displayed information similar to that on the large status board on the firing room wall, supplemented with additional information about range safety status on the stage (Fig. 6.7).

6.7

S-IB operations panel from area A of the firing room. Chrysler’s S-IB test conductor monitored the status of key discrete events on his stage from this panel. Note that there are no switches or controls (Author’s collection). Source: Ward

The VIPs and top brass in area A tracked the events of a test or launch via four large Eidophor projection screens mounted near the ceiling in the center of the firing room. The data on the screens was generated by the computer and the operational television (OTV) system (Fig. 6.8).

6.8

Lead space vehicle test supervisor Don Phillips (center) talks with KSC security officer Sherman Evans during the CDDT for Apollo 12. Behind Phillips is Fleming Law of spacecraft operations. The large Eidophor projection screens above the firing room show operational television (leftmost display), data from discrete events, and a listing of programs being executed on the RCA 110A ground support computer. Source: NASA/Jerome Bascom-Pipp

Two large status displays were mounted diagonally on the walls on either side of the Eidophor displays. The one on the left side displayed the status of key events in a test or countdown. Frank Bryan was responsible for implementing the key events display board, about which he said: “When we were designing the board, I picked a color for each light. My scheme was: green means you’re go; yellow means something’s going to happen and the light’s going to go off before you get to launch; and red means it shouldn’t be on—you’ve got a problem. We ran through the first test, when all management was up in the top row, and Ike [Rigell] came up to me and said, ‘We’re going to change that board. Every light up there is going to be green.’ Management didn’t like seeing yellow lights flashing.”

The status board indicator lights for the Apollo 11 mission are shown in the table below.

AS-506	LAUNCH SEQUENCE START	S-IC INTERTANK ARM RETRACTED	S-IC IGNITION	S-IVB ENGINE START
	S-IVB LOX TANK PRESSURIZED	S-IC ON INTERNAL POWER	COMMIT	S-IVB CUTOFF
RANGE SAFE	S-II LOX TANK PRESSURIZED	S-II ON INTERNAL POWER	LIFTOFF
LAUNCH SUPPORT PREPS COMPLETE	S-IVB LH2 TANK PRESSURIZED	S-IVB ON INTERNAL POWER		ESE CUTOFF
S-II PREPS COMPLETE	S-IVB PROPELLANTS PRESSURIZED	IU ON INTERNAL POWER	S-IC INBOARD ENGINE CUTOFF
S-IVB PREPS COMPLETE	S-IC FUEL TANK PRESSURIZED	S-IC PROPELLANTS PRESSURIZED	S-IC OUTBOARD ENGINE CUTOFF	R F SILENCE
IU READY	S-IC LOX TANK PRESSURIZED	S-IC CARRIER K/O AND ARM RETRACT	S-IC/S-II SEP LOGIC ZERO
S/C READY	S-II LH2 TANK PRESSURIZED	S-IC FORWARD ARM RETRACTED	S-II ENGINE START	TEST HOLDING
EDS READY	S-II PROPELLANTS PRESSURIZED	LSE READY FOR IGNITION	S-II CUTOFF	TEST COUNTING
S-IC PREPS COMPLETE	S-IC INTERTANK CARRIER RETRACTED	READY FOR S-IC IGNITION	S-II/S-IVB SEP LOGIC ZERO	EVENT SYSTEM CALIBRATING

The status display on the right side of the firing room showed which NASA ground stations in the spaceflight tracking data network (STDN) were receiving downlinked data. The tracking ships and stations on duty varied somewhat between missions. Shown below is the configuration of the board during the Apollo 16 launch:

LOCAL DATA AVAILABLE	STDN DATA AVAILABLE		STDN STATION SUPPLYING DATA
LOCAL DATA NOT AVAILABLE	STDN DATA NOT AVAILABLE
SCREEN 1	SCREEN 1	MILA	MADRID	GUAM
SCREEN 2	SCREEN 2	BERMUDA	ASCENSION	HAWAII
SCREEN 3	SCREEN 3	USNS VANGUARD	CARNARVON	GOLDSTONE
SCREEN 4	SCREEN 4	CANARY	HONEYSUCKLE	TEXAS
	S-IVB LIVE DATA
	S-IVB PLAYBACK DATA
	IU LIVE DATA
	IU PLAYBACK DATA			EVENT SYSTEM CALIBRATING

Area B

Area B covered the first five rows on the main floor level of the firing room. Here were the consoles from which engineers for NASA and the stage contractors directly controlled and monitored launch vehicle tests. The consoles provided inputs into and readouts from the ground computer, launch vehicle stages, ground support equipment, and electrical support equipment.

There were 30 consoles in each of the 5 rows of area B, 15 consoles on each side of the central aisle dividing each row in the area. The section for each stage (S-IC, S-II, S-IVB, and IU) had a printer for the digital events evaluator (DEE-6C) computer, and small OTV monitors sat on top of some of the consoles. Each side of every row had at least one computer terminal that could execute programs on the RCA 110A ground computer complex and call up formatted alphanumeric data on the status of certain systems. Consoles were grouped by stage or by function.

Many of the rows in area B had at least one large events display panel with dozens of indicator lights. These panels provided engineers with an overview of all the discrete commands issued to and the responses received from a stage and its ground support equipment during a test (Fig. 6.9).

6.9

An events display panel from the S-IC section of area B, row A. The indicators on this panel showed the status of more than 200 discrete events associated with the first stage of the Saturn V and its launch support equipment. Source: NASA/Ward

Appendix E contains a detailed listing of the consoles in area B. The panels in each row were generally arranged and staffed by function as follows:

• Row BA, left side of the aisle (positions 1–15): S-IC propulsion and electrical networks. Manned by Boeing personnel. (Note: NASA engineers sat beside or behind the contractor engineers manning consoles throughout area B.)

• Row BA, right side of the aisle (positions 16–30): S-II propulsion and electrical networks. Manned by Rockwell.

• Row BB, left side: Instrument unit networks, mechanical, and emergency detection system. Manned by IBM.

• Row BB, right side: S-IVB propulsion and electrical networks. Manned by McDonnell Douglas.

• Row BC, left side: Stabilization and guidance monitoring (theodolite, ST-124M inertial platform, and guidance computer). Manned by IBM.

• Row BC, right side: Service arms mechanical ground support equipment. Manned by Boeing.

• Row BD, left side: Flight control and IU measuring (engine gimbaling for each stage). Manned by IBM and stage contractors.

• Row BD, right side: Mechanical ground support equipment and launch accessories (tail service masts, holddown arms, environmental control system). Manned by Boeing.

• Row BE, left side: Stage RF measuring and tracking, Q-angle of attack. Manned by IBM.

• Row BE, right side: Propellant loading and propellant tanking computer system monitoring. Manned by Boeing.

After the success of the Apollo 11 mission, firing room 3 was no longer needed to support lunar missions (it was the primary firing room only for processing Apollo 10). Firing room 3 was reconfigured to support launches of the Saturn IB for the Skylab manned missions and the Apollo-Soyuz Test Project. In the new configuration of firing room 3, the S-IC consoles on the left side of row BA were left in place and were not used. The S-II consoles on the right side of row BA were removed, since there was no S-II stage on the Saturn IB. Some S-II panels were replaced by S-IB panels, which were manned by Chrysler engineers.

Area C

Area C was at the far end of the firing room from the window. There were eight rows of equipment racks, with two aisles dividing the area. The middle section of row CA was a large air conditioning equipment cabinet, which visually set off area C from the rest of the firing room. Strip chart recorders on the left and right sides of row CA faced toward areas A and B. Photos of the firing rooms during Apollo launches frequently show mission managers in row AA with binoculars in their hand or on their consoles. Ike Rigell said that launch managers used their binoculars to scan the tracings on the area C strip chart recorders from the other end of the firing room.

The consoles in area C controlled, measured, and recorded the status of various systems on the vehicle and at the launch pad as well as some of the support facilities such as the LOX and LH₂ farms. The last two rows of area C contained patch racks and patch distributors, the timing distributor from the countdown clock, signal conditioning equipment, and monitoring consoles for the digital data acquisition system (DDAS). The DDAS conveyed vehicle stage data and ESE data to the firing room and CIF for real-time monitoring on strip charts, indicators, and meters. DDAS data was also recorded to tape drives (Fig. 6.10).

6.10

Boeing electrical engineer Bill Heink at the LOX electrical systems control station in firing room 1, area C, during the Apollo 8 countdown demonstration test. Source: Frame from NASA movie adapted by author

Areas D, E, and F

Area D was in a separate room off the back end of the firing room. This room contained the firing room’s RCA 110A computer and its associated peripheral equipment (tape drives, drum storage, card punch and card reader, and line printer). Each firing room had its own dedicated RCA 110A computer. The Sanders display computer system was also located in area D.

Area E was separate room off to the left side of the firing room. It held power distribution and monitoring racks. Area F was a terminal equipment room at one back corner of the firing room. Each adjoining pair of firing rooms shared a terminal room.

The Woodshed

A launch vehicle documentation room was off to the side of each firing room. This room contained copies of all procedures, systems drawings, and other materials related to the launch vehicle and the ground support systems. It was affectionately known as the woodshed, where Rocco Petrone caucused his launch vehicle managers if problems arose during tests or launch countdowns.

Operational Intercom System (OIS)

The operational intercom system made it possible for test conductors, engineers, technicians, and managers to stay in contact with each other during a test, no matter where they were at KSC, or even at other centers and contractor facilities.

Each major space vehicle and ground support system was assigned a primary OIS channel for communications during a test. The channel assignments were published in a matrix with each test procedure (Fig. 6.11).

6.11

Matrix of active OIS channels during tests and launch countdown for Apollo 17. Source: NASA/Ward

Listening in on a loop was as simple as dialing in the number of the channel on an OIS communications panel and plugging in a headset. Most OIS stations located around KSC could be dialed into two channels, one that enabled the person at the station to talk on the loop, and a second channel that could be monitored (listen-only).

The OIS system was designed so that each test engineer could coordinate and talk with necessary people to perform his tasks while he listened in his other ear to his test conductor, who was on a separate intercom channel. The stage test conductor maintained contact with all of his stage test engineers, listening to them in one ear on one channel, while listening to the NASA test conductor in his other ear. This way, the entire population of test engineers in the LCC was closely linked in a very hierarchical manner to the NASA launch vehicle test conductor.

Shown in the accompanying photo is a typical OIS panel, one from area B of the firing room. The engineer sitting at the station was responsible for monitoring the pneumatic system that purged the fuel line for the auxiliary propulsion system on the S-IVB. We can see by the numbers on his OIS panel that for this test, he was active on OIS channel 125 (launch vehicle troubleshooting) as his primary loop, meaning that he was most likely involved in diagnosing and resolving issues on the Saturn V that had cropped up during the test. He was also monitoring channel 161 (S-IVB test conductor), listening to any procedural call-outs in the test that were related to the S-IVB. Note that the four jacks on the panel enabled as many as four people to plug in their headsets at this station (Fig. 6.12).

6.12

Typical OIS panel in area B of the firing room. Source: NASA/Ward

Many of the manager and test conductor consoles in area A had large OIS communications control panels that could side-tone up to 18 channels at a time, listening in on the discussions among various teams during the test. Roy Tharpe said he was usually tuned into at least six different conversations at once, and he turned the volume for a channel up or down depending on the importance and relevance of what was being discussed on it at any given time. Lights on the console indicated which OIS channels were active (Fig. 6.13).

6.13

OIS monitoring panel from area A of the firing room. Management personnel could monitor many channels at once. Source: NASA/Ward

Each person with functional responsibility during a test was assigned a call station (call sign) that uniquely identified him or her on the net. Every test procedure included a list of the active call stations for that test. The test procedures included a call and response listing for actions during the test, specifying the call signs of the person responsible for initiating an action and the person who would respond that the action had been accomplished and verified. Many people kept a cheat sheet taped to their consoles, listing the call signs most relevant to their role in the test.

Call sign designations were relatively straightforward. The first letter of the call sign generally identified the location of the person (C was the launch control center, L was the launcher platform, P was the launch pad, U was the umbilical tower, etc.). The letter or digit in the second position sometimes referred to the vehicle stage (1 for S-IC, 2 for S-II, 4 for S-IVB, U for instrument unit), or the major function or system (L for launch vehicle operations, S for swing arms, P for propellants, etc.). The last two letters were usually abbreviations of the station’s function (TC for test conductor, PU for propellant utilization, etc.). So, without the cheat sheet at hand, if one heard a callout for “C2PU” over the net, one could guess that this person was in the firing room and monitored the propellant utilization system on the S-II stage.

Conversation on the command channel of the net was strictly limited to what was called for in the test procedure. Launch vehicle operations instilled a level of discipline during test communications that forbade idle chatter. Tip Talone described the protocol for speaking over the OIS:

In any major test, the rule was – and it was a hard rule – you had “pro” words that you used when you came on the net to talk about something that wasn’t in the procedure. You had to come on and sequentially say who you were by call sign, not by name. Names were forbidden. If you were the launch vehicle test conductor, you were CLTC. If you were the first stage engineer, you were the C1PE. You could recognize voices, obviously, but you used the call signs. But it was very disciplined.

If you had a problem, your report had to be: “This is such-and-such, CLTC, calling on channel such-and-such.” You had to be sure he was really on the command channel and not on his own channel, because we had a lot of channels. Then you’d say, “I’ve detected an increased pressurization in a regulator that seems to be creeping. Our plan is to do this and that, and we will report back to you in the next five minutes. Can we have channel 181 to do our troubleshooting on?” And Norm [Carlson, CLTC] would say, “Yep, you can have channel 181,” and tell everybody on the command net and tell them who you wanted to help with troubleshooting. He’d say, “Call signs,” and he’d rattle off the call signs the guy needed, “go to 181 and support troubleshooting.” And of course he’d tell me, “You go to 181 and listen to what they’re doing.”

The talk on the OIS was very stylized, so that there weren’t any questions about what was really going on. There isn’t any panic, and no one guy’s working a problem that nobody else knew about. There were no wasted words, and we didn’t want to jam up the command net in case somebody had a real problem.

Houston, Huntsville, and the other NASA centers could be patched into the OIS network if the test called for their participation. Generally, these remote locations were only able to listen to the test progress; they did not have an active role during tests. They could talk on the problem resolution channels only if their advice was specifically requested. Skip Chauvin said, “They were strictly in a passive mode. They had all the data being shipped back to them, so they were essentially seeing the same things we were. At time we would maybe see a problem at KSC and somebody then says, ‘Let’s find out what Houston is seeing,’ just as backup data. They were there, hand in glove.”

Contractors were also listening in on tests and could pull in resources from the manufacturing facilities if needed. Rockwell’s Fred Cordia said:

There was a backup firing room, the next one over, and they would listen in on certain comm nets and vehicle nets to ensure success. If your primary guy got in a tight spot, a support guy was just a flip of the switch away to talk to, to ensure that this was the right thing to do. They were on the net, but they were not to speak unless spoken to. We also had the design people on the West Coast plumbed into the network in a similar type of arrangement. They had a mission room in at the Rockwell facility at Seal Beach, and the key design people and so forth were there. They were listening only, but if you wanted to talk to one of them, you could.

NASA had their own net back to Mission Control at Houston, they were piped in, and of course all the boosters came through MSFC, so NASA had a design crew back there who were plumbed in, listening in. We were all in synch. There shouldn’t have been any surprises to anybody, anywhere, because as it was happening, everybody heard it on the net.

Direct-dial telephone handsets were located near many of the consoles in the firing room, as well as at the pad, VAB, and other sites. To avoid having problems being broadly exposed before people were ready to talk about them with the broader test team, engineers sometimes made calls on their phones directly to a person rather than discussing the problem over the OIS. Dave Moja said:

If the stage people had a problem and they put it on the net, we’d hear about it, and people like Frank [Bryan] and I would hover around and try to “help” them. I say “help” with quotation marks, because sometimes we what we did wasn’t helpful. So they would do what they called “black phone.” They had phone connections out at the pad and in the launcher and wherever their equipment was, and when they were supposed to talk about problems on the intercom, sometimes they’d do a black phone thing, we called it. Actually it was to their benefit. Whenever I talked to them, I said, “Fine. If it isn’t an integrated problem, I don’t need to get involved in it,” and I tried not to.

Rocco Petrone disliked the black phone and discouraged people from using it. He wanted all problem-solving conversations to be in the open on the OIS. Frank Bryan recalled that, “Rocco insisted that you have your headset on. We’d have a problem, and I’d have a bunch of guys gathered. My phone had a long cord, and I could stretch it way over from my console to the other guys in my row. That was like a red flag to Petrone. He’d say, ‘Get down there and get your headsets on. Let’s talk about it on the loop so I can hear what’s going on.’”

Finally, the OIS system was tied into a public address system around LC-39. Norm Carlson said, “There was a button on the test conductor console I could push, and just go ahead and talk in my headset like I was normally talking, but it would go on the OIS as well as the public address system. You could go just local in the firing room, or broadcast to all areas.”

Operational Television

The OTV system allowed firing room staff to monitor hazardous or inaccessible operations throughout LC-39 from their consoles. OTV feeds could be viewed on up to 60 monitors at one time in the firing room. The feeds to each screen were controlled in the communications control room.

OTV cameras were placed in the VAB high bay work platforms 1 and 2 (5 cameras on each platform), the LUT (27 cameras on various levels), the mobile service structure (12 cameras), and pads A and B (12 cameras each). The LUT, MSS, and pad cameras could be pointed by remote control from the communications control room (Fig. 6.14).

6.14

Operational television switching room in the LCC complex control center room. Source: NASA/Jerome Bascom-Pipp

The LCC Ran the Show

The ability to operate much of the launch pad equipment remotely during tests and countdown was a key innovation in the design of LC-39. The firing rooms controlled a vast network of relays, circuits, and electrical support equipment in the launch vehicle, LUT, and the launch pad. These systems distributed and executed commands from the RCA 110A computer complex to operate the ground support equipment and the launch vehicle prior to liftoff.

The LC-39 control system operated as two complexes of subsystems: the computer-DDAS-hardwire-terminal countdown sequencer complex, and the transfer-logic-distribution complex.

The Computer-DDAS-Hardwire-TCS Complex

Each control panel in the firing room had a dedicated function. Every panel had a designation, which was stenciled on the panel face, and every indicator light or switch was tied to a single discrete event. There was no automation in the panel, and no flexibility or possibility of multiple uses for a panel other than what was wired into the panel. Each panel was hardwired to the distribution racks in the back room. Any required configuration or functional changes after a panel was certified were made via jumper wires in patch panels in the back room. This was a practice carried over from the telephone industry (Figs. 6.15, 6.16, 6.17, and 6.18).

6.15

S-IB networks control panel, used in Skylab 2, 3, 4 and ASTP tests and launches (Author’s collection). Source: Ward

6.16

Back side of the S-IB networks panel in Fig. 6.15, showing the intricate wiring. Configuration changes were effected by installing jumpers in patch panels in the distributor racks in the terminal room rather than rewiring the control panel (Author’s collection). Source: Ward

6.17

Typical patch distributor panel. Plugged into the back of each patch distributor were as many as 60 cables, each carrying 61 connections. The panels routed signals between the firing room, the RCA 110A computers, and the launch support equipment in the LUT and at the pad. Test points on the outside of the panel enabled engineers to test connections without opening the panel. Source: NASA/Frank Bryan

6.18

Patchboard of the type found inside patch distributor panels. Engineers reconfigured circuitry by moving the patch connectors rather than rewiring cables or equipment. Source: NASA/Ward

The Huntsville-designed firing room panels had two power sources that were looped to all of the switches and indicators. If a wire broke, the panel would still have a source of power. MSFC mandated this redundancy as a safety ground rule in control panel design.

Many of the test engineer control panels in area B sported analog meters and indicator lights to show the status of a particular system on the vehicle or the ground support equipment. Switches on panels commanded various test functions. Most of the switches had three positions: ON, OFF, and AUTO. AUTO was the preferred position, as it allowed the RCA 110A ground control computer to execute a test program or function automatically. Moving a switch to the ON or OFF positions instructed the computer to energize or de-energize a circuit, immediately overriding the automated test program.

Switches on a few control panels were hard-wired to critical ground support equipment. These switches commanded functions that put the vehicle and GSE in a safe condition if the computer malfunctioned, or if the cutoff command was issued in the event of a launch failure. For example, the S-IB firing and S-IC firing consoles had an emergency cutoff button that would immediately halt the automated terminal countdown sequence from T minus 3 min 10 s until launch commit at T-0. This red pushbutton can be seen in Fig. 2.15.

Fourteen computer control consoles were located throughout area B. A switch on a computer control console could initiate the operation of a single component or could start the execution of an entire computer program. The terminal operator first inserted a card key into the console to prevent an improper request to execute a program. Once a test routine was called up, the signal from the terminal went to the patch distributor in the firing room. The signal from the patch distributor was routed through signal conditioning equipment and then into the firing room’s RCA 110A computer. The firing room’s 110A communicated via hardline link to the 110A computer located in the base of the LUT (which was either in the VAB or at the launch pad). The LUT’s computer routed its output through signal conditioning equipment in the LUT and then on to a relay rack for the appropriate stage of the launch vehicle or associated ground equipment. The signal then went to the terminal distribution equipment and a crossover distributor to communicate with sensors on the launch vehicle. The response was sent back to the 110A in the launcher and from there back to the 110A in the firing room, which then routed the information to the appropriate console for display.

All of the firing room console switches were read as discrete inputs by the firing room computer and transmitted over the data link to the LUT computer. The LUT computer would then send a discrete output to the launch vehicle or ground support system that corresponded to the console switch. The response from the vehicle or launch pad system was then sent back to the LUT computer, which in turn relayed it to the firing room computer, which lit an indicator lamp on the engineer’s console. When a switch was thrown, the computer would immediately put that function in the condition the engineer wanted. The software being executed would suspend itself and stop sending any further commands. The engineer therefore had ultimate control of his system (Figs. 6.19, 6.20, and 6.21).

6.19

Sanders Associates display console. Area B of the firing room had 14 of these consoles, and there was one in the computer room in area D. These displayed computer program information and test data. Source: NASA/Ward

6.20

Keyboard used by test engineers in the firing rooms to execute test programs, request channel changes on the OTV system, or request printouts of data on the Sanders display screens (Author’s photo). Source: Ward

6.21

Simplified diagram of the key elements of the Saturn ground computer complex system. Source: NASA/Ward

Terminal Countdown Sequencer

The terminal countdown sequencer (TCS) was a solid-state device housed in the base of the mobile launcher. The TCS provided precisely timed outputs during the final countdown sequence to initiate actions in the electrical support equipment, which then commanded actions in the vehicle and ground support equipment. These included such functions as closing vents to pressurize the propellant tanks, switching from external to internal power, and sending the time-staggered ignition commands to each of the engines on the S-IC. These events happened so rapidly, and in such a critically timed sequence, that humans could not control the process manually.

Frank Bryan said: “The TCS was basically a clock that gave outputs. What the outputs did was controlled in the electrical support system relay logic. If you wanted something to happen at T minus 40 s, you’d take that timed output from the TCS and patch it into the patch panel, and then it would pull in a relay, and then you could do any number of things with that relay logic.”

Relay logic acted like an extension of the TCS for controlling the ground support equipment and the launch vehicle. Rather than polling the state of each of the thousands of circuits in the system, the actions of ESE during terminal countdown was dependent on the presence or absence a few dozen interlocks.

Some interlocks (such as thrust failure) could halt the countdown immediately. Summation interlocks were circuits that were energized only if a specified combination of other circuits was also energized. For any given interlock to be turned on, all of the prerequisite input circuits also had to be on. As an example, the LV ready for launch interlock was only energized if all of the individual stages had their ready for launch interlocks energized. One unready stage would prevent the countdown from proceeding past a decision point that required the presence of the LV ready for launch interlock.

Once TCS started the terminal countdown, it could only be stopped by a cutoff signal, either given manually or generated automatically by a missing interlock. In emergency situations, an override could allow the RCA 110A to provide discrete outputs to the electrical support equipment, but this capability was only used one time in actual Saturn V launch countdowns. Otherwise, the Saturn ground computer complex had no part in the terminal phase of the countdown.

The Transfer-Logic-Distribution Complex

The transfer-logic-distribution complex took commands from the LCC consoles, computer, and test equipment in the LUT, and then activated systems on the LUT, pad, and launch vehicle. The logic in this immense distribution network was implemented entirely in relays and hardwire jumpers in distribution and relay racks in the firing room and the LUT—there was no computerization in the distribution system at all. A simplified version of just one portion of this vast electrical network is shown in Fig. 6.22.

6.22

A portion of the electrical support equipment logic distribution system at LC-39. The RCA 110A computer system routed commands from the firing room to the LUT, where the distribution system then sent the appropriate signals to launch support equipment. Source: NASA/Ward

The System in Operation

This section provides a sample of the range and scope of control of just two of the hundreds of panels in the firing room. One panel we will look at controlled functions on a stage and its supporting equipment, and the other controlled some of the ground support equipment in the LUT that served the entire launch vehicle.

Our first example is the S-IVB APS launch and monitor panel. During the ASTP test and countdown, a McDonnell Douglas engineer, call sign C4AL, ran this control panel from his station in firing room 3, area B, row B, console 16. This panel controlled the vents and valves and monitored the pressures of the hypergolic propellant systems for the APS engines (referred to as APS modules) on the S-IVB. The two APS modules helped steer the S-IVB in flight and also performed ullage burns to settle the propellants in the S-IVB’s main tanks before engine restart at trans-lunar injection. Helium bottles on the stage pressurized the APS propellant tanks (Fig. 6.23).

6.23

The S-IVB APS launch and monitor control panel from area B, row B of the firing room. The McDonnell Douglas engineer at this panel tested and monitored the propellant system on the S-IVB’s auxiliary engines. This particular panel was last used in firing room 3 for the ASTP test and launch (Author’s collection). Source: Ward

Let’s look at the circuits controlled by four of the many switches on this panel. Just to the right of center in the upper half of the panel are 2 meters displaying pressures in two segments in module II’s oxidizer system. Beneath these meters are two switches that controlled valves in the system and two switches that opened vents. The simplified circuit diagram shows the action of these four switches. Any test engineer would have to know many of these circuit diagrams intimately before being certified to operate a control panel (Figs. 6.24 and 6.25).

6.24

An enlargement of a section of the S-IVB APS launch and monitor panel, showing the controls for the oxidizer system on APS engine (module) II. Source: Ward

The leftmost part of the diagram shows the circuits controlled by the oxidizer fill valve switch. If the switch is placed in the CLOSE position, the RCA 110A computer is commanded to energize circuit D12321 in the ESE, which closes the valve. In the OPEN position, the computer energizes circuit D12320 and opens the valve. With the switch left in the AUTO position, the computer has discretion to command the ESE, via relays K800-2 and K952, to operate S-IVB oxidizer tank II’s fill and drain valve as commanded by the computerized test program.

At the right of the circuit diagram is the oxidizer ullage emergency vent valve switch. As shown in the diagram, this switch has a hardwire connection that bypasses the RCA 110A computers in the firing room and the LUT. Throwing the switch directly energizes relay K960 in the ESE to open the emergency vent valve on the S-IVB. This action would only be taken if there were a loss of computer control over the tank pressure on the S-IVB prior to launch. As a safeguard, these hardwire switches have to be pulled back to unlock them before they can be flipped into another position (Figs. 6.24 and 6.25).

6.25

A simplified circuit diagram showing how the switches shown in Fig. 6.24 panel commanded the S-IVB stage and the computer. The circuit at right shows a hardwire connection, which bypasses the computer when necessary to take emergency control over the stage. Source: NASA/Ward

In our second example, we will look at the pneumatic distribution system console, which was located at position BC28 (area B, row C, position 28) in the firing room. The Boeing test engineer who manned this panel had the OIS call sign CPDC, pneumatic distribution complex control console. His control panel was divided into sections that monitored the gaseous helium distribution system, the gaseous nitrogen distribution system, the Q-ball removal system, the nitrogen-fed service module deluge purge system, and valve panels 11 and 12 on the LUT.

Analog meters, fed by the digital data acquisition system and converted to analog information for display, gave the CPDC test engineer real-time information on gas pressures in the various pad systems. CLOSED-AUTO-OPEN switches would normally be left in the AUTO position to allow the RCA 110A computer to run its test programs. However, he could override the program and manually command gas inlets and outlets to open or close if necessary. Indicator lights displayed the status of approximately 50 discretes associated with the systems (e.g., 6,000 psi GN₂ supply valve open, balance valve closed, etc.). Although he could not manually command any actions on valve panels 11 and 12, which were located in the high-pressure pneumatic center compartment 1B in the mobile launcher, he could monitor the overall status of the systems fed by the panels. He could also issue instructions over the OIS to other engineers who had direct control over the valve panels. The diagram illustrating the pneumatics control and distribution system shows the extent of the ESE and GSE operated and monitored by a single test engineer at a single control panel in the firing room (Figs. 6.26 and 6.27).

6.26

The pneumatic distribution system control panel from area B, row C, of firing room 1. Source: NASA/Ward

6.27

Schematic diagram of some of the LUT and pad systems controlled by the pneumatic distribution system panel shown in Fig. 6.26. Source: NASA/Ward

These were just two of about 100 test control panels in area B, each of which was dedicated to controlling, testing, and monitoring its own specific piece of the ESE, GSE, or launch vehicle systems. Using these panels, the computer consoles, and the OIS, test conductors and the engineers in the firing room were able to test and control all critical operations at the launch pads from a facility more than 3 miles away. Similar types of test consoles were also located in the VAB, LUT, propellants facilities, and other locations around LC-39.

Activating the LCC

Simultaneously activating the equipment in both the firing room and the VAB during 1965 and 1966 was a laborious process. Rockwell engineer Rich Robitaille’s experience was shared by hundreds of other workers getting LC-39 ready to support its first launch operations. He recalled:

I got to spend most of my time during that period helping guys from all the contractors install those panels and wire them. There were engineers, technicians, and quality control people. The QC guys and technicians were non-degreed, highly skilled technicians that did most of the grunt work. Our engineer always had to sign off on their work. Most of the technicians were five to ten years older than me. I helped them and worked with them and learned a lot. They really taught me what this stuff was all about, since I had no experience in wiring per se.

We had to go in and work with the 110A and learn about that computer and how it works. We all had to start running test procedures. We each had our subsystem and our panels to work with. If you had a software glitch in the computer, you might not have a connection between your switch and out to your panel and the vehicle. If your meter didn’t go on, you spent maybe three days trying to troubleshoot why the meter didn’t have an indication. And the first thing you blamed was the computer! They had some problems with the computer, but most of the problems were in all that wiring, wires after wires, that went from the computer to the stage. The mechanical guys worried about pipe after pipe, and they absolutely hated the computer. IBM knew what they were doing, but every time something happened, people would say, “That computer screwed up again!”

Everybody was getting honed down to a fine team. Everybody was working together, your procedures are talking on the net, the troubleshooting procedures, everything was starting to become like test pilots – people that have been around for years. The activation was where all the bugs were worked out, because there was no vehicle out there. We spent probably a year just activating.

We activated the launch control center at the same time that we went out to the mobile launcher. When you had a problem, the mobile launcher was inside the high bay, so you just had to go up to the 16th floor, where our offices were. We could walk right out from our office into the high bay, and go work on the system. You’ve got one guy in the LCC and the other one in the high bay, working together on the intercom, trying to figure out “How come when I flip that switch I don’t get a light?” and you have to start working backwards. All that wire mishmash in the panels – you might have a wire that was bad in the panel, and you had to pull the panel. And the technician would find out that a wire wasn’t soldered properly. There was a lot of work to figure out just where the problems were. That’s what a lot of our time was spent doing for most of a year.

Frank Bryan helped shake out the equipment:

When we first fired up the ground equipment out at LC-39, and all the stage contractors were trying to check out their equipment and make their panels talk to the relay racks on the LUT, I got some troubleshooting paper opened up. I went through and functionally operated these on-off-auto switches in the firing room in every sequence you could think of, just to make sure what was going to happen. Every contractor in that firing room dialed in and listened and took notes. That was good, because it actually showed them. It’s one thing to have the drawings say it’s going to do this, and another thing to actually see it happen, particularly with the 110A – nobody trusted it.

Ike Rigell, in his role as chief engineer for launch vehicle operations, was understandably conservative in his assessment of the readiness of the 110A system to support launch operations at LC-39. Kurt Debus sometimes chided him about being too cautious. During the activation of firing room 1, Rigell recalls, “I was hesitant to start one test because I wasn’t sure the control room was ready. Dr. Debus said, ‘Ike, take any switch here. I’ll bet you $50 that if I go flip it, it will work.’ I couldn’t take that bet, because I figured he was right! It was his devil’s advocate thing, to see if you really could support your position. One of the guys made that little panel as a result of our back-and-forth on the amount of testing.” (Fig. 6.28).

6.28

The switch panel made by KSC design engineering to commemorate Dr. Debus’ $50 bet to Ike Rigell on switch reliability in the firing room (Author’s photo). Source: Ward

Dave Moja recounted his impressions of some of the difficulties and pressure associated with activating the firing rooms:

Firing room 1 was activated for the first Saturn V launch. We were trying to get all the electrical equipment ready, and Randy Youmans was the guru of doing all that testing. Ike Rigell and Dr. Debus sometimes would come in. Of course we were always in a hurry to do everything. It took literally months.

And then it became my task to lead the parade to activate firing room 2 for the second Saturn V launch. And all the emphasis was still on firing room 1. I often said, “If we had that problem in firing room 1, we’d be off in a room trying to explain it to 100 people, all of us trying to figure it out.” For firing room 2, a group of us called the Flow Two people were off by ourselves. We had just as many problems, but we were able to just work it out ourselves without the spotlight on us.

About That Spacecraft Ready Light…

A lone test conductor from spaceflight operations/preflight operations branch in the firing room during a launch countdown. This was usually John Heard, although Charlie Stevenson and another test conductor also manned the station at times. While Heard was in the firing room at the Launch Control Center, his colleagues were all at their consoles in the ACE control room in the MSOB, nearly 5 miles (8 km) away.

When instructed by Skip Chauvin at about T minus 7 min, the spacecraft test conductor in the firing room flipped the spacecraft ready switch on his console. Bob Sieck said:

We spacecraft folks had two people in the Launch Control Center. One was our director of spacecraft operations, who was a member the mission management team on the top row. Then we had an engineer whose job it was to throw a switch that lit a light up on the big display board that showed either spacecraft as green or spacecraft as red (which would be no go), or the light was totally off. That was his only job. And he did that only at the direction of Skip Chauvin in the ACE control room in the MSOB. That engineer and the director were the only spacecraft people who were in the Launch Control Center, for all the tests and launches.

Spacecraft ready for launch was one of the interlocks for the electrical support equipment logic. As originally designed and built, that interlock was energized when the switch on the spacecraft operations control panel was flipped. What the people from spacecraft operations may not have known was that during much of the Apollo program, their spacecraft ready switch did absolutely nothing other than light the green light on the big display board.

The story goes that there was a procedural near-miss during one countdown. All indications over the OIS showed that the Apollo spacecraft was clearly go. However, for one reason or another, the spacecraft test conductor in the firing room did not receive instructions from spacecraft operations to throw the switch at the appointed time. The spacecraft ready for launch interlock was not energized in the ESE logic chain, which would have halted the countdown at an upcoming decision point. Test operations director Paul Donnelly was informed of the situation, and he ran over to the spacecraft test engineer and demanded that he throw the switch. After an initial objection, he complied just in time to keep the countdown running. The incident provoked a heated discussion between members of the mission management team after the launch.

In the weeks following the launch, the spacecraft ready switch in the firing room was quietly patched out of the terminal count ESE. A technician installed a jumper in the LCC terminal room to bypass the output from the spacecraft panel to the ESE. The spacecraft ready switch continued to operate the lights on the critical events display, but it no longer had any effect on the countdown sequence. The official explanation given was that the switch in the firing room was redundant to the information coming directly from the ACE control room. The spacecraft test conductor in the firing room could call for cutoff if he needed to stop the countdown, but his one switch was rendered—without his knowledge—merely ceremonial.