Inside a 20-Watt Traveling Wave Tube Amplifier from Apollo

How did the Apollo astronauts communicate on their trip to the Moon, 240,000 miles back to Earth? They used a 32-pound amplifier, built around a special kind of vacuum tube called a traveling-wave tube. In this blog post, I look inside this amplifier and explain how the traveling-wave tube works.

The Collins Radio traveling-wave tube amplifier. The label says "Not for flight" so this amplifier was only used on the ground. Click this photo (or any other) for a larger version.

Surprisingly, this amplifier only produced 20 watts of power, not much more than a handheld walkie-talkie.1 You might wonder how a 20-watt signal could be received all the way from the Moon. To pick up the weak signal, NASA built a network of 26-meter (85-foot) dish antennas that spanned the globe, with ground stations in Spain, Australia, and California. For the signal to the spacecraft, the ground stations broadcast a strong, focused 10,000-watt signal that could be picked up by the spacecraft's small antennas. Additional ground stations with smaller 9-meter (30 foot) antennas filled in coverage gaps, along with tracking ships and airplanes.2

NASA's 26-meter antenna at Honeysuckle Creek, Australia. Photo from NASA.

The communication system on Apollo was very complex, as shown in the diagram below. The amplifier, highlighted in yellow, was just one component of this system (which I'm not going to try to explain here). Most communication went over the "Unified S-Band", which sent voice, data, telemetry, TV, control, and ranging through one unified system. In comparison, the Gemini missions used separate systems for different purposes. (S-band refers to the microwave frequency band used by this system.)

Diagram of the Apollo Block II Telecommunications System. (Click for a larger version.) From "Apollo Logistics Training", courtesy of Spaceaholic.

Inside the amplifier

The amplifier was built by Collins Radio, a company that had a large role in the space program.3 (Collins claims that from Mercury and Gemini to Apollo, every American voice transmitted from space was via Collins Radio equipment.) The photo below shows the amplifier with the cover removed, showing the circuitry inside. Note the tangles of coaxial cables for the high-frequency RF signals. The "Danger High Voltage" warning is due to the thousands of volts required by the traveling-wave tubes.

Inside the amplifier, many coaxial cables connect the RF circuitry.

The block diagram below shows the structure of the amplifier,4 centered on the two traveling-wave tubes that perform the amplification. The amplifier takes two inputs: voice/data and the TV signal. In normal use, one tube amplifies the voice/data signal and the other amplifies the TV signal. An important feature is that either signal can be sent to either tube, or the amplifier can be bypassed entirely. This allows communication if a tube fails, or even if the amplifier entirely fails. The signals are directed by RF relays, electrically-controlled switches. The triplexer sends the two amplified signals to the antenna, and directs the signal from the antenna to the receiver.)

Simplified block diagram of the amplifier. From CSM Functional Integrated System Schematics.

The photo below shows the amplifier with the case removed. (We were unable to disassemble the amplifier completely so this photo is from the documentation.5) The traveling-wave tube is the black cylinder at bottom right, about 10 inches long. The second tube is in the same position on the back of the amplifier.

Photo of the traveling-wave tube amplifier used in Apollo. Photo from Collins S-Band Power Amplifier.

How a traveling-wave tube works

The traveling-wave tube (TWT) is the heart of the amplifier. TWT systems have been popular for satellites because they are compact and provide high amplification with very wide bandwidth.7 They are still widely used in satellites, radar, and other systems.

A traveling-wave tube uses an interesting technique to amplify the input RF signal, different from typical vacuum tubes. It creates a beam of electrons and transfers energy from this beam to the RF signal. In more detail, an electron gun shoots electrons down the tube, (a bit like a CRT). As these electrons travel through the tube, they interact with the RF signal and bunch together, transferring energy to the RF signal.6

The problem is that the electron beam and the RF signal need to travel at approximately the same speed in order to interact, but the electron beam travels at about 10% the speed of light,8 while the RF signal travels at the speed of light. The trick is to put the RF signal through a helix, wrapped around the beam. Because the RF signal travels through the long helix rather than in a straight line, its path through the tube is slowed. With the proper helix design, the RF signal and the electron beam travel at approximately the same net speed down the tube, allowing them to interact.

Diagram of the TWT amplifier. From "Apollo Logistics Training", courtesy of Spaceaholic.

The diagram above shows the components of the traveling wave tube in detail. The heart of the TWT is the drift tube that holds the electron beam, wrapped in the helix for the RF signal. At the left, the electron beam is created by the components of the electron gun (heater, cathode, and electrodes). The RF input and output provide the connections to the helix for the signal that is being amplified. The collector absorbs the weakened electron beam after it has passed through the tube. Finally, the permanent magnets keep the electron beam focused through the tube.

It's hard to see the traveling-wave tube inside the amplifier, since it is mounted at the bottom under a bunch of coaxial cables; the photo below is the best I could do. The tube looks like a black cylinder, but you can see the coaxial cables attached at the left and right.

The traveling-wave tube inside the amplifier.

Other parts of the amplifier

Next, I'll briefly describe the other circuitry inside the amplifier. A traveling wave tube requires high voltage to accelerate the electron beam. The photo below shows two of the power supply transformers. The amplifier was powered with 115 volts AC, 3-phase at 400 cycles per second. It also used 28 volts DC for the control circuitry. Note the circuitry encased in plastic at the bottom of the photo.

The amplifier uses high-voltage transformers to power the traveling-wave tubes.

As described earlier, the RF relays switch signals between the two tubes to provide redundancy. The relays (below) are the fairly large square units with coaxial cables attached. These relays are more complex than typical relays because they must transfer gigahertz RF signals. The internal wiring is constructed from metal strips between double ground planes along with waveguides.

The relays with coaxial cables attached.

Another interesting component of the amplifier is the triplexer, a special RF component that connects the antenna to the amplifier. The idea of the triplexer is that it has three ports, each for a different frequency, and keeps the signals on each port separate from each other. Specifically, it combines the main 2287.5 megahertz signal with the TV's 2272.5 megahertz signal and sends these to the antenna. The signal from the ground is at 2106.4 megahertz; the triplexer directs this signal from the antenna to the receiver. Internally, the triplexer has band-pass filters for each frequency, providing a large amount of isolation (60 dB) between its three ports.

The triplexer.

The triplexer is the metal box in the photo above. Note the coax connections with the antenna connection labeled. Although the triplexer says "Danger High Voltage" on top, this refers to the surrounding power supply circuitry, not the triplexer itself.

Controlling the amplifier

The astronauts had control switches in the Command Module to turn the power amplifier on and off, and switch between the primary and secondary tubes. The diagram below shows the location of these switches, marked PWR AMPL. The PRIM/SEC selects which tube was used for the main signal and which was used for the TV signal. The HIGH/OFF/LOW switch selected the power output level for the amplifier. When the amplifier was off, the input signal was connected directly to the antenna, bypassing the amplifier.

Astronauts controlled the amplifier through switches on the console. Diagram from Command/Service Module Systems Handbook p208.

Conclusion

This power amplifier illustrates the complexity of the communication systems for Apollo.9 Even though the amplifier is complex internally with redundant traveling-wave tubes, it is just one of many pieces of hardware. The diagram below shows the Command Module's equipment bay, with the amplifier highlighted in yellow. (The Apollo Guidance Computer was directly above the amplifier, two rows up.)

Diagram of the Apollo Command Module's equipment bay with the S-band power amplifier highlighted. From Command/Service Module Systems Handbook p212.

We are currently investigating the possibility of powering up this amplifier to see if it still operates. I announce my latest blog posts on Twitter, so follow me @kenshirriff for updates. I also have an RSS feed. Thanks to Steve Jurvetson for loaning me this amplifier. Thanks to Spaceaholic and Mike Stewart for providing diagrams and the Collins Aerospace Museum for additional information.

Notes and references

1. Walkie-talkies typically use 0.5 to 5 watts of power, with some models providing 8 watts, mostly limited by FCC regulations. There are a few 20-watt or even 25-watt handheld radios. ↩

2. This photo shows the Vanguard tracking ship. This ship was a surplus tanker from World War II that was repurposed as a missile tracking ship by covering it with antennas. NASA used the ship for communication with Apollo, and the ship was scrapped in 2013.

   

   The Vanguard ship, from Wikimedia.

    ↩

3. The document Collins S-Band Power Amplifier has technical specifications for the amplifier. The presentation Collins Role in Space Communications describes the Collins equipment used in Mercury, Gemini, and Apollo. Collins built equipment for the spacecraft and transmitting and receiving equipment on the ground. ↩

4. Here's a more detailed diagram of the power amplifier circuitry. This diagram shows the power supply and control circuitry in more detail. In particular, the circuitry lets the tubes heat up for 90 seconds before use. Circuitry also shuts down the power if there is a fault or loss of a power phase.

   

   The power amplifier diagram. From "Apollo Logistics Training", courtesy of Spaceaholic.

    ↩

5. The Collins photo makes it look like you can simply remove the case from the amplifier. However, the photo is misleading since the amplifier doesn't come apart like that. We attempted to remove the amplifier from the case, but it is fastened with many inaccessible screws and some components are glued down. We suspect that the amplifier was assembled inside the case, making it very difficult to perform any pre-launch maintenance. We gave up on disassembling the amplifier completely, which is why all our photos show the view from the top. ↩

6. The interaction between the electron beam and the RF signal in the helix is complex, but the net result is that energy is transferred from the beam to the signal. Specifically, the electric field from the RF signal produces positive and negative waves. These accelerate and decelerate the electrons, causing them to bunch together. (On the whole, the electron beam decelerates more than it accelerates, so it loses energy.) The moving bunches of electrons induce more current in the helix, strengthening the RF signal. The result is a feedback loop, causing the RF signal to grow exponentially as it travels through the tube.

   For more information on how traveling-wave tubes work, see Traveling Wave Tube, Recent theory of traveling-wave tubes, or this long presentation. ↩

7. A traveling-wave tube can amplify a large range of frequencies (i.e. it has a high bandwidth) because it doesn't have any resonant elements (unlike a klystron, for instance). Thus, it doesn't need to be tuned to a particular frequency. ↩

8. Ignoring relativistic effects, the speed of an electron beam accelerated by a voltage is given by

   

   Equation for electron beam speed.

   where v₀ is the velocity, V_b is the voltage, e is the charge of an electron, and m_e is the mass of an electron. For example, applying 6000 volts yields an electron speed of 46,000 km/second, about 15% the speed of light.

   This equation is a rearrangement of the kinetic energy from the velocity and the energy from the voltage potential difference.

   

   Kinetic energy equation.

   In the traveling-wave tube, the electron beam must be slightly faster than the (net) RF signal speed so the beam will transfer energy to the RF signal as the beam is slowed.

    ↩

9. For more information on Apollo communication, see Apollo Experience Report - S-Band System Signal Design and Analysis. See also CSM Functional Integrated System Schematics and Command/Service Module Systems Handbook. ↩

"Space age electronics": Inside a GE thin-film paperweight from the 1960s

In the early 1960s, General Electric developed a technology called thin-film electronics.1 These circuits were built from thin films of material, much more compact than individual components. For weight-sensitive applications such as satellites and military equipment, thin-film electronics could potentially be revolutionary.

The GE paperweight consists of circuitry and a satellite model encased in thick clear plastic. It is labeled "Light Military Electronics Department, Defense Electronics Division, General Electric. Space Age Electronics, thin film circuits."

GE's Light Military Electronics department1 built the paperweight above to showcase their "Space Age Electronics". In the center is a thin-film circuit, next to a model of an early satellite. However, the paperweight contained a surprise: when picked up, the paperweight emitted a beep-beep-beep noise, sounding just like a satellite.2 In this blog post, I reverse-engineer the "Space Age Electronics" inside this paperweight and explain how it works. In brief, the visible thin-film circuit implements a flip flop. The remaining circuitry is hidden in the compartment on the left: two oscillators that produce the beeps. These oscillators are implemented in another unusual 1960s technique called "cordwood'.

The thin-film module

The most visible part of the paperweight is the thin-film module. The idea behind thin film is to build resistors and capacitors as thin layers on a substrate, rather than using individual components. Resistors are formed from thin strips of resistive material, the vertical reddish-brown lines on the module's surface. For higher resistance, these lines zig-zag back and forth.3 Capacitors are formed from two thin layers of metal (the plates), separated by an insulating dielectric material.

This angle view shows how the semiconductor components are mounted above the thin film circuitry.

Thin-film transistors were not commercially practical in the 1960s, so the module has tiny discrete transistors and diodes mounted on top, connected by golden wires. (This must have been expensive to manufacture.) In the photo above, the shadows show that the semiconductor components (black blobs) are slightly above the surface. You can distinguish the diodes by their green dots. At the left, five metallic strips provide power and signal connections to the module, with golden contacts connecting these strips to the thin-film circuitry.

A closeup of the thin-film module.

Interest in thin-film technology declined in the mid-1960s as integrated circuits became commercially available. Integrated circuits were cheaper, could fit more components into a chip, and could be mass-produced. For these reasons, integrated circuits took over the electronics market. Thin-film circuits are still used, but only for specialized applications.

I traced out the paperweight's thin-film circuit and found that it implements a toggle flip flop, a standard electronic circuit. The flip flop stores either a 1 state or a 0 state, like a single bit of memory. When it gets a negative pulse on the trigger input, it flips to the opposite state. Thus, as it receives input pulses, it goes "on", "off", "on", "off", etc. In the paperweight, the flip flop creates the separate beeps. The paperweight generates a beep while the flip flop is on, and is silent when the flip flop is off.

Schematic of the circuit in the thin-film module.

You can match up the components in the schematic with the components in the photo: two transistors, two diodes, four capacitors, and multiple resistors. Note that the two sides of the circuit are symmetrical, both in the schematic and in the photo. One side of the circuit is on and one side is off. Depending on which side is on, the circuit holds a 0 or a 1. See the footnote4 for more details.

Inside the paperweight

The left side of the paperweight has a compartment with some interesting circuitry inside. The paperweight was powered by a 22½ V battery, which was relatively common back then but is now obsolete. It looks a bit like a 9-volt battery, except it has one contact at each end. Next to the battery is a vintage earphone, the round pink component. It acts as the speaker in this device.

Looking inside the paperweight's compartment reveals more circuitry.

Another unusual component is the tilt switch in the lower right, which turns the paperweight on and off. (I don't know if this tilt switch contains mercury or has a rolling ball inside.) When the paperweight is horizontal, the tilt switch is open. But if the paperweight is picked up, the tilt switch closes. This probably added to the "drama" of the paperweight, since someone will think it is just a decoration until they pick it up and it starts beeping.

The tilt switch turns the paperweight on and off.

In the upper right of the compartment, a block of plastic encases the oscillator circuitry. The module is built with "cordwood" construction, a way of building high-density circuits that was popular in the 1960s. Instead of mounting components flat on a circuit board, cordwood puts components between two boards. (They are stacked together like logs, giving cordwood its name.) The photo below shows the components; it isn't as clear as I'd like because the components are embedded in yellowing plastic.

This view of the module shows three resistors (striped) and two capacitors (silver).

On each side of the module, components are wired with point-to-point wiring, as shown below. This photo also shows how the insulated connection wires are also embedded in the module. The large dark circles are the two transistors.

Closeup of the cordwood module, showing the wiring. The transistors and the ends of the resistors and capacitors are visible.

The oscillators use unijunction transistors, a somewhat unusual type of transistor, different from standard NPN and PNP transistors. Oscillators could be easily created from unijunction transistors due to their nonlinear characteristics. The unijunction transistor was invented by General Electric in 1953, so it's not surprising that General Electric made use of them in this paperweight. The GE logo is visible on top of the transistors.

In this view of the module, the script "GE" logo is visible on top of the transistors. These transistors are part number 2N491

The cordwood block holds two oscillators, to control the duration of each beep and to generate the beep sound itself. The first oscillator generates five pulses per second. These pulses go to the thin-film flip-flop circuit, which will change its state between off and on with each pulse. That is, the flip flop is off for 200 milliseconds, on for 200 milliseconds, and so forth. The output from the flip flop powers the second transistor oscillator, which generates a 3.5-kilohertz tone. The result is the repeating beep-beep-beep output from the paperweight.

Schematic of the unijunction transistor oscillators.

The schematic above shows the two oscillators. The idea behind a unijunction transistor oscillator is that the capacitor slowly charges through the resistor. As the capacitor charges, the voltage on the emitter (symbolized by the arrow) increases. When it reaches the trigger voltage, the transistor turns on and the capacitor discharges to ground. The cycle repeats, generating a sequence of pulses on the output.

Conclusion

I think the paperweight is from approximately 1962, based on GE's thin-film research at the time and the appearance of the paperweight's model satellite.6 The paperweight was produced in the midst of the space race; John Glenn became the first American in orbit in 1962. Satellites were still a new "space-age" thing at the time, so the paperweight was a symbol of General Electric's advanced technology.5 The beeps from the paperweight are similar to those produced by Sputnik (1957). At the time, the paperweight must have been an impressive object, a vision of the future.

Thanks to Peter B. Newman, technology collector and educator for sending me the paperweight for analysis. Thanks to Mikes Electric Stuff for identifying the tilt switch for me.

I announce my latest blog posts on Twitter, so follow me @kenshirriff. I also have an RSS feed.

Notes and references

1. The paperweight was built by GE's Light Military Electronics department. In the early 1960s, this department produced aerospace electronics such as digital guidance computers, flight-control systems, and satellite sensors. These were used in weapons including the F-105 fighter-bomber, Sidewinder missile, and Polaris and Atlas ICBMs. ↩↩

2. Sputnik's beeps were approximately 150-300 ms long at 1.5 kilohertz. (The frequency isn't well-defined because the transmission was just a carrier switched on and off, but this is the frequency in typical recordings.) The paperweight's beeps were approximately 200 ms long at 3.47 kilohertz. The point is that the paperweight's beeps were designed to resemble the beeps from a satellite such as Sputnik, and people would have recognized this at the time. You can hear the beeps of the paperweight here; I had to edit the audio a bit because I discovered too late that the doorbell rang in the middle of the recording. ↩

3. In the module, some of the resistors are connected to the metal layer through structures that have teeth kind of like a comb. I'm not sure what the purpose of these structures is. My hypothesis is that by changing the number of "teeth", the active length of the resistor can be changed, adjusting the resistor. (Modifying the metal layer is easier than modifying the thin-film layers.) ↩

4. The two transistors are cross-connected, so when one transistor is on, it forces the other one off. The trigger capacitors are pre-charged through the corresponding output. The result is that the transistor that is currently on (output low) will be pulled lower than the transistor that is currently off (output high). This turns off the first transistor, flipping the state of the circuit. It's a fairly standard flip-flop circuit; more details are here. ↩

5. In 1960, GE hoped to build a commercial communications satellite network, and formed a subsidiary "Communication Satellites Inc" in 1960. However, GE abandoned that goal in 1961 (probably due to antitrust issues) to focus on manufacturing equipment for space vehicles. ↩

6. The satellite in the paperweight resembles the Ariel 1 (1962) and Ariel 2 (1964) satellites, with its paddle-like solar cells. It's not an exact match, so I don't know if the satellite is an artist's conception, or is a different satellite. If you recognize the satellite, please let me know. ↩

Teardown of a PC power supply

Have you ever wondered what's inside your computer's power supply? The task of a PC power supply is to convert the power from the wall (120 or 240 volts AC) into stable power at the DC voltages that the computer requires. The power supply must be compact and low-cost while transforming the power efficiently and safely. To achieve these goals, power supplies use a variety of techniques and are more complex inside than you might expect. In this blog post, I tear down a PC power supply and explain how it works.1

The power supply I examined, like most modern power supplies uses a design known as a "switching power supply." Switching power supplies are now very cheap, but this wasn't always the case. In the 1950s, switching power supplies were complex and expensive, used in aerospace and satellite applications that needed small, lightweight power supplies. By the early 1970s, though, new high-voltage transistors and other technology improvements made switching power supplies much cheaper and they became widely used in computers. Now, you can buy a phone charger for a few dollars that contains a switching power supply.

The ATX power supply that I examined was packaged in a metal box the size of a brick, with a remarkable number of colorful cables emerging from it. Removing the case reveals the components below, tightly packed to keep the power supply compact. Many of the components are hidden by the heat sinks that keep the power semiconductors cool along with the fan at the right.

The power supply, removed from the case. The large bundle of wires at the left is connected to the computer. The large component in the middle that looks like a transformer is a filter inductor. Click this photo (or any other) for a larger version.

I'll start with a quick overview of how the switching power supply works, and then describe the components in detail. Starting at the right, the power supply receives AC power. The input AC is converted to high-voltage DC, with the help of some large filtering components. This DC is switched on and off thousands of times a second to produce pulses that are fed into a transformer, which converts the high-voltage pulses into low-voltage, high-current pulses. These pulses are converted to DC and filtered to provide nice, clean power, which is fed to the computer's motherboard and disk drives through the bundle of wires on the left.

While this process may seem excessively complex, most consumer electronics, from your cell phone to your television, use a switching power supply. The high frequencies allow the use of a small, lightweight transformer. In addition, switching power supplies are very efficient; the pulses are adjusted to supply just the power needed, rather than turning excess power into waste heat as in a "linear" power supply.

Input filtering

The first step is for the input AC to go through an input filter circuit that blocks electrical noise from exiting the power supply. The filter below consists of inductors (the toroidal coils) and capacitors. These boxy gray capacitors are special Class-X capacitors, designed to be connected safely across the AC lines.

The input filter components

Rectification: converting AC to DC

The 60-Hertz AC (alternating current) from the wall oscillates 60 times a second, but the power supply needs steady DC (direct current) that flows in one direction. The full-bridge rectifier below converts the AC to DC. The rectifier below is marked with "-" and "+" for the DC outputs, while the two center pins are the AC input. Internally the rectifier contains four diodes. A diode allows current to pass in one direction and blocks it in the other direction, so the result is that the alternating current is converted to direct current, flowing in the desired direction.

The bridge rectifier is labeled "GBU606". Filter circuitry is to its left. To the right, the large black cylinder is one of the voltage-doubler capacitors. The small yellow capacitor is a special Y capacitor, designed for safety.

The diagram below shows how the bridge rectifier works. In the first schematic, the AC input has the upper side positive. The diodes pass the voltage through to the DC output. In the second schematic, the AC input has reversed direction. However, the configuration of the diodes ensures that the DC output voltage stays the same (positive on top). The capacitors smooth out the output.

The two schematics show the flow of current as the AC input oscillates. The diodes force current to flow in the direction indicated by their arrow shape.

Modern power supplies accept a "universal" input voltage of 85 to 264 volts AC, so they are usable in different countries regardless of the country's voltage. However, the circuitry of this older power supply couldn't handle such a wide input range. Instead, you had to flip a switch (below) to select between 115 V and 230 V.

The 115/230 V switch.

The voltage selection switch used a clever circuit, a voltage doubler. The idea is that with the switch closed (for 115 volts), the AC input bypasses the bottom two diodes in the bridge rectifier and is instead connected directly to the two capacitors. When the AC input is positive on top, the top capacitor is charged with the full voltage. And when the AC input is positive on the bottom, the lower capacitor is charged with the full voltage. Since the DC output is across both capacitors, the DC output has double the voltage. The point of this is that the rest of the power supply receives the same voltage, whether the input is 115 volts or 230 volts, simplifying its design. The downsides of the voltage doubler are that the user must put the switch in the correct position (or risk destroying the power supply), and the power supply requires two large capacitors. For these reasons, the voltage doubler has gone out of style in more recent power supplies.

The voltage doubler circuit. Each capacitor is charged with the full voltage, so the DC output has double the voltage. The grayed-out diodes are not used when the doubler is active.

Primary and secondary

For safety, the high-voltage components and the low-voltage components are separated, both mechanically and electrically. The primary side below contains all the circuitry that is connected to the AC line. The secondary side contains the low-voltage circuitry. The primary and secondary are separated by an "isolation boundary" (shown in green), with no electrical connections across the boundary. The transformers pass power across this boundary through magnetic fields, without a direct electrical connection. Feedback signals are sent from the secondary to the primary by opto-isolators, which transmit signals optically. This separation is a key factor in safe power supply design: a direct electrical connection between the AC line and the output would create a high danger of electric shock.

The power supply with main features labeled. The heat sinks, capacitors, control board, and output wires have been removed to give a better view. (SB indicates the standby supply.)

Pulses to the transformer

At this point, the input AC has been converted to high-voltage DC, about 320 volts.2 The DC is chopped into pulses by the switching transistor above, a power MOSFET.3 Because this transistor gets hot during use, it was mounted on a large heat sink. These pulses are fed into the main transformer above, which in a sense is the heart of the power supply.

The transformer consists of multiple coils of wire wound around a magnetizable core. The high-voltage pulses into the transformer's primary winding produce a magnetic field. The core directs this magnetic field to the other, secondary windings, producing voltages in these windings. This is how the power supply safely produces its output voltages: there is no electrical connection between the two sides of the transformer, just a connection by the magnetic field. The other important aspect of the transformer is that the primary winding has the wire wrapped around the core a large number of times, while the secondary windings are wrapped around a much smaller number of times. The result is a step-down transformer: the output voltage is much smaller than the input, but at a much higher current.

The switching transistor3 is controlled by an integrated circuit, a "UC3842B current mode PWM controller". This chip can be considered the brains of the power supply. It generates pulses at the high frequency of 250 kilohertz. The width of each pulse is adjusted to provide the necessary output voltage: if the voltage starts to drop, the chip produces wider pulses to pass more power through the transformer.4

The secondary side

Now we can look at the secondary side of the power supply, which receives the low-voltage outputs from the transformer. The secondary circuitry produces the four output voltages: 5 volts, 12 volts, -12 volts, and 3.3 volts. Each output voltage has a separate transformer winding and a separate circuit to produce that voltage. Power diodes (below) convert the outputs from the transformer to DC, and then inductors and capacitors filter the output to keep it smooth. The power supply must regulate the output voltages to keep them at the proper level even as the load increases or decreases. Interestingly, the power supply uses several different regulation techniques.

Closeup of the output diodes. At the left are cylindrical diodes mounted vertically. In the middle are pairs of rectangular power Schottky diodes; each package holds two diodes. These diodes were attached to a heat sink for cooling. At right note the two staple-shaped copper wires used as current-sensing resistors.

The main outputs are the 5-volt and 12-volt outputs. These are regulated together by the controller chip on the primary side. If the voltage is too low, the controller chip increases the width of the pulses, passing more power through the transformer and causing the voltage on the secondary side to increase. And if the voltage is too high, the chip decreases the pulse width. (The same feedback circuit controls both the 5-volt and 12-volt output, so the load on one output can affect the voltage on the other. Better power supplies regulate the two outputs separately.5)

Underside of the power supply, showing the printed circuit board traces. Note that wide separation between the secondary-side traces on the left and the primary-side traces on the right. Also note the wide metal traces used for the high-current supply and the thin traces for control circuitry.

You might wonder how the controller chip on the primary side receives feedback about the voltage levels on the secondary side, since there is no electrical connection between the two sides. (In the photo above, you can see the wide gap separating the two sides.) The trick is a clever chip called the opto-isolator. Internally, one side of the chip contains an infra-red LED. The other side of the chip contains a light-sensitive photo-transistor. The feedback signal on the secondary side is sent into the LED, and the signal is detected by the photo-transistor on the primary side. Thus, the opto-isolator provides a bridge between the secondary side and the primary side, communicating by light instead of electricity.6

The power supply also provides a negative voltage output (-12 V). This voltage is mostly obsolete, but was used to power serial ports and PCI slots. Regulation of the -12 V supply is completely different from the 5-volt and 12-volt regulation. The -12V output is controlled by a Zener diode, a special type of diode that blocks reverse voltage until a particular voltage is reached, and then starts conducting. The excess voltage is dissipated as heat through a power resistor (pink), controlled by a transistor and the Zener diode. (Since this approach wastes energy, modern high-efficiency power supplies don't use this regulation technique.)

The -12 V supply is regulated by a tiny Zener diode "ZD6", about 3.6 mm long, on the underside of the circuit board. The associated power resistor and transistor "A1015" are on the top side of the board.

Perhaps the most interesting regulation circuit is for the 3.3-volt output, which is regulated by a magnetic amplifier. A magnetic amplifier is an inductor with special magnetic properties that make it behave like a switch. When a current is fed into the magnetic amplifier inductor, at first the inductor will almost completely block the current as the inductor magnetizes and the magnetic field increases. When the inductor reaches its full magnetization (i.e. it saturates), the behavior suddenly changes and the inductor lets the current flow unimpeded. In the power supply, the magnetic amplifier receives pulses from the transformer. The inductor blocks a variable part of the pulse; by changing the pulse width, the 3.3-volt output is regulated.7

The magnetic amplifier is a ring constructed from ferrite material with special magnetic properties. The ring has a few turns of wire wound around it.

The control board

The power supply has a small board holding the control circuitry. This board compares the voltages against a reference to generate the feedback signals. It also monitors the voltages to generate a "power good" signal.8 This circuitry is mounted on a separate, perpendicular board so it doesn't take up much room in the power supply.

The control board has through-hole components on top and the underside is covered with tiny surface-mount components. Note the "zero-ohm" resistors marked with 0, used as jumpers.

The standby power supply

The power supply contains a second circuit for standby power.9 Even when the computer is supposedly turned off, the 5V standby supply is providing 10 watts. This power is used for features that need to be powered when the computer is "off", such as the real-time clock, the power button, and powering-on via the network ("Wake on LAN"). The standby power circuit is almost a second independent power supply: it uses a separate control IC, separate transformer, and components on the secondary side, although it uses the same AC-to-DC circuitry on the primary side. The standby power circuit provides much less power than the main circuit, so it can use a smaller transformer.

The black and yellow transformers: the transformer for standby power is on the left and the main transformer is on the right. The control IC for standby power is in front of the transformer. The large cylindrical capacitor on the right is part of the voltage doubler. The white blobs are silicone to insulate components and hold them in place.

Conclusion

An ATX power supply is complex internally, with a multitude of components ranging from chunky inductors and capacitors to tiny surface-mount devices.10 This complexity, however, results in power supplies that are efficient, lightweight, and safe. In comparison, I wrote about a power supply from the 1940s that produced just 85 Watts DC, but was suitcase-sized and weighed over 100 pounds. Now, with advanced semiconductors, you can hold a much more powerful power supply for under $50 that you can hold in your hand.

I've written about power supplies before, including a history of power supplies in IEEE Spectrum. You might also like my Macbook charger teardown and iPhone charger teardown. I announce my latest blog posts on Twitter, so follow me at kenshirriff. I also have an RSS feed.

Notes and references

1. Intel introduced the ATX standard for personal computers in 1995. The ATX standard (with some updates) still defines the motherboard, enclosure, and power supply configuration for most PCs. The power supply I examined is from 2005, so newer power supplies are more advanced and more efficient. The basic principles are the same, but there are some changes. For instance, regulation using DC-to-DC converters has mostly replaced the magnetic amplifier.

   

   The label on the power supply.

   The label provides information about the power supply I examined. It was built by Bestec for Hewlett-Packard's Dx5150 desktop PC. This power supply doesn't fit the ATX dimensions; it is longer and more rectangular. ↩

2. You might wonder why an AC input of 230 volts yields 320 volts DC. The reason is that AC voltage is normally measured as root-mean-square which (sort of) averages the varying waveform. As a result, a 230-volt AC signal has peaks of 320 volts. The power supply capacitors charge through the diodes to the peak voltage, so the DC will be approximately 320 volts (although it will sag somewhat through the cycle). ↩

3. The power transistor is an FQA9N90C power MOSFET. It can handle 9 amps and 900 volts. ↩↩

4. The integrated circuit is powered by a separate winding on the transformer that provides 34 volts to run the chip. You might notice a chicken-and-egg problem: the control IC creates the pulses to the transformer, but the transformer powers the control IC. The solution is a startup circuit consisting of a 100 kΩ resistor between the IC and the high-voltage DC. This provides a small current, sufficient to start operation of the IC. Once the IC starts sending pulses to the transformer, it is powered by the transformer. ↩

5. The technique of using one regulation loop for two outputs is called cross-regulation. If the load on one output is much higher than the load on the other, the voltages may diverge from their proper values. For this reason, many power supplies have a minimum load requirement on each output. More advanced power supplies use DC-to-DC converters for all the outputs to make sure they are precise. For more about cross-regulation, see this presentation and this presentation. One technique discussed is DC-stacking the output windings, a technique used in this power supply. Specifically, the 12-volt output is implemented as a 7-volt output "stacked" on top of the 5-volt output, yielding 12-volts. With this configuration, a 10% error (for example) in the 12-volt circuit would be just 0.7 V rather than 1.2 V. ↩

6. The opto-isolators are PC817 components, which provide 5000 volts of isolation between the two sides. Note the slot cut in the circuit board underneath the opto-isolators. This provides additional safety, ensuring that dangerous voltages cannot pass between the two sides of the opto-isolator along the surface of the circuit board, for example if there were contamination or condensation on the board. (Specifically, the slot increases the creepage distance.) ↩

7. The pulse width through the magnetic amplifier is set by a simple control circuit. During the reverse part of each pulse, the inductor is partially demagnetized. A control circuit adjusts the demagnetization voltage. A higher demagnetization voltage produces more demagnetization. This causes the inductor to take longer to re-magnetize and thus it blocks the input pulse for a longer time. With a shorter pulse passing through the circuit, the output voltage is decreased. Conversely, a lower demagnetization voltage produces less demagnetization, so the input pulse is blocked for a shorter time. Thus, the output voltage is regulated by changing the demagnetization voltage. Note that the pulse width into the magnetic amplifier is controlled by the control IC; the magnetic amplifier cuts these pulses shorter as needed to regulate the 3.3 V output. ↩

8. The control board contains multiple ICs including an LM358NA op-amp, a TPS3510P supervisor/reset chip, an LM339N quad differential comparator, and an AZ431 precision reference. The supervisor chip is interesting; it is specifically designed for power supplies and monitors the outputs to make sure they are not too high or too low. The AZ431 is a variant of the TL431 bandgap reference chip, which is very commonly used in power supplies to provide a reference voltage. I've written about the TL431 here. ↩

9. The standby power supply uses a different transformer configuration, called a flyback transformer. The control IC is an A6151, which includes the switching transistor in the IC, simplifying the design.

   

   Power supply circuit using the A6151. This schematic is from the datasheet so it is close to the circuit in the power supply I examined, but not identical.

    ↩

10. If you want to see detailed schematics of a variety of ATX power supplies, see danyk.cz. It's remarkable how many different implementations are used in power supplies: different topologies (half-bridge or forward), absence or presence of power factor conversion (PFC), and different control, regulation, and monitoring systems. The power supply I examined is moderately similar to the forward topology ATX supplies without PFC near the bottom of the page. ↩

Teardown of a logic chip from a vintage IBM ES/9000 mainframe

IBM and its large mainframe computers ruled the computer industry for decades. But during the 1980s, mainframes faced increasing competition from microprocessors, workstations, and super-minicomputers. To meet this challenge, IBM pushed technology to the limit to create the ES/9000 in 1991, a family of powerful mainframes with a price tag to match, from $70,500 up to $22 million. The processor of the ES/9000 wasn't a single chip, but a metal and ceramic package called a Thermal Conduction Module (TCM) that held 121 chips. Recently, Dave Jones of EEVBlog created a popular teardown video of a TCM, showing its complex construction. After disassembling the module, he kindly sent me some of these cutting-edge chips to analyze. In this blog post, I examine the circuitry inside one of these logic chips from the ES/9000.

Detail of a bipolar logic chip from the ES/9000 computer. This closeup of the die shows the four layers of metal and the transistors underneath. Click this photo (or any other) for a larger version.

The ES/9000

The ES/9000 family of computers consisted of three lines with performance spanning two orders of magnitude: small entry-level systems for an office, mid-range air-cooled systems (below), and high-end water-cooled systems that could fill a room. The technology of the ES/9000 was very advanced for its time in many ways. Along with the ceramic thermal conduction modules, IBM created new high-speed integrated circuits with state-of-the-art transistors. At the system level, IBM introduced new operating systems as well as ESCON (Enterprise Systems Connection), a high-speed fiber-optic connection between the mainframe and peripherals. An optional cryptographic feature provided high-speed encryption in tamper-resistant hardware. Even the power supplies were innovative; the water-cooled power supplies could be swapped while the computer was running. The innovations of the ES/9000 generated numerous journal articles and patents.1

The ES/9000 type 9121, from Hampage.

In this article, I'm focusing on the mid-range systems, known as the 9121 processors.2 This system (above) was packaged in a drab frame the size of a large refrigerator.3 It used 7.4 KVA of power, occupied 14.7 square feet of floor space, and weighed 2000 pounds. It could hold up to 1 gigabyte of memory, a large capacity at a time when personal computers typically had 1 to 4 megabytes of RAM. A typical 9121 system cost $1.5 million and had about twice the performance of a contemporary Intel 80486 computer that cost $10,000. This is a bit of an apples-and-oranges comparison, since the mainframe gave you high-speed I/O channels, fast memory access, and an advanced operating system, but it shows the dramatic price/performance advantage of microprocessors.

The TCM (Thermal Conduction Module)

One of the most interesting features of the ES/9000 was the Thermal Conduction Module (TCM) that held the integrated circuits. The high-performance bipolar chips generated a lot of heat, so IBM developed new cooling mechanisms so this computer could function without water cooling. The cut-away photo below shows a TCM with its large heat sink attached. At the bottom, some of the integrated circuit dies are visible along with the copper cooling pistons. The computer's main circuitry consists of five different TCMs.4

Diagram of the TCM with the heat sink on top. Photo from Dr. Chu / IBM, diagram from TCM paper.

The TCM is surprisingly small, 5 inches (127.5mm) on a side, yet it holds 121 integrated circuits. Each integrated circuit has a spring-loaded copper piston on it to remove the heat. These pistons transfer the heat into the TCM's metal case, where the heat passes into the heat sink and then the air flow. The pistons are precision-machined to maximize contact and thus heat transfer. The module is filled with oil (visible below), which also increases heat transfer. The design of the TCM allows it to dissipate 600 watts of heat—imagine holding six 100-watt light bulbs in your hand.

Closeup of the TCM showing the copper cooling pistons on top of the silicon dies. Courtesy of Dave Jones.

The integrated circuits in the TCM are not packaged like regular integrated circuits, but consist of a silicon die soldered upside-down to the ceramic substrate, flip-chip style. This ceramic substrate is an incredible feat of engineering. It's essentially a printed-circuit board made out of ceramic, with 63 layers of wiring inside. It has over 80,000 connections on the top to the integrated circuits, 2 million vias, 400 meters of internal wiring, and 2772 pins on the bottom.

The TCM opened up, showing the chips inside. Most of the chips are the bipolar logic chips described in this blog post. The two slightly-smaller dies on the left are probably also logic chips. The 16 reddish rectangular chips are 128-kilobit static RAM chips. Six of the 121 positions are unused. The small reddish components between the chips are decoupling capacitors. From EEVBlog Flickr album, © Dave Jones, used with permission.

The manufacturing process for the ceramic substrate was very complex. Each ceramic sheet, the thickness of two sheets of paper (0.2mm), has tens of thousands of via holes punched in it. Next, the wiring was applied in the form of a molybdenum metal paste, forming wires just 100µm wide. The stack of 63 sheets was then laminated under heat and pressure. Next, the stack was sintered at 600°C to decompose the polymer binder, followed by hydrogen treatment at 1560°C for densification. During this process, the substrate shrank by 17%, but the millions of vias must remain aligned. After trimming and polishing, two layers of thin-film wiring were placed on top of the substrate. (The thin-film wiring allowed wiring changes to be made to the module for bug fixes.)5 Finally, the module was protected with a layer of polyimide film, with thousands of openings burned in it with a laser for the chip's connections.

The bipolar logic chip

Most of the chips on the TCM are bipolar logic chips; these are the square black chips in the previous photo. The die photo below shows one of these logic chips, 6.5mm on a side.8 This chip has an unusual appearance because it was connected directly to the substrate instead of the typical approach of putting pads around the perimeter with bond wires attached. The black circles are the 549 solder balls in a 27×27 grid that connect the chip to the substrate. Of these connections, 228 of these are used for signals, while 321 are used for power. The chip is covered with metal conductors that connect the solder balls to the circuitry underneath.

Die photo of a bipolar logic chip, showing the solder balls. (Click for a larger version.)

The chip is built from a type of transistor called the bipolar transistor, an older type of transistor than the MOS transistors in modern processors. The transistors in this chip used a cutting-edge design with a complex internal structure.6 IBM used bipolar transistors because they provided higher performance at the time, but they had the disadvantages of using higher power and taking up more area on a chip. (This is why the chip needed 321 connections for power and why the ES/9000 required multi-chip modules with a complex cooling system.) The chip contains approximately 85,000 transistors, 40,000 resistors, 10,000 capacitors, and 1000 Schottky diodes. While this may seem like a large number, contemporary CMOS microprocessors (such as the Intel 486) contained over a million transistors, illustrating the much higher density of MOS transistors.7

As shown in the closeup photo below, the chip has four layers of metal wiring on top of the silicon, a lot of layers for the time. The metal layer on top of the chip (called M4) provides power and signal distribution from the solder bumps. Underneath, layer M3 provides horizontal wiring: thick lines to distribute power across the chip and thin lines for signals. Layer M2 provides vertical wiring for both power and signals. The bottom layer (M1) implements the local wiring of the gate circuitry, connecting the transistors and resistors together. The narrowest metal lines are 1.6µm wide. Power distribution uses a hierarchy: the numerous solder balls feed power into the very wide power lines in the top metal layer. These are interconnected with the wide horizontal lines, which connect to the thinner vertical lines, which connect to the circuitry. This hierarchy ensures that voltage drop is minimized across the chip, while providing the multi-amp current it requires.

The chip has four layers of metal. The silicon circuitry is visible underneath, somewhat obscured by the multiple layers of insulating silicon dioxide and silicon nitride on top.

The architecture of the chip is IBM's "master slice" approach, building the chip from a gate array of identical cells. To avoid the expense of creating fully-custom chips, IBM built the various logic chips from a common grid of cells that was customized by the wiring on top. In the photo above, you can see some of these cells underneath the metal. The master cell approach has the disadvantage of being less dense than a custom chip. It turns out that roughly half of the cells in each logic chip went unused because the number of I/O pins on the chip was too small.12 You can see that most of the cells are unused in the photo above; while the transistors and resistors are present, they aren't connected to anything.

The chip contains 5240 cells, capable of implementing 2620 DCS logic gates. The structure of a cell is shown below. The cells are very flexible: each cell can implement one gate in the ECL (Emitter-Coupled Logic) family,9 two gates in the NTL (Non-Threshold Logic) family,10 or half a gate in the DCS (Differential Current Switch) family (which this chip uses). The key components are the transistors, which I've colored blue. The resistors are colored yellow.11 At the top are two large capacitors (red). The capacitors are unused in this DCS circuitry, but can be used to speed up ECL gates.

Diagram of the cell layout used by the chip. From patent EP0493989A1.

The image below shows six of the chip's 5240 cells after removing the metal layers from the chip. You can see how the layout matches the diagram above. (The cells in the middle are upside down.)

Closeup of the logic cells. I stacked multiple photos after removing the metal layers to get this image.

The logic chips are fabricated with a special technique that allows hundreds of different types of logic chips to be produced from a single set of masks. The transistors and other components in the silicon "master slice" are constructed using masks and photolithography as in most integrated circuits. However, the metal layers are patterned using direct-write electron beam lithography, rather than masks. This electron beam is steered to "write" the desired metal layer patterns on the die to produce the desired type of chip. In other words, the basic pattern of the chip is created using masks, but then the different chip types are manufactured directly from the design files, providing flexibility.

The photo below shows the entire die after dissolving the metal layers. This image shows the grid of cells, as well as three vertical rows holding 360 I/O cells.13 The grid pattern is most clear in the upper-right corner, where I sanded the die down. (Due to the difficulty of removing four layers of metal as well as layers of silicon nitride, I couldn't get the die as clean as I like.)

Die after removing the metal. The rounded corners are from my mechanical planarization processing (by which I mean sanding with 600-grit sandpaper). The original die was not rounded.

Differential Current Switch logic (DCS)

The chip is built with an uncommon logic family called DCS (Differential Current Switch).15 As the name suggests, DCS operates on differential signals: each input signal is expressed by two wires carrying both the signal and its complement. The voltage difference between the two wires represents a 0 or 1. Thus, a three-input logic gate will have six input wires, as well as two output wires.

Most logic families implement a NAND or NOR gate as their basic gate. The basic DCS gate, however, is the SELECT operation: it outputs either input A or input B, selected by the S input. In other words, SELECT implements the function if S then A else B, or in Boolean logic, SA+S'B. The SELECT operation is surprisingly flexible; with appropriate inputs, it can implement AND, XOR, or even a latch.14

A SELECT gate is shown below at the conceptual level. Three toggle switches are controlled by the S, A, and B inputs. These switches will pull one output to ground, while the other output will be pulled high by a resistor. Starting at the bottom, the S switch will direct the ground current to either the "A" side or the "B" side. With the switches in the indicated positions, the output will be pulled to ground, while the complemented output remains high. But if input A is set to 1, the output levels reverse, with the output pulled high. Now, suppose input S is set to 0, so the current is directed to the B side. In this case, the output is controlled by switch B. You can verify that the output matches A if S is 1 and matches B if S is 0. In other words, the circuit selects between inputs A and B, depending on the value of S. Note that this circuit generates differential outputs: both the output and its complement.

Conceptually, a DCS gate consists of toggle switches that pull one output high and the other low.

Next, I'll describe how the current switch is implemented with a pair of transistors. At the bottom, a current sink generates a fixed current, which can be switched to either the left side or the right side of the circuit. The idea is that the transistor with a higher input voltage will direct the current to that side, pulling that output low. Thus, the circuit acts like a toggle switch. An important feature of the circuit is that it provides a high degree of amplification: a slight difference in voltages is enough to switch most of the current to one side. (This circuit is essentially the same as the differential amplifier used in an op-amp.) As a result, a voltage swing of just 200 millivolts is enough to distinguish a logical 0 and 1, reducing power consumption. Another important feature of this circuit is that it is activated by the difference between the input voltages, so it is relatively insensitive to electrical noise. In other words, a voltage fluctuation that affects both inputs will cancel out, rather than causing an erroneous 0 or 1.

A 1 input switches the current through the transistor on the left. A 0 input switches the current through the transistor on the right.

The schematic below shows the implementation of a DCS gate. The three green boxes are current switches, using transistor pairs as described above. The yellow boxes are buffer circuits, called emitter followers. Two emitter followers buffer the outputs, while two more are used on the select inputs. Finally, the blue box is the current sink circuit, providing the fixed current that gets switched by the circuit.

Components of a DCS gate.

The diagram below shows this circuit in action. Starting at the bottom, the S input switches the current to the left. The A input then switches the current to the right. This current pulls the complemented output low, while the pull-up resistor pulls the output high. Note that a 0 input on A would switch the current to the other side, and thus switch the output. The B input has no effect since the current bypasses the B side of the circuit. Pulling the S input low, however, would switch the current to the B side, causing the B input to control the output. Thus, this circuit implements the SELECT operation.

Schematic of a SELECT gate, showing how the current is steered.

Reverse-engineering a DCS gate

In this section, I'll look at how a SELECT gate is implemented on the chip. The diagram below zooms in on a corner of the die, and then zooms again on one logic gate, the rectangle at the bottom. As you can see, each logic gate is very small on the die. Because this gate is at the edge of the die, it has less wiring over it so it is easier to see. Even so, the wiring layers on top partially obscure it. A DCS gate is created from four half-cells; I've highlighted the one I will discuss.

Starting from the die, zooming in on a corner and then a cell logic gate.

The components on the die can be matched against the diagram below. As before, the transistors are colored blue, the resistors yellow, and the unused capacitor red.

A half-cell as shown in the patent.

Below, I've indicated some of the components in the previously-highlighted half-cell. The wiring on the bottom metal layer customizes this cell for a particular function. Looking at this wiring, you can see that the emitters (E) of transistors T-5 and T-6 are connected, as are the emitters of transistors T-7 and T-8. The collectors (C) of transistors T-6 and T-8 are connected to the base of the output transistor T-12. The collector of transistor T-7 is connected to resistor R3. The wiring in the upper metal layers is shadowy and less clear. The vertical wiring along the sides provides power to the circuit. Other faint vertical wires are connected to the bases of transistors T-7 and T-8.

A half-cell as it appears on the die, with components labeled. "B" is a transistor base, "E" emitter, and "C" collector.

By studying the die closely, I traced out the circuitry for the gate and found it was a SELECT gate. The schematic below is from the patent; I modified it to match the gate I traced out. Note that IBM used its own symbol for a transistor as I've indicated at the bottom. I've marked the transistors and resistors from the photo above in red. The circuit has six transistors for testing, in the blue box.16 As you can see, one DCS gate takes a lot of components: 17 transistors and 18 resistors. This is one reason the density of the bipolar logic chips is so low.

Schematic of the DCS logic gate, as implemented on the chip. Vcc and Vee are the power supplies for the collector and emitter respectively. Vx controls the current sink. Vt is the pull-down voltage for the emitter-followers, but I'm not sure what Vt stands for. The original schematic was for an AND gate; I modified it to show a SELECT gate.

This shows the circuitry of one logic gate. Larger functional blocks such as adders were constructed by combining multiple gates. The full computer contains hundreds of thousands of these gates, implementing the processor and its control circuitry.

Conclusion

This bipolar logic chip illustrates the advanced technology of the ES/9000 mainframe.17 IBM pushed the limits of technology in everything from integrated circuit construction to ceramic modules to cooling systems. After all this effort, however, sales of the ES/9000 were underwhelming and couldn't slow the advance of microcomputers. Two years after the announcement, IBM had installed about 3600 of them, largely the lower-end models.18 In comparison, about 20 million personal computers were being sold per year, about 10,000 times the volume. Mainframes were 21.6% of computer industry revenue and dropping, less than half of personal computer revenue (44.5% of the industry). In 1997, IBM's bipolar processors reached the end of the road as IBM fully moved to CMOS processors.

I announce my latest blog posts on Twitter, so follow me @kenshirriff. I also have an RSS feed.

If you're interested in the TCM, you should definitely watch Dave Jones' teardown video below, as well as the videos where he attempts to remove the chips with hot air and a heating plate before finally succeeding. Thanks to Dave for sending me the chips as well as letting me use his photos.

Notes and references

1. For more information, the IBM ES/9000 type 9121 was described in detail in a series of articles in the IBM Journal of Research and Development, May 1991. The most relevant articles: The IBM Enterprise System/9000 Type 9121 air-cooled processor describes how the processor was implemented, Differential current switch—High performance at low power describes the Differential Current Switch logic, IBM System/390 air-cooled alumina thermal conduction module describes the structure and manufacturing of the TCM in detail, IBM Enterprise System/9000 Type 9121 Model 320 air-cooled processor technology. The Sept 1992 issue has other relevant articles, including A four-level VLSI bipolar metallization design with chemical-mechanical planarization, Improved performance of IBM Enterprise System/9000 bipolar logic chips. Also see The Design of the ES/9000 module and High performance packaged electronics for the IBM ES9000 mainframe. IBM's announcement of the ES/9000 provides a good summary. ↩

2. It's a bit tricky to keep track of IBM's naming and numbering schemes. The first distinction is between the architecture and the computers that implement the architecture. Enterprise Systems Architecture/390 (ESA/390) was IBM's mainframe architecture for the 1990s, continuing the path from System/360 and System/370. The ESA/390 architecture was implemented by several families of computers, including ES/9000, the CMOS-based 9672 Parallel Enterprise Server, the microprocessor-based Enhanced S/390 MicroProcessor Complex, S/390 Integrated Server, and S/390 Multiprise. The ES/9000 had three main processor types: the low-end CMOS 9221 in an air-cooled rack, the midrange 9121 in an air-cooled frame, and the large water-cooled 9021. (Confusingly, bigger numbers indicate a smaller system.) The 9121, the processor type in the middle, is the one I'm discussing in this blog post. Each processor type had several model numbers, as described below.

   

   The different ES/9000 models, from the reference guide. The two-way and three-way multiprocessors are called "dyadic" and "triadic".

   The ES/9000 family covered an enormous range of performance levels; the largest model provided over 100 times the performance of the smallest. The sizes varied widely too. The rack-mounted 9221 was designed for an office and took about 6 square feet of floor space, while the 9121 in the middle was roughly refrigerator-sized, occupying 15 to 24 square feet. The water-cooled 9021 was the classic room-filling mainframe, sized at 88 to 180 square feet. Roughly speaking the low-end ES/9000 9221 was a replacement for the IBM 9370 office-environment "super-mini computer", the air-cooled ES/9000 9121 was a replacement for the IBM 4381, while the water-cooled ES/9000 9021 was a replacement for the larger IBM 3090 systems. ↩

3. IBM was a leader in industrial design, from their computers to the architecture of their buildings and even their logo, as discussed in the book The Interface: IBM and the Transformation of Corporate Design. In the 1950s and 1960s, the design for IBM's computers concealed the internal circuitry, rather than showing it off like many other systems. Instead, IBM expressed the "inherent drama" of computing through spinning tape drives and other peripherals.

   

   A large ES/9000 installation with the water-cooled 9021 processor. From IBM ESCON brochure.

   My opinion is that IBM's design style fell apart in the 1980s with the loss of dramatic consoles and tape drives, leaving just the featureless boxes. To make things worse, these boxes were stripped of subtle detailing such as their pedestal bases and accent trim, leaving units that wouldn't look out of place in a Soviet paper mill. The ES/9000 won a 1991 design award, however, so some people must like the design more than I do. ↩

4. The ES/9000 had five TCMs. The Central Processor Element (CPE) is the microcoded CPU, the module that executes the instructions. The Buffer Control Element (BCE) implements a 64- or 128-kilobyte high-speed cache with error correction, and also handles virtual memory. The System Control Element (SCE) manages the flow of data between the different parts of the computer. (The System Control Element is especially important in a two- or three-processor system.) The Channel Control Element (CCE) controls the I/O channels and is essentially a separate I/O processor. The system can also have an optional Vector Control Element (VCE) for vector arithmetic.

   I was unable to conclusively determine the function of Dave's TCM. The large number (16) of memory chips suggests the cache in the Buffer Control Element (BCE), but this paper says the BCE has 26 memory chips. Possibly the chips are holding the microcode for the Central Processor Element (CPE). ↩

5. The ceramic module has two layers of complex thin-film wiring visible on top. This wiring has a surprising purpose: it allows modifications and bug fixes to be made to the module. By cutting wires with a laser and attaching new wires, signals can be re-routed.

   

   Closeup of an IC location, showing the thin-film wiring on top. Courtesy of Dave Jones.

   IBM calls the modification of computer wiring an Engineering Change or EC. Back in the 1950s, an engineer could easily perform an engineering change by adding and removing wires from a mainframe's wire-wrapped backplane. The printed-circuit boards of the System/360 made changes more difficult, but IBM developed a special "delete" tool to drill out a trace on the circuit board, allowing modification.

   

   This diagram shows how an Engineering Change is made to an EC/9000 TCM. Parts of the thin-film wiring are cut with a laser, and a wire is attached to the special EC pads. From this paper.

   The introduction of the ceramic TCM raised the issue of how could engineering changes be made when the wiring was encased in ceramic. (Discarding the expensive module wasn't an attractive choice.) The solution was to put exposed wiring on the surface of the module, wiring that could be modified as necessary. This consisted of two layers of polyimide plastic (Kapton) with thin-film wiring. Instead of connecting the IC to the ceramic wiring directly, each chip signal went to an EC pad on the surface. The original trace could be vaporized with a laser, and a modification wire (gold-plated cadmium-copper alloy) ultrasonically bonded to the EC pad. The photo below shows a chip with some EC wires.

   

   Closeup of the module showing Engineering Change wires next to the die. The smaller reddish-brown objects are capacitors. Courtesy of Dave Jones.

   In some cases it was necessary to remove a chip from the TCM. As Dave Jones found, unsoldering a chip is very difficult due to the thermal mass of the TCM. IBM invented a focused infrared machine to unsolder a chip. It combined a vacuum chip pick-up tool and infrared heater, along with a bias heater underneath the substrate to heat the whole TCM. A special prism ensured alignment of the new chip while a "mirror substrate" provided temperature feedback. This illustrates how the development of the ES/9000 required the invention of new, specialized tools. ↩

6. These bipolar chips were created using an IBM technology called ATX-4 that achieved almost five times the density of IBM's earlier ATX-1 chips. IBM described three advanced features of these transistors. First, they used a polysilicon base contact self-aligned with the emitter, reducing stray capacitance by a factor of 3. Second, the transistors were surrounded by deep trenches that allowed transistors to be closely packed. Third, they used a very thin implant for the base and optimized doping for the collector. These features improved the density and performance of the transistors. ↩

7. It's interesting to compare the complexity of the bipolar chip with a CMOS microprocessor at the same time. I did some rough estimates of transistor and gate counts, comparing the ES/9000 to a contemporary microprocessor. Each bipolar chip has 85,000 transistors. A CMOS processor from 1991, such as the MIPS R4000, has 1,350,000 transistors, almost 16 times as many, showing the huge density advantage of MOS over bipolar.

   Looking at gates shows an even larger advantage for CMOS. The bipolar chip implements 2620 DCS gates, of which about half are used. For the CMOS processor, I'll estimate 6 transistors for a 3-input gate; subtracting the 16-kilobyte cache in the MIPS R4000 yields about 100,000 gates, a factor of 70 more than the bipolar chip.

   Comparing a 121-chip TCM to a microprocessor yields a different story, with a TCM a bit more complex than a microprocessor. The TCM has roughly 144,000 gates and 256 kilobytes of cache, compared to 100,000 gates and 16 kilobytes of cache for the microprocessor. Thus, my estimate is that a TCM has 44% more gates than a contemporary microprocessor. Taking into account the R4000's external, off-chip cache, the cache sizes are comparable. The ES/9000 uses five TCMs for the processor, which works out to about 7 times the gates of the R4000.

   

   Cross-section of a transistor in the IC. From Advancing the state of the art in high performance logic and array technology.

   The four metal layers of the chip are also highly advanced. The wiring in the chips is made from aluminum-copper alloy sandwiched with titanium to support high current density. The wiring layers are double-insulated with silicon dioxide and silicon nitride to prevent shorts from developing over time. Each layer of the chip is polished flat (planarized) with chemical-mechanical polishing. Even the vias between wiring layers are complex, created by a "damascene stud" method. The vias are constructed by creating holes with reactive ion etching, filling them with metal, and then polishing away excess metal. ↩

8. Here's a summary of the chip's parameters from IBM Enterprise System/9000 Type 9121 Model 320 air-cooled processor technology.

   

   Design parameters of the chip.

    ↩

9. If you're familiar with ECL (Emitter-Coupled Logic), DCS is similar except it uses differential inputs instead of reference-controlled inputs. Although ECL and DCS both use a current-switching differential amplifier, ECL inputs are compared to a reference voltage, rather than the complemented input. (In IBM's ECL circuitry, the reference voltage is ground, so a negative signal is a logic 0 and a positive signal is a logic 1.)

   The key performance benefit of ECL and DCS logic is that transistors are never completely turned on, i.e. saturated. A transistor is relatively slow to get out of saturation, so a logic family such as TTL that saturates transistors is slower. ↩

10. One fairly obscure logic family supported by the chip is NTL, Non-Threshold Logic. NTL is similar to ECL, but without the reference voltage and reference transistors. As a result, NTL gates don't switch on and off sharply, but change in a more analog fashion with the input voltage. One advantage of NTL is that it uses one half-cell instead of the two used by ECL, so you can fit more NTL gates on a chip. NTL also consumes less power than ECL. However, its performance was poorer and it was more sensitive to noise, so it was rarely used in the ES/9000. NTL is described in more detail in this patent. ↩

11. Each resistor has multiple taps (gray boxes) allowing 15 different resistance values to be obtained. Gates with various speed/power tradeoffs can be constructed by using different resistances: the DCS family supports high, medium, low, and ultra-low power gates. (Most of the circuitry is low and ultra-low power.) The 0.2 pf capacitors were used for ECL speedup and for delay elements. ↩

12. Why do these chips require so many solder balls? There's some theory behind it. In the 1960s, E. F. Rent at IBM noticed a relationship between the number of components in an integrated circuit and the number of pins it required. Specifically, as the number of components increased, the number of pins required also increased, according to a power law. This became known as Rent's rule. As IBM increased the complexity of the logic chips, the number of solder bumps increased correspondingly. Chips in the IBM 3080 computer (1980) had an 11×11 grid of solder balls, while the IBM 3090's chips (1985) had a 17×17 grid. (Numbers from this paper.) The chip I examined has a 27×27 grid, but since the chips were limited by the number of I/O connections and half the gate were unused, it seems that this was insufficient. ↩

13. The image below shows some cells from the chip's I/O circuitry. These cells have a different structure from the cells for the logic gates. These cells include larger transistors to provide the necessary output current.

    

    Die photo showing I/O cells. This die photo was formed from a stack of images.

     ↩

14. The SELECT operation (SA+S'B) can implement multiple operations. For instance, setting B=0 implements S AND A. (For this gate, the redundant transistors can be omitted.) Setting A=B' implements S XOR A. (XOR is inconvenient to implement in most logic families but simple with DCS). Wiring the output back to A results in a latch: when S is high, the output value is held, but when S is low, the latch is loaded from B. An inverter is trivial with DCS: because of the differential signaling, a signal can be inverted simply by switching the two lines. ↩

15. Curiously, IBM's articles about the ES/9000 expand the DCS acronym as both Differential Current Switch and Differential Cascode Current Switch. The term cascode refers to "a two-stage amplifier that consists of a common-emitter stage feeding into a common-base stage." Essentially, it refers to how DCS has two layers of switching transistors, compared to the single layer in a typical ECL gate. ↩

16. Each DCS gate has about 6 additional transistors for test purposes. The problem is how to detect a faulty logic gate. In most logic families, a faulty gate will typically have the output stuck at 0 or 1. By running various test sequences through the circuit, this stuck bit can be detected. However, since a DCS gate uses differential logic, it can end up with a fault where both differential outputs are approximately the same, for instance, if the current sink fails. This is difficult to detect with tests since it is unpredictable how this signal will be interpreted by other gates. This non-determinism makes it hard to detect a faulty gate. The solution is to add test circuitry to each gate. The test circuitry will force an indeterminate output to a 0 or a 1, depending on which test circuit is activated. This makes the tests deterministic and a faulty gate can be detected. This seems like a weird corner case, but it was important enough for IBM to add a substantial amount of circuitry to each gate. The test circuit is described in more detail in this patent. ↩

17. In the 1980s, IBM faced the problem that it was the reigning computer company with its advanced mainframes, but it was encountering competition from microcomputers. Although microcomputers were technically inferior and much less powerful, they were much cheaper and rapidly increasing in power. The book The Innovator's Dilemma is the classic guide to this sort of problem. Incumbents often ignore the risk from disruptive technologies but IBM took the "right" approach and developed the IBM PC (1981) to take advantage of microprocessors. Although the IBM PC was extremely successful, IBM lost control of the PC architecture and personal computers devoured the mainframe market. It will be interesting to see what happens to Intel in the analogous situation as ARM processors gain functionality and cut into the market for technologically-advanced x86 chips. ↩

18. According to Computerworld, the adoption of ES/9000 was slow, with 3600 installed almost two years after introduction. Of the installations, 47% were low-end rack-mounted systems, 36% were air-cooled frame systems, and 17% were high-end water-cooled systems. IBM had over half the mainframe market, well ahead of Fujitsu, Hitachi, and NEC. ↩