Simulating the IBM 360/50 mainframe from its microcode

The IBM System/360 was a groundbreaking family of mainframe computers announced on April 7, 1964. System/360 was an extremely risky "bet-the-company" project for IBM, costing over $5 billion, but the System/360 ended up as a huge success, setting the direction of the computer industry for decades. The S/360 architecture was so successful that it is still supported by IBM's latest mainframes, almost 60 years later. I'm developing a microcode-level simulator1 for the IBM System/360 Model 50 (link to the simulator); this blog post provides background to understand the Model 50 and the simulator.

Screenshot of the simulator running in a browser.

Screenshot of the simulator running in a browser.

The radical decision behind System/360 was to use a single architecture for the entire product line of computers.3 The name symbolized “360 degrees to cover the entire circle of possible uses.” Using a common architecture seems obvious now (e.g. x86), but prior to the System/360, IBM (like other computer manufacturers) produced multiple computers with entirely incompatible architectures.

Internally, the different System/360 models had completely different implementations to support a wide range of cost and performance levels: the fastest model was over 1000 times as powerful as the slowest. Low-end models used simple hardware and an 8-bit datapath while advanced models used wide datapaths, fast semiconductor registers, out-of-order instruction execution, and caches.2 Despite these internal differences, the models all looked the same to the programmer.

Architecture of System/3604

You might expect a computer architecture from the 1960s to be simple, but System/360 is remarkably complex, partly because it merged six computer families into one architecture. It is a 32-bit architecture that supports many datatypes. As well as 32-bit integers and half words, it supports decimal arithmetic on numbers up to 31 digits long. Floating-point arithmetic supports short (32 bit), long (64 bit), or extended (128 bit) values. The processor also supports character strings up to 256 bytes long.

The System/360 instruction set has about 100 different instructions and several addressing modes. Some of these instructions are straightforward arithmetic, logic, or control operations. Other instructions are more complex, such as the "character move" that copies up to 256 characters in memory, or the floating-point instructions.

One of the most complex instructions is "edit", which formats a sequence of decimal digits for printing, for example inserting commas, a minus sign, or decimal point; removing leading zeroes, or filling leading spaces with characters. The number 1234567 could be "edited" into the string "$***12,345.67" for printing on a check. Keep in mind that this is a single instruction, not a library function like printf.

IBM System/360 Model 50 control panel. The dataflow diagram in the upper right illustrates the system's internal design. Photo by Sandstein, CC BY-SA 3.0.

IBM System/360 Model 50 control panel. The dataflow diagram in the upper right illustrates the system's internal design. Photo by Sandstein, CC BY-SA 3.0.

The System/360 architecture also included I/O, defining IBM's "channel" architecture. A channel is a programmable I/O subsystem with its own instruction set. On larger systems, the channel was an independent unit connected to the computer. But smaller systems such as the Model 50 used the same microcode engine to run CPU programs and channel programs.

The point is that System/360 has a large and complex instruction set. A single instruction could result in hundreds of memory accesses and processing steps. The dense instruction set helped programmers to cram programs into the extremely limited core memory of the 1960s. However, the complex instruction set was a problem for the computer designer, who had to implement the complex circuitry to carry out these instructions. The solution was microcode.

The System/360 Model 50 in a datacenter. The console and processor are at the left. An IBM 1442 card reader/punch is behind the IBM 1052 printer-keyboard that the operator is using. At the back, another operator is loading a tape onto an IBM 2401 tape drive. Photo from IBM.

The System/360 Model 50 in a datacenter. The console and processor are at the left. An IBM 1442 card reader/punch is behind the IBM 1052 printer-keyboard that the operator is using. At the back, another operator is loading a tape onto an IBM 2401 tape drive. Photo from IBM.

Microcode

One of the hardest parts of computer design is creating the control logic that tells each part of the processor how to carry out each instruction. In 1951, Maurice Wilkes came up with the idea of microcode: instead of building the control circuitry from complex logic gates, the control logic could be replaced with code (i. e. microcode) stored in a special memory called a control store. To execute an instruction, the computer internally executes several simpler microinstructions, specified by the microcode. Microcode turns the processor's control logic into a programming task instead of a logic design task.5

Microcode played a key role in the success of the System/360, helping IBM produce a line of computers with the same instruction set architecture but widely different implementations. It also allowed a processor to support different instruction sets; System/360 machines could be backward compatible with customers' older machines6 so customers could keep their existing software. For these reasons, the System/360 computers used microcode unless there was a compelling reason not to.7

Another advantage of microcode is that it provides an easy way to fix design flaws and bugs in the field. Instead of modifying the hardware, a service engineer could replace the microcode with a new version. The photo below shows a copper sheet with microcode etched into it for the Model 50.

A replaceable BCROS sheet, holding 17,600 bits. Photo courtesy of Glenn's Computer Museum.

A replaceable BCROS sheet, holding 17,600 bits. Photo courtesy of Glenn's Computer Museum.

Microcode can be implemented in a variety of ways. Many computers use "vertical microcode", where a microcode instruction is similar to a machine instruction, just less complicated. The System/360 designs, on the other hand, used "horizontal microcode", with complex, wide instructions of up to 100 bits, depending on the model. These microinstructions were more like a collection of fields, each controlling low-level signals. This improved performance since multiple parts of the processor could be controlled in parallel.

Hardware of the Model 508

The Model 50 was roughly in the middle of the System/360 lineup, providing a powerful mainframe that could be used by a medium-sized business or university department. The Model 50 typically rented for about $18,000 - $32,000 per month (equivalent to $120,000-$200,000 a month in current dollars).

IBM S/360 Model 50. The console was attached to the main frame, about 5 feet deep. The storage frame and power frame are the black cabinets at the back. Photo from Pinterest.

IBM S/360 Model 50. The console was attached to the main frame, about 5 feet deep. The storage frame and power frame are the black cabinets at the back. Photo from Pinterest.

The Model 50 occupied three large cabinets, each 5 feet long, about 2 feet wide, 6 feet tall, and weighing nearly a ton each.9 The main frame, behind the console, contained the CPU, I/O channel circuitry, and the microcode storage. Behind this, the power cabinet contained the computer's power supplies. To the left, the cabinet at the back contained the main storage: one or two core memory modules, each with 128 kilobytes of memory. (I wrote in detail about the Model 50's core memory earlier.) The computer's cables ran under a raised floor to the I/O devices, which typically included tape drives, a card reader, printers, disk drives, I/O controllers, and so forth.

This diagram shows the three frames that made up the basic S/360 Model 50. Source: Model 50 Maintenance Manual page 138.

This diagram shows the three frames that made up the basic S/360 Model 50. Source: Model 50 Maintenance Manual page 138.

The System/360 processors weren't implemented with integrated circuits, but with SLT (Solid Logic Technology) modules, hybrid modules that contain a few transistors, diodes, and resistors. A typical module implemented a logic gate, so it takes many circuit boards full of modules to construct the processor.

A logic board using SLT modules. Each square metal can is a module.

A logic board using SLT modules. Each square metal can is a module.

Like most computers of the 1960s, the Model 50 used magnetic core memory, with a tiny ferrite ring to store each bit. The photo below shows a core plane that stores 32768 bits (along with 512 bits for I/O). A stack of 18 planes formed a 64-kilobyte memory module, with two parity bits.10

A Model 50 core plane is arranged as a grid of cores. The Y lines run horizontally. X and sense/inhibit lines run vertically. The sense/inhibit lines form loops at the top and bottom. Each of the four vertical pairs of blocks has separate sense/inhibit lines. Each core plane was about 10¾ × 6¾ × ⅛ inches.

A Model 50 core plane is arranged as a grid of cores. The Y lines run horizontally. X and sense/inhibit lines run vertically. The sense/inhibit lines form loops at the top and bottom. Each of the four vertical pairs of blocks has separate sense/inhibit lines. Each core plane was about 10¾ × 6¾ × ⅛ inches.

The Model 50's internal architecture

To the programmer, all processors within System/360 look the same; internal circuitry, however, may be entirely different.

It's important to keep in mind that the internal architecture of the Model 50 is very different from the architecture that the programmer sees.11 In particular, the processor's internal registers are invisible to the programmer. The programmer instead sees 16 general-purpose registers and 4 floating-point registers, but to the processor these are part of the 64-word local store, a small high-speed core memory.

The diagram below shows the complex data flow through the computer.12 The black boxes are internal registers; the processor has a surprisingly large number of registers, used for a variety of purposes. The internal components are connected by buses. Most of the internal communication is over the 32-bit buses, shown in black. The 8-bit "mover" bus is shown in gray.

This diagram shows the data flow through the IBM 360/50 and appears in the upper-right corner of the console. I drew this version since I couldn't find a clear photo of it.

This diagram shows the data flow through the IBM 360/50 and appears in the upper-right corner of the console. I drew this version since I couldn't find a clear photo of it.

The heart of the computer is the 32-bit adder, which performs addition. For subtraction, the argument is complemented by the True/Complement circuit (TC). The adder has an associated shifter to perform bit-shifts; this is especially important for multiplication, division, and floating-point calculations. Operating in parallel with the adder is the "mover", which operates on bytes. It can extract a byte from a 32-bit word, as well as manipulating 4-bit pieces of the byte. The mover also performs Boolean operations (AND, OR, XOR). (Unlike most processors, the Model 50 separates arithmetic and logical operations, instead of having an ALU perform both.)

The computer's main core-memory storage is on the left. To access memory, an address is put in the Storage Address Register (SAR). Data is then read or written through the Storage Data Register (SDR). To the left of main storage, is the Instruction Address Register (the Program Counter or PC in modern terms). At the top is the Local Store, 64 words of high-speed core memory that holds the programmer's registers as well as some internal storage. The local store is accessed through the Local Store Address Register (LSAR).

At the right are the I/O channels: the low-speed Multiplexor Channel and the high-speed Selector Channel. You can think of these as DMA (direct memory access) paths for I/O. The multiplexor channel communicates over an 8-bit bus through the mover, while the selector channel communicates over a 32-bit bus. Although the channels are conceptually separate from the processor, the channels use the same buses, circuitry, and microcode engine as the processor. This limits I/O performance compared to more advanced System/360 models that have independent circuitry for the channels.

An example of the microcode

As you can see, the processor has many registers and functional units. The microcode needs to control these components to carry out program instructions. The microcode architecture is very complex and takes over 100 pages to explain thoroughly,15 so I'm only able to scratch the surface here. Each microinstruction is 90 bits long and performs multiple tasks. In the documentation, IBM used an 11-line block to represent each microinstruction, showing all the activities that are taking place in parallel.

A sample microinstruction is shown below, part of the microcode that implements an add instruction. At this point, earlier microinstructions have fetched and decoded the instruction and put the arguments into the R and L registers. This microinstruction performs the actual 32-bit addition, but there's a lot more happening than just the addition.

One microinstruction, part of the integer addition code. This microinstruction is at micro-address 0220.

One microinstruction, part of the integer addition code. This microinstruction is at micro-address 0220.

Starting with the line "R+L→R" (red), this indicates that the ALU is taking inputs from registers R and L, and the result is going into the R register. In other words, the two arguments are added. The result R is stored into the desired programmer-visible register in local storage (blue). The processor registers FN and J select the address in local storage. Meanwhile, the SETCRALG line sets the Condition code register based on the sign (i.e. "algebraic" value) of the result, indicating if the result is positive, negative, or zero.

The line "BC⩝C" indicates that signed overflow is detected and used as the carry flag14 while CAR (yellow) indicates the microcode branches on this carry (overflow) value. Thus, the microcode will take one path if the addition was valid and a second error path if overflow occurred. A microinstruction can "emit" an arbitrary 4-bit value (green) which can be used in a variety of ways. In this case, the binary value 1000 is emitted, fed into the W register, and then the M register, for use by the next microinstruction. As you can see, the CPU performs many activities in parallel for one microinstruction, which increases the computer's performance.

All the activities of a microinstruction are encoded into a 90-bit word consisting of 28 fields.13 The microinstruction discussed above (micro-address 0220) is highlighted in the documentation below. A single microinstruction is very complex, which is why it takes an 11-line block of text to represent it.

Part of the microcode listing. The previously-discussed microinstruction is highlighted. Note that the micro-address 0220 matches the address in the upper-left corner of the microinstruction diagram.

Part of the microcode listing. The previously-discussed microinstruction is highlighted. Note that the micro-address 0220 matches the address in the upper-left corner of the microinstruction diagram.

The processor documentation contains hundreds of pages of microcode;16 one page of the floating-point multiply code is below. Each box is one microinstruction, and the lines between them indicate the complex control paths. I'm not going to explain this microcode,17 but I wanted to show its complexity.

Part of the floating-point multiply microcode. (Click for a larger view.) From ALD vol 18.

Part of the floating-point multiply microcode. (Click for a larger view.) From ALD vol 18.

The console

The discussion above has shown the complex internal architecture of the Model 50. The numerous lights and controls on the console19 provide a view into this internal state. There were three main uses for the console. The first use was basic "operator control" tasks such as turning the system on, booting it, or powering it off, using the controls in the lower section of the console. These controls were consistent across the S/360 line and were usually the only controls the operator needed. The three hexadecimal dials in the lower right selected the I/O unit that held the boot software. Once the system had booted, the operator generally typed commands into the system rather than using the console.

Control panel of the IBM System/360 Model 50. This panel has marginal check controls for auxiliary storage in the upper right, replacing the dataflow diagram.

Control panel of the IBM System/360 Model 50. This panel has marginal check controls for auxiliary storage in the upper right, replacing the dataflow diagram.

The second console function was "operator intervention": program debugging tasks such as examining and modifying memory or registers and setting breakpoints. The lights and toggle switches in the lower half of the console were used for operator intervention. The operator could enter a 24-bit address using the row of 24 toggle switches, and enter a 32-bit data value using the row of 32 toggle switches above. The lights allowed the contents of memory to be examined. With other switches, the operator could set a breakpoint, single-step through a program, and perform other debugging operations.

The third console function was system maintenance and repair performed by an IBM customer engineer. The customer engineering displays took up the top half of the console and provided detailed access to the computer's complex internal state. To save space, the Model 50 had four roller knobs on the right side, with 8 positions for each knob. Each knob position selected a different function for the row of 36 lights (32 bits plus parity). The legends above the lights rotate with the knobs, showing the meaning of each light. For example, one position would display the L register, while another position would display the current microinstruction. In the photo below, the upper roller and lights are displaying part of the microcode currently being executed (ROS = Read Only Store). The roller below shows some of the internal registers and counters.

Closeup of two rollers and the associated lights.

Closeup of two rollers and the associated lights.

Finally, the voltmeter and voltage control knobs in the upper left of the console were used by an IBM customer engineer for "marginal checking". By raising and lowering the voltage levels, borderline components could be detected and replaced before they caused problems.

The simulator

The simulator is at righto.com/360 and the code is on Github. I implemented the simulator in JavaScript so it can run in a browser. It runs a sample program by executing the Model 50's microcode, simulating each microinstruction and the hardware. Each microinstruction is displayed graphically, along with the current instruction, the registers, the local storage, and core memory. It displays the console lights accurately based on the internal state, on a zoomable virtual console. Each row of lights can display 8 different elements, which you can change by clicking on a roller. You can step also through the microcode, one microinstruction at a time.

This simulator is still under development so don't expect it to work perfectly. I also haven't implemented the toggle switches, so you can't enter a program from the console yet. I also need to implement the I/O system, which has its own registers and a different microcode format.

To build the simulator, I extracted the binary microcode from the listings using a custom OCR tool. I implemented the hundreds of micro-operations, which were tricky to get correct. While most micro-operations are simple operations such as moving a register to the bus, some microinstructions are much more complex, especially for floating-point operations.20 Another complication is that a microinstruction performs many tasks in parallel and it was hard to determine the exact order in which to perform them.

My eventual goal with the simulator is to move it into the physical world. Specifically, I plan to drive the lights on CuriousMarc's Model 50 control panel to make the panel operate accurately. We also plan to hook up his IBM tape drives and card reader so we can have all the pieces of a Model 50 mainframe working together, except for the processor itself. I plan to port the simulator to C so I can run it in a microcontroller to drive the physical console. An FPGA implementation is another possibility; this would provide the maximum speed, but would be harder to implement.

I announce my latest blog posts on Twitter, so follow me @kenshirriff for updates and future articles. I also have an RSS feed. Thanks to Richard Cornwell for discussion and data.

Notes and references

  1. My simulator is not particularly useful unless you really care about the microcode in the Model 50. If you want to run software on a simulated System/360, you probably want to use the Hercules system

  2. I'll briefly summarize some of the different implementations used in System/360 computers.

    The low-end Model 30 uses an 8-bit bus and ALU, so 32-bit operations take four steps. It uses 60-bit microcode.

    The Model 40 also has an 8-bit bus and ALU, but it has 16-bit registers and a 16-bit bus to memory, improving the performance. It has 60-bit microcode.

    The Model 50 (discussed in this blog post) has 32-bit registers, memory bus, and adder. It also has the 8-bit mover that can operate in parallel with the adder.

    The Model 65 has a 64-bit bus, and multiple adders (60 and 8-bit) that allow a floating-point fraction and exponent to be processed in parallel. It also has an 8-byte instruction buffer and external channels. It uses 100-bit microcode.

    The Model 75 has a 64-bit main adder, 8-bit exponent adder, 8-bit decimal adder, and a 24-bit addressing adder. It overlaps instruction fetching and execution, with 16 bytes of instruction prefetching and 8 bytes of data prefetching.

    The high-end Model 91 has an advanced superscalar architecture with out-of-order execution, instruction pipelining, and multiple arithmetic execution units. Higher models support memory interleaving for faster access: 2-way on the Model 65 up to 16-way on the Model 195.

    The models 44, 75, 91 and above used hardwired control instead of microcode to squeeze out more performance.

    As you can see, the System/360 line has a wide variety of implementations. At the low end, the hardware is kept to a minimum to reduce costs, while at the high end, more hardware boosts performance, with wider datapaths and multiple functional units providing parallelism. 

  3. The System/360 line didn't completely meet the goal of a compatible architecture. IBM split out the business and scientific markets on the low-end machines by marketing subsets of the instruction set. The basic instructions were provided in the "standard" instruction set. On top of this, decimal instructions (for business) were in the "commercial" instruction set and floating-point was in the "scientific" instruction set. The "universal" instruction set provided all these instructions plus storage protection (i.e. memory protection between programs). Additionally, cost-cutting on the low-end Model 20 made it incompatible with the S/360 architecture, and the Model 44 was somewhat incompatible to improve performance on scientific applications. 

  4. IBM defined the System/360 architecture in great detail in a document called the IBM System/360 Principles of Operation. It describes not only the instruction set, but also the datatypes, input/output model, the interrupt model, and even the basic structure of the system control panel. To learn more about System/360, see A Programmer's Introduction to the IBM System/360 Architecture, Instructions, and Assembler Language. A bunch of assembly examples are at rosettacode

  5. The primary benefit of microcode for IBM was economic. As described in Microprogram Control for System/360, the cost of a non-microcoded processor is roughly linear in the size of the instruction set. However, a microcoded system has a roughly fixed cost, with a small overhead for additional instructions. Thus, as instruction sets get more complex (as in System/360), there is a crossover point where microcode is more efficient. This is especially the case for smaller systems where the base cost is lower. The lower marginal cost also makes emulating other systems more feasible. The IBM System/360 was one of the first commercial computers to make extensive use of microcode. 

  6. Various System/360 machines supported compatibility features with earlier IBM computers including the 1401, 1440, 1620, 7070, 7074, 7080, 709, 7090, 7094. Generally, a smaller System/360 machine could replace a smaller IBM computer such as the 1401, while a larger mainframe such as the 7090 needed to be replaced by a larger System/360 computer such as the Model 65.  

  7. A few System/360 models did not use microcode. The Model 44 was designed as a high-performance computer for scientific applications, so it used hardwired control. The Model 85 was partially microcoded, while the Models 75 and 91 were completely hardwired. 

  8. The book IBM's 360 and Early 370 Systems describes the history of the S/360 in great detail. IBM lists data on each model, including dates, data flow width, cycle time, storage, and microcode size. Another list with model details is here. The article System/360 and Beyond has lots of info. A list of 360 models and brief descriptions is here. For information on the Model 50 specifically, see the Functional Characteristics manual, Field Engineering manuals, Wikipedia, photos here and here, CuriousMarc video

  9. For detailed dimensions of the System/360 components, see the Physical Planning Manual For more memory, another 1500-pound frame could be added to the Model 50, boosting it from 256 kilobytes of memory to 512 kilobytes. Up to four Large Capacity Storage units (IBM 2361) could be added, each providing two more megabytes. 

  10. I wrote in detail about the Model 50's core memory system here

  11. The quote is from System/360 Model 40 comprehensive introduction

  12. The Model 50 Field Engineering Diagram Manual contains the detailed data flow diagram below. This diagram corresponds to the diagram discussed earlier, but provides much more detail. In particular, it shows the exact bit widths of the various data paths and registers.

    The detailed data flow diagram. Click for a larger version.

    The detailed data flow diagram. Click for a larger version.

     

  13. The table below shows how a microinstruction is encoded into a 90-bit word.

    BitsNameMeaning
    0PParity
    1-3LUMover input left side
    4-5MVMover input right side
    6-11ZPROAR address (Read Only storage Address Register)
    12-15ZFROAR branch control
    16-18ZNAddress control field
    19-23TRAdder control
    24Unused
    25-27WSLocal store address control
    28-30SFLocal store functions
    31PParity
    32-34IVInvalid digit test
    35-39ALAdder latch gating
    40-43WMMover destination
    44-45UPByte counter function
    46MDMD counter control
    47LBL byte counter control
    48MBM byte counter control
    49-51DGLength counter
    52-53ULMover function left digit
    54-55URMover function right digit
    56PParity
    57-60CEEmit field
    61-63LXLeft adder input
    64TCTrue or complement control
    65-67RYRight adder input
    68-71ADAdder function control
    72-77ABA branch control
    78-82BBB branch control
    83Unused
    84-89SSStat setting control

    For channel instructions, the microcode format is slightly different since some of the fields need to control the channel circuitry. However, most of the fields are the same as for the CPU. The table below shows the microcode format for the channel; the highlighted entries are different from the CPU microcode.

    BitsNameMeaning
    0PParity
    1-3LUMover input left side
    4-5MVMover input right side
    6-11ZPROAR address
    12-15ZFROAR branch control
    16-18ZNAddress control field
    19-23TRAdder control
    24Unused
    25CSLocal storage address selector
    26-27SALocal storage address
    28-30SFLocal storage function
    31PParity
    32-34CTTiming signals to channel
    35-39ALAdder latch gating
    40-42WLMover destination
    43-46HCMultiplexor channel stat setting
    47-48CGControl signals to channel
    49-51MGMultiplexor channel gate control
    52-53ULMover function left digit
    54-55URMover function right digit
    56PParity
    57-60CEEmit field
    61-63LXLeft adder input
    64TCTrue or complement control
    65-67RYRight adder input
    68-70CLSelector channel adder latch tests
    71Unused
    72-77ABA branch control
    78-82BBB branch control
    83Unused
    84-89SSStat setting control
     

  14. When adding twos-complement signed numbers, an overflow occurs if the carry out of the most significant bit is different from the carry out of the second-most-significant bit. (I explain this in detail here.) IBM numbers the bits in a word "backward" with bit 0 the most significant. Thus, an overflow occurs if the carry from bit 0 XOR'd with the carry from bit 1 is nonzero. IBM uses ⩝ to indicate an exclusive or. Thus, CARRY(0) ⩝ CARRY(1) indicates an overflow, represented as BC⩝C in the microcode. 

  15. For a description of how the Model 50 microcode works, see the book "Microprogramming: Principles and Practices", S. Husson (1970), pages 295 to 411. Bitsavers has a lot of Model 50 documents, but not everything. If you have additional documentation, such as the IBM Automated Logic Diagrams, please let me know. 

  16. The Model 50's microcode listing is available in three volumes on bitsavers. The binary microcode listings are difficult to read with OCR because pages were printed on different printers; some use serif fonts and others use sans-serif fonts. I made my own OCR program designed to process binary, which was able to read the listings for the most part. The presence of parity in the microcode helped catch errors. 

  17. Ok, I'll give a brief explanation of that page of microcode, which is part of the implementation of floating-point multiplication. The implementation is designed with tradeoffs between speed, code length, and temporary memory usage. The idea is to multiply the multiplicand by the multiplier, kind of like long multiplication on paper, where you multiply a digit at a time and add the partial sums. This code processes a hex digit of the multiplier at a time, with a separate case for each digit. The multiplicand is multiplied by the digit and this is added to the running total, shifting as appropriate. To make this fast, multiples of the multiplicand are pre-computed. However, pre-computing 16 multiples (one for each hex digit value) would take too much temporary (local) storage. So the only pre-computed multiples are 1, 2, and 6, and these are combined for other digits. To multiply by the digit 7, for instance, the multiples for 1 and 6 are added. To multiply by the digit 4, the multiple for 6 is added and the multiple for 2 is subtracted.

    But what about multiplying by 9 through 15? The trick is to "borrow" 16 from the next-higher digit. For instance, to multiply by the digit 11, you borrow 16, subtract the multiple for 6, and add the multiple for 1. Then the value one less is used for the next digit to account for the borrow. Thus, all 16 possibilities can be handled by adding or subtracting at most two of the pre-computed values. With borrowing, the code needs to handle 32 cases; the included page implements 22 of these cases. This implementation makes multiplication rapid, but the microcode is complex with many paths. (There is also a bunch more code to handle the floating-point exponent, normalizing values, overflow, underflow, and so forth.) 

  18. Different System/360 models used a variety of methods to store microcode.18 An important feature of IBM's microcode storage was that the microcode could be replaced in the field. The low-end Model 25 held microcode in a 16-kilobyte section of core memory called Control Storage. The Model 30 used CCROS (Card Capacitor Read-only Store), storing the microcode on special metalized punch cards that were read capacitively. Transformer Read-Only Storage (TROS, below) was used by the System/360 Model 20 and Model 40. I wrote an article about microcode storage if you want more information.

    A TROS module from an IBM System/360 Model 20.

    A TROS module from an IBM System/360 Model 20.

    The Model 50 (as well as 65 and 67) stored microcode in BCROS (Balanced Capacitor Read-Only Storage), using copper-clad epoxy glass laminate boards, each 20″×8½″. Each sheet plane held 176 words of 100 bits, and the Model 50 used 16 sheets to store 2816 words. (Only 90 of the 100 bits in each word were used.) The data in BCROS was etched into the copper wiring (below). Each bit is represented by two squares: one connected to the upper wire and one connected to the lower wire (or vice versa), forming the balanced capacitors.

    Closeup of a BCROS sheet from a System/360 Model 50.

    Closeup of a BCROS sheet from a System/360 Model 50.

     

  19. The features of the system control panel were carefully defined in the System/360 Principles of Operation pages 117-121, providing a consistent operator experience across the S/360 line. (The customer engineering part of the panel, on the other hand, was not specified and wildly different across the product line.) Diagrams of S/360 consoles are at quadibloc. For more details on the consoles, see my article on System/360 consoles

  20. The micro-operation that caused me the most difficulty is ED*FP, which computes the difference between two exponents for floating-point, but also computes four floating-point flags including the sign depending on the type of operation. Not only is this operation complex, but I think there is a typo in the description.

    A description of the ED*FP micro-operation.

    A description of the ED*FP micro-operation.

    Another complex micro-operation is MLJK, which performs multiple actions as part of instruction decoding:

    Gate adder latch to L reg and M reg. Gate latch bits 12-15 to J reg. Gate latch bits 16-19 to MD counter. Turn off refetch stat.
    If latch bits 12-15 all zero, turn on stat 0. Otherwise turn off stat 0.
    If latch bits 16-19 all zero, turn on stat 1. Otherwise turn off stat 1.
    If latch bits 16-17 all zero, turn on one-syllable stat. Otherwise turn off one-syllable stat.
    If latch bits 0-1 equal 00, set ILC to 01.
    If latch bits 0-1 equal 01 or 10, set ILC to 10.
    If latch bits 0-1 equal 11, set ILC to 11. 

23 comments:

JRJ said...

The Edit instruction actually goes quite a bit further back, at least as far back as the IBM 1401.

Chris Bigos said...

Great work Ken! You may find this useful if you haven't already got it...

http://www.ibm360.info/2050%20FETOM%20CROS.pdf

Jay Maynard said...

What would really be cool is if you could get this to the point that OS/360 could be genned and run on it. (ISTR you could run MFT on a /50; MVT might be a bit much for it.) If you do get that far along, let me know and I'll be happy to help.

John Atkinson said...

Thank you Ken, I started our maintenance company with a contract on a 360 50 So it has a warm spot in my memory. This was in 1979 and there were still quite a number of 360 systems still operational. Great piece.

Ray Jarrett said...

During the 1960`s to 1990`s the FAA used the IBM 360/370(9020 system) for the nations air traffic control system.I was involved in the implementation of these systems. During the 1970s, I was an instructor for the FAA and I taught the micro code for the IBM 370. This was a beast of a computer system that run for 30 plus years to support air traffic control. Great article brings back alot of "core memories"

Unknown said...

WOW - I'm impressed. I'm probably one of the few that programmed a model 20. Short story, General Telephone had one that was just sitting there doing nothing and they wanted something to run on it. I guess to justify the thing. It's pretty easy once you know the other models. Ran TOS (I think). My only frustration was that the card reader required an EXTRA card for data vs the other card reader. I think I spent a week before I accidentally added one more card and my program worked.

Unknown said...

I had a site with a 360-50 and a 360-65, fortunately the only one that broke was the 65 and I could fix that.

If the 50 had gone down I would have had to call my father, Howard Thompson, he was Tech support for the company.

That site also had 2401 Tape Drives, 2314 DASD, and 1403-N1printers, and they probably had 2540 Card Reader/Punches, but I can't remember for sure.

I've also worked on other vintage IBM I/O.

We had one site with six 360-65 Mainframes, one had a 7080 emulator installed. That site also had a 1401. And I still remember looking inside a 729 Tape Drive and being amazed at how they drove the reels.

Unknown said...

I had a site with a 360-50 and a 360-65, fortunately the only one that broke was the 65 and I could fix that.

If the 50 had gone down I would have had to call my father, Howard Thompson, he was Tech support for the company.

That site also had 2401 Tape Drives, 2314 DASD, and 1403-N1printers, and they probably had 2540 Card Reader/Punches, but I can't remember for sure.

I've also worked on other vintage IBM I/O.

We had one site with six 360-65 Mainframes, one had a 7080 emulator installed. That site also had a 1401. And I still remember looking inside a 729 Tape Drive and being amazed at how they drove the reels.

Stephen Thompson

Cdp1802 said...

Great work as always! I am trying to make microcoding popular again for hobbyists, so I developed a horizontal microcode compiler, it is actually very similar in some ways to /360 one.

https://hackaday.io/project/172073-microcoding-for-fpgas

Bill Frantz said...

You mentioned, "The Model 30 used CCROS (Card Capacitor Read-only Store), storing the microcode on special metalized punch cards that were read capacitively."

I watched an IBM FE change the microcode of a Model 30 by taking the card and repunching it in a standard IBM keypunch.

nemo said...

This seems to be a document (at bitsavers) for Autmated Logic Diagrams.
If it applies to the 360 or your demands, I wouldn't be able to ascertain...

http://www.bitsavers.org/pdf/ibm/logic/SY22-2798-2_LogicBlocks_AutomatedLogicDiagrams_SLT,SLD,ASLT,MST_TO_Oct71.pdf

Mark Jeronimus said...

I'm surprised the quadibloc website still around today. I scraped it literal ages ago to preserve it for myself, as I think the pixel-art is just stunning!

Though it does start showing signs of age. Secure browsers refuse to plainly load it because http is insecure, and while https seems to be supported by quadibloc, the certificate is corrupt to boot.

Anonymous said...

My first job was in the operations department for a large oil company on the dual 360/50 systems. It was the first large system I became familiar with and taught myself the assembler and produced a few programs for fun, including one which produced an accurate yearly calendar under a picture of Brigitte Bardot sitting on a high stool in character graphics all printed out on the IBM 1403 line printers (which were very good). Everyone computer centre was printing out a new one of these in December and my name became mud with the operations manager for IBM so I was transferred to work on the Univac 1108 equipment instead.

The IBM 360/50 was a nice system with regular architecture but a terrible OS (MFT-II in my days). The Univac architecture was much less regular but was fast and has a super multitasking OS: Exec 8. In addition it had an interactive TTY terminal in the tape library, ostensibly for use by the tape librarian, but that job had been eliminated so I annexed the terminal to learn how the Univac worked. The additional benefit of this was that access to the tape library was retricted so I could play writing assembler programs and my supervisors usually couldn't get at me.

I lasted at the Big Oil company for about 2 years before I moved on and worked on Univac equipment for most of my career.

Unknown said...

Fantastic article, thanks!

On a related note, I used the model 40 microcode for the second example in this Wikipedia article:

https://en.wikipedia.org/wiki/Microsequencer

Kalle Pihlajasaari said...

How are the capacitive microcode memories read?

Unknown said...

The IBM/360 was IMHO a bit of a dissappointment-- the architecture as first released had the very clumsy base plus 12 bit offset addressing which made addressing anything over 4K bytes quite a pain.

Even the FORTRAN compiler would sometimes give up with a "BASE REGISTER OUT OF BOUNDS" error. Also the lower models did not have memory protection or a supervisor mode as such so they were pretty much limited to running one task and often crashing and needing a complete rebooting from tape.

The later FORTRAN compilers would by default allocate a HUGE symbol table which would swamp out everything out of core memory, leading to thrashing the disks constantly during a compilation. Most sites could not afford the 128K of memory that was needed to make things run semi swiftly.



JerseyBob said...

WOW! Sure brings back memories. I worked for IBM from 1967 to 2005. I was trained on the 360/50 in 1968. Eight weeks of school. I was also trained on the 360 models 25, 30, 91, and 195. Later on the 370/155, 158, 168 and 3032, 3033, 3081, 3083, 3084 and 3090. Then I transitioned to UNIX based servers. Quite a ride.

Unknown said...

Great article! The first time I coded on a mainframe was after graduating from college in 1987 on a 3090 in…what else…COBOL. 😁. I got into homebrew CPU several years ago, and my first design and build was a microcoded 16-bit CPU. I drew inspiration for it from the M68000 (instruction set) and DG MV10000. I could have never done that with hardwired logic. I’m presently laying out a PCB for an integer multiply coprocessor. That one uses a hardwired state machine. It’s only 7 states! 😆

I’m working on a new design, but this time I’m looking to S/360 for inspiration. It’s a much more ambitious design with ideas taken from Model 65, 75, and 91. It’s a 16/32-bit design and I’m hoping to have independent I- and E-units to overlap instruction/operand fetches with execution.

I found Bitsavers and have been devouring everything there I can find. For a 1960s machine, the 91 blows my mind. I like the 75 and 65. I’d particularly like to find the ALD diagrams for the 75 so I can better understand the descriptions in the 2075 FEMM. Still trying to get my head around how IBM uses triggers and latches. It seems like triggers are what we know today as SR flip-flops and latches are similar to today’s transparent latches.

Chris Bigos said...

For an explanation of Latches and Triggers see Figure 2.1 on page 115 and the related text.
http://ibm360.info/2065%20FETOM.pdf

CEO said...

The Space Shuttle AP-101/S was based on the 360!-architecture. I worked on FCOS, the flight computer OS for years and its assembler code and was all so cool. Low-level stuff. FCOS has a great macro implementation on top of the assembly language that simplified coding. At one time I found a bug in the microcode, the MHW, something that doesn’t much ever, and we had to implement workarounds prior to the “next launch” which put a lot of pressure on the team.

Thanks for these articles, they are fabulous.

Jay Maynard said...

CEO, you must have worked there about the same time I did. I was a VTAM syprog at Rockwell Shuttle Operations Company during the time they were working to return to flight after the Challenger disaster, though I left before they actually flew again. I was the guy responsible for RSOC's corporate network in Houston.

Unknown said...

Cool article. I have a few verilog projects that migith be of interest to you: https://github.com/mdw-at-linuxbox/x2150 is the control unit for the 1052-7 (including for the 360/50), https://github.com/mdw-at-linuxbox/x2821 is the control unit for the 1403 and 2540. These should be more or less complete and functional.

Also inspired by this article, I made a stab and I think I'm half-way through this: https://github.com/mdw-at-linuxbox/x2050/ . Still a long ways to go, but I think I more or less understand the multiplexor channel logic. Lots more to do overall and certainly not functional yet.

Joe Van Steen said...

Very much enjoyed your article and discussion of the internals of the 360. I learned to program on a 24K 360/25 running DOS in 1970, before moving to an environment that had 2 360/65's for production and a 360/50 for development, all originally running MFT. Then rapidly escalating to MVT and eventually MVS as soon as those became available. Our high end 360 was a 360/95 which was the first one I remember with a pipeline architecture; and prior to the Intel floating point debacle, a hardware multiple error when it was first introduced. (Diagnosed when saving account computations on the 95 wouldn't match the calculation on a 65 which did the next, or prior[?], step in processing.) Then we moved to the 370 and 390 series with their own OS peculiarities, but that old 360/65 MFT core code still continued to run, and similar code is still going today, in COBOL.

There's something to be said for stability of core frameworks over time.

Thank you for the history and details of the internals of the hardware side. The 360 assembler and macro language were my forte and made a lot of sense to be. (Software rather than hardware engineering.) I had a hard time translating that to PC's when the Intel 8008, 8080 and 8086 became popular because of the stack orientation of the assembler on the Intel versus the linked register system used on the 360 for chaining programs through a call structure. It's always fun to see through the looking glass to what was on the other side of those assembler instructions.