Section 1.2 Programs Are Translated by Other Programs into Different Forms 5 printf.o hello.c Pre- hello.s processor hello.i Compiler Assembler hello.o Linker hello Source (cpp) (cc1) (a8） (1d) Modified Assembly Relocatable Executable program source program object object (text) program (text) programs program (text) (binary) (binary） Figure 1.3 The compilation system. linux>gcc -o hello hello.c Here,the Gcc compiler driver reads the source file hello.c and translates it into an executable object file hello.The translation is performed in the sequence of four phases shown in Figure 1.3.The programs that perform the four phases (preprocessor,compiler,assembler,and linker)are known collectively as the compilation system. Preprocessing phase.The preprocessor(cpp)modifies the original C program according to directives that begin with the '#character.For example,the #include <stdio.h>command in line 1 of hello.c tells the preprocessor to read the contents of the system header file stdio.h and insert it directly into the program text.The result is another C program,typically with the.i suffix. Compilation phase.The compiler (cc1)translates the text file hello.i into the text file hello.s,which contains an assembly-language program.This program includes the following definition of function main: main: 2 subq $8,%r8p 3 movl $.LC0,%ed1 ” call puts 5 movl $o,%eax 6 addq $8,%rsp 7 ret Each of lines 2-7 in,this definition describes one low-level machine- language instruction in a textual form.Assembly language is useful because it provides a common output language for different compilers for different high-level languages.For example,C compilers and Fortran compilers both generate output files in the same assembly language. Assembly phase.Next,.the assembler (as)translates hello.s into machine- language instructions,packages them in a form known as a relocatable object program,and stores the result in the object file hello.o.This file is a binary file containing 17 bytes to encode the instructions for function main.If we were to view hello.o with a text editor,it would appear to be gibberish
Section 1.2 Programs Are Translated by Other Programs into Different Forms S hello.c Pre- hello.i Compiler processor (cc!) hello.s Assembler (as) printf.o hello.a Linker (ld) hello Source (cpp) Modified program source (text) program (text) Assembly program (text) RelocatBble object programs (binary) '---~ Executable object program (binary) Figure 1.3 The compilation system. linux> gee -o hello hello. c Here, the Gee compiler driver reads ~he source file hello. c and translates it into an executable object file hel~o. The translation is performed in the sequence of four phases shown in Figure 1.3. The programs that perform the four phases (preprocessor, compiler, assembler, and linker) are known collectively as the compilation system. • Preproce~~ing phase. The preprocessor ( cpp) modifies the original C program according to directives that begi;i with the '#' character. Fpr example, the #include <stdio .h> command in line 1 of hello. c tells the preprocessor to read the contents of the system head,er file stdio. h and insert it directly into the program text. The result is another C program, typically with the . i suffix. • Compilation phase. The compiler (eel) translates the text file hello. i into the text file hello. s, which contains an assembly-language program. This program includes the following definition of function main: 1 main: 2 sub'q $8, %rsp 3 movl $.LCO, %edi 4 call puts 5 movl $0, %eax 6 addq $8, %rsp 7 ret Each of lines 2-7 in. tllis definition describes one low-level \Ilachine- . language instruction in a textual form. Assembly language is useful because it provides JI cpmmon output language for different compilers for different high-level languages. For example, C compil<;rs and Fortran compilers both generate output files in the same assembly language. • Assembly phase. Nextdhe assembler (as) translates hello. s into machinelanguage instructions, packages them in a form known as a relocatable object program, and stores the result in the object file hello. o. This file is a binary file containing 17 bytes to encode the instructions for function main. If we were to view hello. o with a text editor, it would appear to be gibberish
6 Chapter 1 A Tour of Computer Systems Aside The GNU project Gcc is one of many useful tools developed by the GNU (short-for GNU's Not Unix)project.The GNU project is a tax-exempt charity started by Richard Stallman in 1984,with the ambitious goal of developing a complete Unix-like system whose source code is unencumbered by restrictions on how it can be modified or distributed.The GNU project has developed an environment with all the major components'of a Unix operating system,except for the kernel,which was developed separately by the Linux project.The GNU nvronment includes the EMAcseditor sccmiler,GD debugger assembler,linker,utilities for manipulating binaries,and other components.The Gcc compiler has grown to support many different languages,with the ability to generate code for many different machines.Supported languages include C,C++,Fortran,Java,Pascal,Objective:C,and Ada. The GNU project is a remarkable achievement,and yet it is often overlooked.The modern open- source movement(commonly associated with Linux)owes its intellectual origins to the GNU project's notion of free software ("free"as in "free speech,"not "free beer").Further,Linux owes much of its popularity to the GNU tools,which provide the environment for the Linux kernel. Linking phase.Notice that our hello program calls the printf function,which is part of the standard C library provided by every C compiler.The printf function resides in a separate precompiled object file called printf.o,which must somehow be merged with our hello.o program.The linker(1d)handles this merging.The result is the hello file,which is an executable object file(or simply executable)that is ready to be loaded into memory and executed by the system. 1.3 It Pays to Understand How Compilation Systems Work For simple programs such as hello.c,we can rely on.the compilation system to produce correct and efficient machine code.However,there are some important reasons why programmers need to understand how compilation systems work: .Optimizing program performance.Modern compilers are sophisticated tools that usually produce good code.As programmers,we do not need to know the inner workings of the compiler in order to write efficient code.However, in order to make good coding decisions in our C programs,we do need a basic understanding of machine-level code and how the compiler translates different Cstatements into machine code.For example,is a switch statement always more efficient than a sequence of if-else statements?How much overhead is incurred by a function call?Is a while loop more efficient than a for loop?Are pointer references more efficient than array indexes?Why does our loop run so much faster if we sum into a local variable instead of an argument that is passed by reference?How can a function run faster when we simply rearrange the parentheses in an arithmetic expression?
• -.:'"' - - ---- 6 Chapter 1 A Tour of Computer Systems r A;.;.; -~~;~NU project - ~--- -* - -- --1 ' Gee is one of many useful topls developed by the GNU (short-for GNU's Not Unix) project. The 1 GNU project is a tax-exempt charity started by Richard Stallman in 1984, with the ambitious goal of I developing a: complete Unix-like system whose source code is unencumbered by restrictfons on how it can be modified or ,distributed. The GNU project has developed a~ environment with all the major I components ·of a Urux operatmg system, except for the kernel, which was developed separately by 1· the Linux project. The GNU environment includes the EMACS editor, pee compiler, GDB debugger, assembler, linker, utilities for manipulating binaries, and other components. The qcc compiler has t grown to support many different languages, with the ability to generate code for many different machines. Supported languages.include C, C++, Fortran, Java, Pascal, Objective,C, and Ada. The GNU project is a remarkable achievement, and yet it is often overlooked. The modern open- ! source movement (commonly associated with Linux) owes its intellectual origins to the GNU project's notion of free software ("free" as in "free speech," not ''free beer"). Further, Linux owes much of its popularity to the GNU tools, which provide the environment for the Linux kernel. ~ ... _""" .. ,.,___.....,...,.,..,.._ ____ ..;,_._~---""-'---""~ • Linking phase. Notice that our hello program calls the printf function, which is part of the standard C library provided by every C compiier. The printf function resides in a separate precompiled object file called printf . o, which must somehow be merged with our hello. o program. The linker (ld) handles this merging. The result is the hello file, which is an executable object file (or simply executable) that is ready to be loaded into memory and executed by the system. 1.3 It Pays to Understand How Compilation Systems Work For simple programs such as hello. c, we can rely on.the compilation system to produce correct and efficient machine code. However, there are some important reasons why programmers need to understand how compilation systems work: • Optimizing program performance. Modern compilers are sophisticated tools that usually produce good code. As programmers, we do not need to know the inner workings of the compiler in order to write efficient code. However, in order to make good coding decisions in our C programs, we do need a basic understanding of machine-level code and how the compiler translates different C statements into machine code. For example, is a switch statement always more efficient than a sequence of if-else state'tnents? How much overhead is incurred by a function call? Is a while loop more efficient than a for loop? Are pointer references more efficient than array indexes? Why does our loop run so much faster if .we sum into a local variable instead of an argument that is passed by reference? How<:an a function run faster when we simply rearrange the parentheses in an arithmetic expression? '' '• \
Section 1.4 Processors Read and Interpret Instructions Stored in Memory 7 In Chapter 3,we introduce x86-64,the machine language of recent gen- erations of Linux,Macintosh,and Windows computers.We describe how compilers translate different C constructs into this language.In Chapter 5, you will learn how to tune the performance of yqur C programs by making simple transformations to the C code that help the-compiler do its job better. In Chapter 6,you will learn about the hierarchical nature of the memory sys- tem,how Ccompilers store data arrays in memory,and how your Cprograms can exploit this knowledge to run more efficiently. Understarrding link-time errors.In our experience,some of the most perplex- ing programming errors are related to the operation of the linker,especially when you are trying to build large software systems.For example,what does it mean'when the linker reports that it cannot resolve a reference?What is the difference between a static variable and a.global variable?What happens if you define two global.variables in different C files with the same name?What is the difference between a static library and a dynamic library?Why does it matter what order we list libraries on the command line?And scariest of all, why do some linker-related errors not appear until run time?You will learn the answers to these kinds of questions in Chapter 7. .Avoiding security holes.For many years,buffer overflow vulnerabilities have accounted for many of the security holes in network and Internet servers. These vulnerabilities exist because too few programmers understand the need to carefully restrict the quantity and forms of data they accept from untrusted sources.A first step in learning secyre programming is to understand the con- sequences of the way data and control information are stored on the program stack.We cover the stack discipline and buffer overflow vulnerabilities in Chapter 3 as part of our study of assembly language.We will also learn about methods that can be used by the programmer,compiler,and operating system to reduce the threat of attack. 1.4 Processors.Read and Interpret Instructions Stored in Memory At this point,our hello.c source program has been translated by the compilation system into an executable object file called hello that is stored on disk.'To run the executable file on a Unix system,we type its name to an application program known as a shell: linux>./hello hello,world linuk> 13 The shell.is a command-line interpreter that prints a prompt,waits for you to type a command-line;and then performs the command.If the first word of the command line does not correspond to a built-in shell command,then the shell
I I I I I I I I I I I I I I I I I I I I I I I I Section 1.4 Processors Read and Interpret Instructions Stored in Memory 7 In Chapter 3, we introduce x86-64, the machine language of recent generations of Linux, Macintosh, ,and Windows co111puters. We describe how compilers translate different C constructs into this language. In Chapter 5, you will learn ho)" to tune the perforlljance of yqur C progquns by making simple transforni,~ti'?9s tq}he C coC!e th,ai help the.compiler do its job better. In Chapter 6, you will learn about the hierarchical nature of the memory system, ho'w C compilers store data arrays in memory, and how your C programs can expl~it-this knowledge to run i;nore efficiently. • Understartding link-time errors. In our experience;·some of the most perplexing programming errors are related to the operation of the linker, especially whep you are trying to bµild large software systems. For example, what does it meal)'wherithe linker reports that it cannot resolve a reference? What is the diff'<);ence between a static variable and a.global variable? What happens if you define two glob,aJ.variables in dif{ere!Jt C files with the same name? What is the ~erence between a static library and a dynamic library? Why does it mat~~r wlj_at orqer 'Ye list libraries on the command line? And scariest of all, why do some· linker-related errors not appear until run time? You will !earn the answers to these kinds of questions in Chapter 7. • Avoiding security holes. For many years, buffer overflow vulnerabilities have accounted for many of the security holes in network and Internet servers. ., ']1ie~.'i ".11\ner~bilities i;xist pecause top few p,rogrammers understanq the need t\) carefully ~~strjct ti/,~, quantity and forms pf data they acci;ptfrom untrusted sources.,A fir,~~,:;tep,\n le<l;rning seciµ7 progr?,l'Il1)lil\g is to understand the conseq1;1enc~s of t~e wa:r, datp ~d. controj information are stored on the program stack. We cover the stack discipline and buffer overflow vulnerabilities in Chapter 3 'as part of our study of assembly language. We will also learn about methods that can be used by the programmer, compiler, and operating system to reduce the threat of attack. () . 1.4 Processors. Read and Interpret Instructions Stored-in Memory • • At this point, our hello. c source program has been translated by the compilation system into an executable object file called hello that is stored on disk. To run the executable file on a Unix system, we type its name to an application program known as a shell: linux> ./hello· hell~, world n linuX> ' ' The shell.is a command-line interpreter that prints a prompt, waits for you to type a command-line;·and then performs the command. If the first word of the command line does not correspond to a built-in shell command, then the shell
8 Chapter 1 A Tour of Computer Systems Figure 1.4 CPU Hardware organization Register file of a typical system.CPU: central processing unit, PC ALU ALU:arithmetic/logic unit, PC:program counter,USB: System bus Memory bus Universal Serial Bus. VO Main Bus interface bridge memory 1/0 bus 00-→ Expansion slots for other devices such USB Graphics Disk as network adapters controller adapter controller Mouse Keyboard Display hello executable Disk stored on disk assumes that it is the name of an executable file that it should load'and run.So in this case,the'shell loads and runs the hello program and then waits for it to terminate.The hello program prints its message to the screen and then terminates. The shell then prints a prompt and waits for the next input command line. 1.4.1 Hardware Organization of a System To understand what happens to our hello program when we run it,we need to understand the hardware organization of a typical system,which is shown in Figure 1.4.This particular pictureis modeled after the family of recent Intel systems,but all systems have a similar look and feel.Don'tworry about the complexity of this figure just now.We will get to its various details in stages throughout the course of the book. Buses Running throughout the system is a collection of electrical conduits called buses that carry bytes of information back and forth between the components.Buses are typically designed to transfer fixed-size chunks of bytes known as words.The number of bytes in a word (the word size)is a.fundamental system parameter that varies across systems.Most machines today have word sizes of either 4 bytes(32 bits)or 8 bytes (64 bits).In this book,we do not assume any fixed definition of word size.Instead,we will specify what we mean by a"word"in any context that requires this to be defined
I, '' I I I I ~- __ .. -- -- -·-- 8 Chapter 1 A Tour of Computer Systems Figure 1.4 CPU y Hardware organization of a typical system. CPU: central processing unit, ALU: arithmetic/logic unit, PC: program counter, USB: Sys{em bus Memoi'y buS • 110 /' r, h.) Main Universal Serial Bus. Bus interface Graphics adapter i Mouse Keyboard Display bridge "'I-_... -"'"I' memory ' 110 bus l;:xpansion slots for ~--,.-'"---~ other devices such as network" ~dapters hell:'O executable stol"ed on disk assumes that it is the name of an executable file that it should load' and run. So in this case, the"shell loads and runs the hello program and ihen"waits for it to terminate. The hello program pfints iis.inessage'to the screen and ihen terminates. 'The shell then prints a prompt aiid ":'.ait~ for the next input command line. 1.4.1 i-lardware Organization of a System To understand what happens to our hello program when we run it, we µeed to understand the hardware organization of a typical system, which is shown in Figure 1.4. This particular picture·fS inodel~d after the· family of recent Intei systems, but all system's have a similar look and feel. Don't ·worry about the complexity of this figure just now. We will get to its various details in stages throughout the course of the book. Buses Running throughout the system is a collection of electrical condµits called buses that carry bytes of information back and forth between the componepts. Buses are typically designed to transfer fixed-size chunks of bytes known as wor~. The number of bytes in a word (the word size) is aiundamental system parameter that varies across systems. Most machines todcyy have word-sizes of either 4 bytes (32 bits) or 8 bytes (64 bits). In this book, we do not assume any fixed definition of word size..Instead, we will specify what we mean by a "word" in any context that requires this to be defined. I 1 l . • ' I 1 I
Section 1.4 Processors Read and Interpret Instructions Stored in Memory 9 I/O Devices Input/output(I/O)devices are the system's connection to the external world.Our example system has four I/O devices:a keyboard and mouse for user input,a display for user output,and a disk drive (or simply disk)for long-term storage of data and programs.Initially,the executable hello program resides on the disk. Each I/O device is connected to the 1O bus by either a controller or an adapter. Thedistinction between the two is mainly one of packaging.Controllers are chip sets in the device itself or on the system's main printed circuit board (often called the motherboard).An adapter is a card that plugs into a slot on the motherboard. Regardless,the purpose of each is to transfer information back and forth between the I/O bus and an I/O device. Chapter 6 has more to say about how 1/O devices such as disks work.In Chapter 10,you will learn how to use the Unix I/O interface to access devices from your application programs.We focus on the especially interesting class of devices known as networks,but the techniques generalize to other kinds of devices as well. Main Memory The main memory is a temporary storage device that holds both a program and the data it manipulates while the processor is executing the program.Physically, main memory consists of a collection of dynamic random access memory (DRAM) chips.Logically,memory is organized as a linear array of bytes,each with its own unique address (array index)starting at zero.In general,each of the machine instructions that constitute a program can consist of a variable number of bytes. The sizes of data items that correspond to C program variables vary according to type.For example,on an x86-64 machine running Linux,data of type short require 2 bytes,types int and float 4 bytes,and types long and double 8 bytes. Chapter 6 has more to say about how memory technologies such as DRAM chips work,and how they are combined to form main memory. Processor The central processing unit (CPU),or simply processor,is the engine that inter- prets (or executes)instructions stored in main memory.At its core is a word-size storage device (or register)called the program counter (PC).At any point in time, the PC points at(contains the address of)some machine-language instruction in main memory.2 From the time that power is applied to the system until the time that the power is shut off,a processorrepeatedly executes the instruction pointed at by the program counter and updates the program counter to point to the next instruction. A processor appears to operate according to a very simple instruction execution model,defined by its instruction set architecture.In this model;instructions execute 2.PC is also a commonly used acronym for "personal computer."However,the distinction between the two should be clear from the context
Section 1.4 Processors Read and Interpret Instructions Stored in Memory 9 1/0 Devices Input/output (I/O) devices are the system's connection to the external world. Our example system has four I/O devices: a keyboard and mouse for user input, a display for user output, and a disk drive (or simply disk) fo~ long-t<;rm storage of data and programs. Initially, the executable hello program resides on the disk . . ,Each I/O device is connected to the IIO bus by either a controller or an adapter. 'Die lf\stinction between the two is mainly one of packaging. Controllers are chip sets in the device itself or on the system's main printed circuit board (often called the motherboarlfJ. An adapter is a card that plugs into a slot on the motherboard. Regardles~, the purpose of each is to transfer information back and forth between the I/O 0us 'and an iio device. Chapter 6 has more to say about how I/O devices such as disks work. In Chapter 10, you will learn how to use the Unix 110 interface to access devices from your application programs. We focus on the especially interesting class of devices known as networks, but the techniques generalize to other kinds of devices as well. Main Memory The main memory is a temporary storage device that holds both a program and the data it manipulates while the processor is executing the program. Physically, main memory consists of a collection of dynamic random access memory (DRAM) chips. Logically, memory is organized as a linear array of bytes, each with its own unique address (array index) starting at zero. In general, each of the machine instructions that constitute a program can consist of a variable number of bytes. The sizes of data items that correspond to C program variables vary according to type. For example, on an x86-64 machine running Linux, data of type short require 2 bytes, types int and float 4 bytes, and types long and double 8 bytes. Chapter 6 has more to say about how memory technologies such as DRAM chips work, and how they are combined to form main memory. Processor The central processing unit (CPU), or simply processor, is the engine that interprets (or executes) instructions stored in main memory. At its core is a word-size storage device (or register) called the program counter (PC). At any point in time, the PC points at (contains the address of) some machine-language instruction in main memory.2 From the time that power is applied to the system until the time that the power is shut off, a processor. repeatedly executes the instruction pointed at by the program counter and updates the program counter to point to the next instruction. A processor appears to operate according to a very simple instruction execution model, defined by its instruction set architecture. In this model; instructions execute 2. PC is also a commonly used acronym for "personal computer." However, the distinction between the two should be clear from the context