CONTENTS 13.3.3 Comparing Sorts 234 l3.4 Hard Problems··················· 235 13.4.1 Towers of Hanoi.. 236 l3.4.2 The Halting Problem.,,,.,..,.·...·.·.·.·.············ 239 l343 Conclusion········· 241
CONTENTS v 13.3.3 Comparing Sorts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 13.4 Hard Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 13.4.1 Towers of Hanoi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 13.4.2 The Halting Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 13.4.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Chapter 1 Computers and Programs Almost everyone has used a computer at one time or another.Perhaps you have played computer games or used a computer to write a paper or balance your checkbook.Computers are used to predict the weather, design airplanes,make movies,run businesses,perform financial transactions,and control factories. Have you ever stopped to wonder what exactly a computer is?How can one device perform so many different tasks?These basic questions are the starting point for learning about computers and computer programming. 1.1 The Universal Machine A modern computer might be defined as"a machine that stores and manipulates information under the con- trol of a changeable program."There are two key elements to this definition.The first is that computers are devices for manipulating information.This means that we can put information into a computer,and it can transform the information into new,useful forms,and then output or display the information for our interpretation. Computers are not the only machines that manipulate information.When you use a simple calculator to add up a column of numbers,you are entering information(the numbers)and the calculator is processing the information to compute a running sum which is then displayed.Another simple example is a gas pump.As you fill your tank,the pump uses certain inputs:the current price of gas per gallon and signals from a sensor that reads the rate of gas flowing into your car.The pump transforms this input into information about how much gas you took and how much money you owe. We would not consider either the calculator or the gas pump as full-fledged computers,although modern versions of these devices may actually contain embedded computers.They are different from computers in that they are built to perform a single,specific task.This is where the second part of our definition comes into the picture:computers operate under the control of a changeable program.What exactly does this mean? A computer program is a detailed,step-by-step set of instructions telling a computer exactly what to do. If we change the program,then the computer performs a different sequence of actions,and hence,performs a different task.It is this flexibility that allows your PC to be at one moment a word processor,at the next moment a financial planner,and later on,an arcade game.The machine stays the same,but the program controlling the machine changes. Every computer is just a machine for executing(carrying out)programs.There are many different kinds of computers.You might be familiar with Macintoshes and PCs,but there are literally thousands of other kinds of computers both real and theoretical.One of the remarkable discoveries of computer science is the realization that all of these different computers have the same power;with suitable programming,each computer can basically do all the things that any other computer can do.In this sense,the PC that you might have sitting on your desk is really a universal machine.It can do anything you want it to,provided you can describe the task to be accomplished in sufficient detail.Now that's a powerful machine!
Chapter 1 Computers and Programs Almost everyone has used a computer at one time or another. Perhaps you have played computer games or used a computer to write a paper or balance your checkbook. Computers are used to predict the weather, design airplanes, make movies, run businesses, perform financial transactions, and control factories. Have you ever stopped to wonder what exactly a computer is? How can one device perform so many different tasks? These basic questions are the starting point for learning about computers and computer programming. 1.1 The Universal Machine A modern computer might be defined as “a machine that stores and manipulates information under the control of a changeable program.” There are two key elements to this definition. The first is that computers are devices for manipulating information. This means that we can put information into a computer, and it can transform the information into new, useful forms, and then output or display the information for our interpretation. Computers are not the only machines that manipulate information. When you use a simple calculator to add up a column of numbers, you are entering information (the numbers) and the calculator is processing the information to compute a running sum which is then displayed. Another simple example is a gas pump. As you fill your tank, the pump uses certain inputs: the current price of gas per gallon and signals from a sensor that reads the rate of gas flowing into your car. The pump transforms this input into information about how much gas you took and how much money you owe. We would not consider either the calculator or the gas pump as full-fledged computers, although modern versions of these devices may actually contain embedded computers. They are different from computers in that they are built to perform a single, specific task. This is where the second part of our definition comes into the picture: computers operate under the control of a changeable program. What exactly does this mean? A computer program is a detailed, step-by-step set of instructions telling a computer exactly what to do. If we change the program, then the computer performs a different sequence of actions, and hence, performs a different task. It is this flexibility that allows your PC to be at one moment a word processor, at the next moment a financial planner, and later on, an arcade game. The machine stays the same, but the program controlling the machine changes. Every computer is just a machine for executing (carrying out) programs. There are many different kinds of computers. You might be familiar with Macintoshes and PCs, but there are literally thousands of other kinds of computers both real and theoretical. One of the remarkable discoveries of computer science is the realization that all of these different computers have the same power; with suitable programming, each computer can basically do all the things that any other computer can do. In this sense, the PC that you might have sitting on your desk is really a universal machine. It can do anything you want it to, provided you can describe the task to be accomplished in sufficient detail. Now that’s a powerful machine! 1
2 CHAPTER 1.COMPUTERS AND PROGRAMS 1.2 Program Power You have already learned an important lesson of computing:Soffware (programs)rules the hardware(the physical machine).It is the software that determines what any computer can do.Without programs,comput- ers would just be expensive paperweights.The process of creating software is called programming,and that is the main focus of this book. Computer programming is a challenging activity.Good programming requires an ability to see the big picture while paying attention to minute detail.Not everyone has the talent to become a first-class program- mer,just as not everyone has the skills to be a professional athlete.However,virtually anyone can learn how to program computers.With some patience and effort on your part,this book will help you to become a programmer. There are lots of good reasons to learn programming.Programming is a fundamental part of computer science and is,therefore,important to anyone interested in becoming a computer professional.But others can also benefit from the experience.Computers have become a commonplace tool in our society.Understanding the strengths and limitations of this tool requires an understanding of programming.Non-programmers often feel they are slaves of their computers.Programmers,however,are truly the masters.If you want to become a more intelligent user of computers,then this book is for you. Programming can also be loads of fun.It is an intellectually engaging activity that allows people to express themselves through useful and sometimes remarkably beautiful creations.Believe it or not,many people actually write computer programs as a hobby.Programming also develops valuable problem-solving skills,especially the ability to analyze complex systems by reducing them to interactions of understandable subsystems. As you probably know,programmers are in great demand.More than a few liberal arts majors have turned a couple computer programming classes into a lucrative career option.Computers are so commonplace in the business world today that the ability to understand and program computers might just give you the edge over your competition,regardless of your occupation. 1.3 What is Computer Science? You might be surprised to learn that computer science is not the study of computers.A famous computer scientist named Edsgar Dijkstra once quipped that computers are to computer science what telescopes are to astronomy.The computer is an important tool in computer science,but it is not itself the object of study. Since a computer can carry out any process that we can describe,the real question is What processes can we describe?Put another way,the fundamental question of computer science is simply What can be computed? Computer scientists use numerous techniques of investigation to answer this question.The three main ones are design,analysis,and experimentation. One way to demonstrate that a particular problem can be solved is to actually design a solution.That is. we develop a step-by-step process for achieving the desired result.Computer scientists call this an algorithm. That's a fancy word that basically means"recipe."The design of algorithms is one of the most important facets of computer science.In this book you will find techniques for designing and implementing algorithms. One weakness of design is that it can only answer the question What is computable?in the positive.IfI can devise an algorithm,then the problem is solvable.However,failing to find an algorithm does not mean that a problem is unsolvable.It may mean that I'm just not smart enough,or I haven't hit upon the right idea yet.This is where analysis comes in. Analysis is the process of examining algorithms and problems mathematically.Computer scientists have shown that some seemingly simple problems are not solvable by any algorithm.Other problems are in- tractable.The algorithms that solve these problems take too long or require too much memory to be of practical value.Analysis of algorithms is an important part of computer science;throughout this book we will touch on some of the fundamental principles.Chapter 13 has examples of unsolvable and intractable problems. Some problems are too complex or ill-defined to lend themselves to analysis.In such cases,computer scientists rely on experimentation;they actually implement systems and then study the resulting behavior. Even when theoretical analysis is done,experimentation is often needed in order to verify and refine the
2 CHAPTER 1. COMPUTERS AND PROGRAMS 1.2 Program Power You have already learned an important lesson of computing: Software (programs) rules the hardware (the physical machine). It is the software that determines what any computer can do. Without programs, computers would just be expensive paperweights. The process of creating software is called programming, and that is the main focus of this book. Computer programming is a challenging activity. Good programming requires an ability to see the big picture while paying attention to minute detail. Not everyone has the talent to become a first-class programmer, just as not everyone has the skills to be a professional athlete. However, virtually anyone can learn how to program computers. With some patience and effort on your part, this book will help you to become a programmer. There are lots of good reasons to learn programming. Programming is a fundamental part of computer science and is, therefore, important to anyone interested in becoming a computer professional. But others can also benefit from the experience. Computers have become a commonplace tool in our society. Understanding the strengths and limitations of this tool requires an understanding of programming. Non-programmers often feel they are slaves of their computers. Programmers, however, are truly the masters. If you want to become a more intelligent user of computers, then this book is for you. Programming can also be loads of fun. It is an intellectually engaging activity that allows people to express themselves through useful and sometimes remarkably beautiful creations. Believe it or not, many people actually write computer programs as a hobby. Programming also develops valuable problem-solving skills, especially the ability to analyze complex systems by reducing them to interactions of understandable subsystems. As you probably know, programmers are in great demand. More than a few liberal arts majors have turned a couple computer programming classes into a lucrative career option. Computers are so commonplace in the business world today that the ability to understand and program computers might just give you the edge over your competition, regardless of your occupation. 1.3 What is Computer Science? You might be surprised to learn that computer science is not the study of computers. A famous computer scientist named Edsgar Dijkstra once quipped that computers are to computer science what telescopes are to astronomy. The computer is an important tool in computer science, but it is not itself the object of study. Since a computer can carry out any process that we can describe, the real question is What processes can we describe? Put another way, the fundamental question of computer science is simply What can be computed? Computer scientists use numerous techniques of investigation to answer this question. The three main ones are design, analysis, and experimentation. One way to demonstrate that a particular problem can be solved is to actually design a solution. That is, we develop a step-by-step process for achieving the desired result. Computer scientists call this an algorithm. That’s a fancy word that basically means “recipe.” The design of algorithms is one of the most important facets of computer science. In this book you will find techniques for designing and implementing algorithms. One weakness of design is that it can only answer the question What is computable? in the positive. If I can devise an algorithm, then the problem is solvable. However, failing to find an algorithm does not mean that a problem is unsolvable. It may mean that I’m just not smart enough, or I haven’t hit upon the right idea yet. This is where analysis comes in. Analysis is the process of examining algorithms and problems mathematically. Computer scientists have shown that some seemingly simple problems are not solvable by any algorithm. Other problems are intractable. The algorithms that solve these problems take too long or require too much memory to be of practical value. Analysis of algorithms is an important part of computer science; throughout this book we will touch on some of the fundamental principles. Chapter 13 has examples of unsolvable and intractable problems. Some problems are too complex or ill-defined to lend themselves to analysis. In such cases, computer scientists rely on experimentation; they actually implement systems and then study the resulting behavior. Even when theoretical analysis is done, experimentation is often needed in order to verify and refine the
1.4.HARDWARE BASICS Output Devices CPL Input Devices Main Memory Secondary Memory Figure 1.1:Functional View of a Computer. analysis.For most problems,the bottom-line is whether a working,reliable system can be built.Often we require empirical testing of the system to determine that this bottom-line has been met.As you begin writing your own programs,you will get plenty of opportunities to observe your solutions in action. 1.4 Hardware Basics You don't have to know all the details of how a computer works to be a successful programmer,but under- standing the underlying principles will help you master the steps we go through to put our programs into action.It's a bit like driving a car.Knowing a little about internal combustion engines helps to explain why you have to do things like fill the gas tank,start the engine,step on the accelerator,etc.You could learn to drive by just memorizing what to do,but a little more knowledge makes the whole process much more understandable.Let's take a moment to "look under the hood"of your computer. Although different computers can vary significantly in specific details,at a higher level all modern digital computers are remarkably similar.Figure 1.1 shows a functional view of a computer.The central processing unit(CPU)is the "brain"of the machine.This is where all the basic operations of the computer are carried out.The CPU can perform simple arithmetic operations like adding two numbers and can also do logical operations like testing to see if two numbers are equal. The memory stores programs and data.The CPU can only directly access information that is stored in main memory(called RAM for Random Access Memory).Main memory is fast,but it is also volatile.That is, when the power is turned off,the information in the memory is lost.Thus,there must also be some secondary memory that provides more permanent storage.In a modern personal computer,this is usually some sort of magnetic medium such as a hard disk(also called a hard drive)or floppy. Humans interact with the computer through input and output devices.You are probably familiar with common devices such as a keyboard,mouse,and monitor(video screen).Information from input devices is processed by the CPU and may be shuffled off to the main or secondary memory.Similarly,when information needs to be displayed,the CPU sends it to one or more output devices. So what happens when you fire up your favorite game or word processing program?First,the instructions that comprise the program are copied from the(more)permanent secondary memory into the main memory of the computer.Once the instructions are loaded,the CPU starts executing the program. Technically the CPU follows a process called the fetch execute cycle.The first instruction is retrieved from memory,decoded to figure out what it represents,and the appropriate action carried out.Then the next instruction is fetched,decoded and executed.The cycle continues,instruction after instruction.This is really all the computer does from the time that you turn it on until you turn it off again:fetch,decode,execute.It doesn't seem very exciting,does it?But the computer can execute this stream of simple instructions with blazing speed,zipping through millions of instructions each second.Put enough simple instructions together in just the right way,and the computer does amazing things
1.4. HARDWARE BASICS 3 Input Devices CPU Secondary Memory Main Memory Output Devices Figure 1.1: Functional View of a Computer. analysis. For most problems, the bottom-line is whether a working, reliable system can be built. Often we require empirical testing of the system to determine that this bottom-line has been met. As you begin writing your own programs, you will get plenty of opportunities to observe your solutions in action. 1.4 Hardware Basics You don’t have to know all the details of how a computer works to be a successful programmer, but understanding the underlying principles will help you master the steps we go through to put our programs into action. It’s a bit like driving a car. Knowing a little about internal combustion engines helps to explain why you have to do things like fill the gas tank, start the engine, step on the accelerator, etc. You could learn to drive by just memorizing what to do, but a little more knowledge makes the whole process much more understandable. Let’s take a moment to “look under the hood” of your computer. Although different computers can vary significantly in specific details, at a higher level all modern digital computers are remarkably similar. Figure 1.1 shows a functional view of a computer. The central processing unit (CPU) is the “brain” of the machine. This is where all the basic operations of the computer are carried out. The CPU can perform simple arithmetic operations like adding two numbers and can also do logical operations like testing to see if two numbers are equal. The memory stores programs and data. The CPU can only directly access information that is stored in main memory (called RAM for Random Access Memory). Main memory is fast, but it is also volatile. That is, when the power is turned off, the information in the memory is lost. Thus, there must also be some secondary memory that provides more permanent storage. In a modern personal computer, this is usually some sort of magnetic medium such as a hard disk (also called a hard drive) or floppy. Humans interact with the computer through input and output devices. You are probably familiar with common devices such as a keyboard, mouse, and monitor (video screen). Information from input devices is processed by the CPU and may be shuffled off to the main or secondary memory. Similarly, when information needs to be displayed, the CPU sends it to one or more output devices. So what happens when you fire up your favorite game or word processing program? First, the instructions that comprise the program are copied from the (more) permanent secondary memory into the main memory of the computer. Once the instructions are loaded, the CPU starts executing the program. Technically the CPU follows a process called the fetch execute cycle. The first instruction is retrieved from memory, decoded to figure out what it represents, and the appropriate action carried out. Then the next instruction is fetched, decoded and executed. The cycle continues, instruction after instruction. This is really all the computer does from the time that you turn it on until you turn it off again: fetch, decode, execute. It doesn’t seem very exciting, does it? But the computer can execute this stream of simple instructions with blazing speed, zipping through millions of instructions each second. Put enough simple instructions together in just the right way, and the computer does amazing things
4 CHAPTER 1.COMPUTERS AND PROGRAMS 1.5 Programming Languages Remember that a program is just a sequence of instructions telling a computer what to do.Obviously,we need to provide those instructions in a language that a computer can understand.It would be nice if we could just tell a computer what to do using our native language,like they do in science fiction movies.("Computer, how long will it take to reach planet Alphalpha at maximum warp?")Unfortunately,despite the continuing efforts of many top-flight computer scientists(including your author),designing a computer to understand human language is still an unsolved problem. Even if computers could understand us,human languages are not very well suited for describing complex algorithms.Natural language is fraught with ambiguity and imprecision.For example,if I say:"I saw the man in the park with the telescope,"did I have the telescope,or did the man?And who was in the park?We understand each other most of the time only because all humans share a vast store of common knowledge and experience.Even then,miscommunication is commonplace. Computer scientists have gotten around this problem by designing notations for expressing computa- tions in an exact,and unambiguous way.These special notations are called programming languages.Every structure in a programming language has a precise form (its syntax)and a precise meaning (its semantics).A programming language is something like a code for writing down the instructions that a computer will follow. In fact,programmers often refer to their programs as computer code,and the process of writing an algorithm in a programming language is called coding. Python is one example of a programming language.It is the language that we will use throughout this book.You may have heard of some other languages,such as C++,Java,Perl,Scheme,or BASIC.Although these languages differ in many details,they all share the property of having well-defined,unambiguous syntax and semantics. All of the languages mentioned above are examples of high-level computer languages.Although they are precise,they are designed to be used and understood by humans.Strictly speaking,computer hardware can only understand very low-level language known as machine language. Suppose we want the computer to add two numbers.The instructions that the CPU actually carries out might be something like this. load the number from memory location 2001 into the CPU load the number from memory location 2002 into the CPU Add the two numbers in the CPU store the result into location 2003 This seems like a lot of work to add two numbers,doesn't it?Actually,it's even more complicated than this because the instructions and numbers are represented in binary notation (as sequences of Os and Is). In a high-level language like Python,the addition of two numbers can be expressed more naturally:c a b.That's a lot easier for us to understand,but we need some way to translate the high-level language into the machine language that the computer can execute.There are two ways to do this:a high-level language can either be compiled or interpreted. A compiler is a complex computer program that takes another program written in a high-level language and translates it into an equivalent program in the machine language of some computer.Figure 1.2 shows a block diagram of the compiling process.The high-level program is called source code,and the resulting machine code is a program that the computer can directly execute.The dashed line in the diagram represents the execution of the machine code An interpreter is a program that simulates a computer that understands a high-level language.Rather than translating the source program into a machine language equivalent,the interpreter analyzes and executes the source code instruction by instruction as necessary.Figure 1.3 illustrates the process The difference between interpreting and compiling is that compiling is a one-shot translation;once a program is compiled,it may be run over and over again without further need for the compiler or the source code.In the interpreted case,the interpreter and the source are needed every time the program runs.Compiled programs tend to be faster,since the translation is done once and for all,but interpreted languages lend themselves to a more flexible programming environment as programs can be developed and run interactively. The translation process highlights another advantage that high-level languages have over machine lan- guage:portability.The machine language of a computer is created by the designers of the particular CPU
4 CHAPTER 1. COMPUTERS AND PROGRAMS 1.5 Programming Languages Remember that a program is just a sequence of instructions telling a computer what to do. Obviously, we need to provide those instructions in a language that a computer can understand. It would be nice if we could just tell a computer what to do using our native language, like they do in science fiction movies. (“Computer, how long will it take to reach planet Alphalpha at maximum warp?”) Unfortunately, despite the continuing efforts of many top-flight computer scientists (including your author), designing a computer to understand human language is still an unsolved problem. Even if computers could understand us, human languages are not very well suited for describing complex algorithms. Natural language is fraught with ambiguity and imprecision. For example, if I say: “I saw the man in the park with the telescope,” did I have the telescope, or did the man? And who was in the park? We understand each other most of the time only because all humans share a vast store of common knowledge and experience. Even then, miscommunication is commonplace. Computer scientists have gotten around this problem by designing notations for expressing computations in an exact, and unambiguous way. These special notations are called programming languages. Every structure in a programming language has a precise form (its syntax) and a precise meaning (its semantics). A programming language is something like a code for writing down the instructions that a computer will follow. In fact, programmers often refer to their programs as computer code, and the process of writing an algorithm in a programming language is called coding. Python is one example of a programming language. It is the language that we will use throughout this book. You may have heard of some other languages, such as C++, Java, Perl, Scheme, or BASIC. Although these languages differ in many details, they all share the property of having well-defined, unambiguoussyntax and semantics. All of the languages mentioned above are examples of high-level computer languages. Although they are precise, they are designed to be used and understood by humans. Strictly speaking, computer hardware can only understand very low-level language known as machine language. Suppose we want the computer to add two numbers. The instructions that the CPU actually carries out might be something like this. load the number from memory location 2001 into the CPU load the number from memory location 2002 into the CPU Add the two numbers in the CPU store the result into location 2003 This seems like a lot of work to add two numbers, doesn’t it? Actually, it’s even more complicated than this because the instructions and numbers are represented in binary notation (as sequences of 0s and 1s). In a high-level language like Python, the addition of two numbers can be expressed more naturally: c = a + b. That’s a lot easier for us to understand, but we need some way to translate the high-level language into the machine language that the computer can execute. There are two ways to do this: a high-level language can either be compiled or interpreted. A compiler is a complex computer program that takes another program written in a high-level language and translates it into an equivalent program in the machine language of some computer. Figure 1.2 shows a block diagram of the compiling process. The high-level program is called source code, and the resulting machine code is a program that the computer can directly execute. The dashed line in the diagram represents the execution of the machine code. An interpreter is a program that simulates a computer that understands a high-level language. Rather than translating the source program into a machine language equivalent, the interpreter analyzes and executes the source code instruction by instruction as necessary. Figure 1.3 illustrates the process. The difference between interpreting and compiling is that compiling is a one-shot translation; once a program is compiled, it may be run over and over again without further need for the compiler or the source code. In the interpreted case, the interpreter and the source are needed every time the program runs. Compiled programs tend to be faster, since the translation is done once and for all, but interpreted languages lend themselves to a more flexible programming environment as programs can be developed and run interactively. The translation process highlights another advantage that high-level languages have over machine language: portability. The machine language of a computer is created by the designers of the particular CPU