Programming from the Ground Up
Jonathan Bartlett
Edited by
Dominick Bruno, Jr.
Programming from the Ground Up by Jonathan Bartlett Edited by Dominick Bruno, Jr. Copyright © 2003 by Jonathan Bartlett Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in Appendix H. In addition, you are granted full rights to use the code examples for any purpose without even having to credit the authors. All trademarks are property of their respective owners.
This book can be purchased at http://www.bartlettpublishing.com/
This book is not a reference book, it is an introductory book. It is therefore not suitable by itself to learn how to professionally program in x86 assembly language, as some details have been left out to make the learning process smoother. The point of the book is to help the student understand how assembly language and computer programming works, not to be a reference to the subject. Reference information about a particular processor can be obtained by contacting the company which makes it.
To receive a copy of this book in electronic form, please visit the website http://savannah.nongnu.org/projects/pgubook/ This site contains the instructions for downloading a transparent copy of this book as defined by the GNU Free Documentation License.
Table of Contents 1. Introduction........................................................................................................1 Welcome to Programming ...............................................................................1 Your Tools ........................................................................................................3 2. Computer Architecture .....................................................................................7 Structure of Computer Memory.......................................................................7 The CPU...........................................................................................................9 Some Terms ...................................................................................................11 Interpreting Memory......................................................................................13 Data Accessing Methods................................................................................14 Review ...........................................................................................................16 3. Your First Programs ........................................................................................19 Entering in the Program .................................................................................19 Outline of an Assembly Language Program..................................................22 Planning the Program.....................................................................................28 Finding a Maximum Value.............................................................................31 Addressing Modes .........................................................................................41 Review ...........................................................................................................45 4. All About Functions.........................................................................................49 Dealing with Complexity...............................................................................49 How Functions Work .....................................................................................50 Assembly-Language Functions using the C Calling Convention ..................52 A Function Example ......................................................................................59 Recursive Functions.......................................................................................64 Review ...........................................................................................................71 5. Dealing with Files.............................................................................................75 The UNIX File Concept.................................................................................75 Buffers and .bss ...........................................................................................76 Standard and Special Files .............................................................................78 Using Files in a Program................................................................................79
iii
Review ...........................................................................................................93 6. Reading and Writing Simple Records ...........................................................95 Writing Records ...........................................................................................100 Reading Records ..........................................................................................104 Modifying the Records ................................................................................111 Review .........................................................................................................114 7. Developing Robust Programs........................................................................117 Where Does the Time Go?...........................................................................117 Some Tips for Developing Robust Programs...............................................118 Handling Errors Effectively .........................................................................121 Making Our Program More Robust .............................................................123 Review .........................................................................................................126 8. Sharing Functions with Code Libraries.......................................................129 Using a Shared Library ................................................................................130 How Shared Libraries Work ........................................................................133 Finding Information about Libraries............................................................134 Useful Functions ..........................................................................................140 Building a Shared Library............................................................................141 Review .........................................................................................................143 9. Intermediate Memory Topics........................................................................147 How a Computer Views Memory ................................................................147 The Memory Layout of a Linux Program....................................................149 Every Memory Address is a Lie ..................................................................151 Getting More Memory .................................................................................155 A Simple Memory Manager ........................................................................157 Using our Allocator......................................................................................174 More Information.........................................................................................177 Review .........................................................................................................178
iv
10. Counting Like a Computer .........................................................................181 Counting.......................................................................................................181 Truth, Falsehood, and Binary Numbers .......................................................186 The Program Status Register .......................................................................195 Other Numbering Systems...........................................................................196 Octal and Hexadecimal Numbers ................................................................199 Order of Bytes in a Word .............................................................................201 Converting Numbers for Display .................................................................204 Review .........................................................................................................210 11. High-Level Languages .................................................................................213 Compiled and Interpreted Languages ..........................................................213 Your First C Program ...................................................................................215 Perl ...............................................................................................................218 Python ..........................................................................................................219 Review .........................................................................................................220 12. Optimization.................................................................................................223 When to Optimize ........................................................................................223 Where to Optimize.......................................................................................224 Local Optimizations.....................................................................................225 Global Optimization.....................................................................................229 Review .........................................................................................................230 13. Moving On from Here .................................................................................233 From the Bottom Up ....................................................................................234 From the Top Down .....................................................................................234 From the Middle Out ...................................................................................235 Specialized Topics .......................................................................................236 Further Resources on Assembly Language..................................................237
v
A. GUI Programming ........................................................................................239 B. Common x86 Instructions ............................................................................257 C. Important System Calls ................................................................................271 D. Table of ASCII Codes ...................................................................................275 E. C Idioms in Assembly Language .................................................................277 F. Using the GDB Debugger..............................................................................289 G. Document History .........................................................................................299 H. GNU Free Documentation License..............................................................301 I. Personal Dedication........................................................................................311 Index....................................................................................................................313
vi
Chapter 1. Introduction Welcome to Programming I love programming. I enjoy the challenge to not only make a working program, but to do so with style. Programming is like poetry. It conveys a message, not only to the computer, but to those who modify and use your program. With a program, you build your own world with your own rules. You create your world according to your conception of both the problem and the solution. Masterful programmers create worlds with programs that are clear and succinct, much like a poem or essay. One of the greatest programmers, Donald Knuth, describes programming not as telling a computer how to do something, but telling a person how they would instruct a computer to do something. The point is that programs are meant to be read by people, not just computers. Your programs will be modified and updated by others long after you move on to other projects. Thus, programming is not as much about communicating to a computer as it is communicating to those who come after you. A programmer is a problem-solver, a poet, and an instructor all at once. Your goal is to solve the problem at hand, doing so with balance and taste, and teach your solution to future programmers. I hope that this book can teach at least some of the poetry and magic that makes computing exciting. Most introductory books on programming frustrate me to no end. At the end of them you can still ask "how does the computer really work?" and not have a good answer. They tend to pass over topics that are difficult even though they are important. I will take you through the difficult issues because that is the only way to move on to masterful programming. My goal is to take you from knowing nothing about programming to understanding how to think, write, and learn like a programmer. You won’t know everything, but you will have a background for how everything fits together. At the end of this book, you should be able to do the following:
1
Chapter 1. Introduction •
Understand how a program works and interacts with other programs
•
Read other people’s programs and learn how they work
•
Learn new programming languages quickly
•
Learn advanced concepts in computer science quickly
I will not teach you everything. Computer science is a massive field, especially when you combine the theory with the practice of computer programming. However, I will attempt to get you started on the foundations so you can easily go wherever you want afterwards. There is somewhat of a chicken and egg problem in teaching programming, especially assembly language. There is a lot to learn - it’s almost too much to learn almost at once, but each piece depends on all the others. Therefore, you must be patient with yourself and the computer while learning to program. If you don’t understand something the first time, reread it. If you still don’t understand it, it is sometimes best to take it by faith and come back to it later. Often after more exposure to programming the ideas will make more sense. Don’t get discouraged. It’s a long climb, but very worthwhile. At the end of each chapter are three sets of review exercises. The first set is more or less regurgitation - they check to see if can you give back what you learned in the chapter. The second set contains application questions - they check to see if you can apply what you learned to solve problems. The final set is to see if you are capable of broadening your horizons. Some of these questions may not be answerable until later in the book, but they give you some things to think about. Other questions require some research into outside sources to discover the answer. Still others require you to simply analyze your options and explain a best solution. Many of the questions don’t have right or wrong answers, but that doesn’t mean they are unimportant. Learning the issues involved in programming, learning how to research answers, and learning how to look ahead are all a major part of a programmer’s work. If you have problems that you just can’t get past, there is a mailing list for this
2
Chapter 1. Introduction book where readers can discuss and get help with what they are reading. The address is
[email protected]. This mailing list is open for any type of question or discussion along the lines of this book. You can subscribe to this list by going to http://mail.nongnu.org/mailman/listinfo/pgubook-readers.
Your Tools This book teaches assembly language for x86 processors and the GNU/Linux operating system. Therefore we will be giving all of the examples using the GNU/Linux standard GCC tool set. If you are not familiar with GNU/Linux and the GCC tool set, they will be described shortly. If you are new to Linux, you should check out the guide available at http://rute.sourceforge.net/1 What I intend to show you is more about programming in general than using a specific tool set on a specific platform, but standardizing on one makes the task much easier. Those new to Linux should also try to get involved in their local GNU/Linux User’s Group. User’s Group members are usually very helpful for new people, and will help you from everything from installing Linux to learning to use it most efficiently. A listing of GNU/Linux User’s Groups is available at http://www.linux.org/groups/ All of these programs have been tested using Red Hat Linux 8.0, and should work with any other GNU/Linux distribution, too.2 They will not work with non-Linux operating systems such as BSD or other systems. However, all of the skills learned in this book should be easily transferable to any other system. If you do not have access to a GNU/Linux machine, you can look for a hosting provider who offers a Linux shell account, which is a command-line only interface 1. This is quite a large document. You certainly don’t need to know everything to get started with this book. You simply need to know how to navigate from the command line and how to use an editor like pico, emacs, or vi (or others). 2. By "GNU/Linux distribution", I mean an x86 GNU/Linux distribution. GNU/Linux distributions for the Power Macintosh, the Alpha processor, or other processors will not work with this book.
3
Chapter 1. Introduction to a Linux machine. There are many low-cost shell account providers, but you have to make sure that they match the requirements above (i.e. - Linux on x86). Someone at your local GNU/Linux User’s Group may be able to give you one as well. Shell accounts only require that you already have an Internet connection and a telnet program. If you use Windows®, you already have a telnet client - just click on start, then run, then type in telnet. However, it is usually better to download PuTTY from http://www.chiart.greenend.co.uk/~sgtatham/putty/ because Windows’ telnet has some weird problems. There are a lot of options for the Macintosh, too. NiftyTelnet is my favorite. If you don’t have GNU/Linux and can’t find a shell account service, then you can download Knoppix from http://www.knoppix.org/ Knoppix is a GNU/Linux distribution that boots from CD so that you don’t have to actually install it. Once you are done using it, you just reboot and remove the CD and you are back to your regular operating system. So what is GNU/Linux? GNU/Linux is an operating system modeled after UNIX®. The GNU part comes from the GNU Project (http://www.gnu.org/)3, which includes most of the programs you will run, including the GCC tool set that we will use to program with. The GCC tool set contains all of the programs necessary to create programs in various computer languages. Linux is the name of the kernel. The kernel is the core part of an operating system that keeps track of everything. The kernel is both an fence and a gate. As a gate, it allows programs to access hardware in a uniform way. Without the kernel, you would have to write programs to deal with every device model ever made. The kernel handles all device-specific interactions so you don’t have to. It also handles file access and interaction between processes. For example, when you type, your typing goes through several programs before it hits your editor. First, the kernel is what handles your hardware, so it is the first to receive notice about the keypress. The keyboard sends in scancodes to the kernel, which then converts them to the actual letters, numbers, and symbols they represent. If you are using a windowing 3. The GNU Project is a project by the Free Software Foundation to produce a complete, free operating system.
4
Chapter 1. Introduction system (like Microsoft Windows® or the X Window System), then the windowing system reads the keypress from the kernel, and delivers it to whatever program is currently in focus on the user’s display. Example 1-1. How the computer processes keyboard sigals Keyboard -> Kernel -> Windowing system -> Application program
The kernel also controls the flow of information between programs. The kernel is a program’s gate to the world around it. Every time that data moves between processes, the kernel controls the messaging. In our keyboard example above, the kernel would have to be involved for the windowing system to communicate the keypress to the application program. As a fence, the kernel prevents programs from accidentally overwriting each other’s data and from accessing files and devices that they don’t have permission to. It limits the amount of damage a poorly-written program can do to other running programs. In our case, the kernel is Linux. Now, the kernel all by itself won’t do anything. You can’t even boot up a computer with just a kernel. Think of the kernel as the water pipes for a house. Without the pipes, the faucets won’t work, but the pipes are pretty useless if there are no faucets. Together, the user applications (from the GNU project and other places) and the kernel (Linux) make up the entire operating system, GNU/Linux. For the most part, this book will be using the computer’s low-level assembly language. There are essentially three kinds of languages: Machine Language This is what the computer actually sees and deals with. Every command the computer sees is given as a number or sequence of numbers.
5
Chapter 1. Introduction Assembly Language This is the same as machine language, except the command numbers have been replaced by letter sequences which are easier to memorize. Other small things are done to make it easier as well. High-Level Language High-level languages are there to make programming easier. Assembly language requires you to work with the machine itself. High-level languages allow you to describe the program in a more natural language. A single command in a high-level language usually is equivalent to several commands in an assembly language. In this book we will learn assembly language, although we will cover a bit of high-level languages. Hopefully by learning assembly language, your understanding of how programming and computers work will put you a step ahead.
6
Chapter 2. Computer Architecture Before learning how to program, you need to first understand how a computer interprets programs. You don’t need a degree in electrical engineering, but you need to understand some basics. Modern computer architecture is based off of an architecture called the Von Neumann architecture, named after its creator. The Von Neumann architecture divides the computer up into two main parts - the CPU (for Central Processing Unit) and the memory. This architecture is used in all modern computers, including personal computers, supercomputers, mainframes, and even cell phones.
Structure of Computer Memory To understand how the computer views memory, imagine your local post office. They usually have a room filled with PO Boxes. These boxes are similar to computer memory in that each are numbered sequences of fixed-size storage locations. For example, if you have 256 megabytes of computer memory, that means that your computer contains roughly 256 million fixed-size storage locations. Or, to use our analogy, 256 million PO Boxes. Each location has a number, and each location has the same, fixed-length size. The difference between a PO Box and computer memory is that you can store all different kinds of things in a PO Box, but you can only store a single number in a computer memory storage location.
7
Chapter 2. Computer Architecture
Memory locations are like PO Boxes You may wonder why a computer is organized this way. It is because it is simple to implement. If the computer were composed of a lot of differently-sized locations, or if you could store different kinds of data in them, it would be difficult and expensive to implement. The computer’s memory is used for a number of different things. All of the results of any calculations are stored in memory. In fact, everything that is "stored" is stored in memory. Think of your computer at home, and imagine what all is stored in your computer’s memory. •
The location of your cursor on the screen
•
The size of each window on the screen
•
The shape of each letter of each font being used
•
The layout of all of the controls on each window
•
The graphics for all of the toolbar icons
8
Chapter 2. Computer Architecture •
The text for each error message and dialog box
•
The list goes on and on...
In addition to all of this, the Von Neumann architecture specifies that not only computer data should live in memory, but the programs that control the computer’s operation should live there, too. In fact, in a computer, there is no difference between a program and a program’s data except how it is used by the computer. They are both stored and accessed the same way.
The CPU So how does the computer function? Obviously, simply storing data doesn’t do much help - you need to be able to access, manipulate, and move it. That’s where the CPU comes in. The CPU reads in instructions from memory one at a time and executes them. This is known as the fetch-execute cycle. The CPU contains the following elements to accomplish this: •
Program Counter
•
Instruction Decoder
•
Data bus
•
General-purpose registers
•
Arithmetic and logic unit
The program counter is used to tell the computer where to fetch the next instruction from. We mentioned earlier that there is no difference between the way data and programs are stored, they are just interpreted differently by the CPU. The program counter holds the memory address of the next instruction to be executed. The CPU begins by looking at the program counter, and fetching whatever number is stored in memory at the location specified. It is then passed on to the instruction
9
Chapter 2. Computer Architecture decoder which figures out what the instruction means. This includes what process needs to take place (addition, subtraction, multiplication, data movement, etc.) and what memory locations are going to be involved in this process. Computer instructions usually consist of both the actual instruction and the list of memory locations that are used to carry it out. Now the computer uses the data bus to fetch the memory locations to be used in the calculation. The data bus is the connection between the CPU and memory. It is the actual wire that connects them. If you look at the motherboard of the computer, the wires that go out from the memory are your data bus. In addition to the memory on the outside of the processor, the processor itself has some special, high-speed memory locations called registers. There are two kinds of registers - general registers and special-purpose registers. General-purpose registers are where the main action happens. Addition, subtraction, multiplication, comparisions, and other operations generally use general-purpose registers for processing. However, computers have very few general-purpose registers. Most information is stored in main memory, brought in to the registers for processing, and then put back into memory when the processing is completed. special-purpose registers are registers which have very specific purposes. We will discuss these as we come to them. Now that the CPU has retrieved all of the data it needs, it passes on the data and the decoded instruction to the arithmetic and logic unit for further processing. Here the instruction is actually executed. After the results of the computation have been calculated, the results are then placed on the data bus and sent to the appropriate location in memory or in a register, as specified by the instruction. This is a very simplified explanation. Processors have advanced quite a bit in recent years, and are now much more complex. Although the basic operation is still the same, it is complicated by the use of cache hierarchies, superscalar processors, pipelining, branch prediction, out-of-order execution, microcode translation, coprocessors, and other optimizations. Don’t worry if you don’t know what those words mean, you can just use them as Internet search terms if you want
10
Chapter 2. Computer Architecture to learn more about the CPU.
Some Terms Computer memory is a numbered sequence of fixed-size storage locations. The number attached to each storage location is called it’s address. The size of a single storage location is called a byte. On x86 processors, a byte is a number between 0 and 255. You may be wondering how computers can display and use text, graphics, and even large numbers when all they can do is store numbers between 0 and 255. First of all, specialized hardware like graphics cards have special interpretations of each number. When displaying to the screen, the computer uses ASCII code tables to translate the numbers you are sending it into letters to display on the screen, with each number translating to exactly one letter or numeral.1 For example, the capital letter A is represented by the number 65. The numeral 1 is represented by the number 49. So, to print out "HELLO", you would actually give the computer the sequence of numbers 72, 69, 76, 76, 79. To print out the number 100, you would give the computer the sequence of numbers 49, 48, 48. A list of ASCII characters and their numeric codes is found in Appendix D. In addition to using numbers to represent ASCII characters, you as the programmer get to make the numbers mean anything you want them to, as well. For example, if I am running a store, I would use a number to represent each item I was selling. Each number would be linked to a series of other numbers which would be the ASCII codes for what I wanted to display when the items were scanned in. I would have more numbers for the price, how many I have in inventory, and so on. 1. With the advent of international character sets and Unicode, this is not entirely true anymore. However, for the purposes of keeping this simple for beginners, we will use the assumption that one number translates directly to one character. For more information, see Appendix D.
11
Chapter 2. Computer Architecture So what about if we need numbers larger than 255? We can simply use a combination of bytes to represent larger numbers. Two bytes can be used to represent any number between 0 and 65536. Four bytes can be used to represent any number between 0 and 4294967295. Now, it is quite difficult to write programs to stick bytes together to increase the size of your numbers, and requires a bit of math. Luckily, the computer will do it for us for numbers up to 4 bytes long. In fact, four-byte numbers are what we will work with by default. We mentioned earlier that in addition to the regular memory that the computer has, it also has special-purpose storage locations called registers. Registers are what the computer uses for computation. Think of a register as a place on your desk - it holds things you are currently working on. You may have lots of information tucked away in folders and drawers, but the stuff you are working on right now is on the desk. Registers keep the contents of numbers that you are currently manipulating. On the computers we are using, registers are each four bytes long. The size of a typical register is called a computer’s word size. x86 processors have four-byte words. This means that it is most natural on these computers to do computations four bytes at a time. This gives us roughly 4 billion values. Addresses are also four bytes (1 word) long, and therefore also fit into a register. x86 processors can access up to 4294967296 bytes if enough memory is installed. Notice that this means that we can store addresses the same way we store any other number. In fact, the computer can’t tell the difference between a value that is an address, a value that is a number, a value that is an ASCII code, or a value that you have decided to use for another purpose. A number becomes an ASCII code when you attempt to display it. A number becomes an address when you try to look up the byte it points to. Take a moment to think about this, because it is crucial to understanding how computer programs work. Addresses which are stored in memory are also called pointers, because instead of having a regular value in them, they point you to a different location in memory. As we’ve mentioned, computer instructions are also stored in memory. In fact,
12
Chapter 2. Computer Architecture they are stored exactly the same way that other data is stored. The only way the computer knows that a memory location is an instruction is that a special-purpose register called the instruction pointer points to them at one point or another. If the instruction pointer points to a memory word, it is loaded as an instruction. Other than that, the computer has no way of knowing the difference between programs and other types of data.2
Interpreting Memory Computers are very exact. Because they are exact, programmers have to be equally exact. A computer has no idea what your program is supposed to do. Therefore, it will only do exactly what you tell it to do. If you accidentally print out a regular number instead of the ASCII codes that make up the number’s digits, the computer will let you - and you will wind up with jibberish on your screen (it will try to look up what your number represents in ASCII and print that). If you tell the computer to start executing instructions at a location containing data instead of program instructions, who knows how the computer will interpret that but it will certainly try. The computer will execute your instructions in the exact order you specify, even if it doesn’t make sense. The point is, the computer will do exactly what you tell it, no matter how little sense it makes. Therefore, as a programmer, you need to know exactly how you have your data arranged in memory. Remember, computers can only store numbers, so letters, pictures, music, web pages, documents, and anything else are just long sequences of numbers in the computer, which particular programs know how to interpret. For example, say that you wanted to store customer information in memory. One way to do so would be to set a maximum size for the customer’s name and address - say 50 ASCII characters for each, which would be 50 bytes for each. Then, after 2. Note that here we are talking about general computer theory. Some processors and operating systems actually mark the regions of memory that can be executed with a special marker that indicates this.
13
Chapter 2. Computer Architecture that, have a number for the customer’s age and their customer id. In this case, you would have a block of memory that would look like this:
Start of Record: Customer’s name (50 bytes) - start of record Customer’s address (50 bytes) - start of record + 50 bytes Customer’s age (1 word - 4 bytes) - start of record + 100 bytes Customer’s id number (1 word - 4 bytes) - start of record + 104 byte
This way, given the address of a customer record, you know where the rest of the data lies. However, it does limit the customer’s name and address to only 50 ASCII characters each. What if we didn’t want to specify a limit? Another way to do this would be to have in our record pointers to this information. For example, instead of the customer’s name, we would have a pointer to their name. In this case, the memory would look like this: Start of Record: Customer’s name pointer (1 word) - start of record Customer’s address pointer (1 word) - start of record + 4 Customer’s age (1 word) - start of record + 8 Customer’s id number (1 word) - start of record + 12
The actual name and address would be stored elsewhere in memory. This way, it is easy to tell where each part of the data is from the start of the record, without explicitly limitting the size of the name and address. If the length of the fields within our records could change, we would have no idea where the next field started. Because records would be different sizes, it would also be hard to find where the next record began. Therefore, almost all records are of fixed lengths. Variable-length data is usually store separately from the rest of the record.
14
Chapter 2. Computer Architecture
Data Accessing Methods Processors have a number of different ways of accessing data, known as addressing modes. The simplest mode is immediate mode, in which the data to access is embedded in the instruction itself. For example, if we want to initialize a register to 0, instead of giving the computer an address to read the 0 from, we would specify immediate mode, and give it the number 0. In the register addressing mode, the instruction contains a register to access, rather than a memory location. The rest of the modes will deal with addresses. In the direct addressing mode, the instruction contains the memory address to access. For example, I could say, please load this register with the data at address 2002. The computer would go directly to byte number 2002 and copy the contents into our register. In the indexed addressing mode, the instruction contains a memory address to access, and also specifies an index register to offset that address. For example, we could specify address 2002 and an index register. If the index register contains the number 4, the actual address the data is loaded from would be 2006. This way, if you have a set of numbers starting at location 2002, you can cycle between each of them using an index register. On x86 processors, you can also specify a multiplier for the index. This allows you to access memory a byte at a time or a word at a time (4 bytes). If you are accessing an entire word, your index will need to be multiplied by 4 to get the exact location of the fourth element from your address. For example, if you wanted to access the fourth byte from location 2002, you would load your index register with 3 (remember, we start counting at 0) and set the multiplier to 1 since you are going a byte at a time. This would get you location 2005. However, if you wanted to access the fourth word from location 2002, you would load your index register with 3 and set the multiplier to 4. This would load from location 2014 - the fourth word. Take the time to calculate these yourself to make sure you understand how it works. In the indirect addressing mode, the instruction contains a register that contains a pointer to where the data should be accessed. For example, if we used indirect
15
Chapter 2. Computer Architecture addressing mode and specified the %eax register, and the %eax register contained the value 4, whatever value was at memory location 4 would be used. In direct addressing, we would just load the value 4, but in indirect addressing, we use 4 as the address to use to find the data we want. Finally, there is the base pointer addressing mode. This is similar to indirect addressing, but you also include a number called the offset to add to the register’s value before using it for lookup. We will use this mode quite a bit in this book. In the Section called Interpreting Memory we discussed having a structure in memory holding customer information. Let’s say we wanted to access the customer’s age, which was the eighth byte of the data, and we had the address of the start of the structure in a register. We could use base pointer addressing and specify the register as the base pointer, and 8 as our offset. This is a lot like indexed addressing, with the difference that the offset is constant and the pointer is held in a register, and in indexed addressing the offset is in a register and the pointer is constant. There are other forms of addressing, but these are the most important ones.
Review Know the Concepts •
Describe the fetch-execute cycle.
•
What is a register? How would computation be more difficult without registers?
•
How do you represent numbers larger than 255?
•
How big are the registers on the machines we will be using?
•
How does a computer know how to interpret a given byte or set of bytes of memory?
16
Chapter 2. Computer Architecture •
What are the addressing modes and what are they used for?
•
What does the instruction pointer do?
Use the Concepts •
What data would you use in an employee record? How would you lay it out in memory?
•
If I had the pointer the the beginning of the employee record above, and wanted to access a particular piece of data inside of it, what addressing mode would I use?
•
In base pointer addressing mode, if you have a register holding the value 3122, and an offset of 20, what address would you be trying to access?
•
In indexed addressing mode, if the base address is 6512, the index register has a 5, and the multiplier is 4, what address would you be trying to access?
•
In indexed addressing mode, if the base address is 123472, the index register has a 0, and the multiplier is 4, what address would you be trying to access?
•
In indexed addressing mode, if the base address is 9123478, the index register has a 20, and the multiplier is 1, what address would you be trying to access?
Going Further •
What are the minimum number of addressing modes needed for computation?
•
Why include addressing modes that aren’t strictly needed?
•
Research and then describe how pipelining (or one of the other complicating factors) affects the fetch-execute cycle.
17
Chapter 2. Computer Architecture •
Research and then describe the tradeoffs between fixed-length instructions and variable-length instructions.
18
Chapter 3. Your First Programs In this chapter you will learn the process for writing and building Linux assembly-language programs. In addition, you will learn the structure of assembly-language programs, and a few assembly-language commands. As you go through this chapter, you may want to refer also to Appendix B and Appendix F. These programs may overwhelm you at first. However, go through them with diligence, read them and their explanations as many times as necessary, and you will have a solid foundation of knowledge to build on. Please tinker around with the programs as much as you can. Even if your tinkering does not work, every failure will help you learn.
Entering in the Program Okay, this first program is simple. In fact, it’s not going to do anything but exit! It’s short, but it shows some basics about assembly language and Linux programming. You need to enter the program in an editor exactly as written, with the filename exit.s. The program follows. Don’t worry about not understanding it. This section only deals with typing it in and running it. In the Section called Outline of an Assembly Language Program we will describe how it works. #PURPOSE: # #
Simple program that exits and returns a status code back to the Linux kernel
#INPUT: #
none
#OUTPUT: # # # #
returns a status code. by typing
This can be viewed
echo $?
19
Chapter 3. Your First Programs # #
after running the program
#VARIABLES: # %eax holds the system call number # %ebx holds the return status # .section .data .section .text .globl _start _start: movl $1, %eax
# this is the linux kernel command # number (system call) for exiting # a program
movl $0, %ebx
# # # # #
this is the status number we will return to the operating system. Change this around and it will return different things to echo $?
int $0x80
# this wakes up the kernel to run # the exit command
What you have typed in is called the source code. Source code is the human-readable form of a program. In order to transform it into a program that a computer can run, we need to assemble and link it. The first step is to assemble it. Assembling is the process that transforms what you typed into instructions for the machine. The machine itself only reads sets of numbers, but humans prefer words. An assembly language is a more human-readable form of the instructions a computer understands. Assembling
20
Chapter 3. Your First Programs transforms the human-readable file into a machine-readable one. To assembly the program type in the command as exit.s -o exit.o as is the command which runs the assembler, exit.s is the source file, and -o exit.o tells the assemble to put it’s output in the file exit.o. exit.o is an
object file. An object file is code that is in the machine’s language, but has not been completely put together. In most large programs, you will have several source files, and you will convert each one into an object file. The linker is the program that is responsible for putting the object files together and adding information to it so that the kernel knows how to load and run it. In our case, we only have one object file, so the linker is only adding the information to enable it to run. To link the file, enter the command ld exit.o -o exit ld is the command to run the linker, exit.o is the object file we want to link, and -o exit instructs the linker to output the new program into a file called exit.1 If
any of these commands reported errors, you have either mistyped your program or the command. After correcting the program, you have to re-run all the commands. You must always re-assemble and re-link programs after you modify the source file for the changes to occur in the program. You can run exit by typing in the command ./exit
The ./ is used to tell the computer that the program isn’t in one of the normal program directories, but is the current directory instead2. You’ll notice when you type this command, the only thing that happens is that you’ll go to the next line. 1. If you are new to Linux and UNIX®, you may not be aware that files don’t have to have extensions. In fact, while Windows® uses the .exe extension to signify an executable program, UNIX executables usually have no extension. 2. . refers to the current directory in Linux and UNIX systems.
21
Chapter 3. Your First Programs That’s because this program does nothing but exit. However, immediately after you run the program, if you type in echo $?
It will say 0. What is happening is that every program when it exits gives Linux an exit status code, which tells it if everything went all right. If everything was okay, it returns 0. UNIX programs return numbers other than zero to indicate failure or other errors, warnings, or statuses. The programmer determines what each number means. You can view this code by typing in echo $?. In the following section we will look at what each part of the code does.
Outline of an Assembly Language Program Take a look at the program we just entered. At the beginning there are lots of lines that begin with hashes (#). These are comments. Comments are not translated by the assembler. They are used only for the programmer to talk to anyone who looks at the code in the future. Most programs you write will be modified by others. Get into the habit of writing comments in your code that will help them understand both why the program exists and how it works. Always include the following in your comments: •
The purpose of the code
•
An overview of the processing involved
•
Anything strange your program does and why it does it3
After the comments, the next line says 3. You’ll find that many programs end up doing things strange ways. Usually there is a reason for that, but, unfortunately, programmers never document such things in their comments. So, future programmers either have to learn the reason the hard way by modifying the code and watching it break, or just leaving it alone whether it is still needed or not. You should always document any strange behavior your program performs. Unfortunately, figuring out what is strange and what is straightforward comes mostly with experience.
22
Chapter 3. Your First Programs .section .data
Anything starting with a period isn’t directly translated into a machine instruction. Instead, it’s an instruction to the assembler itself. These are called assembler directives or pseudo-operations because they are handled by the assembler and are not actually run by the computer. The .section command breaks your program up into sections. This command starts the data section, where you list any memory storage you will need for data. Our program doesn’t use any, so we don’t need the section. It’s just here for completeness. Almost every program you write in the future will have data. Right after this you have .section .text
which starts the text section. The text section of a program is where the program instructions live. The next instruction is .globl _start
This instructs the assembler that _start is important to remember. _start is a symbol, which means that it is going to be replaced by something else either during assembly or linking. Symbols are generally used to mark locations of programs or data, so you can refer to them by name instead of by their location number. Imagine if you had to refer to every memory location by it’s address. First of all, it would be very confusing because you would have to memorize or look up the numeric memory address of every piece of code or data. In addition, every time you had to insert a piece of data or code you would have to change all the addresses in your program! Symbols are used so that the assembler and linker can take care of keeping track of addresses, and you can concentrate on writing your program.
23
Chapter 3. Your First Programs .globl means that the assembler shouldn’t discard this symbol after assembly, because the linker will need it. _start is a special symbol that always needs to be marked with .globl because it marks the location of the start of the program.
Without marking this location in this way, when the computer loads your program it won’t know where to begin running your program. The next line _start:
defines the value of the _start label. A label is a symbol followed by a colon. Labels define a symbol’s value. When the assembler is assembling the program, it has to assign each data value and instruction an address. Labels tell the assembler to make the symbol’s value be wherever the next instruction or data element will be. This way, if the actual physical location of the data or instruction changes, you don’t have to rewrite any references to it - the symbol automatically gets the new value. Now we get into actual computer instructions. The first such instruction is this: movl $1, %eax
When the program runs, this instruction transfers the number 1 into the %eax register. In assembly language, many instructions have operands. movl has two operands - the source and the destination. In this case, the source is the literal number 1, and the destination is the %eax register. Operands can be numbers, memory location references, or registers. Different instructions allow different types of operands. See Appendix B for more information on which instructions take which kinds of operands. On most instructions which have two operands, the first one is the source operand and the second one is the destination. Note that in these cases, the source operand is not modified at all. Other instructions of this type are, for example, addl, subl, and imull. These add/subtract/multiply the source operand from/to/by the destination operand and and save the result in the destination operand. Other
24
Chapter 3. Your First Programs instructions may have an operand hardcoded in. idivl, for example, requires that the dividend be in %eax, and %edx be zero, and the quotient is then transferred to %eax and the remainder to %edx. However, the divisor can be any register or memory location. On x86 processors, there are several general-purpose registers4 (all of which can be used with movl): • %eax • %ebx • %ecx • %edx • %edi • %esi
In addition to these general-purpose registers, there are also several special-purpose registers, including: • %ebp • %esp • %eip • %eflags
We’ll discuss these later, just be aware that they exist.5 Some of these registers, 4. Note that on x86 processors, even the general-purpose registers have some special purposes, or used to before it went 32-bit. However, these are general-purpose registers for most instructions. Each of them has at least one instruction where it is used in a special way. However, for most of them, those instructions aren’t covered in this book. 5. You may be wondering, why do all of these registers begin with the letter e? The reason is that early generations of x86 processors were 16 bits rather than 32 bits. Therefore, the registers were only half the length they are now. In later generations of x86 processors, the size of the registers doubled. They kept the old names to refer to the first half of the register,
25
Chapter 3. Your First Programs like %eip and %eflags can only be accessed through special instructions. The others can be accessed using the same instructions as general-purpose registers, but they have special meanings, special uses, or are simply faster when used in a specific way. So, the movl instruction moves the number 1 into %eax. The dollar-sign in front of the one indicates that we want to use immediate mode addressing (refer back to the Section called Data Accessing Methods in Chapter 2). Without the dollar-sign it would do direct addressing, loading whatever number is at address 1. We want the actual number 1 loaded in, so we have to use immediate mode. The reason we are moving the number 1 into %eax is because we are preparing to call the Linux Kernel. The number 1 is the number of the exit system call . We will discuss system calls in more depth soon, but basically they are requests for the operating system’s help. Normal programs can’t do everything. Many operations such as calling other programs, dealing with files, and exiting have to be handled by the operating system through system calls. When you make a system call, which we will do shortly, the system call number has to be loaded into %eax (for a complete listing of system calls and their numbers, see Appendix C). Depending on the system call, other registers may have to have values in them as well. Note that system calls is not the only use or even the main use of registers. It is just the one we are dealing with in this first program. Later programs will use registers for regular computation. The operating system, however, usually needs more information than just which call to make. For example, when dealing with files, the operating system needs to know which file you are dealing with, what data you want to write, and other details. The extra details, called parameters are stored in other registers. In the case of the exit system call, the operating system requires a status code be loaded and added an e to refer to the extended versions of the register. Usually you will only use the extended versions. Newer models also offer a 64-bit mode, which doubles the size of these registers yet again and uses an r prefix to indicate the larger registers (i.e. %rax is the 64-bit version of %eax). However, these processors are not widely used, and are not covered in this book.
26
Chapter 3. Your First Programs in %ebx. This value is then returned to the system. This is the value you retrieved when you typed echo $?. So, we load %ebx with 0 by typing the following: movl $0, %ebx
Now, loading registers with these numbers doesn’t do anything itself. Registers are used for all sorts of things besides system calls. They are where all program logic such as addition, subtraction, and comparisons take place. Linux simply requires that certain registers be loaded with certain parameter values before making a system call. %eax is always required to be loaded with the system call number. For the other registers, however, each system call has different requirements. In the exit system call, %ebx is required to be loaded with the exit status. We will discuss different system calls as they are needed. For a list of common system calls and what is required to be in each register, see Appendix C The next instruction is the "magic" one. It looks like this: int $0x80
The int stands for interrupt. The 0x80 is the interrupt number to use.6 An interrupt interrupts the normal program flow, and transfers control from our program to Linux so that it will do a system call.7. You can think of it as like signaling Batman(or Larry-Boy8, if you prefer). You need something done, you send the signal, and then he comes to the rescue. You don’t care how he does his work - it’s more or less magic - and when he’s done you’re back in control. In this case, all we’re doing is asking Linux to terminate the program, in which case we 6. You may be wondering why it’s 0x80 instead of just 80. The reason is that the number is written in hexadecimal. In hexadecimal, a single digit can hold 16 values instead of the normal 10. This is done by utilizing the letters a through f in addition to the regular digits. a represents 10, b represents 11, and so on. 0x10 represents the number 16, and so on. This will be discussed more in depth later, but just be aware that numbers starting with 0x are in hexadecimal. Tacking on an H at the end is also sometimes used instead, but we won’t do that in this book. For more information about this, see Chapter 10 7. Actually, the interrupt transfers control to whoever set up an interrupt handler for the interrupt number. In the case of Linux, all of them are set to be handled by the Linux kernel. 8. If you don’t watch Veggie Tales, you should. Start with Dave and the Giant Pickle.
27
Chapter 3. Your First Programs won’t be back in control. If we didn’t signal the interrupt, then no system call would have been performed. Quick System Call Review: To recap - Operating System features are accessed through system calls. These are invoked by setting up the registers in a special way and issuing the instruction int $0x80. Linux knows which system call we want to access by what we stored in the %eax register. Each system call has other requirements as to what needs to be stored in the other registers. System call number 1 is the exit system call, which requires the status code to be placed in %ebx.
Now that you’ve assembled, linked, run, and examined the program, you should make some basic edits. Do things like change the number that is loaded into %ebx, and watch it come out at the end with echo $?. Don’t forget to assemble and link it again before running it. Add some comments. Don’t worry, the worse thing that would happen is that the program won’t assemble or link, or will freeze your screen. That’s just part of learning!
Planning the Program In our next program we will try to find the maximum of a list of numbers. Computers are very detail-oriented, so in order to write the program we will have to have planned out a number of details. These details include: •
Where will the original list of numbers be stored?
•
What procedure will we need to follow to find the maximum number?
•
How much storage do we need to carry out that procedure?
•
Will all of the storage fit into registers, or do we need to use some memory as well?
28
Chapter 3. Your First Programs You might not think that something as simple as finding the maximum number from a list would take much planning. You can usually tell people to find the maximum number, and they can do so with little trouble. However, our minds are used to putting together complex tasks automatically. Computers need to be instructed through the process. In addition, we can usually hold any number of things in our mind without much trouble. We usually don’t even realize we are doing it. For example, if you scan a list of numbers for the maximum, you will probably keep in mind both the highest number you’ve seen so far, and where you are in the list. While your mind does this automatically, with computers you have to explicitly set up storage for holding the current position on the list and the current maximum number. You also have other problems such as how to know when to stop. When reading a piece of paper, you can stop when you run out of numbers. However, the computer only contains numbers, so it has no idea when it has reached the last of your numbers. In computers, you have to plan every step of the way. So, let’s do a little planning. First of all, just for reference, let’s name the address where the list of numbers starts as data_items. Let’s say that the last number in the list will be a zero, so we know where to stop. We also need a value to hold the current position in the list, a value to hold the current list element being examined, and the current highest value on the list. Let’s assign each of these a register: • %edi
will hold the current position in the list.
• %ebx
will hold the current highest value in the list.
• %eax
will hold the current element being examined.
When we begin the program and look at the first item in the list, since we haven’t seen any other items, that item will automatically be the current largest element in the list. Also, we will set the current position in the list to be zero - the first element. From then, we will follow the following steps:
29
Chapter 3. Your First Programs 1. Check the current list element (%eax) to see if it’s zero (the terminating element). 2. If it is zero, exit. 3. Increase the current position (%edi). 4. Load the next value in the list into the current value register (%eax). What addressing mode might we use here? Why? 5. Compare the current value (%eax) with the current highest value (%ebx). 6. If the current value is greater than the current highest value, replace the current highest value with the current value. 7. Repeat. That is the procedure. Many times in that procedure I made use of the word "if". These places are where decisions are to be made. You see, the computer doesn’t follow the exact same sequence of instructions every time. Depending on which "if"s are correct, the computer may follow a different set of instructions. The second time through, it might not have the highest value. In that case, it will skip step 6, but come back to step 7. In every case except the last one, it will skip step 2. In more complicated programs, the skipping around increases dramatically. These "if"s are a class of instructions called flow control instructions, because they tell the compute which steps to follow and which paths to take. In the previous program, we did not have any flow control instructions, as there was only one possible path to take - exit. This program is much more dynamic in that it is directed by data. Depending on what data it receives, it will follow different instruction paths. In this program, this will be accomplished by two different instructions, the conditional jump and the unconditional jump. The conditional jump changes paths based on the results of a previous comparison or calculation. The unconditional jump just goes directly to a different path no matter what. The unconditional jump may seem useless, but it is very necessary since all of the instructions will be laid
30
Chapter 3. Your First Programs out on a line. If a path needs to converge back to the main path, it will have to do this by an unconditional jump. We will see more of both of these jumps in the next section. Another use of flow control is in implementing loops. A loop is a piece of program code that is meant to be repeated. In our example, the first part of the program (setting the current position to 0 and loading the current highest value with the current value) was only done once, so it wasn’t a loop. However, the next part is repeated over and over again for every number in the list. It is only left when we have come to the last element, indicated by a zero. This is called a loop because it occurs over and over again. It is implemented by doing unconditional jumps to the beginning of the loop at the end of the loop, which causes it to start over. However, you have to always remember to have a conditional jump to exit the loop somewhere, or the loop will continue forever! This condition is called an infinite loop. If we accidentally left out step 1, 2, or 3, the loop (and our program) would never end. In the next section, we will implement this program that we have planned. Program planning sounds complicated - and it is, to some degree. When you first start programming, it’s often hard to convert our normal thought process into a procedure that the computer can understand. We often forget the number of "temporary storage locations" that our minds are using to process problems. As you read and write programs, however, this will eventually become very natural to you. Just have patience.
Finding a Maximum Value Enter the following program as maximum.s: #PURPOSE: # #
This program finds the maximum number of a set of data items.
31
Chapter 3. Your First Programs #VARIABLES: The registers have the following uses: # # %edi - Holds the index of the data item being examined # %ebx - Largest data item found # %eax - Current data item # # The following memory locations are used: # # data_items - contains the item data. A 0 is used # to terminate the data # .section .data data_items: #These are the data items .long 3,67,34,222,45,75,54,34,44,33,22,11,66,0 .section .text .globl _start _start: movl $0, %edi # move 0 into the index register movl data_items(,%edi,4), %eax # load the first byte of data movl %eax, %ebx # since this is the first item, %eax is # the biggest start_loop: cmpl $0, %eax je loop_exit incl %edi movl data_items(,%edi,4), cmpl %ebx, %eax jle start_loop movl %eax, %ebx
32
# start loop # check to see if we’ve hit the end # load next value %eax # compare values # jump to loop beginning if the new # one isn’t bigger # move the value as the largest
Chapter 3. Your First Programs jmp start_loop
# jump to loop beginning
loop_exit: # %ebx is the status code for the exit system call # and it already has the maximum number movl $1, %eax #1 is the exit() syscall int $0x80
Now, assemble and link it with these commands: as maximum.s -o maximum.o ld maximum.o -o maximum
Now run it, and check it’s status. ./maximum echo $?
You’ll notice it returns the value 222. Let’s take a look at the program and what it does. If you look in the comments, you’ll see that the program finds the maximum of a set of numbers (aren’t comments wonderful!). You may also notice that in this program we actually have something in the data section. These lines are the data section: data_items: #These are the data items .long 3,67,34,222,45,75,54,34,44,33,22,11,66,0
Lets look at this. data_items is a label that refers to the location that follows it. Then, there is a directive that starts with .long. That causes the assembler to reserve memory for the list of numbers that follow it. data_items refers to the location of the first one. Because data_items is a label, any time in our program where we need to refer to this address we can use the data_items symbol, and the assembler will substitute it with the address where the numbers start during assembly. For example, the instruction movl data_items, %eax would move
33
Chapter 3. Your First Programs the value 3 into %eax. There are several different types of memory locations other than .long that can be reserved. The main ones are as follows: .byte
Bytes take up one storage location for each number. They are limited to numbers between 0 and 255. .int
Ints (which differ from the int instruction) take up two storage locations for each number. These are limitted to numbers between 0 and 65535.9 .long
Longs take up four storage locations. This is the same amount of space the registers use, which is why they are used in this program. They can hold numbers between 0 and 4294967295. .ascii
The .ascii directive is to enter in characters into memory. Characters each take up one storage location (they are converted into bytes internally). So, if you gave the directive .ascii "Hello there\0", the assembler would reserve 12 storage locations (bytes). The first byte contains the numeric code for H, the second byte contains the numeric code for e, and so forth. The last character is represented by \0, and it is the terminating character (it will never display, it just tells other parts of the program that that’s the end of the characters). Letters and numbers that start with a backslash represent characters that are not typeable on the keyboard or easily viewable on the screen. For example, \n refers to the "newline" character which causes the 9. Note that no numbers in assembly language (or any other computer language I’ve seen) have commas embedded in them. So, always write numbers like 65535, and never like 65,535.
34
Chapter 3. Your First Programs computer to start output on the next line and \t refers to the "tab" character. All of the letters in an .ascii directive should be in quotes. In our example, the assembler reserves 14 .longs, one right after another. Since each long takes up 4 bytes, that means that the whole list takes up 56 bytes. These are the numbers we will be searching through to find the maximum. data_items is used by the assembler to refer to the address of the first of these values. Take note that the last data item in the list is a zero. I decided to use a zero to tell my program that it has hit the end of the list. I could have done this other ways. I could have had the size of the list hard-coded into the program. Also, I could have put the length of the list as the first item, or in a separate location. I also could have made a symbol which marked the last location of the list items. No matter how I do it, I must have some method of determining the end of the list. The computer knows nothing - it can only do what its told. It’s not going to stop processing unless I give it some sort of signal. Otherwise it would continue processing past the end of the list into the data that follows it, and even to locations where we haven’t put any data. Notice that we don’t have a .globl declaration for data_items. This is because we only refer to these locations within the program. No other file or program needs to know where they are located. This is in contrast to the _start symbol, which Linux needs to know where it is so that it knows where to begin the program’s execution. It’s not an error to write .globl data_items, it’s just not necessary. Anyway, play around with this line and add your own numbers. Even though they are .long, the program will produce strange results if any number is greater than 255, because that’s the largest allowed exit status. Also notice that if you move the 0 to earlier in the list, the rest get ignored. Remember that any time you change the source file, you have to re-assemble and re-link your program. Do this now and see the results. All right, we’ve played with the data a little bit. Now let’s look at the code. In the comments you will notice that we’ve marked some variables that we plan to use. A variable is a dedicated storage location used for a specific purpose, usually
35
Chapter 3. Your First Programs given a distinct name by the programmer. We talked about these in the previous section, but didn’t give them a name. In this program, we have several variables: •
a variable for the current maximum number found
•
a variable for which number of the list we are currently examining, called the index
•
a variable holding the current number being examined
In this case,we have few enough variables that we can hold them all in registers. In larger programs, you have to put them in memory, and then move them to registers when you are ready to use them. We will discuss how to do that later. When people start out programming, they usually underestimate the number of variables they will need. People are not used to having to think through every detail of a process, and therefore leave out needed variables in their first programming attempts. In this program, we are using %ebx as the location of the largest item we’ve found. %edi is used as the index to the current data item we’re looking at. Now, let’s talk about what an index is. When we read the information from data_items, we will start with the first one (data item number 0), then go to the second one (data item number 1), then the third (data item number 2), and so on. The data item number is the index of data_items. You’ll notice that the first instruction we give to the computer is: movl $0, %edi
Since we are using %edi as our index, and we want to start looking at the first item, we load %edi with 0. Now, the next instruction is tricky, but crucial to what we’re doing. It says: movl data_items(,%edi,4), %eax
Now to understand this line, you need to keep several things in mind:
36
Chapter 3. Your First Programs • data_items •
is the location number of the start of our number list.
Each number is stored across 4 storage locations (because we declared it using .long)
• %edi
is holding 0 at this point
So, basically what this line does is say, "start at the beginning of data_items, and take the first item number (because %edi is 0), and remember that each number takes up four storage locations." Then it stores that number in %eax. This is how you write indexed addressing mode instructions in assembly language. The instruction in a general form is this: movl
BEGINNINGADDRESS(,%INDEXREGISTER,WORDSIZE)
In our case data_items was our beginning address, %edi was our index register, and 4 was our word size. This topic is discussed further in the Section called Addressing Modes. If you look at the numbers in data_items, you will see that the number 3 is now in %eax. If %edi was set to 1, the number 67 would be in %eax, and if it was set to 2, the number 34 would be in %eax, and so forth. Very strange things would happen if we used a number other than 4 as the size of our storage locations.10 The way you write this is very awkward, but if you know what each piece does, it’s not too difficult. For more information about this, see the Section called Addressing Modes Let’s look at the next line: movl %eax, %ebx 10. The instruction doesn’t really use 4 for the size of the storage locations, although looking at it that way works for our purposes now. It’s actually what’s called a multiplier. basically, the way it works is that you start at the location specified by data_items, then you add %edi*4 storage locations, and retrieve the number there. Usually, you use the size of the numbers as your multiplier, but in some circumstances you’ll want to do other things.
37
Chapter 3. Your First Programs We have the first item to look at stored in %eax. Since it is the first item, we know it’s the biggest one we’ve looked at. We store it in %ebx, since that’s where we are keeping the largest number found. Also, even though movl stands for move, it actually copies the value, so %eax and %ebx both contain the starting value.11 Now we move into a loop. A loop is a segment of your program that might run more than once. We have marked the starting location of the loop in the symbol start_loop. The reason we are doing a loop is because we don’t know how many data items we have to process, but the procedure will be the same no matter how many there are. We don’t want to have to rewrite our program for every list length possible. In fact, we don’t even want to have to write out code for a comparison for every list item. Therefore, we have a single section of code (a loop) that we execute over and over again for every element in data_items. In the previous section, we outlined what this loop needed to do. Let’s review: •
Check to see if the current value being looked at is zero. If so, that means we are at the end of our data and should exit the loop.
•
We have to load the next value of our list.
•
We have to see if the next value is bigger than our current biggest value.
•
If it is, we have to copy it to the location we are holding the largest value in.
•
Now we need to go back to the beginning of the loop.
Okay, so now lets go to the code. We have the beginning of the loop marked with start_loop. That is so we know where to go back to at the end of our loop. Then we have these instructions: cmpl $0, %eax je end_loop 11. Also, the l in movl stands for move long since we are moving a value that takes up four storage locations.
38
Chapter 3. Your First Programs The cmpl instruction compares the two values. Here, we are comparing the number 0 to the number stored in %eax This compare instruction also affects a register not mentioned here, the %eflags register. This is also known as the status register, and has many uses which we will discuss later. Just be aware that the result of the comparison is stored in the status register. The next line is a flow control instruction which says to jump to the end_loop location if the values that were just compared are equal (that’s what the e of je means). It uses the status register to hold the value of the last comparison. We used je, but there are many jump statements that you can use: je
Jump if the values were equal jg
Jump if the second value was greater than the first value12 jge
Jump if the second value was greater than or equal to the first value jl
Jump if the second value was less than the first value jle
Jump if the second value was less than or equal to the first value jmp
Jump no matter what. This does not need to be preceeded by a comparison. 12. notice that the comparison is to see if the second value is greater than the first. I would have thought it the other way around. You will find a lot of things like this when learning programming. It occurs because different things make sense to different people. Anyway, you’ll just have to memorize such things and go on.
39
Chapter 3. Your First Programs The complete list is documented in Appendix B. In this case, we are jumping if %eax holds the value of zero. If so, we are done and we go to loop_exit.13 If the last loaded element was not zero, we go on to the next instructions: incl %edi movl data_items(,%edi,4), %eax
If you remember from our previous discussion, %edi contains the index to our list of values in data_items. incl increments the value of %edi by one. Then the movl is just like the one we did beforehand. However, since we already incremented %edi, %eax is getting the next value from the list. Now %eax has the next value to be tested. So, let’s test it! cmpl %ebx, %eax jle start_loop
Here we compare our current value, stored in %eax to our biggest value so far, stored in %ebx. If the current value is less or equal to our biggest value so far, we don’t care about it, so we just jump back to the beginning of the loop. Otherwise, we need to record that value as the largest one: movl %eax, %ebx jmp start_loop
which moves the current value into %ebx, which we are using to store the current largest value, and starts the loop over again. Okay, so the loop executes until it reaches a 0, when it jumps to loop_exit. This part of the program calls the Linux kernel to exit. If you remember from the last 13. The names of these symbols can be anything you want them to be, as long as they only contain letters and the underscore character(_). The only one that is forced is _start, and possibly others that you declare with .globl. However, if its a symbol you define and only you use, feel free to call it anything you want that is adequately descriptive (remember that others will have to modify your code later, and will have to figure out what your symbols mean).
40
Chapter 3. Your First Programs program, when you call the operating system (remember it’s like signaling Batman), you store the system call number in %eax (1 for the exit call), and store the other values in the other registers. The exit call requires that we put our exit status in %ebx We already have the exit status there since we are using %ebx as our largest number, so all we have to do is load %eax with the number one and call the kernel to exit. Like this: movl $1, %eax int 0x80
Okay, that was a lot of work and explanation, especially for such a small program. But hey, you’re learning a lot! Now, read through the whole program again, paying special attention to the comments. Make sure that you understand what is going on at each line. If you don’t understand a line, go back through this section and figure out what the line means. You might also grab a piece of paper, and go through the program step-by-step, recording every change to every register, so you can see more clearly what is going on.
Addressing Modes In the Section called Data Accessing Methods in Chapter 2 we learned the different types of addressing modes available for use in assembly language. This section will deal with how those addressing modes are represented in assembly language instructions. The general form of memory address references is this: ADDRESS_OR_OFFSET(%BASE_OR_OFFSET,%INDEX,MULTIPLIER)
All of the fields are optional. To calculate the address, simply perform the following calculation:
41
Chapter 3. Your First Programs
FINAL ADDRESS = ADDRESS_OR_OFFSET + %BASE_OR_OFFSET + MULTIPLIER * %INDEX ADDRESS_OR_OFFSET and MULTIPLIER must both be constants, while the other
two must be registers. If any of the pieces is left out, it is just substituted with zero in the equation. All of the addressing modes mentioned in the Section called Data Accessing Methods in Chapter 2 except immediate-mode can be represented in this fashion. direct addressing mode This is done by only using the ADDRESS_OR_OFFSET portion. Example: movl ADDRESS, %eax
This loads %eax with the value at memory address ADDRESS. indexed addressing mode This is done by using the ADDRESS_OR_OFFSET and the %INDEX portion. You can use any general-purpose register as the index register. You can also have a constant multiplier of 1, 2, or 4 for the index register, to make it easier to index by bytes, double-bytes, and words. For example, let’s say that we had a string of bytes as string_start and wanted to access the third one (an index of 2 since we start counting the index at zero), and %ecx held the value 2. If you wanted to load it into %eax you could do the following: movl string_start(,%ecx,1), %eax
This starts at string_start, and adds 1 * %ecx to that address, and loads the value into %eax. indirect addressing mode Indirect addressing mode loads a value from the address indicated by a register. For example, if %eax held an address, we could move the value at
42
Chapter 3. Your First Programs that address to %ebx by doing the following: movl (%eax), %ebx
base pointer addressing mode Base-pointer addressing is similar to indirect addressing, except that it adds a constant value to the address in the register. For example, if you have a record where the age value is 4 bytes into the record, and you have the address of the record in %eax, you can retrieve the age into %ebx by issuing the following instruction: movl
4(%eax), %ebx
immediate mode Immediate mode is very simple. It does not follow the general form we have been using. Immediate mode is used to load direct values into registers or memory locations. For example, if you wanted to load the number 12 into %eax, you would simply do the following: movl $12, %eax
Notice that to indicate immediate mode, we used a dollar sign in front of the number. If we did not, it would be direct addressing mode, in which case the value located at memory location 12 would be loaded into %eax rather than the number 12 itself. register addressing mode Register mode simply moves data in or out of a register. In all of our examples, register addressing mode was used for the other operand. These addressing modes are very important, as every memory access will use one of these. Every mode except immediate mode can be used as either the source or destination operand. Immediate mode can only be a source operand.
43
Chapter 3. Your First Programs In addition to these modes, there are also different instructions for different sizes of values to move. For example, we have been using movl to move data a word at a time. in many cases, you will only want to move data a byte at a time. This is accomplished by the instruction movb. However, since the registers we have discussed are word-sized and not byte-sized, you cannot use the full register. Instead, you have to use a portion of the register. Take for instance %eax. If you only wanted to work with two bytes at a time, you could just use %ax. %ax is the least-significant half (i.e. - the last part of the number) of the %eax register, and is useful when dealing with two-byte quantities. %ax is further divided up into %al and %ah. %al is the least-significant byte of %ax, and %ah is the most significant byte.14 Loading a value into %eax will wipe out whatever was in %al and %ah (and also %ax, since %ax is made up of them). Similarly, loading a value into either %al or %ah will corrupt any value that was formerly in %eax. Basically, it’s wise to only use a register for either a byte or a word, but never both at the same time. 14. When we talk about the most or least significant byte, it may be a little confusing. Let’s take the number 5432. In that number, 54 is the most significant half of that number and 32 is the least significant half. You can’t quite divide it like that for registers, since they operate on base 2 rather than base 10 numbers, but that’s the basic idea. For more information on this topic, see Chapter 10.
44
Chapter 3. Your First Programs
Layout of the %eax register For a more comprehensive list of instructions, see Appendix B.
Review Know the Concepts •
What does if mean if a line in the program starts with the ’#’ character?
•
What is the difference between an assembly language file and an object code file?
•
What does the linker do?
•
How do you check the result status code of the last program you ran?
•
What is the difference between movl $1, %eax and movl 1, %eax?
45
Chapter 3. Your First Programs •
Which register holds the system call number?
•
What are indexes used for?
•
Why do indexes usually start at 0?
•
If I issued the command movl data_items(,%edi,4), %eax and data_items was address 3634 and %edi held the value 13, what address would you be using to move into %eax?
•
List the general-purpose registers.
•
What is the difference between movl and movb?
•
What is flow control?
•
What does a conditional jump do?
•
What things do you have to plan for when writing a program?
•
Go through every instruction and list what addressing mode is being used for each operand.
Use the Concepts •
Modify the first program to return the value 3.
•
Modify the maximum program to find the minimum instead.
•
Modify the maximum program to use the number 255 to end the list rather than the number 0
•
Modify the maximum program to use an ending address rather than the number 0 to know when to stop.
•
Modify the maximum program to use a length count rather than the number 0 to know when to stop.
46
Chapter 3. Your First Programs •
What would the instruction movl _start, %eax do? Be specific, based on your knowledge of both addressing modes and the meaning of _start. How would this differ from the instruction movl $_start, %eax?
Going Further •
Modify the first program to leave off the int instruction line. Assemble, link, and execute the new program. What error message do you get. Why do you think this might be?
•
So far, we have discussed three approaches to finding the end of the list - using a special number, using the ending address, and using the length count. Which approach do you think is best? Why? Which approach would you use if you knew that the list was sorted? Why?
47
Chapter 3. Your First Programs
48
Chapter 4. All About Functions Dealing with Complexity In Chapter 3, the programs we wrote only consisted of one section of code. However, if we wrote real programs like that, it would be impossible to maintain them. It would be really difficult to get multiple people working on the project, as any change in one part might adversely affect another part that another developer is working on. To assist programmers in working together in groups, it is necessary to break programs apart into separate pieces, which communicate with each other through well-defined interfaces. This way, each piece can be developed and tested independently of the others, making it easier for multiple programmers to work on the project. Programmers use functions to break their programs into pieces which can be independently developed and tested. Functions are units of code that do a defined piece of work on specified types of data. For example, in a word processor program, I may have a function called handle_typed_character which is activated whenever a user types in a key. The data the function uses would probably be the keypress itself and the document the user currently has open. The function would then modify the document according to the keypress it was told about. The data items a function is given to process are called it’s parameters. In the word processing example, the key which was pressed and the document would be considered parameters to the handle_typed_characters function. The parameter list and the processing expectations of a function (what it is expected to do with the parameters) are called the function’s interface. Much care goes into designing function interfaces, because if they are called from many places within a project, it is difficult to change them if necessary. A typical program is composed of hundreds or thousands of functions, each with a
49
Chapter 4. All About Functions small, well-defined task to perform. However, ultimately there are things that you cannot write functions for which must be provided by the system. Those are called primitive functions (or just primitives) - they are the basics which everything else is built off of. For example, imagine a program that draws a graphical user interface. There has to be a function to create the menus. That function probably calls other functions to write text, to write icons, to paint the background, calculate where the mouse pointer is, etc. However, ultimately, they will reach a set of primitives provided by the operating system to do basic line or point drawing. Programming can either be viewed as breaking a large program down into smaller pieces until you get to the primitive functions, or incrementally building functions on top of primitives until you get the large picture in focus. In assembly language, the primitives are usually the same thing as the system calls, even though system calls aren’t true functions as we will talk about in this chapter.
How Functions Work Functions are composed of several different pieces: function name A function’s name is a symbol that represents the address where the function’s code starts. In assembly language, the symbol is defined by typing the the function’s name as a label before the function’s code. This is just like labels you have used for jumping. function parameters A function’s parameters are the data items that are explicitly given to the function for processing. For example, in mathematics, there is a sine function. If you were to ask a computer to find the sine of 2, sine would be the function’s name, and 2 would be the parameter. Some functions have
50
Chapter 4. All About Functions many parameters, others have none.1 local variables Local variables are data storage that a function uses while processing that is thrown away when it returns. It’s kind of like a scratch pad of paper. Functions get a new piece of paper every time they are activated, and they have to throw it away when they are finished processing. Local variables of a function are not accessible to any other function within a program. static variables Static variables are data storage that a function uses while processing that is not thrown away afterwards, but is reused for every time the function’s code is activated. This data is not accessible to any other part of the program. Static variables are generally not used unless absolutely necessary, as they can cause problems later on. global variables Global variables are data storage that a function uses for processing which are managed outside the function. For example, a simple text editor may put the entire contents of the file it is working on in a global variable so it doesn’t have to be passed to every function that operates on it.2 Configuration values are also often stored in global variables. return address The return address is an "invisible" parameter in that it isn’t directly used during the function. The return address is a parameter which tells the function 1. Function parameters can also be used to hold pointers to data that the function wants to send back to the program. 2. This is generally considered bad practice. Imagine if a program is written this way, and in the next version they decided to allow a single instance of the program edit multiple files. Each function would then have to be modified so that the file that was being manipulated would be passed as a parameter. If you had simply passed it as a parameter to begin with, most of your functions could have survived your upgrade unchanged.
51
Chapter 4. All About Functions where to resume executing after the function is completed. This is needed because functions can be called to do processing from many different parts of your program, and the function needs to be able to get back to wherever it was called from. In most programming languages, this parameter is passed automatically when the function is called. In assembly language, the call instruction handles passing the return address for you, and ret handles using that address to return back to where you called the function from. return value The return value is the main method of transferring data back to the main program. Most programming languages only allow a single return value for a function. These pieces are present in most programming languages. How you specify each piece is different in each one, however. The way that the variables are stored and the parameters and return values are transferred by the computer varies from language to language as well. This variance is known as a language’s calling convention, because it describes how functions expect to get and receive data when they are called.3 Assembly language can use any calling convention it wants to. You can even make one up yourself. However, if you want to interoperate with functions written in other languages, you have to obey their calling conventions. We will use the calling convention of the C programming language for our examples because it is the most widely used, and because it is the standard for Linux platforms. 3. A convention is a way of doing things that is standardized, but not forcibly so. For example, it is a convention for people to shake hands when they meet. If I refuse to shake hands with you, you may think I don’t like you. Following conventions is important because it makes it easier for others to understand what you are doing, and makes it easier for programs written by multiple independent authors to work together.
52
Chapter 4. All About Functions
Assembly-Language Functions using the C Calling Convention You cannot write assembly-language functions without understanding how the computer’s stack works. Each computer program that runs uses a region of memory called the stack to enable functions to work properly. Think of a stack as a pile of papers on your desk which can be added to indefinitely. You generally keep the things that you are working on toward the top, and you take things off as you are finished working with them. Your computer has a stack, too. The computer’s stack lives at the very top addresses of memory. You can push values onto the top of the stack through an instruction called pushl, which pushes either a register or memory value onto the top of the stack. Well, we say it’s the top, but the "top" of the stack is actually the bottom of the stack’s memory. Although this is confusing, the reason for it is that when we think of a stack of anything - dishes, papers, etc. - we think of adding and removing to the top of it. However, in memory the stack starts at the top of memory and grows downward due to architectural considerations. Therefore, when we refer to the "top of the stack" remember it’s at the bottom of the stack’s memory. You can also pop values off the top using an instruction called popl. This removes the top value from the stack and places it into a register or memory location of your choosing.. When we push a value onto the stack, the top of the stack moves to accomodate the additional value. We can actually continually push values onto the stack and it will keep growing further and further down in memory until we hit our code or data. So how do we know where the current "top" of the stack is? The stack register, %esp, always contains a pointer to the current top of the stack, wherever it is. Every time we push something onto the stack with pushl, %esp gets subtracted by 4 so that it points to the new top of the stack (remember, each word is four bytes long, and the stack grows downward). If we want to remove something from the stack, we simply use the popl instruction, which adds 4 to %esp and puts the previous top value in whatever register you specified. pushl and popl each take
53
Chapter 4. All About Functions one operand - the register to push onto the stack for pushl, or receive the data that is popped off the stack for popl. If we simply want to access the value on the top of the stack without removing it, we can simply use the %esp register in indirect addressing mode. For example, the following code moves whatever is at the top of the stack into %eax: movl (%esp), %eax
If we were to just do this: movl %esp, %eax
then %eax would just hold the pointer to the top of the stack rather than the value at the top. Putting %esp in parenthesis causes the computer to go to indirect addressing mode, and therefore we get the value pointed to by %esp. If we want to access the value right below the top of the stack, we can simply issue this instruction: movl 4(%esp), %eax
This instruction uses the base pointer addressing mode (see the Section called Data Accessing Methods in Chapter 2) which simply adds 4 to %esp before looking up the value being pointed to. In the C language calling convention, the stack is the key element for implementing a function’s local variables, parameters, and return address. Before executing a function, a program pushes all of the parameters for the function onto the stack in the reverse order that they are documented. Then the program issues a call instruction indicating which function it wishes to start. The call instruction does two things. First it pushes the address of the next instruction, which is the return address, onto the stack. Then it modifies the instruction pointer (%eip) to point to the start of the function. So, at the time the function starts, the stack looks like this (the "top" of the stack is at the bottom on
54
Chapter 4. All About Functions this example): Parameter #N ... Parameter 2 Parameter 1 Return Address
