9LinuxKernel VersionsLinux kernels distinguish between stable and development kernels with a simple nam-ing scheme (see Figure 1.2).Three or four numbers, delineated with a dot, representLinux kernel versions.The first value is the major release, the second is the minor release,and the third is the revision.An optional fourth value is the stable version.The minorrelease also determines whether thekernel is a stable or developmentkernel;an evennumber is stable, whereas an odd number is development.For example, thekernel version2.6.30.1 designates a stable kernel.This kernel has a major version of two, a minor versionof six,a revision of 30,and a stable version of one.The first two values describe theker-nel series"in this case,the 2.6kernel series.The Major Version The Revision2.6.26.1TheMinor Version The Stable VersionFigure1.2Kernelversion namingconvention.Developmentkernels have a series of phases.Initially,the kernel developers work onnewfeatures and chaos ensues.Over time, thekernel matures and eventually a featurefreeze is declared.At that point, Linus will not accept new features.Work on existing features, however, can continue.After Linus considers thekernel nearly stabilized, a codefreeze is put into effect. When that occurs, only bug fixes are accepted. Shortly thereafter(hopefully),Linus releases the first version of a new stable series.For example, the devel-opment series 1.3 stabilized into 2.0 and 2.5 stabilized into2.6.Within a given series, Linus releases new kernels regularly, with each version earning anew revision.For example, the first version of the 2.6 kernel series was 2.6.0.The nextwas 2.6.1.These revisions contain bug fixes, new drivers, and new features, but the differ-encebetweentworevisions—say,2.6.3and2.6.4isminorThis is how development progressed until 2004, when at the invite-only KernelDevelopers Summit, the assembled kernel developers decided to prolong the 2.6 kernelseries and postpone the introduction of a 2.7 development series.The rationale was thatthe 2.6 kernel was well received, stable, and sufficiently mature such that new destabiliz-ing features were unneeded.This course has proven wise,as the ensuing years have shown2.6 is a mature and capable beast.As of this writing,a 2.7 development series is not onthe table and seems unlikely. Instead, the development cycle of each 2.6 revision hasgrown longer,each release incorporating a mini-development series.Andrew Morton,Linus's second-in-command, has repurposed his 2.6-mm treeonce a testing ground formemory management-related changes—into a general-purpose test bed.Destabilizingwww.it-ebooks.info
ptg Linux Kernel Versions 9 Linux kernels distinguish between stable and development kernels with a simple naming scheme (see Figure 1.2).Three or four numbers, delineated with a dot, represent Linux kernel versions.The first value is the major release, the second is the minor release, and the third is the revision.An optional fourth value is the stable version.The minor release also determines whether the kernel is a stable or development kernel; an even number is stable, whereas an odd number is development. For example, the kernel version 2.6.30.1 designates a stable kernel.This kernel has a major version of two, a minor version of six, a revision of 30, and a stable version of one.The first two values describe the “kernel series”—in this case, the 2.6 kernel series. Development kernels have a series of phases. Initially, the kernel developers work on new features and chaos ensues. Over time, the kernel matures and eventually a feature freeze is declared.At that point, Linus will not accept new features.Work on existing features, however, can continue.After Linus considers the kernel nearly stabilized, a code freeze is put into effect.When that occurs, only bug fixes are accepted. Shortly thereafter (hopefully), Linus releases the first version of a new stable series. For example, the development series 1.3 stabilized into 2.0 and 2.5 stabilized into 2.6. Within a given series, Linus releases new kernels regularly, with each version earning a new revision. For example, the first version of the 2.6 kernel series was 2.6.0.The next was 2.6.1.These revisions contain bug fixes, new drivers, and new features, but the difference between two revisions—say, 2.6.3 and 2.6.4—is minor. This is how development progressed until 2004, when at the invite-only Kernel Developers Summit, the assembled kernel developers decided to prolong the 2.6 kernel series and postpone the introduction of a 2.7 development series.The rationale was that the 2.6 kernel was well received, stable, and sufficiently mature such that new destabilizing features were unneeded.This course has proven wise, as the ensuing years have shown 2.6 is a mature and capable beast.As of this writing, a 2.7 development series is not on the table and seems unlikely. Instead, the development cycle of each 2.6 revision has grown longer, each release incorporating a mini-development series.Andrew Morton, Linus’s second-in-command, has repurposed his 2.6-mm tree—once a testing ground for memory management-related changes—into a general-purpose test bed. Destabilizing The Major Version The Revision The Minor Version The Stable Version 2.6.26.1 Figure 1.2 Kernel version naming convention. www.it-ebooks.info
10Chapter1IntroductiontotheLinuxKernelchanges thus flow into 2.6-mm and, when mature, into one of the 2.6 mini-developmentseries. Thus, over the last few years, each 2.6 release—for example, 2.6.29—has taken sev-eral months,boasting significant changes over its predecessor.This“development series inminiature"has proven rather successful,maintaining high levels of stability while still intro-ducing new features and appears unlikely to change in the near future. Indeed, the consen-sus amongkernel developers is that this new release process will continue indefinitely.To compensate for the reduced frequency of releases, the kernel developers have intro-duced the aforementioned stable release.This release (the 8 in 2.6.32.8) contains crucial bugfixes, often back-ported from the under-development kernel (in this example, 2.6.33).Inthis manner, the previous release continues to receive attention focused on stabilization.The LinuxKernel DevelopmentCommunityWhen you begin developing code for the Linux kernel, you become a part of the globalkernel development community.The main forumfor this community is the Linux KernelMailing List (oft-shortened to lkml).Subscription information is available at http://vgerkernel.org. Note that this is a high-traffic list with hundreds of messages a day and thatthe other readers—who include all the core kernel developers,including Linus—are notopen to dealing with nonsense.The list is, however, a priceless aid during developmentbecause it is where you can find testers,receive peer review,and ask questions.Later chapters provide an overview of the kernel development process and a morecomplete description of participating successfully in the kernel development communityIn the meantime,however, lurking on (silently reading) the Linux Kernel Mailing List isas good a supplement to this book as you can find.BeforeWeBeginThis book is about the Linux kernel: its goals, the design that fulfills those goals, and theimplementation that realizes that design.The approach is practical, taking a middle roadbetween theory and practice when explaining how everything works.My objective is togive you an insider's appreciation and understanding for the design and implementationof the Linux kernel.This approach,coupled with some personal anecdotes and tips onkernel hacking,should ensure that this book gets you off theground running, whetheryou are looking to develop core kernel code,a new device driver, or simply better under-stand the Linux operating system.While reading this book, you should have access to a Linux system and the kernelsource.Ideally, by this point, you are a Linux user and have poked and prodded at thesource, but require some help making it all come together.Conversely,you might neverhave used Linux but just want to learn the design of the kernel out of curiosity.However.if yourdesire is to write some code of your own,there is no substitute for the source.Thesource code is freely available; use it!Oh, and above all else, have fun!www.it-ebooks.info
ptg 10 Chapter 1 Introduction to the Linux Kernel changes thus flow into 2.6-mm and, when mature, into one of the 2.6 mini-development series.Thus, over the last few years, each 2.6 release—for example, 2.6.29—has taken several months, boasting significant changes over its predecessor.This “development series in miniature” has proven rather successful, maintaining high levels of stability while still introducing new features and appears unlikely to change in the near future. Indeed, the consensus among kernel developers is that this new release process will continue indefinitely. To compensate for the reduced frequency of releases, the kernel developers have introduced the aforementioned stable release.This release (the 8 in 2.6.32.8) contains crucial bug fixes, often back-ported from the under-development kernel (in this example, 2.6.33). In this manner, the previous release continues to receive attention focused on stabilization. The Linux Kernel Development Community When you begin developing code for the Linux kernel, you become a part of the global kernel development community.The main forum for this community is the Linux Kernel Mailing List (oft-shortened to lkml). Subscription information is available at http://vger. kernel.org. Note that this is a high-traffic list with hundreds of messages a day and that the other readers—who include all the core kernel developers, including Linus—are not open to dealing with nonsense.The list is, however, a priceless aid during development because it is where you can find testers, receive peer review, and ask questions. Later chapters provide an overview of the kernel development process and a more complete description of participating successfully in the kernel development community. In the meantime, however, lurking on (silently reading) the Linux Kernel Mailing List is as good a supplement to this book as you can find. Before We Begin This book is about the Linux kernel: its goals, the design that fulfills those goals, and the implementation that realizes that design.The approach is practical, taking a middle road between theory and practice when explaining how everything works. My objective is to give you an insider’s appreciation and understanding for the design and implementation of the Linux kernel.This approach, coupled with some personal anecdotes and tips on kernel hacking, should ensure that this book gets you off the ground running, whether you are looking to develop core kernel code, a new device driver, or simply better understand the Linux operating system. While reading this book, you should have access to a Linux system and the kernel source. Ideally, by this point, you are a Linux user and have poked and prodded at the source, but require some help making it all come together. Conversely, you might never have used Linux but just want to learn the design of the kernel out of curiosity. However, if your desire is to write some code of your own, there is no substitute for the source.The source code is freely available; use it! Oh, and above all else, have fun! www.it-ebooks.info
2Getting Started with the Kerneln this chapter, we introduce some of the basics of the Linux kernel: where to get itssource, how to compile it, and how to install the new kernel. We then go over the differ-ences between thekernel and user-space programs and common programming constructsused in the kernel.Although the kernel certainly is unique in many ways, at the end ofthe day it is little different from any other large software project.Obtainingthe KernelSourceThe current Linux source code is always available in both a complete tarball (an archivecreated with the tar command) and an incremental patch from the official home of theLinux kernel, http://www.kernel.org.Unless you havea specific reasonto work with anolderversion ofthe Linux source,you alvays want the latest code.The repository at kernel.org is the place to get it, alongwith additional patches from a number of leadingkernel developers.Using GitOver the last couple of years, the kernel hackers, led by Linus himself, have begun using anew version control system to manage the Linuxkernel source.Linus created this systemcalled Git, with speed in mind. Unlike traditional systems such as CvS, Git is distributed,and its usage and workflow is consequently unfamiliar to many developers.I strongly rec-ommend using Git to download and manage the Linux kernel source.You can use Git to obtain a copy of the latest"pushed"version of Linus's tree$ git clonegit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/iinux-2.6.gitWhen checked out, you can update your tree to Linus's latest:$ gitpullWith these two commands,you can obtain and subsequentlykeep up to date with theofficial kernel tree.To commit and manage your own changes, see Chapter 20,"Patches,www.it-ebooks.info
ptg 2 Getting Started with the Kernel In this chapter, we introduce some of the basics of the Linux kernel: where to get its source, how to compile it, and how to install the new kernel.We then go over the differences between the kernel and user-space programs and common programming constructs used in the kernel.Although the kernel certainly is unique in many ways, at the end of the day it is little different from any other large software project. Obtaining the Kernel Source The current Linux source code is always available in both a complete tarball (an archive created with the tar command) and an incremental patch from the official home of the Linux kernel, http://www.kernel.org. Unless you have a specific reason to work with an older version of the Linux source, you always want the latest code.The repository at kernel.org is the place to get it, along with additional patches from a number of leading kernel developers. Using Git Over the last couple of years, the kernel hackers, led by Linus himself, have begun using a new version control system to manage the Linux kernel source. Linus created this system, called Git, with speed in mind. Unlike traditional systems such as CVS, Git is distributed, and its usage and workflow is consequently unfamiliar to many developers. I strongly recommend using Git to download and manage the Linux kernel source. You can use Git to obtain a copy of the latest “pushed” version of Linus’s tree: $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git When checked out, you can update your tree to Linus’s latest: $ git pull With these two commands, you can obtain and subsequently keep up to date with the official kernel tree.To commit and manage your own changes, see Chapter 20,“Patches, www.it-ebooks.info
12Chapter2 Getting Started with theKernelHacking,and the Community"A completediscussion of Git is outside thescopeof thisbook; many online resources provide excellent guides.InstallingtheKernelSourceThe kernel tarball is distributed in both GNU zip (gzip) and bzip2 format. Bzip2 is thedefault and preferred format because it generally compresses quite a bit better than gzipThe Linux kernel tarball in bzip2 format is named 1inux-x.y.z.tar.bz2, where x.y.zis the version of that particular release of the kernel source.After downloading the source,uncompressing and untarring it is simple.If your tarball is compressed with bzip2,run$ tar xvjf linux-x.y.z.tar.bz2If it is compressed with GNU zip, run$ tar xvzf linux-x.y.z.tar.gzThis uncompresses and untars the source to the directory 1inux-x.y.z. If you use gitto obtain and manage the kernel source, you do not need to download the tarball Justrun thegit clone command as described and git downloads and unpacks the latest source.Whereto Installand Hack ontheSourceThe kernel source is typically installed in /usr/src/1inux.You should not use this sourcetree for development because the kernel version against whichyour C library is compiled isoften linked to this tree.Moreover,you should not require root in order to make changes tothe kernelinstead, work out of your home directory and use root only to install new ker-nels.Evenwhen installinganewkernel,/usr/src/1inuxshouldremainuntouched.UsingPatchesThroughout the Linux kernel community,patches are the lingua franca of communicationYou will distribute your code changes in patches and receive code from others as patches.Incremental patches provide an easy wayto move from onekernel tree to the next.Insteadof downloading each large tarball of thekernel source, you can simply apply an incremen-tal patch to go from one version to the next.This saves everyone bandwidth and you timeToapplyan incremental patch,from insideyourkernel sourcetree,simply run$ patch -p1 < ../patch-x.y.zGenerallya patch to a given version of thekernel is appliedagainst the previous versionGenerating and applying patches is discussed in much more depth in later chapters.TheKernelSource TreeThe kernel source tree is divided into a number of directories, most ofwhich containmany more subdirectories.The directories in the root of the source tree, along with theirdescriptions, are listed in Table 2.1.www.it-ebooks.info
ptg 12 Chapter 2 Getting Started with the Kernel Hacking, and the Community.”A complete discussion of Git is outside the scope of this book; many online resources provide excellent guides. Installing the Kernel Source The kernel tarball is distributed in both GNU zip (gzip) and bzip2 format. Bzip2 is the default and preferred format because it generally compresses quite a bit better than gzip. The Linux kernel tarball in bzip2 format is named linux-x.y.z.tar.bz2, where x.y.z is the version of that particular release of the kernel source.After downloading the source, uncompressing and untarring it is simple. If your tarball is compressed with bzip2, run $ tar xvjf linux-x.y.z.tar.bz2 If it is compressed with GNU zip, run $ tar xvzf linux-x.y.z.tar.gz This uncompresses and untars the source to the directory linux-x.y.z. If you use git to obtain and manage the kernel source, you do not need to download the tarball. Just run the git clone command as described and git downloads and unpacks the latest source. Where to Install and Hack on the Source The kernel source is typically installed in /usr/src/linux. You should not use this source tree for development because the kernel version against which your C library is compiled is often linked to this tree. Moreover, you should not require root in order to make changes to the kernel—instead, work out of your home directory and use root only to install new kernels. Even when installing a new kernel, /usr/src/linux should remain untouched. Using Patches Throughout the Linux kernel community, patches are the lingua franca of communication. You will distribute your code changes in patches and receive code from others as patches. Incremental patches provide an easy way to move from one kernel tree to the next. Instead of downloading each large tarball of the kernel source, you can simply apply an incremental patch to go from one version to the next.This saves everyone bandwidth and you time. To apply an incremental patch, from inside your kernel source tree, simply run $ patch –p1 < ./patch-x.y.z Generally, a patch to a given version of the kernel is applied against the previous version. Generating and applying patches is discussed in much more depth in later chapters. The Kernel Source Tree The kernel source tree is divided into a number of directories, most of which contain many more subdirectories.The directories in the root of the source tree, along with their descriptions, are listed in Table 2.1. www.it-ebooks.info
Building the Kernel13Table 2.1DirectoriesintheRootoftheKernelSourceTreeDirectoryDescriptionarchArchitecture-specific sourceblockBlock/olayerCrypto APIcryptoDocumentationKernel sourcedocumentationdriversDevice driversfirmwareDevicefirmwareneededtousecertaindriversfsThe VFS and the individual filesystemsincludeKernel headersinitKernel boot and initializationipcInterprocess communication codekernelCore subsystems,suchasthe scheduler1ibHelper routinesmmMemorymanagementsubsystemandtheVMnetNetworking subsystemsamplesSample,demonstrativecodescriptsScripts used to build the kernelLinux Security ModulesecuritysoundSound subsystemusrEarly user-space code (called initramfs)toolsTools helpful for developing LinuxvirtVirtualization infrastructureA number of files in the root of the source tree deserve mention.The file copYING isthe kernel license (the GNU GPL v2).CREDITs is a listing of developers with more than atrivial amount of code in the kernel.MAINTAINERs lists the names of the individuals whomaintain subsystems and drivers in the kernel. Makefile is the base kernel Makefile.Building the KernelBuilding the kernel is easy.It is surprisingly easier than compiling and installing other sys-tem-level components, such as glibc.The 2.6 kernel series introduced a new configurationand build system, which made the job even easier and is a welcome improvement overearlier releases.www.it-ebooks.info
ptg Building the Kernel 13 A number of files in the root of the source tree deserve mention.The file COPYING is the kernel license (the GNU GPL v2). CREDITS is a listing of developers with more than a trivial amount of code in the kernel. MAINTAINERS lists the names of the individuals who maintain subsystems and drivers in the kernel. Makefile is the base kernel Makefile. Building the Kernel Building the kernel is easy. It is surprisingly easier than compiling and installing other system-level components, such as glibc.The 2.6 kernel series introduced a new configuration and build system, which made the job even easier and is a welcome improvement over earlier releases. Table 2.1 Directories in the Root of the Kernel Source Tree Directory Description arch Architecture-specific source block Block I/O layer crypto Crypto API Documentation Kernel source documentation drivers Device drivers firmware Device firmware needed to use certain drivers fs The VFS and the individual filesystems include Kernel headers init Kernel boot and initialization ipc Interprocess communication code kernel Core subsystems, such as the scheduler lib Helper routines mm Memory management subsystem and the VM net Networking subsystem samples Sample, demonstrative code scripts Scripts used to build the kernel security Linux Security Module sound Sound subsystem usr Early user-space code (called initramfs) tools Tools helpful for developing Linux virt Virtualization infrastructure www.it-ebooks.info