LookingattheSourcesBefore we start wetting our toes in the kernel, let's download the sources, learn to applya patch, and look atthelayoutofthecodetree.First, go to www.kernel.org and get the latest stable tree. The sources are archived as tar files compressed inboth gzip (.gz)and bzip2(.bz2)formats.Obtain the sourcefilesby uncompressing and untarring thezippedtarball.Inthefollowingcommands,replaceX.Y.Zwiththelatestkernelversion,suchas2.6.23:bash> cd /usr/srcbash>wgetwww.kernel.org/pub/linux/kernel/vx.Y/linux-x.Y.z.tar.bz2bash>tar xvfj linux-x.Y.z.tar.bz2Now that you have the unpacked source tree in /usr/src/linux-X.y.z/ on your system, let's enable someexperimentaltestfeaturesintothetreebygettingacorresponding-mm(AndrewMorton)patchCode View:bash> cd /usr/srcbash>wgetwww.kernel.org/pub/linux/kernel/people/akpm/patches/x.Y/x.Y.z/x.Y.z-mm2/x.Y.Z-mm2.bz2Apply the patch:bash>cd/usr/src/iinux-x.Y.z/bash>bzip2-dc../x.Y.z-mm2.bz2Ipatch-plThe -dc option asks bzip2 to uncompress the specified files to standard output. This is piped to the patch utilitywhich applies changes to each modified file in the code tree.If youneed to applymultiple patches,do so in the right sequence.To generateakernel enabled withthex.Y.z-aa-bb patch, first download thefull x.Y.z kernel sources, apply the x.Y.z-aa patch, and then apply theX.Y.z-aa-bb patch
Looking at the Sources Before we start wetting our toes in the kernel, let's download the sources, learn to apply a patch, and look at the layout of the code tree. First, go to www.kernel.org and get the latest stable tree. The sources are archived as tar files compressed in both gzip (.gz) and bzip2 (.bz2) formats. Obtain the source files by uncompressing and untarring the zipped tar ball. In the following commands, replace X.Y.Z with the latest kernel version, such as 2.6.23: bash> cd /usr/src bash> wget www.kernel.org/pub/linux/kernel/vX.Y/linux-X.Y.Z.tar.bz2 . bash> tar xvfj linux-X.Y.Z.tar.bz2 Now that you have the unpacked source tree in /usr/src/linux-X.Y.Z/ on your system, let's enable some experimental test features into the tree by getting a corresponding -mm (Andrew Morton) patch: Code View: bash> cd /usr/src bash> wget www.kernel.org/pub/linux/kernel/people/akpm/patches/X.Y/X.Y.Z/X.Y.Zmm2/X.Y.Z-mm2.bz2 Apply the patch: bash> cd /usr/src/linux-X.Y.Z/ bash> bzip2 -dc ./X.Y.Z-mm2.bz2 | patch -p1 The -dc option asks bzip2 to uncompress the specified files to standard output. This is piped to the patch utility, which applies changes to each modified file in the code tree. If you need to apply multiple patches, do so in the right sequence. To generate a kernel enabled with the X.Y.Z-aa-bb patch, first download the full X.Y.Z kernel sources, apply the X.Y.Z-aa patch, and then apply the X.Y.Z-aa-bb patch
PatchSubmissionTogenerateakernel patchout of yourchanges,usethediff command:Code View:bash> diff-Nur /path/to/original/kernel /path/to/your/kernel>changes.patchNote that the original kernel precedes the changed version in the diff-ing order.As per 2.6 kernelpatch submission conventions, you also need to add a line at the end of the patch that says this:signed-off-by:Name<Email>With this, you certify that you wrote the code yourself and that you have the right to contribute it.You are now all set to post your patch to the relevant mailing list, such as LKML.LookatDocumentation/SubmittingPatchesforaguideoncreatingpatchesforsubmissionandatDocumentation/applying-patches.txt for a tutorial on applying patches.Nowthatyourpatched/usr/src/linux-X.Y.Z/treeisreadyforuse,let'stakeamomenttoobservehowthesource layout is organized.Go to the root of the source tree and list its contents.The directories branching outfromtherootofthecodetreeareasfollows:1.arch.This directory contains architecture-specific files.You will see separate subdirectories under arch/forprocessorssuchasARM,Motorola68K,s390,MIPS,Alpha,SPARC,andIA64.2.block.ThisprimarilycontainstheimplementationofI/Oschedulingalgorithmsforblockstoragedevices3.crypto.Thisdirectoryimplements cipheroperationsandthecryptographicAPI,used,forexample,bysomeWiFidevicedriversforimplementingencryptionalgorithmsDocumentation.Thisdirectoryhasbrief descriptionsof variouskernel subsystems.Thiscanbeyourfirst4stoptodigforanswerstokernel-relatedqueries5.drivers.Device drivers fornumerous device classes and peripheral controllers reside in this directory.Thedeviceclassesincludecharacter,serial,Inter-IntegratedCircuit(12C),PersonalComputerMemoryCardInternationalAssociation(PCMCIA),PeripheralComponentInterconnect(PCI),UniversalSerialBus(USB),video,audio,block,IntegratedDriveElectronics(IDE),SmallComputerSystemInterface(SCSI),CDROM,networkadapters,AsynchronousTransferMode(ATM),Bluetooth,and MemoryTechnologyDevices(MTD).Each of these classes live in a separate subdirectory under drivers/.You will,for instance,findPCMCIAdriversourcesinsidethedrivers/pcmcia/directoryandMTDdrivers insidethedrivers/mtd/directory.Thesubdirectoriespresent underdrivers/constitute theprimarysubjectsforthisbook
Patch Submission To generate a kernel patch out of your changes, use the diff command: Code View: bash> diff –Nur /path/to/original/kernel /path/to/your/kernel > changes.patch Note that the original kernel precedes the changed version in the diff-ing order. As per 2.6 kernel patch submission conventions, you also need to add a line at the end of the patch that says this: Signed-off-by: Name <Email> With this, you certify that you wrote the code yourself and that you have the right to contribute it. You are now all set to post your patch to the relevant mailing list, such as LKML. Look at Documentation/SubmittingPatches for a guide on creating patches for submission and at Documentation/applying-patches.txt for a tutorial on applying patches. Now that your patched /usr/src/linux-X.Y.Z/ tree is ready for use, let's take a moment to observe how the source layout is organized. Go to the root of the source tree and list its contents. The directories branching out from the root of the code tree are as follows: arch. This directory contains architecture-specific files. You will see separate subdirectories under arch/ for processors such as ARM, Motorola 68K, s390, MIPS, Alpha, SPARC, and IA64. 1. 2. block. This primarily contains the implementation of I/O scheduling algorithms for block storage devices. crypto. This directory implements cipher operations and the cryptographic API, used, for example, by some WiFi device drivers for implementing encryption algorithms. 3. Documentation. This directory has brief descriptions of various kernel subsystems. This can be your first stop to dig for answers to kernel-related queries. 4. drivers. Device drivers for numerous device classes and peripheral controllers reside in this directory. The device classes include character, serial, Inter-Integrated Circuit (I2C), Personal Computer Memory Card International Association (PCMCIA), Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), video, audio, block, Integrated Drive Electronics (IDE), Small Computer System Interface (SCSI), CDROM, network adapters, Asynchronous Transfer Mode (ATM), Bluetooth, and Memory Technology Devices (MTD). Each of these classes live in a separate subdirectory under drivers/. You will, for instance, find PCMCIA driver sources inside the drivers/pcmcia/ directory and MTD drivers inside the drivers/mtd/ directory. The subdirectories present under drivers/ constitute the primary subjects for this book. 5. 6
6. fs. This directory contains the implementation of filesystems such as EXT3, EXT4, reiserfs, FAT, VFAT,sysfs,procfs,isofs,JFFS2,XFS,NTFS,andNFS.7.include.Kernel headerfiles livehere.Subdirectoriesprefixed withasm containheaders specificto theparticulararchitecture.Sothedirectoryinclude/asm-x86/containsheaderfilespertainingtothex86architecture,whereasinclude/asm-arm/holdsheadersfortheARMarchitecture.8.init.This directory contains high-level initialization and startup code9.ipc.This contains supportfor Inter-ProcessCommunication(IPC)mechanisms suchasmessagequeues,semaphores,andsharedmemory1o.kernel.Thearchitecture-independent portions of the basekernel can befound here.11. lib. Library routines such as generic kernel object (kobject) handlers and Cyclic Redundancy Code (CRC)computationfunctionsstayhere12.mm.Thememorymanagementimplementationliveshere.13.net.Networking protocols reside under this directory.Protocols implemented include Internet Protocolversion4(IPv4),IPv6,InternetworkProtocoleXchange(IPx),Bluetooth,ATM,Infrared,LinkAccessProcedureBalanced (LAPB),and Logical LinkControl (LLC).14.scripts.Scriptsusedduringkernelbuildresidehere.15.security.This directorycontainstheframeworkforsecurity16.sound.The Linux audio subsystem is based in this directory.17.usr.This currently containstheinitramfs implementation.Unifiedx86ArchitectureTreeStartingwiththe2.6.24kernel release,thei386andthex86_64(the64-bitcousinofthe32-biti386)architecture-specific treeshave been unified intoa commonarch/x86/directory.Ifyouareusingapre-2.6.24kernel, replacereferencestoarch/x86/inthisbookwitharch/i386/.Similarly,changeanyoccurrenceofinclude/asm-x86/toinclude/asm-i386/.Somefilenameswithinthesedirectories havealso changed.Wading throughthese large directories in search of symbols and othercodeelements can bea tough task.ThetoolsinTable1.1areworthyaidsasyounavigatethekernelsourcetree
fs. This directory contains the implementation of filesystems such as EXT3, EXT4, reiserfs, FAT, VFAT, sysfs, procfs, isofs, JFFS2, XFS, NTFS, and NFS. 6. include. Kernel header files live here. Subdirectories prefixed with asm contain headers specific to the particular architecture. So the directory include/asm-x86/ contains header files pertaining to the x86 architecture, whereas include/asm-arm/ holds headers for the ARM architecture. 7. 8. init. This directory contains high-level initialization and startup code. ipc. This contains support for Inter-Process Communication (IPC) mechanisms such as message queues, semaphores, and shared memory. 9. 10. kernel. The architecture-independent portions of the base kernel can be found here. lib. Library routines such as generic kernel object (kobject) handlers and Cyclic Redundancy Code (CRC) computation functions stay here. 11. 12. mm. The memory management implementation lives here. net. Networking protocols reside under this directory. Protocols implemented include Internet Protocol version 4 (IPv4), IPv6, Internetwork Protocol eXchange (IPX), Bluetooth, ATM, Infrared, Link Access Procedure Balanced (LAPB), and Logical Link Control (LLC). 13. 14. scripts. Scripts used during kernel build reside here. 15. security. This directory contains the framework for security. 16. sound. The Linux audio subsystem is based in this directory. 17. usr. This currently contains the initramfs implementation. Unified x86 Architecture Tree Starting with the 2.6.24 kernel release, the i386 and the x86_64 (the 64-bit cousin of the 32-bit i386) architecture-specific trees have been unified into a common arch/x86/ directory. If you are using a pre-2.6.24 kernel, replace references to arch/x86/ in this book with arch/i386/. Similarly, change any occurrence of include/asm-x86/ to include/asm-i386/. Some filenames within these directories have also changed. Wading through these large directories in search of symbols and other code elements can be a tough task. The tools in Table 1.1 are worthy aids as you navigate the kernel source tree
Table 1.1.Tools That Aid Source Tree NavigationToolDescriptionIxrTheLinuxcross-referencer,Ixr,downloadablefromhttp://lxr.sourceforge.net/,lets you traverse the kernel tree usingawebbrowserbyprovidinghyperlinkstoconnectkernelsymbolswith their definitions and uses.cscope,hostedathttp://cscope.sourceforge.net/,builds a symboliccscopedatabasefromall files inasourcetree,soyoucanquicklylocatedeclarations,definitions,regular expressions,andmore.Cscopemight notbeas versatileas Ixr,but itgives youtheflexibilityofusingthesearchfeaturesofyourfavoritetexteditorratherthanabrowser.From the root of your kernel tree, issuethe cscope -qkRvcommand to build the cross-referencedatabase.The-g optiongeneratesmoreindexing information,so searchesbecomenoticeablyfasterattheexpenseofextra initial startuptime.The-koptionrequestscscopetotuneitsbehaviorto suitkernel sources,-Rasksforrecursivesubdirectorytraversal,and-vappealsforverbosemessages.You can obtainthedetailed invocation syntaxfromthemanpage.The ctags utility,downloadable from http://ctags.sourceforge.net/,ctags/etagsgeneratescross-referencetagsformanylanguages,soyoucanlocatesymbolandfunctiondefinitionsinasourcetreefromwithinaneditor such as vi.Do make tags from the root of yourkerneltree to ctag all source files. The etags utility generates similarindexinginformationunderstoodbytheemacseditor.IssuemakeTAGs to etag your kernel sourcefiles.UtilitiesTools such as grep, find, sdiff, strace, od, dd, make, tar, file, andobjdump.GCCoptionsYou may ask GcC to generate preprocessed source code using the-E option.Preprocessed code contains headerfile expansions andreducesthe need to hop-skip through nested include files toexpand multiplelevels of macros.Hereisa usage exampletopreprocessdrivers/char/mydrv.candproduceexpandedoutputinmydrv.i::-Edrivers/char/mydrv.c-DKERNEL-Iincludebash>gcc-Iinclude/asm-x86/mach-default>mydrv.iThe include paths specified using the -I option depend on theheaderfilesincludedbyyourcode.GCCgeneratesassembly listings ifyouusethe-s option.Togenerate an assembly listing in mydrv.sfordrivers/char/mydrv.c,do this:bash>gcc-sdrivers/char/mydrv.cKERNEL-Iinclude-D-Ianother/include/path
Table 1.1. Tools That Aid Source Tree Navigation Tool Description lxr The Linux cross-referencer, lxr, downloadable from http://lxr.sourceforge.net/, lets you traverse the kernel tree using a web browser by providing hyperlinks to connect kernel symbols with their definitions and uses. cscope cscope, hosted at http://cscope.sourceforge.net/, builds a symbolic database from all files in a source tree, so you can quickly locate declarations, definitions, regular expressions, and more. Cscope might not be as versatile as lxr, but it gives you the flexibility of using the search features of your favorite text editor rather than a browser. From the root of your kernel tree, issue the cscope -qkRv command to build the cross-reference database. The -q option generates more indexing information, so searches become noticeably faster at the expense of extra initial startup time. The –k option requests cscope to tune its behavior to suit kernel sources, -R asks for recursive subdirectory traversal, and –v appeals for verbose messages. You can obtain the detailed invocation syntax from the man page. ctags/etags The ctags utility, downloadable from http://ctags.sourceforge.net/, generates cross-reference tags for many languages, so you can locate symbol and function definitions in a source tree from within an editor such as vi. Do make tags from the root of your kernel tree to ctag all source files. The etags utility generates similar indexing information understood by the emacs editor. Issue make TAGS to etag your kernel source files. Utilities Tools such as grep, find, sdiff, strace, od, dd, make, tar, file, and objdump. GCC options You may ask GCC to generate preprocessed source code using the -E option. Preprocessed code contains header file expansions and reduces the need to hop-skip through nested include files to expand multiple levels of macros. Here is a usage example to preprocess drivers/char/mydrv.c and produce expanded output in mydrv.i: bash> gcc -E drivers/char/mydrv.c -D_KERNEL_ -Iinclude -Iinclude/asm-x86/mach-default > mydrv.i The include paths specified using the -I option depend on the header files included by your code. GCC generates assembly listings if you use the -S option. To generate an assembly listing in mydrv.s for drivers/char/mydrv.c, do this: bash> gcc -S drivers/char/mydrv.c -D_KERNEL_ -Iinclude -Ianother/include/path
BuildingtheKernelNow that you have an idea of the sourcetree layout, let'smake a trivial code change,compile, and get itrunning.Go to thetop-level init/directory and ventureto make a small code changeto the initialization filemain.c.Add a print statement to the beginning of thefunction, start_kernel (),declaring yourlove for polarbears:asmlinkage voidinit start kernel(void)char *command line;extern struct kernel_paramstart_param[],param[];stopprintk("Penguins are cute, but so are polar bearsln");/*+.*/rest init();子You're now ready to kick off the build process. Go to the root of the source tree and start with a clean slate:bash>cd /usr/src/linux-x.Y.z/bash> make cleanConfigure the kernel.This is when you pick and choose the pieces that form part of the operating system.Youmay specify whether each desired component is to be statically or dynamically linked to the kernel:bash>make menuconfigmenuconfig isa text interfaceto thekernel configuration menu.Usemake xconfigtoget a graphical interfaceThe configuration information that you choose is saved in a file named .config in the root of your source tree.Ifyou don't want to weave the configuration from scratch, use the file arch/your-arch/defconfig (or arch/your-arch/configs/your-machine_defconfigifthereareseveralsupportedplatformsforyourarchitecture)asthestartingpoint.So,ifyouarecompilingthekernelforthe32-bitx86architecture,dothis:bash>cparch/x86/configs/i386_defconfig.configCompilethekernel andgenerateacompressed boot image:bash> make bzImageThe kernel image is produced in arch/x86/boot/bzlmage. Update your boot partition:bash>cparch/x86/boot/bzImage/boot/vmlinuzYou might need to alert your bootloader about the arrival of the new boot image. If you are using the GRUBbootloader,it figures this out automatically; but if youare using LILO,raise a flag:
Building the Kernel Now that you have an idea of the source tree layout, let's make a trivial code change, compile, and get it running. Go to the top-level init/ directory and venture to make a small code change to the initialization file main.c. Add a print statement to the beginning of the function, start_kernel(), declaring your love for polar bears: asmlinkage void _init start_kernel(void) { char *command_line; extern struct kernel_param _start_param[], _stop_param[]; + printk("Penguins are cute, but so are polar bears\n"); /* . */ rest_init(); } You're now ready to kick off the build process. Go to the root of the source tree and start with a clean slate: bash> cd /usr/src/linux-X.Y.Z/ bash> make clean Configure the kernel. This is when you pick and choose the pieces that form part of the operating system. You may specify whether each desired component is to be statically or dynamically linked to the kernel: bash> make menuconfig menuconfig is a text interface to the kernel configuration menu. Use make xconfig to get a graphical interface. The configuration information that you choose is saved in a file named .config in the root of your source tree. If you don't want to weave the configuration from scratch, use the file arch/your-arch/defconfig (or arch/yourarch/configs/your-machine_defconfig if there are several supported platforms for your architecture) as the starting point. So, if you are compiling the kernel for the 32-bit x86 architecture, do this: bash> cp arch/x86/configs/i386_defconfig .config Compile the kernel and generate a compressed boot image: bash> make bzImage The kernel image is produced in arch/x86/boot/bzImage. Update your boot partition: bash> cp arch/x86/boot/bzImage /boot/vmlinuz You might need to alert your bootloader about the arrival of the new boot image. If you are using the GRUB bootloader, it figures this out automatically; but if you are using LILO, raise a flag: