Underscore Shortcuts and Previous Outputs The standard Python shell contains just one simple shortcut for accessing previous output;the variable(i.e.,a single underscore)is kept updated with the previous out- put;this works in IPython as well: In[]:print(_) But IPython takes this a bit further you can use a double underscore toacces the second-to-last output,and a triple underscore to access the third-to-last output(skip- ping any commands with no output): IPython stops there:more than three underscores starts to get a bit hard to count, and at that point it's easier to refer to the output by line number There is one more shortcut we should mention,however-a shorthand for out[x]is X(i.e.,a single underscore followed by the line number): 2974268256817 1n[13]:2 0uti1j:0.9092974268256817 Suppressing Output Sometimes you might wish to suppress the output of a statement(this is perhap mostcommo with the plotting commands that well explore Chapter). maybe the command you're executing produces a result that you'd prefer not to store in your output history,perhaps so that it can be deallocated when other references are 0ediiaiyosprstcoptofaommndsoadamioba In [14]:math.sin(2)+math.cos(2); Note that the result is computed silently,and the output is neither displayed on the screen or stored in the Out dictionary: ho Input and Output History 15
Underscore Shortcuts and Previous Outputs The standard Python shell contains just one simple shortcut for accessing previous output; the variable _ (i.e., a single underscore) is kept updated with the previous out‐ put; this works in IPython as well: In [9]: print(_) 1.0 But IPython takes this a bit further—you can use a double underscore to access the second-to-last output, and a triple underscore to access the third-to-last output (skip‐ ping any commands with no output): In [10]: print(__) -0.4161468365471424 In [11]: print(___) 0.9092974268256817 IPython stops there: more than three underscores starts to get a bit hard to count, and at that point it’s easier to refer to the output by line number. There is one more shortcut we should mention, however—a shorthand for Out[X] is _X (i.e., a single underscore followed by the line number): In [12]: Out[2] Out[12]: 0.9092974268256817 In [13]: _2 Out[13]: 0.9092974268256817 Suppressing Output Sometimes you might wish to suppress the output of a statement (this is perhaps most common with the plotting commands that we’ll explore in Chapter 4). Or maybe the command you’re executing produces a result that you’d prefer not to store in your output history, perhaps so that it can be deallocated when other references are removed. The easiest way to suppress the output of a command is to add a semicolon to the end of the line: In [14]: math.sin(2) + math.cos(2); Note that the result is computed silently, and the output is neither displayed on the screen or stored in the Out dictionary: In [15]: 14 in Out Out[15]: False Input and Output History | 15
Related Magic Commands For accessing a batch of previous inputs at once,the history magic command is very helpful.Here is how you can print the first four inputs In [6]:xhistory-n 1-4 3:math.cos(2) 4:print(In) As usual,you can type %history?for more information and a description of options available.Other similar magic commands are rerun(which will re-execute some portion of the command history)and %save(which saves some set of the command history to a file).For more information,I suggest exploring these using the help functionality discussed in"Help and Documentation in IPython"on page 3. IPython and Shell Commands When working interactively with the standard Python interpreter,one of the frustra- tions you'll face is the need to switch between multiple windows to access Pythor tools and system command-line tools.IPython bridges this gap,and gives you a syn- tax for executing shell commands directly from within the IPython terminal.The The following assumes you're on a Unix-like system,such as Linux or Mac OS X Some of the examples that follow will fail on Windows,which uses a different type of shell by default(though with the 2016 announcement of native Bash shells on Win- dows, oon this may no longe r be an issue!).If you're unfamiliar with shell co mands.Id suggest reviewing the Shell Tutorial put together by the always excellen Software Carpentry Foundation. Quick Introduction to the Shell A full intro to using the shell/terminal/command line is well beyond the scope of this chapter,but for the uninitiated we will offer a quick introduction here.The shell is a way to interact textually with your computer.Ever since the mid-1980s,when Micro soft and Apple introduced the first versions of their now ubiquitous graphical operat ing systems,most computer users have interacted with their operating system through familiar clicking of menus and drag-and-drop movements.But operating ore the graphical ser interfac and d were primarily cor t input:at the prompt,the user would type a com mand,and the computer would do what the user told it to.Those early prompt 16Chapter 1:IPython:Beyond Normal Python
Related Magic Commands For accessing a batch of previous inputs at once, the %history magic command is very helpful. Here is how you can print the first four inputs: In [16]: %history -n 1-4 1: import math 2: math.sin(2) 3: math.cos(2) 4: print(In) As usual, you can type %history? for more information and a description of options available. Other similar magic commands are %rerun (which will re-execute some portion of the command history) and %save (which saves some set of the command history to a file). For more information, I suggest exploring these using the ? help functionality discussed in “Help and Documentation in IPython” on page 3. IPython and Shell Commands When working interactively with the standard Python interpreter, one of the frustra‐ tions you’ll face is the need to switch between multiple windows to access Python tools and system command-line tools. IPython bridges this gap, and gives you a syn‐ tax for executing shell commands directly from within the IPython terminal. The magic happens with the exclamation point: anything appearing after ! on a line will be executed not by the Python kernel, but by the system command line. The following assumes you’re on a Unix-like system, such as Linux or Mac OS X. Some of the examples that follow will fail on Windows, which uses a different type of shell by default (though with the 2016 announcement of native Bash shells on Win‐ dows, soon this may no longer be an issue!). If you’re unfamiliar with shell com‐ mands, I’d suggest reviewing the Shell Tutorial put together by the always excellent Software Carpentry Foundation. Quick Introduction to the Shell A full intro to using the shell/terminal/command line is well beyond the scope of this chapter, but for the uninitiated we will offer a quick introduction here. The shell is a way to interact textually with your computer. Ever since the mid-1980s, when Micro‐ soft and Apple introduced the first versions of their now ubiquitous graphical operat‐ ing systems, most computer users have interacted with their operating system through familiar clicking of menus and drag-and-drop movements. But operating systems existed long before these graphical user interfaces, and were primarily con‐ trolled through sequences of text input: at the prompt, the user would type a com‐ mand, and the computer would do what the user told it to. Those early prompt 16 | Chapter 1: IPython: Beyond Normal Python
systems are the precursors of the shells and terminals that most active data scientists still use today. Someone unfamiliar with the shell might ask why you would bother with this,when you can accomplish many results by simply clicking on icons and menus.A shell user might reply with another question:why hunt icons and click menus when you can accomplish things much n ore easily by typing?While it might sound like a typical tech preference impasse,when moving beyond basic tasks it quickly be ecomes clea that the shell offers much more control of advanced tasks,though admittedly the learning curve can intimidate the average computer user. As an example,here is a sample of a Linux/OS X shell session where a user explores creates,and modifies directories and files on their system(osx:~$is the prompt, and everything after the s sign is the typed command;text that is preceded by a is meant just as description,rather than something you would actually type in): elo o "hello world" echo is like Python's print function horake pwd print working directory this is the "path"that we're in oroec #ls list working directory contents osx:-$cd projects/ cd change directory eeT9as8r8gc myproject.txt osx:projects mkdir myproject nkdir make new directory osx:projects cd myproject/ osx:myproject mv ../myproject.txt./#nv nove file.Here we're mving the up (.to the current directory (/ osx:myproject ls myproject.txt Notice that all of this is just a compact way to do familiar operations (navigating a directory structure,creating a directory,moving a file,etc.)by typing commands rather than clicking icons and menus.Note that with just a few commands(pwd,ls, cd.nkdir,and p)you can do many of the most common file e operation It's wher you go beyond these basics that the shell approach becomes really powerful. IPython and Shell Commands 17
systems are the precursors of the shells and terminals that most active data scientists still use today. Someone unfamiliar with the shell might ask why you would bother with this, when you can accomplish many results by simply clicking on icons and menus. A shell user might reply with another question: why hunt icons and click menus when you can accomplish things much more easily by typing? While it might sound like a typical tech preference impasse, when moving beyond basic tasks it quickly becomes clear that the shell offers much more control of advanced tasks, though admittedly the learning curve can intimidate the average computer user. As an example, here is a sample of a Linux/OS X shell session where a user explores, creates, and modifies directories and files on their system (osx:~ $ is the prompt, and everything after the $ sign is the typed command; text that is preceded by a # is meant just as description, rather than something you would actually type in): osx:~ $ echo "hello world" # echo is like Python's print function hello world osx:~ $ pwd # pwd = print working directory /home/jake # this is the "path" that we're in osx:~ $ ls # ls = list working directory contents notebooks projects osx:~ $ cd projects/ # cd = change directory osx:projects $ pwd /home/jake/projects osx:projects $ ls datasci_book mpld3 myproject.txt osx:projects $ mkdir myproject # mkdir = make new directory osx:projects $ cd myproject/ osx:myproject $ mv ../myproject.txt ./ # mv = move file. Here we're moving the # file myproject.txt from one directory # up (../) to the current directory (./) osx:myproject $ ls myproject.txt Notice that all of this is just a compact way to do familiar operations (navigating a directory structure, creating a directory, moving a file, etc.) by typing commands rather than clicking icons and menus. Note that with just a few commands (pwd, ls, cd, mkdir, and cp) you can do many of the most common file operations. It’s when you go beyond these basics that the shell approach becomes really powerful. IPython and Shell Commands | 17
Shell Commands in IPython You can use any command that works at the command line in IPython by prefixing it with the!character.For example,the ls,pwd,and echo commands can be run as follows: In [1]:!ls myproject.txt holginkerojcts/Awprojet In [3]:!echo "printing from the shell' printing from the shel Passing Values to and from the Shell Shell commands can not only be called from IPython,but can also be made to inter act with the IPython namespace.For example,you can save the output of any shell command to a Python list using the assignment operator: In [4]:contents =!ls In [6]:directory =!pwd In [7]:print(directory) ['/Users/jakevdp/notebooks/tmp/nyproject'] Note that these results are not returned as lists,but as a special shell return type defined in IPython: In [8]:type(directory) IPython.utils.text.SList This looks and acts a lot like a Python list,but has additional functionality,such as the grep and fields methods and the s,n,and p properties that allow you to search, filter,and display the results in convenient ways.For more information on these,you can use IPython's built-in help features. Communication in the other direction-passing Python variables into the shell-is possible through the {varname}syntax: In [9]:message ="hello from Python' het) 18 Chapter 1:IPython:Beyond Normal Python
Shell Commands in IPython You can use any command that works at the command line in IPython by prefixing it with the ! character. For example, the ls, pwd, and echo commands can be run as follows: In [1]: !ls myproject.txt In [2]: !pwd /home/jake/projects/myproject In [3]: !echo "printing from the shell" printing from the shell Passing Values to and from the Shell Shell commands can not only be called from IPython, but can also be made to inter‐ act with the IPython namespace. For example, you can save the output of any shell command to a Python list using the assignment operator: In [4]: contents = !ls In [5]: print(contents) ['myproject.txt'] In [6]: directory = !pwd In [7]: print(directory) ['/Users/jakevdp/notebooks/tmp/myproject'] Note that these results are not returned as lists, but as a special shell return type defined in IPython: In [8]: type(directory) IPython.utils.text.SList This looks and acts a lot like a Python list, but has additional functionality, such as the grep and fields methods and the s, n, and p properties that allow you to search, filter, and display the results in convenient ways. For more information on these, you can use IPython’s built-in help features. Communication in the other direction—passing Python variables into the shell—is possible through the {varname} syntax: In [9]: message = "hello from Python" In [10]: !echo {message} hello from Python 18 | Chapter 1: IPython: Beyond Normal Python
The curly braces contain the variable name,which is replaced by the variable's con- tents in the shell command. Shell-Related Magic Commands If you play with IPython's shell commands for a while,you might notice that you can- not use !cd to navigate the filesystem In[12]:cd… In [13]:!pwd /home/jake/projects/myproject The reason is that shell commands in the notebook are executed in a temporary sub shell.If you'd like to change the working directory in a more enduring way,you can use the %cd magic command: In fact,by default you can even use this without the%sign: This is known as an automagic function,and this behavior can be toggled with the sautomagic magic function. Besides %cd,other available shell-like magic functions are %cat,%cp,%env,%s,%man, xmkdir,more,xmv,pwd,m and %rmdir,any of which can be used without the sign if automagic is on.This makes it so that you can almost treat the IPython prompt as if it's a normal shell: In [16]:mkdir tmp In[17]:s myproject.txt tmp/ In [1]:cp myproject.txt tmp/ In [19]:ls tmp myproject.txt In [20]:rm-r tmp This access to the shell from within the same terminal window as your Python ses sio Shell-Related MagicCommands19
The curly braces contain the variable name, which is replaced by the variable’s con‐ tents in the shell command. Shell-Related Magic Commands If you play with IPython’s shell commands for a while, you might notice that you can‐ not use !cd to navigate the filesystem: In [11]: !pwd /home/jake/projects/myproject In [12]: !cd .. In [13]: !pwd /home/jake/projects/myproject The reason is that shell commands in the notebook are executed in a temporary sub‐ shell. If you’d like to change the working directory in a more enduring way, you can use the %cd magic command: In [14]: %cd .. /home/jake/projects In fact, by default you can even use this without the % sign: In [15]: cd myproject /home/jake/projects/myproject This is known as an automagic function, and this behavior can be toggled with the %automagic magic function. Besides %cd, other available shell-like magic functions are %cat, %cp, %env, %ls, %man, %mkdir, %more, %mv, %pwd, %rm, and %rmdir, any of which can be used without the % sign if automagic is on. This makes it so that you can almost treat the IPython prompt as if it’s a normal shell: In [16]: mkdir tmp In [17]: ls myproject.txt tmp/ In [18]: cp myproject.txt tmp/ In [19]: ls tmp myproject.txt In [20]: rm -r tmp This access to the shell from within the same terminal window as your Python ses‐ sion means that there is a lot less switching back and forth between interpreter and shell as you write your Python code. Shell-Related Magic Commands | 19