Useful IPython magic commands

Buy Me a Coffee☕

An IPython magic command is the special code which basically starts with % or %% but sometimes ! or !!:

*Memos:

• Basically, % or %% works more than ! or !! so only if % or %% doesn't work use ! or !!.
• There are the magic commands which work only with % or ! but not with %% or !!.
• I have never seen the magic commands which work with !!.
• The magic commands with % or ! are for one line of code.
• The magic commands with %% or !! are for a cell(one or more lines of code).
• IPython masic commands can be used on IPython and the IPython-based editors such as Google Colab, JupyterLab, Jupyter Notebook, etc.
• The IPython doc doesn't cover all magic commands.

<% with unix(linux) commands>

*For unix(linux) commands, % should be used because ! doesn't sometimes work.

• %pwd can show the current directory:

%pwd
# '/home/superkai'

• %ls can show the files and folders. *-a can also show hidden files and folders:

%ls -a
# ./  ../ file1 .file2 folder1/ .folder2/

• %cd can change the current directory:

%cd folder1

• %rm can remove(delete) files and folders. *-r enables to remove folders:

%rm -r file1 folder1

<!git commands>

*Memos:

• %git doesn't work.
• Basically, you better use these commands on IPython-based editors so that you don't need to learn their own specific ways to do these operations below. *Colab doesn't support to clone a private repository using API authentication.

1. Clone a public or private repository:

*Memos:

• % doesn't work with git commands.
• My post explains how to clone a private repository with FGPAT(Fine-Grained Personal Access Token) from Github.
• My post explains how to clone a private repository with PAT(Personal Access Token) from Github.

# Clone a public repository

!git clone https://github.com/<usr or org>/<repo>.git

# Clone a private repository

!git clone https://<fgpat or pat>@github.com/<usr or org>/<repo>.git

2. Do add operation:
*Memos:

• -A can stage all (new, modified, deleted) files.
• -a(lowercase) doesn't work.

!git add -A

3. May need config operation to add an email and username:

git config --global user.email "you@example.com"
git config --global user.name "Your Name"

4. Do commit operation. *The command without the message -m "..." gets error:

!git commit -m "New"

5. Do push operation. *The command without main will work:

<%timeit and %%timeit>

*Memos:

• %timeit works for one line of code.
• %%timeit works for a cell(one or more lines of code)
• You can set -r <number> after %timeit and %%timeit e.g. %timeit -r 4 and %%timeit -r 4 which can change the number of repeats from 7 to 4. *You can check more options (flags) for %timeit and %%timeit.

%timeit can calculate the mean(average) and standard deviation(std) of one line of code, repeating 7 times with the suitable number of loops by default as shown below:

%timeit sum(range(1000000))
# 21.1 ms ± 4.79 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)

%%timeit can calculate the mean(average) and standard deviation(std) of a cell(one or more lines of code), repeating 7 times with the suitable number of loops by default as shown below:

%%timeit
sum(range(1000000))
sum(range(1000000))
sum(range(1000000))
# 57.5 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

<%time and %%time>

*Memos:

• %time works for one line of code.
• %%time works for a cell(one or more lines of code)
• CPU times is the total of User CPU time and System CPU time: *Memos:
  • User CPU time is the amount of the time which program uses CPU on user space. *User space is the memory space for applications.
  • System CPU time is the amount of the time which program uses CPU on kernel space. *Kernel space is the memory space for kernel which is the core of the operating system.
• Real is Real time which is the time from when program starts to when program finishes. *Real time is also called wall-clock time or wall time.

%time can calculate CPU time and wall(real) time of one line of code as shown below:

%time sum(range(1000000))
# CPU times: user 19.7 ms, sys: 0 ns, total: 19.7 ms
# Wall time: 19.6 ms
# 499999500000

%%time can calculate CPU time and wall (real) time of a cell(one or more lines of code) as shown below:

%%time
sum(range(1000000))
sum(range(1000000))
sum(range(1000000))
# CPU times: user 60.6 ms, sys: 0 ns, total: 60.6 ms
# Wall time: 63.4 ms
# 499999500000