HPCC – Linux Basics

Manuals: Command Line Basics

Mon, 01 Jan 0001 00:00:00 +0000

Basics

Syntax and Notes

Remember the UNIX/Linux command line is case sensitive!
The hash (pound) sign # indicates end of a command and the start of a comment.
The notation <...> refers to variables and file names that need to be specified by the user. The symbols < and > need to be excluded.
No need to memorize all of these commands, by using these commands you will naturally memorize the most frequently used.
When specifying file names:
- The . (dot) refers to the present working directory
- The ~ (tilde) refers to user’s home directory

Commands

pwd # "Print working directory"; show your current path
ls # "List" contents of current directory
ls -l # Similar to ls, but provides additional info on files and directories
ls -a # List all files, including hidden files (.name) as well
ls -R # Lists subdirectories recursively
ls -t # Lists files in chronological order
cd <dir_name> # "Change directory" to specified path
cd # Brings you to your home directory
cd ~ # Also bring you to your home directory
cd .. # Moves one directory up
cd ../../ # Moves two directories up (and so on)
cd - # Go back to you were previously (before the last directory change)

Informative

file <file-name> # Show type of file (text, binary, compressed, etc...)
id # Shows your user name and associated groups
hostname # Shows the name of the machine your shell is currently on

Files and Directories

mkdir <dir_name> # Creates specified directory
rmdir <dir_name> # Removes empty directory
rm <file_name> # Removes file_name
rm -r <dir_name> # Removes directory including its contents, but asks for confirmation
rm -rf <dir_name> # Same as above, but turns confirmation off. Use with caution
cp <name> <path> # Copy file/directory as specified in path (-r to include content in directories)
mv <name1> <name2> # Renames directories or files
mv <name> <path> # Moves file/directory as specified in path

Copy and paste

The methods to copy and paste on the command line differ depending on your operating systems (ie. Mac OSX, MS Windows, Linux) and your SSH application (ie. Terminal, MobaXTerm).

Linux (xterm)

# Copy
CTRL+SHIFT+C
# Paste
CTRL+SHIFT+V

MS Windows (MobaXTerm)

# Copy by highlighting with mouse
# Paste
SHIFT+INSERT

Mac OSX (Terminal)

# Copy
COMMAND+c
# Paste
COMMAND+v

Shortcuts

Command History

↑ # Up arrow key scrolls backwards through command history
↓ # Down arrow key scrolls forwards through command history
history # Shows all commands you have used recently

Auto-completion

The tab (⇥) key auto completes commands or file names if there is only one option. Hitting the tab (⇥) key twice will list multiple options. Keep in mind that there are no spaces between the tab (⇥) keys and the partial names of commands or files.

Show all directories under my home that I can cd into:

cd ~/⇥⇥

Show all files that I can ls with names that start with “myfile”:

ls myfile⇥⇥

Show all commands that I can run with names that start with “sp”:

sp⇥⇥

Cursor

Ctrl+a # Cursor to beginning of command line
Ctrl+e # Cursor to end of command line
Ctrl+w # Cut last word
Ctrl+k # Cut to the end of the line
Ctrl+y # Paste ("yank") content that was cut earlier (by Ctrl-w or Ctrl-k)

Other Useful Unix Commands

df -h /scratch # Show local disk space for /scratch, do not use for /rhome or /bigdata
free -h # Show memory of current machine
bc # Command-line calculator (to exit type 'quit')
wget <URL> # Download a file or directory from the web
ln -s <FILENAME1> <FILENAME2> # Creates symbolic link (shortcut, or alias) for file or directory
du -sh . # Shows size of current directory
du -sh <FILENAME> # Shows size of individual file
du -s * | sort -nr # Shows size of each file within current directory, sorted by size

Help

Not all command have help documentation available, however one of these methods will likely work:

help <COMMAND> # Show help for a Bash command
man <COMMAND> # Show the manual page for a program (press the 'q' key to exit)
<COMMAND> --help # Show help documentation for command
<COMMAND> -h # Show help documentation for command

Online help: Google is your friend.

Universally available Linux commands, with detailed examples and explanations: https://www.linuxconfig.org/linux-commands

Manuals: File Systems and Transfers

Mon, 01 Jan 0001 00:00:00 +0000

File Systems

The file system in Linux is where you can save data, files, scripts, etc. There are different storage pools based on the path. In Linux you can provide any storage pool from any directory, not like MS Windows systems, where a drive letter is assigned to each storage pool (ie. “C:”,“D:"). This means that by navigating through nested directories, you may find different capacity limits, depending on where you are.

Locations

Most unix system, including Linux, have a common directory hierarchy. The following is called the root level, since it is at the “top” like roots of a inverted tree:

/
|-- bigdata
|-- bin
|-- boot
|-- dev
|-- etc
|-- home
|-- lib
|-- lib64
|-- media
|-- mnt
|-- opt
|-- proc
|-- rhome
|-- root
|-- run
|-- sbin
|-- srv
|-- sys
|-- tmp
|-- usr
`-- var

The two most important directories are /rhome and /bigdata, since this is where your code and data will be stored. These two directories are IBM Spectrum Scale (GPFS) pools, so storage quotas apply. Your home directory lives directly under /rhome and your groups shared storage lives under /bigdata (if extra storage was purchased). These two “bigdata” directories /bigdata/groupname/username and /bigdata/groupname/shared are symlinked (alias/shortcut) to your home directory for convenience, as seen here:

/
|-- bigdata
|-- groupname (Quota based on purchase)
|-- username <-------------|
|-- shared <----------| |
|-- bin | |
|-- boot | |
|-- dev | |
|-- etc | |
|-- home | |
|-- lib | |
|-- lib64 | |
|-- media | |
|-- mnt | |
|-- opt | |
|-- proc | |
|-- rhome | |
|-- username (20GB Quota) | |
|-- shared ---------->| |
|-- bigdata -------------->|
|-- root
|-- run
|-- sbin
|-- srv
|-- sys
|-- tmp
|-- usr
`-- var

For more information regarding these locations, and others, visit HPCC Cluster: Data Storage.

Case sensitive

All paths and commands are case sensitive, an uppercase letter is not the same as a lowercase letter.

Path Types

An absolute path is a full path from top to bottom, from the root to the leaf:

/rhome/username/example_dir/example_file

A relative path is a partial path with the current working directory is the starting point:

example_dir/example_file

Commands

Here are many common commands related to files and file systems (run man <command> for more information):

pwd # Print working directory
ls # List files in directory
touch # Make an empty file
mkdir # Make a directory
cd # Change to directory
cp # Copy file[s] from a directory to a directory
mv # Move file[s] from a directory to a directory
rm # Remove a file
rmdir # Remove an empty directory
df # Check size of storage pool
du # Check size of file or directory
check_quota # Check quota for home and bigdata

Note: CTRL+c will cancel a running command

File Transfers

This section has moved to the Data Sharing page.

Manuals: Permissions and Ownership

Mon, 01 Jan 0001 00:00:00 +0000

Overview

In Linux (and Unix systems in general), access to files and directories is controlled by a system of owners, groups, and permission bits. Changing these settings is necessary to control access by other users. The permission system also affects what files can be executed.

Ownership Levels

user (u) - User ownership of a file/directory. This user has the special right to change the permission bits and group ownership.
group (g) - Group ownership of a file/directory. Members of this group may be assigned greater access rights than non-members.
other (o) - Everyone else that isn’t the owning user or from the owning group.

Permission Bits

The elemental permissions in Linux/Unix are read, write, and execute. Users and groups can have one many, or none of these rights. Their meanings are as follows:

	Letter	Number	File	Directory
Read	r	4	View the contents	View the listings
Write	w	2	Modify the contents	Create a new file, or rename or delete existing files
Execute	x	1	Execute a program/script	Traversal rights

Checking Permissions

Annotated output for ls -la:

---------- File type (d = directory, - = regular file, l = symlink)
|--------- User permission triplet
|| ------ Group permission triplet
|| | --- Other permission triplet
|| | |
|| | | [user] [group]
drwx-----x 61 username groupname 4096 Feb 24 16:39 ./
drwxr-xr-x 688 root root 262144 Feb 24 11:05 ../
drwx------ 2 username groupname 4096 Feb 2 22:45 .ssh/
drwxr-xr-x 5 username groupname 4096 Dec 12 15:57 Downloads/
drwxr-xr-x 2 username groupname 4096 Jan 9 16:29 bin/
-rw------- 1 username groupname 7960 Feb 23 18:37 .bash_history
-rw-r--r-- 1 username groupname 306 Nov 3 15:08 .bashrc
-rw-r--r-- 1 username groupname 677 Apr 8 2013 .profile
-rw-r--r-- 1 username groupname 128 Nov 30 12:38 .tmux.conf
-rw-r--r-- 1 username groupname 12126 Nov 2 13:14 .vimrc
lrwxrwxrwx 1 username groupname 23 Sep 12 10:49 bigdata -> /bigdata/groupname/username/
-rw-r--r-- 1 username groupname 5657 Sep 19 11:31 bookmarks.html
lrwxrwxrwx 1 username groupname 23 Sep 12 10:49 shared -> /bigdata/groupname/shared/

Assign write and execute permissions to user and group

chmod ug+rx my_file

To remove all permissions from all three user groups

chmod ugo-rwx my_file
# '+' causes the permissions selected to be added
# '-' causes them to be removed
# '=' causes them to be the only permissions that the file has.
chmod +rx public_html/ or $ chmod 755 public_html/ # Example for number system:

Change ownership

chown <user> <file or dir> # changes user ownership
chgrp <group> <file or dir> # changes group ownership
chown <user>:<group> <file or dir> # changes user & group ownership

Manuals: Finding Things

Mon, 01 Jan 0001 00:00:00 +0000

Find Files

find ~ -name "*pattern*" # Searches for *pattern* in and below your home directory
find ~ -iname "*pattern*" # Same as above, but case insensitive
find ~ -type f -mtime -2 # Searches for files you have modified in the last two days

Useful find arguments:

-user <userName>
-group <groupName>
-ctime <number of days ago changed>
-exec <command to run on each file> {} \;

Find Text

grep "pattern" <FILENAME> # Provides lines in a file where "pattern" appears
grep -H "pattern" # -H prints out file name in front of pattern
find ~ -name "*.txt" -exec grep -H "pattern" {} \; # Search lines where "pattern" appears in files with names that end with ".txt"

Find Applications

which <APPLICATION_NAME> # Location of application
whereis <APPLICATION_NAME> # Searches for executables in set of directories
rpm -qa | grep "pattern" # List all RPM packages and filter based on "pattern"

Manuals: Text Editors

Mon, 01 Jan 0001 00:00:00 +0000

Text Viewing

Here are a few commands that are used to just display the text within a file:

more <FILENAME> # Views text, use space bar to browse, 'q' to quit
less <FILENAME> # Also views text, uses arrow keys to browse, 'q' to quit
cat <FILENAME> # Concatenates files and prints content to screen

Text Editors

Nano
- A simple terminal-based editor.
Neovim
- Non-graphical (terminal-based) editor. Neovim is an improved version of vim.
Vim Gvim
- Non-graphical (vim) or window-based editor (gvim). Vim is the improved version of vi.
Emacs
- Non-graphical or window-based editor.
Visual Studio Code
- Graphical editor that runs on your local machine that supports different plugins.

Nano

The nano editor is the simplest to use and can be good for beginners:

nano <FILENAME> # Open file if it exists, or create it

Navigation in nano uses the arrow keys, and all other commands are noted at the bottom of the screen. The CTRL key is used in combination with other keys to execute commands in nano.

For example, at the bottom of the nano screen it is noted that ^X is used to exit. This means you will need to hold the CTRL key and then press x in order to quit. After that, just follow the on screen prompts at the bottom.

For more nano commands, please visit Overview of nano shortcuts.

Neovim / Vim / GVim / VI

All of these editors follow the same principals.

nvim <FILENAME> # Open file if it exists, or create it
vim <FILENAME> # Open file if it exists, or create it
gvim <FILENAME> # Open file if it exists, or create it (must have XForwarding or VNC)
vi <FILENAME> # Open file if it exists, or create it

For more information please visit Vim Manual.

Emacs

Navigation in emacs also uses the arrow keys. It is similar to nano, in that, CTRL is combined with other keys to execute commands.

For example, to open a file, simply run the command with a file name:

emacs <FILENAME> # Open file if it exists, or create it

Then, after you have made some changes, exit by holding the CTRL key and then pressing c, releasing and then holding the CTRL key once more and pressing c again. After that, just follow the on screen prompts at the bottom.

For more commands in emacs please visit GNU Emacs Reference Card

Visual Studio Code

Install

This editor should be installed on your local machine (ie. workstation, laptop). Please visit Visual Studio Code for software download.

Remote Editing

To setup Visual Studio Code to remotely edit files on the cluster, please refer to our guide to VSCode Usage on HPCC.

RStudio Server

The shared web instances of RStudio will soon be deprecated and replaced with Open OnDemand. Please start transitioning there to receive the best performance from RStudio.

Two options exist to access the RStudio Server IDE:

Shared Web Instance
Compute Node Instance

While the Shared Web Instance is easier for less experienced users, it does not allow to load a specific R version nor access more extensive computing resources. Thus, experienced users may prefer the Compute Node Instance as it does not share these limitations.

1. Shared Web Instance

R users can log in to their HPCC accounts via an RStudio Server instance. To do so, visit the HPCC RStudio Server. Next provide your HPCC login credentials and click the Sign In button.

2. Compute Node Instance

a. Interactive

Alternatively, an RStudio Server instances can be started directly on a compute node and accessed via an SSH tunnel. This involves the following steps.

SSH into the cluster as outlined here.
Log in to a compute node interactively via srun, where the proper parameters need to be specified by the user. These include partition name, RAM, number of CPU cores, wall time limit, etc. Additional details on using srun are available here.

srun --partition=short --mem=8gb --cpus-per-task=2 --ntasks=1 --time=2:00:00 --pty bash -l

Load the latest versions of R and RStudio Server using the module system:

module unload R
module load R/4.3.0 # Or latest version (previously it was R/4.2.2)
module load rstudio-server/2023.09.1-494 # Or latest version (previously it was rstudio-server/2022.02.0-443)

Start RStudio Server:

start-rserver.sh

Next follow the instructions printed to the terminal after running start-rserver.sh. The command-lines given in these instructions need be executed in a terminal of a user’s local system, and not on the remote system where start-rserver.sh was exectuted.

b. Non-Interactive

The steps for launching an interactive job can be integrated into a script and submitted non-interactvely for a quicker deployment of a RStudio Server instance on a compute node. Instructions outling how to do this are located here.

Jupyter Server

The shared web instance of Jupyter will soon be deprecated and replaced with Open OnDemand. Please start transitioning there to receive the best performance from Jupyter.

Two options exist to access JupyterLab:

Web Instance
Compute Node Instance

1. Jupyter Web Instance

Users can log into their HPCC accounts via the JupyterHub server instance. To do so, visit the HPCC JupyterHub server. Next provide your HPCC login credentials and click the Sign In button.

Account changes can sometimes lead to users needing to restart active cluster sessions, and Jupyter is no different. To restart your Jupyter session, from the “File” tab click “Hub Control Panel”. From the new screen click “Stop My Server”, then “Start My Server”. After a few seconds your session will be restarted.

Choosing A Profile

There are a handful of profiles that can be selected from, below is a chart displaying the existing profiles.

Profile Name	Partition	Resources	Time Limit	Notes
Immediate Server	-	Shared	None	This will run Jupyter on a shared machine with all other users who choose this option. Choose this if you want a server with no time limit at the expense of worse performance.
Quick Server	Short	2 cores, 8GB memory	2 Hours	This is for a quick session lasting under 2 hours. Good for testing or debugging work quickly.
Compute Intel	Intel	8 cores, 32GB memory	24 hours	This will run on an Intel machine with a 24 hour time limit.
Compute Batch	Batch	8 cores, 32GB memory	24 hours	This will run on an Batch (AMD) machine with a 24 hour time limit.
Compute Epyc	Epyc	8 cores, 32GB memory	24 hours	This will run on an AMD machine with a 24 hour time limit.
Highmem	Highmem	8 cores, 128GB memory	24 hours	This will run on the highmem partition, good for jobs that require a lot of memory.

NOTE that when your job reaches the specified time limit the job will be terminated and jupyter session closed. Jupyter autosaves every 2 minutes, but any new work will be lost and any running jobs cancelled once the time limit is reached.

If your job requires more flexible resources, please refer to the below section for running Jupyter directly on a compute node.

2. Jupyter Compute Node Instance

a. Steps

First, start an interactive session on a compute node

srun -p epyc -t 4:00:00 -c 4 --mem=10GB --pty bash -l # Customize as needed

After your job has been scheduled, activate the jupyterlab module and start the server.

module load jupyterlab/
start-jupyter.sh

A fed seconds after running start-jupyter.sh it will prompt you for a password. Enter a password that you would like to use to access the notebook. NOTE: Text will not show up when you type your password, this is expected.

After entering a password, it will print some text guiding you on creating a tunnel.

NOTE: The port and node will likely be different than pictured. This is expected, and unique to each session.

As the text suggests, enter the ssh -NL command in your terminal or setup MobaXTerm with the supplied details. If using the ssh method, the terminal is expected to hang after logging in and no further output should be generated.

After a few seconds the server will start. At this point you can navigate to “http://127.0.0.1:PORT/lab” on your local machine, **replacing PORT with the port assigned when running the start-jupyter.sh command (9345 in the example above).

b. Logging In

After navigating to the login page, you will be prompted for the password you originally gave the notebook.

After logging in, you can use the notebook as you would on our hosted JupyterLab server.

c. Shutting Down

When you are finished with your session, you can stop the Jupyter server from running by going to “File > Shut Down” in the notebook, or by entering Ctrl+C in the terminal window.

Manuals: Streams

Mon, 01 Jan 0001 00:00:00 +0000

Streams

On the command line, or terminal, there are three very important lanes where information can be sent, we call these streams. A single command can take information in from STDIN and then send information out on both STDOUT and STDERR simultaneously.

STDIN

For example, we can send the contents of a file as a STDIN steam to the wc command in order to count the lines:

wc -l < file.txt

STDOUT

The STDOUT steam is probably the most often used, since this is how commands send information to the screen. However, if we do not want the information printed to the screen, we can send it into a file for later review:

ls > output.txt # Overwrite contents in output file with `ls` results

You can also append to the same file, if more information is to be saved:

ls >> output.txt # Append results from `ls` to the bottom of the file

STDERR

The error stream is very useful to separate error messages (or warnings) from real output (your results). Since there is no -e flag for the ls command this will generate an error. We can then store this error in a by redirecting the error stream with 2>.

ls -e 2> errors.txt

Tips

Combined streams

If you want to combined your STDOUT with your STDERR stream and store it into a file, you can do this with &>, like so:

command &> output_and_errors.txt

Trash Streams

If you want to ignore all information from STDOUT and STDERR you can send both of these streams to the trash (/dev/null):

command &> /dev/null

This can be useful when you are only interested in the resulting file that your command will create.

Manuals: Piping

Mon, 01 Jan 0001 00:00:00 +0000

Piping

One of the the most powerful things you can do in Linux is piping. This allows chaining of commands so that the output (STDOUT) of one command is the input (STDIN) for another. This is done by placing a | (pipe) character between the commands. Please note that not all commands support this, for example if your command is not taking input from STDIN.

As an example, let’s collect all the lines where pattern is found in a file, then count how many lines were found:

grep 'pattern' filename | wc -l

You can pipe as many commands together as you like, not just two. For example, you can combined two CSV files and extract the first column, then filter for only unique values:

cat filename1.csv filename2.csv | cut -f 1 | sort | uniq

For a few more simple examples, please visit here Pipe, Grep and Sort Command in Linux/Unix with Examples. Or you can try searching Google for even more complex examples, the possibilities are endless.

Manuals: Variables

Mon, 01 Jan 0001 00:00:00 +0000

Variables

The HPCC cluster uses bash as the default shell environment. Within this environment, variables can be set and reused.

For example:

MYVAR=’Something’
export MYVAR=’Something’
echo $MYVAR

Default Variables

Some softwares utilize this feature and require that specific environment variables be set. For example, every time you login, the following variables are set by default:

echo $HOME #Contains your home path
echo $USER #Contains your username
echo $PATH #Contains paths of executables
echo $LD_LIBRARY_PATH #Contains paths of library dependencies

Finding Variables

To see a list of all variables currently set in your shell, use the env command. You can also grep through this list to find variables, like so:

env | grep -i home

Or if you are in a Slurm job, you can find all related Slurm variables:

env | grep -i slurm

Setting variables

Try to choose unique names when setting variables. It is best to not overwrite a variable that is already set, unless on purpose.

To set a variable in your current shell, you can do so like this:

MYVAR='Something Important'

Notice that there is no spaces around the = sign.

If you would like to set a variable that is carried over to all other commands or sub-shells, then it must be exported:

export MYVAR='Something Important'

Manuals: Scripting

Mon, 01 Jan 0001 00:00:00 +0000

Scripting

Converting code into a script is useful, and almost necessary when running jobs on the cluster.

There are many benifits of doing this:

Easy to run - blackbox
Easy to maintain - consolidated code
Easy to distribute - capsulated code
Easy to automate (crontab?) - does not require interaction

Breakdown

There are four basic parts that are needed to convert your commands into a script:

First save all your commands into a file and call it myscript.sh, you can do this with a Text Editor or Transferring it from your computer.
Add the #! (SheBang) as the first line in file, this defines the interpreter. In this example, we are using bash as the interpreter, which will run all subsequent lines in this file:

#!/bin/bash

Add the proper permissions to the script, allow user (or group) execution:

chmod u+x myscript.sh

chmod g+x myscript.sh

You can pass arguments via command line into a script, this step is optional, but important to note.

For example if I want to call my script, like so:

/where/my/script/lives/myscript.sh username number

Then inside my script I can capture the command line arguments into variables, like this:

username=$1
number=$2

Lastly adding the path to a script to the PATH environment variable, allows us to call the script without a prefixed path:

export PATH=/where/my/script/lives/:$PATH # Can be added to .bashrc for convenience

After we have exported PATH with the new path of our script, we call it like so:

myscript.sh username number

Walkthrough

My bash commands:

sacct -n -p -u jhayes -S 2020-01-01 -l > myjobs.txt
cut -d'|' -f4 myjobs.txt > partitions.txt
wc -l partitions.txt > count.txt

Convert the above commands into a script named myscript.sh, with the following contents:

#!/bin/bash
# Gather Slurm job information
sacct -n -p -u jhayes -S 2020-01-01 -l > myjobs.txt
# Filter on parittion column
cut -d'|' -f4 myjobs.txt > partitions.txt
# Count how many records per partition
cat partitions.txt | sort | uniq -c > count.txt

Optional, we can alter the above commands by adding some pipes, as well as adding some variables to make this script count records for only a given partition:

#!/bin/bash -l
# Gather Slurm job information
# filter on partition column
# count how many records for given partition
sacct -n -p -u $1 -S 2020-01-01 -l | cut -d'|' -f4 | grep $2 | wc -l > count.txt

Add correct permissions:

chmod u+x myscript.sh

Add to my PATH:

mkdir -p ~/bin
mv myscript.sh ~/bin
export PATH=~/bin:$PATH

Now run my new script:

myscript.sh

Or, if we did the optional step of adding variables, we can do this:

myscript.sh johndoe001 intel # Arguments are <USERNAME> and <PARTITION>

Manuals: Process Management

Mon, 01 Jan 0001 00:00:00 +0000

Process Management

Basic Linux process management commands only apply to processes that are running on the current machine you are logged into. This means that you cannot use these commands to manage jobs. Jobs on the cluster are managed through Slurm, see Cluster Jobs for more details. However, these commands are still useful for pausing, backgrounding, killing processes on a login node directly. This commands could also be useful when running an interactive session on a compute node.

User Management

top # view top consumers of memory and CPU (press 1 to see per-CPU statistics)
who # Shows who is logged into system
w # Shows which users are logged into system and what they are doing

Process Management

Processes

ps # Shows processes running by user
ps -e # Shows all processes on system; try also '-a' and '-x' arguments
ps ux -u <USERNAME> # Shows all processes owned by user
ps axjf # Shows the child-parent hierarchy of all processes
ps -o %t -p <PID> # Shows how long a particular process was running.
# (E.g. 6-04:30:50 means 6 days 4 hours ...)

Here are two common utilities for displaying processes, sorting, and even killing them:

top # Basic text based interface for exploring and managing processes
htop # Text based interface for exploring and managing processes

Note q to quit and ? to see help

Background Resume Cancel

CTRL+z ENTER # Suspend a process in the background
fg # Resume a suspended process and brings it into foreground
bg # Resume a suspended process but keeps it running in the background
CTRL+c # Cancel the process that is currently running in the foreground

PID

echo $! # Get PID of last executed command

Killing

kill -l # List all of the signals that can be sent to a process
kill <PID> # Kill a specific process with process ID using SIGTERM
kill -9 <PID> # Violently kill process with process ID using SIGKILL, may corrupt files

More on Terminating Processes

DigitalOcean - How To Use ps, kill, and nice to Manage Processes in Linux

Manuals: Shell Bootcamp

Mon, 01 Jan 0001 00:00:00 +0000

The Unix Shell

When you log into UNIX/LINUX system, then is starts a program called the Shell. It provides you with a working environment and interface to the operating system. Usually there are several different shell programs installed. The shell program bash is one of the most common ones.

finger <user_name> # shows which shell you are using
chsh -l # gives list of shell programs available on your system (does not work on all UNIX variants)
<shell_name> # switches to different shell

STDIN, STDOUT, STDERR, Redirections, and Wildcards

See LINUX HOWTOs

By default, UNIX commands read from standard input (STDIN) and send their output to standard out (STDOUT).

You can redirect them by using the following commands:

<beginning-of-filename>* # * is wildcard to specify many files
ls > file # prints ls output into specified file
command < my_file # uses file after '<' as STDIN
command >> my_file # appends output of one command to file
command | tee my_file # writes STDOUT to file and prints it to screen
command > my_file; cat my_file # writes STDOUT to file and prints it to screen
command > /dev/null # turns off progress info of applications by redirecting
# their output to /dev/null
grep my_pattern my_file | wc # Pipes (|) output of 'grep' into 'wc'
grep my_pattern my_non_existing_file 2 > my_stderr # prints STDERR to file

Useful shell commands

cat <file1> <file2> > <cat.out> # concatenate files in output file 'cat.out'
paste <file1> <file2> > <paste.out> # merges lines of files and separates them by tabs (useful for tables)
cmp <file1> <file2> # tells you whether two files are identical
diff <fileA> <fileB> # finds differences between two files
head -<number> <file> # prints first lines of a file
tail -<number> <file> # prints last lines of a file
split -l <number> <file> # splits lines of file into many smaller ones
csplit -f out fasta_batch "%^>%" "/^>/" "{*}" # splits fasta batch file into many files
# at '>'
sort <file> # sorts single file, many files and can merge (-m)
# them, -b ignores leading white space, ...
sort -k 2,2 -k 3,3n input_file > output_file # sorts in table column 2 alphabetically and
# column 3 numerically, '-k' for column, '-n' for
# numeric
sort input_file | uniq > output_file # uniq command removes duplicates and creates file/table
# with unique lines/fields
join -1 1 -2 1 <table1> <table2> # joins two tables based on specified column numbers
# (-1 file1, 1: col1; -2: file2, col2). It assumes
# that join fields are sorted. If that is not the case,
# use the next command:
sort table1 > table1a; sort table2 > table2a; join -a 1 -t "$(echo -e '\t')" table1a table2a > table3 # '-a <table>' prints all lines of specified table!
# Default prints only all lines the two tables have in
# common. '-t "$(echo -e '\t')" ->' forces join to
# use tabs as field separator in its output. Default is
# space(s)!!!
cat my_table | cut -d , -f1-3 # cut command prints only specified sections of a table,
# -d specifies here comma as column separator (tab is
# default), -f specifies column numbers.
grep # see chapter 4
egrep # see chapter 4

Screen

Screen references

Starting a New Screen Session

screen # Start a new session
screen -S <some-name> # Start a new session and gives it a name

Commands to Control Screen

Ctrl-a d # Detach from the screen session
Ctrl-a c # Create a new window inside the screen session
Ctrl-a Space # Switch to the next window
Ctrl-a a # Switch to the window that you were previously on
Ctrl-a " # List all open windows. Double-quotes " are typed with the Shift key
Ctrl-d or type exit # Exit out of the current window. Exiting form the last window will end the screen session
Ctrl-a [ # Enters the scrolling mode. Use Page Up and Page Down keys to scroll through the window. Hit the Enter key twice to return to normal mode.

Attaching to Screen Sessions

From any computer, you can attach to a screen session after SSH-ing into a server.

screen -r # Attaches to an existing session, if there is only one
screen -r # Lists available sessions and their names, if there are more then one session running
screen -r <some-name> # Attaches to a specific session
screen -r <first-few-letters-of-name> # Type just the first few letters of the name
# and you will be attached to the session you need

Destroying Screen Sessions

Terminate all programs that are running in the screen session. The standard way to do that is: Ctrl-c
Exit out of your shell: exit
Repeat steps 1 and 2 until you see the message: [screen is terminating]

There may be programs running in different windows of the same screen session. That’s why you may need to terminate programs and exit shells multiple time.

Tabs and a Reasonably Large History Buffer

For a better experience with screen, run

cp ~/.screenrc ~/.screenrc.backup 2> /dev/null
echo 'startup_message off
defscrollback 10240
caption always "%{=b dy}{ %{= dm}%H %{=b dy}}%={ %?%{= dc}%-Lw%?%{+b dy}(%{-b r}%n:%t%{+b dy})%?(%u)%?%{-dc}%?%{= dc}%+Lw%? %{=b dy}}"
' > ~/.screenrc

Simple One-Liner Shell Scripts

Web page for script download.

Renames many files *.old to *.new. To test things first, replace ‘do mv’ with ‘do echo mv’:

for i in *.input; do mv $i ${i/\.old/\.new}; done
for i in *\ *; do mv "$i" "${i// /_}"; done # Replaces spaces in files by underscores

Run an application in loops on many input files:

for i in *.input; do ./application $i; done

Run fastacmd from BLAST program in loops on many *.input files and create corresponding *.out files:

for i in *.input; do fastacmd -d /data/../database_name -i $i > $i.out; done

Run SAM’s target99 on many input files:

for i in *.pep; do target99 -db /usr/../database_name -seed $i -out $i; done
Search in many files for a pattern and print occurrences together with file names.
for j in 0 1 2 3 4 5 6 7 8 9; do grep -iH <my_pattern> *$j.seq; done

Example of how to run an interactive application (tmpred) that asks for file name input/output:

for i in *.pep; do echo -e "$i\n\n17\n33\n\n\n" | ./tmpred $i > $i.out; done

Run BLAST2 for all .fasa1/.fasta2 file pairs in the order specified by file names and write results into one file:

for i in *.fasta1; do blast2 -p blastp -i $i -j ${i/_*fasta1/_*fasta2} >> my_out_file; done

This example uses two variables in a for loop. The content of the second variable gets specified in each loop by a replace function.

Runs BLAST2 in all-against-all mode and writes results into one file ('-F F' turns low-complexity filter off):

for i in *.fasta; do for j in *.fasta; do blast2 -p blastp -F F -i $i -j $j >> my_out_file; done; done;

How to write a real shell script

Create file which contains an interpreter as the first line:
```
#!/bin/bash
```
Place shell commands in file below the interpreter line using a text editor.
Make file executable:
```
chmod +x my_shell_script
```
Run shell script like this:
```
./my_shell_script
```

Place it into your /rhome//bin directory

mkdir -p ~/bin
mv my_shell_script ~/bin/

Add the bin path to your shell permanently:

echo 'export PATH=~/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

Simple One-Liner Perl Scripts

Small collection of useful one-liners:

perl -p -i -w -e 's/pattern1/pattern2/g' my_input_file
# Replaces a pattern in a file by a another pattern using regular expressions.
# $1 or \1: back-references to pattern placed in parentheses
# -p: lets perl know to write program
# -i.bak: creates backup file *.bak, only -i doesn't
# -w: turns on warnings
# -e: executable code follows

Parse lines based on patterns:

perl -ne 'print if (/my_pattern1/ ? ($c=1) : (--$c > 0)); print if (/my_pattern2/ ? ($d = 1) : (--$d > 0))' my_infile > my_outfile
# Parses lines that contain pattern1 and pattern2.
# The following lines after the pattern can be specified in '$c=1' and '$d=1'.
# For logical OR use this syntax: '/(pattern1|pattern2)/'.

Remote Copy: wget, scp, ncftp

Wget

Use wget to download a file from the web:

wget ftp://ftp.ncbi.nih.... # file download from www; add option '-r' to download entire directories

SCP

Use scp to copy files between machines (ie. laptop to server):

scp source target # Use form 'userid@machine_name' if your local and remote user ids are different.
# If they are the same you can use only 'machine_name'.

Here are more scp examples:

scp user@remote_host:file.name . # Copies file from server to local machine (type from local
# machine prompt). The '.' copies to pwd, you can specify # here any directory, use wildcards to copy many files.
scp file.name user@remote_host:~/dir/newfile.name
# Copies file from local machine to server.
scp -r user@remote_host:directory/ ~/dir
# Copies entire directory from server to local machine.

Nice FTP

From the linux command line run ncftp and use it to get files:

ncftp
ncftp> open ftp.ncbi.nih.gov
ncftp> cd /blast/executables
ncftp> get blast.linux.tar.Z (skip extension: @)
ncftp> bye

Archiving and Compressing

Creating Archives

tar -cvf my_file.tar mydir/ # Builds tar archive of files or directories. For directories, execute command in parent directory. Don't use absolute path.
tar -czvf my_file.tgz mydir/ # Builds tar archive with compression of files or directories. For
# directories, execute command in parent directory. Don't use absolute path.
zip -r mydir.zip mydir/ # Command to archive a directory (here mydir) with zip.
tar -jcvf mydir.tar.bz2 mydir/ # Creates *.tar.bz2 archive

Viewing Archives

tar -tvf my_file.tar
tar -tzvf my_file.tgz

Extracting Archives

tar -xvf my_file.tar
tar -xzvf my_file.tgz
gunzip my_file.tar.gz # or unzip my_file.zip, uncompress my_file.Z,
# or bunzip2 for file.tar.bz2
find -name '*.zip' | xargs -n 1 unzip # this command usually works for unzipping
# many files that were compressed under Windows
tar -jxvf mydir.tar.bz2 # Extracts *.tar.bz2 archive

Try also:

tar zxf blast.linux.tar.Z
tar xvzf file.tgz

Important options:

f: use archive file
p: preserve permissions
v: list files processed
x: exclude files listed in FILE
z: filter the archive through gzip

Simple Installs

Systems-wide installations

Applications in user accounts

Installation of RPMs

Environment Variables

xhost user@host # adds X permissions for user on server.
echo $DISPLAY # shows current display settings
export DISPLAY=<local_IP>:0 # change environment variable
unsetenv DISPLAY # removes display variable
env # prints all environment variables

List of directories that the shell will search when you type a command:

echo $PATH

You can edit your default DISPLAY setting for your account by adding it to file .bash_profile

Exercises

Exercise 1

Download proteome of Halobacterium spec. with wget and look at it:

module load ncbi-blast/2.2.26 # Loads legacy blastall
wget ftp://ftp.ncbi.nlm.nih.gov/genomes/genbank/archaea/Halobacterium_salinarum/representative/GCA_000069025.1_ASM6902v1/GCA_000069025.1_ASM6902v1_protein.faa.gz
gunzip GCA_000069025.1_ASM6902v1_protein.faa.gz
mv GCA_000069025.1_ASM6902v1_protein.faa AE004437.faa
less AE004437.faa # press q to quit

Simple Analysis:

a. How many predicted proteins are there?
```
grep '^>' AE004437.faa --count
```
b. How many proteins contain the pattern “WxHxxH” or “WxHxxHH”?
```
egrep 'W.H..H{1,2}' AE004437.faa --count
```
c. Use the find function (/) in ‘less’ to fish out the protein IDs containing the pattern or more elegantly do it with awk:
```
awk --posix -v RS='>' '/W.H..(H){1,2}/ { print ">" $0;}' AE004437.faa | less # press q to quit
```

Create a BLASTable database with formatdb:

ls # before
formatdb -i AE004437.faa -p T -o T
ls # after
'-p F' for nucleotide and '-p T' for protein database; '-o T' parse SeqId and create indexes

Generate myseq.fasta

a. Generate list of sequence IDs for the above pattern match result (i.e. retrieve my_IDs from step 2c). Alternatively, download the pre-generated file with wget.

b. Retrieve the corresponding sequences for these IDs with the fastacmd command from the blastable database:
```
wget https://cluster.hpcc.ucr.edu/~tgirke/Documents/UNIX/my_IDs
fastacmd -d AE004437.faa -i my_IDs > myseq.fasta
less myseq.fasta # press q to quit
```
(Optional) Looking at several different patterns:

a. Generate several lists of sequence IDs from various pattern match results (i.e. retrieve a.my_ids, b.my_ids, and c.my_ids from step 2c).

b. Retrieve the sequences in one step using the fastacmd in a for-loop:
```
for i in *.my_ids; do fastacmd -d AE004437.faa -i $i > $i.fasta; done
```

Run blastall with a few proteins in myseq.fasta against your newly created Halobacterium proteome database.

Create first a complete blast output file including alignments. In a second step use the ’m -8' option to obtain a tabular output (i.e. tab separated values):

blastall -p blastp -i myseq.fasta -d AE004437.faa -o blastp.out -e 1e-6 -v 10 -b 10
blastall -p blastp -i myseq.fasta -d AE004437.faa -m 8 -e 1e-6 > blastp.tab
less blastp.out # press q to quit
less -S blastp.tab # -S disables line wrapping, press q to quit

The filed descriptions of the Blast tabular output (from the “-m 8” option) are available here:

1 Query (The query sequence id)
2 Subject (The matching subject sequence id)
3 % id
4 alignment length
5 mismatches
6 gap openings
7 q.start
8 q.end
9 s.start
10 s.end
11 e-value
12 bit score

Is your blastp.out file equivalent to this one?

Parse blastall output into Excel spread sheet

a. Using biocore parser
```
blastParse -i blastp.out -o blast.xls -c 5
```
b. Using BioPerl parser
```
bioblastParse.pl blastp.out > blastparse.txt
```

Exercise 2

Split sample fasta batch file with csplit (use sequence file myseq.fasta from Exercise 1).

csplit -z myseq.fasta '/>/' '{*}'

Delete some of the files generated by csplit Concatenate single fasta files from (step 1) into to one file with cat (e.g. cat file1 file2 file3 > bigfile). BLAST two related sequences, retrieve the result in tabular format and use comm to identify common hit IDs in the two tables.

Exercise 3

Run HMMPFAM search with proteins from Exercise 1 against Pfam database (will take ~3 minutes)

hmmscan -E 0.1 --acc /srv/projects/db/pfam/2011-12-09-Pfam26.0/Pfam-A.hmm myseq.fasta > output.pfam

Easier to parse/process tabular output

hmmscan -E 0.1 --acc --tblout output.pfam /srv/projects/db/pfam/2011-12-09-Pfam26.0/Pfam-A.hmm myseq.fasta # also try --domtblout

Which query got the most hits? How many hits were found that query?

Exercise 4

Create multiple alignment with ClustalW (e.g. use sequences with ‘W.H..HH’ pattern)

clustalw myseq.fasta
mv myseq.aln myalign.aln

Exercise 5

Reformat alignment into PHYILIP format using ‘seqret’ from EMBOSS

seqret clustal::myalign.aln phylip::myalign.phylip

Exercise 6

Create neighbor-joining tree with PHYLIP

cp myalign.phylip infile
protdist # creates distance matrix (you may need to press 'R' and then 'Y')
cp outfile infile
neighbor # use default settings (press 'Y')
cp outtree intree

retree # displays tree and can use midpoint method for defining root of tree, my typical command sequence is: ‘N’ (until you see PHYLIP) ‘Y’ ‘M’ ‘W’ ‘R’ ‘R’ ‘X’

cp outtree tree.dnd

View your tree in TreeBrowse or open it in TreeView

Manuals: Linux Basics - Vim Manual

Mon, 01 Jan 0001 00:00:00 +0000

Note: Neovim has replaced vim on the cluster, however most (if not all) items here still apply. Load Neovim via the module system, like so module load neovim. It may also be useful to alias nvim to vim, like so alias vim=nvim. Thia alias can be added to your ~/.bashrc file for convenience.

Vim Manual

This is an extensive overview of vim features and operations.

Basics

vim <my_file_name> # open/create file with vim

Once you are in Vim the most important commands are i , : and ESC. The i key brings you into the insert mode for typing. ESC brings you out of there. And the : key starts the command mode at the bottom of the screen. In the following text, all commands starting with : need to be typed in the command mode. All other commands are typed in the normal mode after hitting the ESC key.

Modifiers

i # INSERT MODE
ESC # NORMAL (NON-EDITING) MODE
: # Commands start with ':'
:w # Save command; if you are in editing mode you have to hit ESC first!!
:q # Quit file, don't save
:q! # Exits WITHOUT saving any changes you have made
:wq # Save and quit
R # Replace MODE
r # Replace only one character under cursor
q: # History of commands (from NORMAL MODE!), to reexecute one of them, select and hit enter!
:w new_filename # Saves into new file
:#,#w new_filename # Saves specific lines (#,#) to new file
:# # Go to specified line number

Moving Around

$ # Moves cursor to end of line
A # Same as $, but switches to insert mode
0 # Zero moves cursor to beginning of line
CTRL-g # Shows file name and current line you are on
SHIFT-G # Brings you to bottom of file

Lines

:set wrap # Wrap lines around the screen if too long
:set nowrap # No line wrapping
:set number # Shows line numbers
:set nonumber # No line numbers

Multiple Files

vim -o *.txt # Opens many files at once and displays them with horizontal
# Split, '-O' does vertical split
vim *.txt # Opens many files at once; ':n' switches between files

:wall or :qall # Write or quit all open files
:args *.txt # Places all the relevant files in the argument list
:all # Splits all files in the argument list (buffer) horizontally
CTRL-w # Switch between windows
:split # Shows same file in two windows
:split <file-to-open> # Opens second file in new window
:vsplit # Splits windows vertically, very useful for tables, ":set scrollbind" let's you scroll all open windows simultaneously
:close # Closes current window
:only # Closes all windows except current one

Spell Checking

:set spell # Turns on spell checking
:set nospell # Turns spell checking off
:! dict <word> # Meaning of word
:! wn 'word' -over # Synonyms of word

Syntax Highlighting

:set filetype=perl # Turns on syntax coloring for a chosen programming language.
:syn on # Turns syntax highlighting on
:syn off # Turns syntax highlighting off

Undo and Redo

u # Undo last command
U # Undo all changes on current line
CTRL-R # Redo one change which was undone

Deleting

x # Deletes what is under cursor
dw # Deletes from curser to end of word including the space
de # Deletes from curser to end of word NOT including the space
cw # Deletes rest of word and lets you then insert, hit ESC to continue with NORMAL mode
c$ # Deletes rest of line and lets you then insert, hit ESC to continue with with NORMAL mode
d$ # Deletes from cursor to the end of the line
dd # Deletes entire line
2dd # Deletes next two lines, continues: 3dd, 4dd and so on.

Copy and Paste

yy # Copies line, for copying several lines do 2yy, 3yy and so on
p # Pastes clipboard behind cursor

Search

/my_pattern # Searches for my_pattern downwards, type n for next match
?my_pattern # Searches for my_pattern upwards, type n for next match
:set ic # Switches to ignore case search (case insensitive)
:set hls # Switches to highlight search (highlights search hits)

Replacements

Great intro: A Tao of Regular Expressions

Quick reference to some replacement techniques:

:s/old_pat/new_pat/ # Replaces first occurrence in a line
:s/old_pat/new_pat/g # Replaces all occurrence in a line
:s/old_pat/new_pat/gc # Add 'c' to ask for confirmation
:#,#s/old_pat/new_pat/g # Replaces all occurrence between line numbers: #,#
:%s/old_pat/new_pat/g # Replaces all occurrence in file
:%s/\(pattern1\)\(pattern2\)/\1test\2/g # Regular expression to insert, you need here '\' in front of parentheses (<# Perl)
:%s/\(pattern.*\)/\1 my_tag/g # Appends something to line containing pattern (<# .+ from Perl is .* in VIM)
:%s/\(pattern\)\(.*\)/\1/g # Removes everything in lines after pattern
:%s/\(At\dg\d\d\d\d\d\.\d\)\(.*\)/\1\t\2/g # Inserts tabs between At1g12345.1 and Description
:%s/\n/new_pattern/g # Replaces return signs
:%s/pattern/\r/g # Replace pattern with return signs!!
:%s/\(\n\)/\1\1/g # Insert additional return signs
:%s/\(^At\dg\d\d\d\d\d.\d\t.\{-}\t.\{-}\t.\{-}\t.\{-}\t\).\{-}\t/\1/g # Replaces content between 5th and 6th tab (5th column), '{-}' turns off 'greedy' behavior
:#,#s/\( \{-} \|\.\|\n\)/\1/g # Performs simple word count in specified range of text
:%s/\(E\{6,\}\)/<font color="green">\1<\/font>/g # Highlight pattern in html colors, here highlighting of >= 6 occurences of Es
:%s/\([A-Z]\)/\l\1/g # Change uppercase to lowercase, '%s/\([A-Z]\)/\u\1/g' does the opposite

Uses ‘global’ command to apply replace function only on those lines that match a certain pattern. The ‘copy $’ command after the pipe ‘|’ prints all matching lines at the end of the file.

:g/my_pattern/ s/\([A-Z]\)/\l\1/g | copy $

Command ‘args’ places all relevant files in the argument list (buffer); ‘all’ displays each file in separate split window; command ‘argdo’ applies replacement to all files in argument list (buffer); flag ‘e’ is necessary to avoid stop at error messages for files with no matches; command ‘update’ saves all changes to files that were updated.

:args *.txt | all | argdo %s/\old_pat/new_pat/ge | update

Utilities

Matching Parentheses
- Place cursor on (, [ or { and type % # cursor moves to matching parentheses
Printing and Inserting Files
- :ha # Prints entire file
- :#,#ha # Prints specified lines: #,#
- :r <filename> # Inserts content of specified file after cursor
Convert Text File to HTML Format
- :runtime! syntax/2html.vim # Run this command with open file in Vim
Shell Commands in Vim
- :!<SHELL_COMMAND> <ENTER> # Executes any shell command, hit <enter> to return
- :sh # Switches window to shell, 'exit' switches back to vim
Using Vim as Table Editor
- v starts visual mode for selecting characters
- V starts visual mode for selecting lines`
- CTRL-V starts visual mode for selecting blocks (use CTRL-q in gVim under Windows). This allows column-wise selections and operations like inserting and deleting columns. To restrict substitute commands to a column, one can select it and switch to the command-line by typing :. After this the substitution syntax for a selected block looks like this: '<,'>s///.
- :set scrollbind starts simultaneous scrolling of ‘vsplitted’ files. To set to horizontal binding of files, use command :set scrollopt=hor (after first one). Run all these commands before the :split command.
- :AlignCtrl I= \t then :%Align This allows to align tables by column separators (here ‘\t’) when the Align utility from Charles Campbell’s is installed. To sort table rows by selected lines or block, perform the visual select and then hit F3 key. The rest is interactive. To enable this function, one has to include in the .vimrc file the Vim sort script from Gerald Lai.

Settings

The default settings in Vim are controlled by the .vimrc file in your home directory.

see last chapter of vimtutor (start from shell)
useful .vimrc sample
when vim starts to respond very slowly then one may need to delete the .viminf* files in home directory

Help

Online Help
- Find help on the web. Google will find answers to most questions on vi and vim (try searching for both terms).
- Purdue University Vi Tutorial
- Animated Vim Tutorial: https://linuxconfig.org/vim-tutorial
- Useful list of vim commands:
  - Vim Commands Cheat Sheet
  - VimCard

You can run a tutor from the command Line:

vimtutor # Open vim tutorial from shell, ":q" to quit

You can also get help from within Vim:

:help # opens help within vim, hit :q to get back to your file
:help <topic> # opens help on specified topic
:help_topic| CTRL-] # when you are in help this command opens help topic specified between |...|,
# CTRL-t brings you back to last topic
:help <topic> CTRL-D # gives list of help topics that contain key word
: <up-down keys> # like in shell you get recent commands!!!!

HPCC – Linux Basics

Manuals: Command Line Basics

Basics

Syntax and Notes

Commands

Navigation and Exploration

Informative

Files and Directories

Copy and paste

Shortcuts

Command History

Auto-completion

Cursor

Other Useful Unix Commands

Help

Manuals: File Systems and Transfers

File Systems

Locations

Case sensitive

Path Types

Commands

File Transfers

Manuals: Permissions and Ownership

Overview

Ownership Levels

Permission Bits

Checking Permissions

Change ownership

Manuals: Finding Things

Find Files

Find Text

Find Applications

Manuals: Text Editors

Text Viewing

Text Editors

Nano

Neovim / Vim / GVim / VI

Emacs

Visual Studio Code

Install

Remote Editing

RStudio Server

1. Shared Web Instance

2. Compute Node Instance

a. Interactive

b. Non-Interactive

Jupyter Server

1. Jupyter Web Instance

Choosing A Profile

2. Jupyter Compute Node Instance

a. Steps

b. Logging In

c. Shutting Down

Manuals: Streams

Streams

STDIN

STDOUT

STDERR

Tips

Combined streams

Trash Streams

Manuals: Piping

Piping

Manuals: Variables

Variables

Default Variables

Finding Variables

Setting variables

Manuals: Scripting

Scripting

Breakdown

Walkthrough

Manuals: Process Management

Process Management

User Management

Process Management

Processes

Background Resume Cancel

PID

Killing