Shell Scripts

So far you have been typing commands one at a time into the terminal. Shell scripts let you save a sequence of commands in a file and run them all at once. This is essential for reproducible bioinformatics workflows.

What is a shell script

A shell script is a plain text file containing commands. The shell reads the file and executes each command in order, just as if you had typed them yourself.

Every bash script should start with a shebang line. This tells your system which program should interpret the file.

#!/bin/bash

Create your first script. Open a new file called hello.sh in your text editor and add the following:

#!/bin/bash

echo "Starting analysis..."
date
echo "Done."

Save the file. Before you can run it, you need to make it executable:

$ chmod +x hello.sh

Now run the script:

$ ./hello.sh
Starting analysis...
Fri Mar 28 10:00:00 EDT 2026
Done.

The ./ tells the shell to look for the script in the current directory.

Variables

Variables let you store values and reuse them throughout a script. This avoids repetition and makes your scripts easy to update.

Assign a variable with = and no spaces around the equals sign:

#!/bin/bash

SAMPLE="sample_01"
GENOME="hg38"

echo "Processing sample: $SAMPLE"
echo "Reference genome: $GENOME"

Use curly braces ${} when the variable is next to other text. Without them, the shell cannot tell where the variable name ends.

#!/bin/bash

SAMPLE="sample_01"

# Correct: curly braces separate the variable from the suffix
echo "Read 1: ${SAMPLE}_R1.fastq.gz"
echo "Read 2: ${SAMPLE}_R2.fastq.gz"

# Wrong: the shell looks for a variable called SAMPLE_R1
echo "Read 1: $SAMPLE_R1.fastq.gz"

You can also capture command output in a variable:

#!/bin/bash

TODAY=$(date +%Y-%m-%d)
echo "Analysis date: $TODAY"

Command line arguments

Hard-coding file names into a script limits its usefulness. Command line arguments let you pass values into a script when you run it.

Symbol	Meaning
$1	First argument
$2	Second argument
$@	All arguments
$#	Number of arguments

Here is a script that takes an input FASTQ file and prints its line count:

#!/bin/bash

INPUT_FILE=$1

echo "Counting reads in: $INPUT_FILE"
LINES=$(wc -l < "$INPUT_FILE")
READS=$((LINES / 4))
echo "Total reads: $READS"

Run it by passing a file name:

$ ./count_reads.sh sample_01_R1.fastq
Counting reads in: sample_01_R1.fastq
Total reads: 25000

You can check that the user provided the right number of arguments:

#!/bin/bash

if [ $# -ne 1 ]; then
    echo "Usage: $0 <input.fastq>"
    exit 1
fi

INPUT_FILE=$1
echo "Processing: $INPUT_FILE"

Conditionals

Conditionals let your script make decisions. The basic structure is if/then/else/fi.

#!/bin/bash

INPUT_FILE=$1

if [ -f "$INPUT_FILE" ]; then
    echo "Found input file: $INPUT_FILE"
    echo "Starting analysis..."
else
    echo "Error: file not found: $INPUT_FILE"
    exit 1
fi

Common test operators:

Test	Meaning
-f FILE	True if FILE exists and is a regular file
-d DIR	True if DIR exists and is a directory
-s FILE	True if FILE exists and is not empty
-z STRING	True if STRING is empty
-n STRING	True if STRING is not empty

Here is a practical example. This script checks for a required input file and output directory before running an alignment:

#!/bin/bash

FASTQ=$1
OUTDIR=$2

if [ ! -f "$FASTQ" ]; then
    echo "Error: input file not found: $FASTQ"
    exit 1
fi

if [ ! -d "$OUTDIR" ]; then
    echo "Output directory does not exist. Creating: $OUTDIR"
    mkdir -p "$OUTDIR"
fi

echo "Aligning $FASTQ..."
echo "Results will be saved to $OUTDIR"

For loops

For loops let you repeat a command for each item in a list. This is one of the most useful features in bioinformatics scripting.

Loop over all FASTQ files in a directory:

#!/bin/bash

for FASTQ in data/*.fastq.gz; do
    echo "Running FastQC on: $FASTQ"
    fastqc "$FASTQ" --outdir results/fastqc/
done

Loop over a list of sample IDs:

#!/bin/bash

for SAMPLE in sample_01 sample_02 sample_03; do
    echo "Processing: $SAMPLE"
    echo "  Read 1: ${SAMPLE}_R1.fastq.gz"
    echo "  Read 2: ${SAMPLE}_R2.fastq.gz"
done

You can also read sample IDs from a file:

#!/bin/bash

for SAMPLE in $(cat samples.txt); do
    echo "Processing: $SAMPLE"
done

The file samples.txt would contain one sample ID per line:

sample_01
sample_02
sample_03

While loops

While loops are useful for reading a file line by line. This is common when processing sample sheets or metadata files.

#!/bin/bash

while read -r LINE; do
    echo "Processing: $LINE"
done < samples.txt

A more realistic example reads a CSV sample sheet with multiple columns:

#!/bin/bash

# Skip the header line, then read each row
tail -n +2 samplesheet.csv | while IFS=',' read -r SAMPLE_ID FASTQ_R1 FASTQ_R2; do
    echo "Sample: $SAMPLE_ID"
    echo "  R1: $FASTQ_R1"
    echo "  R2: $FASTQ_R2"
done

The sample sheet might look like this:

sample_id,fastq_r1,fastq_r2
sample_01,sample_01_R1.fastq.gz,sample_01_R2.fastq.gz
sample_02,sample_02_R1.fastq.gz,sample_02_R2.fastq.gz

Exit codes

Every command returns an exit code when it finishes. An exit code of 0 means success. Any other number means something went wrong. You can check the exit code of the last command with $?.

$ ls data/
sample_01_R1.fastq.gz  sample_02_R1.fastq.gz
$ echo $?
0

$ ls nonexistent_folder/
ls: cannot access 'nonexistent_folder/': No such file or directory
$ echo $?
2

In scripts, you should handle errors early. Bash provides special options for this:

Option	Effect
set -e	Exit immediately if any command fails
set -u	Exit if an undefined variable is used
set -o pipefail	Catch errors in piped commands

The recommended practice is to start every script with all three:

#!/bin/bash
set -euo pipefail

Without set -e, a failing command is silently ignored and the script continues. This can lead to corrupted output files or misleading results. Always include this line at the top of your scripts.

A complete example

Here is a realistic script that runs FastQC on every FASTQ file in a directory. It checks inputs, creates an output folder, processes each file, and writes a log.

#!/bin/bash
set -euo pipefail

# Check arguments
if [ $# -ne 1 ]; then
    echo "Usage: $0 <fastq_directory>"
    exit 1
fi

FASTQ_DIR=$1
OUTDIR="results/fastqc"
LOGFILE="results/fastqc_log.txt"

# Verify input directory
if [ ! -d "$FASTQ_DIR" ]; then
    echo "Error: directory not found: $FASTQ_DIR"
    exit 1
fi

# Create output directory
mkdir -p "$OUTDIR"

# Initialize the log file
echo "FastQC analysis log" > "$LOGFILE"
echo "Date: $(date)" >> "$LOGFILE"
echo "Input directory: $FASTQ_DIR" >> "$LOGFILE"
echo "---" >> "$LOGFILE"

# Count input files
FILE_COUNT=$(ls "$FASTQ_DIR"/*.fastq.gz 2>/dev/null | wc -l)
if [ "$FILE_COUNT" -eq 0 ]; then
    echo "Error: no .fastq.gz files found in $FASTQ_DIR"
    exit 1
fi

echo "Found $FILE_COUNT FASTQ files. Starting analysis..."

# Process each file
for FASTQ in "$FASTQ_DIR"/*.fastq.gz; do
    FILENAME=$(basename "$FASTQ")
    echo "Processing: $FILENAME"
    fastqc "$FASTQ" --outdir "$OUTDIR"
    echo "Completed: $FILENAME" >> "$LOGFILE"
done

echo "---" >> "$LOGFILE"
echo "All files processed successfully." >> "$LOGFILE"

echo "Done. Results saved to $OUTDIR"
echo "Log file: $LOGFILE"

Save this as run_fastqc.sh, make it executable, and run it:

$ chmod +x run_fastqc.sh
$ ./run_fastqc.sh data/raw_reads/
Found 6 FASTQ files. Starting analysis...
Processing: sample_01_R1.fastq.gz
Processing: sample_01_R2.fastq.gz
Processing: sample_02_R1.fastq.gz
Processing: sample_02_R2.fastq.gz
Processing: sample_03_R1.fastq.gz
Processing: sample_03_R2.fastq.gz
Done. Results saved to results/fastqc
Log file: results/fastqc_log.txt

Quick reference

Concept	Syntax	Example
Shebang	#!/bin/bash	First line of every script
Make executable	chmod +x script.sh	Run once after creating
Variable	VAR="value"	SAMPLE="sample_01"
Use variable	$VAR or ${VAR}	echo ${SAMPLE}_R1.fastq.gz
First argument	$1	INPUT=$1
All arguments	$@	echo $@
Argument count	$#	if [ $# -ne 1 ]
File exists	[ -f FILE ]	if [ -f "$INPUT" ]
Directory exists	[ -d DIR ]	if [ -d "$OUTDIR" ]
For loop	for X in LIST; do ... done	for F in *.fastq.gz
While read	while read -r LINE; do ... done < file	Read line by line
Exit on error	set -euo pipefail	Top of every script
Last exit code	$?	echo $?

When bash scripts are not enough

Bash scripts are great for automating simple tasks. Running FastQC on a folder of files, downloading data, or setting up a server are all good uses. But bash has real limitations for bioinformatics pipelines.

What bash cannot do well

Bash has no built-in way to handle parallel execution across many samples. If one step in a 10-step pipeline fails at step 7, you have to figure out manually where to restart. Bash does not track which steps have already finished. It also cannot distribute work across multiple machines or cloud instances.

Error handling in bash is fragile. Even with set -euo pipefail, complex pipelines break in subtle ways. A missing file might not cause an error until three steps later, making debugging difficult.

Bash scripts also do not capture the software environment. Your script might work today but fail in six months when a tool gets updated. The script says “run samtools” but does not say which version.

What to use instead

For reproducible bioinformatics pipelines, use a workflow manager like Nextflow with nf-core pipelines. Workflow managers handle:

• Parallelization: Run 50 samples at the same time across multiple CPUs or machines.
• Resume: If step 7 fails, fix the issue and restart from step 7. All previous results are cached.
• Containers: Each step runs inside a container with a specific tool version. The results are the same on any machine.
• Portability: The same pipeline runs on your laptop, an HPC cluster, or AWS.

Bash scripts are a building block. They teach you how commands work together. But a real analysis pipeline needs the structure and safety that workflow managers provide. The Reproducibility section covers this in detail.

Next steps

Now that you can write scripts, the next page covers installing software with apt.