Files & Permissions

Working with files is the core of everything you do on the command line. This page covers creating, viewing, copying, moving, and deleting files. It also explains the Unix permissions system, which controls who can read, write, and execute each file.

Creating files

The simplest way to create an empty file is with touch.

$ touch samplesheet.csv
$ touch notes.txt

If the file already exists, touch updates its timestamp without changing its contents.

You can also create a file by redirecting output into it with >. This writes text directly to a new file.

$ echo "sample_id,condition,replicate" > samplesheet.csv

If the file already exists, > overwrites it completely. To append instead, use >>.

$ echo "sample_01,treated,1" >> samplesheet.csv
$ echo "sample_02,control,1" >> samplesheet.csv

Viewing file contents

cat

cat prints the entire contents of a file to the screen. It works well for small files.

$ cat samplesheet.csv
sample_id,condition,replicate
sample_01,treated,1
sample_02,control,1

For large files like FASTQ or BAM outputs, cat floods your terminal. Use head or less instead.

head

head shows the first 10 lines by default. Use -n to specify a different number.

$ head -n 4 sample_01.fastq
@SRR123456.1 1 length=150
ATCGATCGATCGATCGATCGATCGATCGATCGATCG
+
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FF

This is useful for quickly checking the format of a FASTQ file.

tail

tail shows the last 10 lines by default. It is the opposite of head.

$ tail -n 2 samplesheet.csv
sample_01,treated,1
sample_02,control,1

tail -f follows a file in real time. This is handy for watching a log file as a pipeline runs.

$ tail -f nextflow.log

Press Ctrl+C to stop following.

less

less opens a file in a scrollable viewer. It loads only what fits on screen, so it handles huge files without trouble.

$ less alignment_results.sam

Inside less:

Press Space to scroll down one page.
Press b to scroll up one page.
Press / then type a word to search forward.
Press q to quit.

wc

wc counts lines, words, and characters. The -l flag counts only lines.

$ wc -l samplesheet.csv
3 samplesheet.csv

This is a quick way to check how many records a file contains. For a FASTQ file, divide the line count by 4 to get the number of reads.

$ wc -l sample_01.fastq
4000000 sample_01.fastq

That file has 1,000,000 reads.

Copying files

cp copies a file to a new location.

$ cp samplesheet.csv samplesheet_backup.csv
$ cp sample_01.fastq.gz /data/raw_reads/

To copy an entire directory, add the -r flag for recursive.

$ cp -r results/ results_backup/

Without -r, cp refuses to copy a directory and prints an error.

Moving and renaming

mv moves a file to a different location. It also serves as the rename command.

Rename a file:

$ mv sample1.fastq.gz sample_01.fastq.gz

Move a file into another directory:

$ mv sample_01.fastq.gz raw_reads/

Move and rename at the same time:

$ mv results/counts.tsv final_output/gene_counts.tsv

Deleting files

rm removes files permanently. There is no trash or recycle bin. Once a file is deleted, it is gone.

$ rm old_alignment.bam

To delete a directory and everything inside it, use -r for recursive.

$ rm -r temp_files/

If you want a confirmation prompt before each deletion, use -i.

$ rm -i *.bam
rm: remove regular file 'sample_01.bam'? y
rm: remove regular file 'sample_02.bam'? n

Wildcards

Wildcards let you match multiple files at once.

The `*` wildcard

* matches any number of characters, including zero.

$ ls *.fastq.gz
sample_01.fastq.gz  sample_02.fastq.gz  sample_03.fastq.gz

Copy all BAM files into a results directory:

$ cp data/*.bam results/

List all CSV files that start with “sample”:

$ ls sample*.csv
samplesheet.csv  sample_metadata.csv

The `?` wildcard

? matches exactly one character.

$ ls sample_0?.fastq.gz
sample_01.fastq.gz  sample_02.fastq.gz  sample_03.fastq.gz

This would not match sample_10.fastq.gz because ? only replaces a single character.

File permissions

Every file and directory on a Linux system has permissions that control who can do what. There are three types of access:

r = read the file
w = write to the file
x = execute the file as a program

These permissions are assigned to three categories of users:

User = the file’s owner
Group = members of the file’s group
Other = everyone else

Reading ls -l output

Run ls -l to see permissions.

$ ls -l
-rw-r--r-- 1 jsmith labgroup  2048 Mar 15 10:30 samplesheet.csv
-rwxr-xr-x 1 jsmith labgroup   512 Mar 14 09:00 run_pipeline.sh
drwxr-xr-x 2 jsmith labgroup  4096 Mar 15 11:00 results

The first column shows the permission string. Here is how to read it:

-rwxr-xr-x
│└┬┘└┬┘└┬┘
│ │  │  └── other: r-x (read and execute)
│ │  └───── group: r-x (read and execute)
│ └──────── user:  rwx (read, write, and execute)
└────────── file type: - means regular file, d means directory

For samplesheet.csv above, the permissions -rw-r--r-- mean:

The owner can read and write.
The group can only read.
Everyone else can only read.

chmod with numeric notation

chmod changes file permissions. The numeric notation uses three digits, one each for user, group, and other. Each digit is the sum of:

Value	Permission
4	read
2	write
1	execute

Common permission combinations:

Number	Permissions	Meaning
755	rwxr-xr-x	Owner full, others read and execute
644	rw-r–r–	Owner read/write, others read only
700	rwx——	Owner full, no access for others
600	rw—––	Owner read/write, no access for others

Set a data file so only you can read and write it:

$ chmod 600 patient_data.csv

Give everyone read access to your results:

$ chmod 644 gene_counts.tsv

Making scripts executable

When you write a shell script, it is not executable by default.

$ ls -l run_pipeline.sh
-rw-r--r-- 1 jsmith labgroup 512 Mar 15 10:00 run_pipeline.sh

$ ./run_pipeline.sh
bash: ./run_pipeline.sh: Permission denied

You need to add execute permission first.

$ chmod +x run_pipeline.sh

$ ls -l run_pipeline.sh
-rwxr-xr-x 1 jsmith labgroup 512 Mar 15 10:00 run_pipeline.sh

$ ./run_pipeline.sh
Starting alignment pipeline...

chmod +x adds execute permission for user, group, and other. Use chmod 755 for the same result with explicit control over all permissions.

Changing ownership

chown changes who owns a file. You will rarely need this on a personal machine. On a shared lab server, it comes up when files need to belong to a specific group so that collaborators can access them.

$ chown jsmith:labgroup samplesheet.csv

This sets the owner to jsmith and the group to labgroup.

To change ownership of a directory and everything inside it, add -R for recursive.

$ chown -R jsmith:labgroup project_data/

Note that chown usually requires administrator privileges. See the sudo section below.

Running commands as administrator: `sudo`

Some commands require administrator privileges. Installing software, changing system files, and modifying ownership of files you do not own all fall into this category.

sudo stands for “superuser do.” It runs a single command with administrator privileges.

# Install a package
$ sudo apt install samtools

# Edit a system file
$ sudo nano /etc/hosts

# Change ownership of another user's file
$ sudo chown labgroup:labgroup shared_data/

After typing sudo, the system asks for your password. It remembers the password for a few minutes so you do not have to type it every time.

When to use sudo

Use sudo only when necessary. Common situations include:

Installing or updating software with apt
Editing files outside your home directory
Starting or stopping system services
Changing ownership of files you do not own

When not to use sudo

Never use sudo for your normal work. Do not run analysis scripts or pipelines as root. If a bioinformatics tool asks you to use sudo, something is probably wrong with the installation. Running as root can overwrite important system files and creates security risks.

A better approach is to install tools in your home directory using package managers like conda or uv. This avoids the need for sudo entirely.

The root user

The root user has full control over the system. On most machines, you do not log in as root directly. Instead, you use sudo to run individual commands with root privileges. This limits the damage if you make a mistake.

On cloud servers like AWS EC2, the default user already has sudo access. On shared HPC clusters, you typically do not have sudo at all. Your system administrator handles software installation.

Quick reference

Command	What it does
`touch file.txt`	Create an empty file
`echo "text" > file.txt`	Create a file with content
`cat file.txt`	Print entire file
`head -n 20 file.txt`	Show first 20 lines
`tail -n 20 file.txt`	Show last 20 lines
`less file.txt`	Open file in scrollable viewer
`wc -l file.txt`	Count lines in a file
`cp src dest`	Copy a file
`cp -r dir/ dest/`	Copy a directory recursively
`mv old new`	Move or rename a file
`rm file.txt`	Delete a file permanently
`rm -r dir/`	Delete a directory permanently
`rm -i file.txt`	Delete with confirmation prompt
`chmod 644 file.txt`	Set read/write for owner, read for others
`chmod +x script.sh`	Make a script executable
`chown user:group file`	Change file ownership
`sudo command`	Run a command as administrator

Next steps

Now that you can manage files and permissions, learn how to chain commands together in Pipes & Redirection.