Managing files in Linux using the Terminal

The terminal isn't as difficult as it's made out to be

People are sometimes afraid of Linux because they associate Linux with a command line window or a terminal. And while it's true a lot of things can be done in a terminal window, these days most tasks can also be done graphically. Why use a terminal at all then if most jobs can be done graphically? The terminal uses fewer resources to accomplish the same task. The terminal is also almost always a faster way to do the same job.

Consider the example of updating the system. If I want to check for updates graphically in Xubuntu I need to:

  1. Click the whisker menu icon in the top left
  2. Click the settings menu
  3. Clicking on the software updater program
  4. Wait for it to check for updates
  5. Once this is done click the Install Now button.
  6. Enter the password for the super user (your password)
  7. Click authenticate for the updates to start.

During this time the updater needs to draw both graphical and text elements to the screen. There are keyboard shortcuts that can save a little time, but they involve a bit of typing. Now consider the same process from the terminal:

  1. Press the super key (Windows key on most keyboards) and the T key to open a terminal window.
  2. In the terminal window type: sudo apt update && sudo apt -y upgrade
  3. Enter the password for the super user (your password)

Once the password is entered the upgrade starts. Fewer steps, and updates happen faster since there are fewer elements to draw on the screen (more text elements, but fewer graphical elements, so this goes by quicker).

Next I'm going to jump into basic navigation commands, but you may want to look at the Very helpful terminal commands section since many of the tips in that section make it a lot easier to type long filenames and quickly recall previously typed commands.

Basic file and directory/folder navigation commands

There are a few basic commands that are helpful to know in order to work with files and directories (folders) on a system:

  • ls - This is a small L and small S. Think of the ls command as list (though typing list won't get you the same result). The ls command lists the contents of a directory/folder.
  • cd - Change Directory. The cd command can be used to change into a different directory/folder from the current one.
  • pwd - Print Working Directory. The pwd command prints to the screen the directory the user is currently in. Although most terminal "shells" list the directory at the prompt, some do not, and sometimes the pwd command can be used as part of multiple commands or a script of commands.
  • cp - Copy. As you might imagine cp is used to copy files and directories.
  • mv - Move. The mv command is used to move or rename files and directories

The command I probably type most often is ls. It's worth mentioning now that Linux, and other UNIX-like operating systems, are case sensitive. Typing Ls or LS will result in a command not found error. The ls command is typed in lower case. Use ls to list the contents of a directory, including any files and directories within the directory you're in.

The ls command run in my home directory
The results of the ls command run in my home directory

In Xubuntu 20.04 the default behaviour of the ls command is to list files and directories alphabetically. When you type the ls command you might not actually see all the files and directories because by default ls does not list "hidden files" (also known as dot files) or "hidden directories." More on how to list hidden files and directories below under the title Switches - things that modify commands.

The next common command I tend to use is cd, or change directory. To change into a directory just type cd and the directory path/name. In the example above to change into the Downloads directory type:

cd Downloads

The command above assumes the Downloads directory is in the current directory. If you've just run the ls command and Downloads is in the list of directories then the command should work.

All the Linux distributions I know normally* start the terminal in the current user's "home" directory. Consider the following example: a Xubuntu Linux desktop system shared by 3 users named Bob, Doug, and Larry. On that Xubuntu Linux desktop there would be 3 home directories at the following locations:

/home/bob
/home/doug
/home/larry

Each of these users has their own "home" directory so they can all keep their own work separate from one another, and have their own specific bookmarks, shortcuts, documents, photographs, etc. Now let's say Bob is interested in beer and has a folder named beer in his home directory. The "full path" to that folder would be:

/home/bob/beer

If Doug wanted to access the beer directory he could not just open a terminal and type cd beer because the directory is in bob's home folder, not his. Instead doug would have to type:

cd /home/bob/beer

Using this command doug would change into the beer directory in bob's home directory. There is a small exception that would prevent doug from changing into bob's beer directory: permissions. Normally when a user creates a directory a certain set of permissions are set on that directory that allow other users to access (list, but not not delete items from) that directory. The owner, in this case bob, can change the permissions on files or directories to prevent other users from listing files within the directory. Permissions are a pretty big topic, and while they're important when you're working with many users on one system, they're too big a topic for this post. A good place to learn more about permissions is Learning the shell - Lesson 9.

The pwd command prints/echos the directory the user is currently in. Sometimes this is displayed in the terminal prompt:

The print working directory (pwd) command
Print Working Directory (PWD) displays the full directory path

The prompt above shows charm@lion:~/Downloads. The first part is the username, charm. After the @ symbol is the hostname/name of the computer (in this case lion). After the colon : is the directory the user is currently in. Recall that ~ is a shortcut for the user's home directory. ~/Downloads in this case is /home/charm/Downloads. If we were the user bob ~/Downloads would be /home/bob/Downloads.

The copy (cp) and move (mv) command work in a similar fashion in that they both need 2 additional arguments: the file or directory your're copying and where you're copying/moving them to. Copy makes an additional copy of the file/directory. Move moves the file or directory to another filename directory. Move can be used to rename files/directories. Consider the following examples:

mv /home/bob/beer /home/bob/molson_canadian
cp /home/bob/beer /home/bob/molson_export
cp /home/bob/beer /home/bob/labatt_50

The first example changes the /home/bob/beer directory to /home/bob/molson_canadian. The second example copies the /home/bob/beer directory as /home/bob/molson_export, but leaves the /home/bob/beer directory in tact. The move mv command actually moves the directory so it only exists as /home/bob/molson_canadian. If we actually ran these commands in order the second two commands wouldn't be possible since we already moved the beer directory to /home/bob/molson_canadian. To actually change the directories we'd need to do the following:

mv /home/bob/beer /home/bob/molson_canadian
cp /home/bob/molson_canadian /home/bob/molson_export
cp /home/bob/molson_canadian /home/bob/labatt_50

In the first command we move the beer directory to molson_canadian, then we copy that directory as two other directories molson_export and labatt_50. If we ran ls we'd see the following:

molson_canadian
molson_export
labatt_50

One thing that sometimes trips new users up is the fact that Linux (and other UNIX-like operating systems) treat small characters and capitals as different entities. The directories /home/bob/beer and /home/bob/Beer are different entities. You can have both in the same directory. It's important to note that most commands are lower case.

Switches - things that modify commands

Switches are useful characters that modify the effects of a command. Switches often beautify or limit output of a command. For example, If I type ls in my home directory the result is:

There are two switches that I often use when I list files, the -a all switch, and the -l use long listing format. These switches could be used separately, ls -a -l, but most commands let you combine switches, ls -al does the same thing as ls -a -l.

Using ls -al in my home directory.
Using ls -al or ls -a -l results in a very different listing in the same home directory

The screenshot above shows that ls -al results in a very different looking listing that just the ls command. What's going on? The ls command on it's own (earlier screenshot) is displaying all the files and directories (folders). The ls -al command and switches list all the files and directories, but also list "hidden files and directories" as well as the permissions, owner of the file/directory, group the file/directory belongs to, size of the file/directory, and the date the last time the file/directory was accessed. Files shown in blue are generally directories, whereas grey files are just that files on the system.

In a directory that has a lot of files or folders typing ls -al might result in many of the files/directories scrolling off the screen. We can tell the terminal to stop at the bottom of the terminal and prompt us for more using the more command. The more command isn't a switch, but a command. To have it act on ls -al we need to use the pipe symbol |. The pipe symbol looks like a small "L" but it's the symbol usually access by pressing SHIFT along with the \ key on the keyboard. So the ls command would look like: ls -al | more and would result in the following:

The ls -al switch using with pipe | and the more command
The results of ls -al | more executed in my home directory

Notice the --More-- at the end, this means there's more to display. Pressing the SPACE key will display the next screen of the listing, pressing the ENTER key would display the next single line of the listing. Also notice how our nice blue directories are no longer blue. How do you know what's a directory and what's a file since everything is grey? The answer is the first character of the display, if it starts with a d it's a directory. If it starts with a - it's a file. By the way, the rwx stuff after are the permissions on the file or directory. Permissions are a bit advanced for this introduction, but we'll cover them later.

Other useful commands

  • man - See the "manual" page for a command
  • more / less - Used to page through files and listings.
  • wget - Used to download files via http, https, and ftp.
  • grep - The grep command is used to match patterns. Grep is often used in conjunction with other commands to limit the results, or to find a particular result.
  • find - Used to find files within a directory structure.
  • tail - See the tail end of a file. For example: tail -n 5 sysinfo.txt shows the last 5 lines from the text file sysinfo.txt
  • rm - Remove, used to delete files, and sometimes directories. Caution needs to be used when using rm since it's possible to remove more than you want.
  • rmdir - Remove Directory. As it looks rmdir is used to remove directories.
  • mkdir - Make Directory. Used to make/create directories.
  • echo - Show or echo something to the screen, can be text, a file or a variable.
  • whoami - Echos to the screen the current user
  • du - Disk Usage, used to estimate file size usage.
  • df - Report Disk Filesystem usage

Manual pages, or man pages, are helpful descriptions of terminal commands. To see a man page type the word man followed by the command name. For example:

man ls

The man (manual) page for the ls command
Part of the man page for the ls command

Manual pages are divided into several sections. Not all sections are available for every man page, it really depends on who created/maintains the man page for each command. Every man page typically includes at least the NAME of the command, a very short SYNOPSIS, and a longer DESCRIPTION of the command, including switches and modifiers. The name is the name of the command with a short description of what it does. The synopsis shows very briefly the switches/options the command takes plus any other parts of the command. In the screenshot above it shows ls [OPTION]... [FILE]. If there's a file named beer in the current directory and you type ls beer the ls command will echo to the screen the word beer (the filename). Directories are also actually files. If you type ls Downloads (a directory in most home directories) you'll see the contents of the Downloads directory.

Listing the Downloads directory from my recent Xubuntu install
Listing the ~/Downloads directory from my recent Xubuntu install, I've downloaded a few files and made directories

The description of the command gives a longer explanation of the command. This description is where to look when trying to figure out how different switches/options affect the output of a command. Switches/options sometimes have multiple ways of typing the switch. For example, if you typing ls -a gets you the same result as typing ls --all. The description of a man page can be interesting in that it can help you learn things about a file you might not learn from examining the file using the graphical user interface. If you right click on the Downloads folder/directory and select properties you get the following information about the folder:

  • Name: Downloads
  • Kind: folder
  • Location: /home/<your_username>
  • Modified: Yesterday
  • Accessed: Yesterday:
  • Size: number of items followed by a space total
  • Usage: space used on the drive the folder resides on

There are more tabs, but the point here I want to make is that more precise information can be found using terminal commands. From your home directory if you type:

ls -al -d Downloads

You'll get a 1 line that shows the permissions (third tab of the graphical user window), the owner of the file, the group the file belongs to, the size of the file, the date the file was created and the file/directory name. It's worth mentioning the -d switch shows the directory itself (Downloads) rather than the contents of the directory. Without the -d switch the information and contents of Downloads would be displayed rather than the information about the Downloads folder/directory. This is where the description of a man page can help add even more precision. The ls command can take a --full-time switch. If you type:

ls -al --full-time -d Downloads/

Very precise information about when the Downloads directory was created appears. Instead of a time of 17:32 you'd see something like 17:32:07.994758992 -0400, down to the precise milliseconds along with the timezone. This is just one example of how the description field is handy (but often overlooked).

Listing my Downloads directory with the --full-time switch and comparing it to the graphical method
Getting more information about the Downloads folder using the terminal vs. graphically

The AUTHOR section of a manual page displays the programmers of the command. This is pretty cool as it allows you to potentially track down the specific author of the command. For ls we can see this is Richard M. Stallman and David MacKenzie. Sometimes the author section even includes the email address(es) of the author(s). Below the author section is the REPORTING BUGS section which displays where to reports bugs with software. The COPYRIGHT information is important if you intend to use a piece of software for a specific purpose. SEE ALSO is handy for learning more information outside of the manual page about a command.

The ls command does not have an EXAMPLES section. I find examples very handy. Sadly few manual pages these days seem to include the EXAMPLES section. The time command includes an examples section. If more program authors included an EXAMPLES section there would be less of a need for blog posts like this.

The EXAMPLES section of the time manual (man) page
The EXAMPLES section of the time man page is particularly helpful

The more and less commands are similar in that they're both used to display the contents of a text file. According to the manual page for the less command it only reads part of a file at a time. The more command reads the entire file into memory before displaying the contents. This difference makes the less command more useful for really long documents. Personally I like the more primitive more command because there are a couple of useful things it does differently which make it handy:

  • for really short files more displays the contents on the command line rather than in an editor of sorts
  • more displays the percentage of a document read for longer documents (use space to page through the more pages)

The space bar can be used in both more and less to page through further pages of a document.

The wget command is used to retrieve files on the Internet from the command line. For example:

wget https://www.gnu.org/licenses/gpl-3.0.txt

The wget command above retrieves the textfile gpl-3.0.txt from the website www.gnu.org's licenses directory. The file doesn't need to be a textfile, it can be a file of any type as long as it's retrievable using http, https, or ftp.

wget https://openclipart.org/download/119419/new-penguin-charles-mcc-01r.svg

The command above grabs a public domain image I created for openclipart.org (a funny looking penguin). If you know the exact location of a file wget can be used with the URL to download the file. Wget can be used to grab more than one piece of content on a website and can traverse the web site using the -r recursive switch:

wget -r https://slashdot.org

Note that wget doesn't grab everything. You won't be able to grab content blocked by robots.txt and other directives, but it can be really handy. I know one person who used wget to scrap political donations from a web site in order to put them in a local database where they were able to run their own analysis on the data.

The grep command is used in conjunction with other commands. To join commands you need a pipe | symbol between the commands. If you had gpl-3.0.txt in your current directory (if you used wget to download it a couple of commands above) you could type:

less gpl-3.0.txt | grep freedom
Grepping freedom from the gpl-3.0.txt document
How many times is freedom mentioned in the GPL 3.0 document?

Less would start to parse through gpl-3.0.txt but the grep command would limit the display only to lines of the textfile containing the word freedom. This is kind of like search, except most graphical searches require you click a next button or text to go to the next result. The grep command will just display everything. This means if the word freedom appears more times than a page will display the top results will scroll off the screen. You can limit this by adding a pipe symbol | and the more or less command at the end. For example:

less gpl-3.0.txt | grep License | more

Even though we use less at the beginning, it won't paginate the License results because grep was used after less. In this case I used more to paginate after the grep command (because I like the -more- prompt rather than less's : prompt). Both would work at then end.

The tail command is used to display the tail end of a document. By default tail displays the last 10 lines of the document.

tail gpl-3.0.txt
Using the tail command on the gpl-3.0.txt document
Tail showing the last 10 lines of gpl-3.0.txt

Using switches you can expand or limit the number of lines displayed:

tail -2 gpl-3.0.txt

Would result in:

Public License instead of this License.  But first, please read
https://www.gnu.org/licenses/why-not-lgpl.html

The rm command is a command to be careful with. It's possible to remove files and directories you don't want to. By default rm only removes files, but if you use certain switches, like the -r recursive switch, it's possible to remove directories and files you don't want to remove, so be careful using the rm command. We can remove the gpl-3.0.txt file from the ~/Downloads folder with the following command:

rm ~/Downloads/gpl-3.0.txt

If you misspell the filename you'll get an error stating the file doesn't exist, but again, if you use switches it's possible to do damage, so again be careful using rm. (I like using a semi-graphical tool that runs in the command line called midnight commander, or mc, when moving/deleting files). If you run the command successfully nothing will be displayed on the screen, but the file will be removed.

The rmdir command removes directories. Directories need to be empty for rmdir to remove them. Running rmdir on ~/Downloads if ~/Downloads was not empty would result in the following error (rmdir ~/Downloads):

rmdir: failed to remove '/home/charm/Downloads/': Directory not empty

If the ~/Downloads directory was empty rmdir would silently remove it the same way rm removes files silently. You can use the -v (--verbose) switch to display more information when using rmdir.

mkdir makes directories if they don't already exist. Type:

mkdir downloads

If you typed this exactly the directory downloads would be made in the current directory. Even if the Downloads directory already existed downloads would be made because in the world of Linux and UNIX Downloads and downloads are different - case matters. (You can then use rmdir downloads to remove the downloads directory you just created).

The echo command is used to echo things to the screen. This is different from more or less in that those commands display the contents of a file. Echo just displays a line of text:

echo ~/Documents/gpl-3.0.txt

Results in:

/home/charm/Documents/gpl-3.0.txt

On your system it will be displayed slightly differently, whatever username you use will be displayed in place of 'charm.' Echo is frequently used to display the value of system variables. A good example is the $PATH variable. The $PATH variable stores information about which paths the terminal has stored in RAM. Why does this matter? This matters because if you're trying to run a command that isn't in one of the directories in $PATH it won't run.

echo $PATH
Echoing the $PATH variable
Stored in path are /usr/local/sbin, /usr/local/bin, /usr/sbin, and so on.

Not sure who you're logged in as, use echo with the $USER variable: echo $USER.

Another command that will display the identity of the current user is the whoami command. Both echo $USER and whoami result in the same (current) username displayed.

df - Disk Filesystem usage

df -hH -x squashfs

The command above displays the disk filesystem sizes in human readable form, but excludes squashfs-based file systems like Ubuntu's snaps.

Commands can be stacked

Commands can be used together to accomplish a task. More than one command can exist on the same line and completely change the output of the initial command that was started with.

Very helpful terminal shortcuts

Shortcuts make learning terminal commands easier. In particular the TAB key is extremely useful. TAB can help complete long commands and filenames. It works something like this:

  1. Start by entering part of a command (for example: fir)
  2. Press the TAB key to complete the rest of the name/command (firefox)

In cases where there is more than one program/file with the same letter sequence pressing TAB a second time will list all the programs/files that begin with that sequence of letters. For example:

  1. Start by entering part of a command (for example: xfce4)
  2. Press the TAB key twice (there are over 20 matches - see image below)
Using TAB filename completion to list command that begin with xfce4- on Xubuntu 20.04
Using double TAB on xfce4- results in many programs

In a case like this where there are many programs/files, typing a few extra characters after can help complete the command. For example: xfce4-ta plus tab will complete the program name xfce4-taskmanager. If you were to press enter after this command it would bring up the graphical task manager for Xubuntu.

But it can get a bit tricky. Take xfce4-sc for example. Typing xfce4-sc and pressing the TAB key results in xfce4-screens. If you pressed enter at this point you would get a command not found error. But if you were to press TAB again after xfce4-screens you would see the 5 or so other commands that begin with xfce4-screens (as opposed to the 20+ we started with just typing xfce4- and hitting TAB). TAB will keep going in a command/filename until you type a character that's different from the others. Consider my list of programs that begin with xfce4-screens:

xfce4-screensaver xfce4-screensaver-preferences xfce4-screensaver-command xfce4-screenshooter xfce4-screensaver-configure

If I then typed the character h and pressed the TAB key the sequence would complete to xfce4-screenshooter, a graphical program to capture a screenshot. I eliminated the 4 other commands. However if I typed the character a and pressed TAB nothing would happen since there are 4 commands that being with xfce4-screensa, hitting TAB again would show me the 4 xfce4-screesaver commands.

TAB filename/command completion is really handy. It might seem a bit complex, but if you just keep pressing TAB and typing characters you'll get used to how easy it really is.

Two other very useful keys when using the terminal are the up and down arrows. Pressing the up arrow after command cycles through your command history. Pressing the down arrow backs down through the history. For example: Consider the following 3 commands I typed and pressed enter after were:

xfce4-screenshooter xfce4-screensaver-preferences clear

I would take quite a bit of time to type each one of these out again, even with filename completion. But if you've already typed them out pressing the UP arrow redraws each command on the screen. When you type a command in the terminal it's kept in a history list. The UP arrow goes through that history list starting with the last command in the list. To see your history list type history in a terminal window.

If you type in a few commands and press the DOWN arrow key notice nothing happens. UP arrow will start to cycle backwards through the commands, but DOWN arrow doesn't do anything because you're already at the end of the list of commands. If you use UP arrow to cycle through the last 5 commands, then use DOWN arrow it will retype those last 5 commands in reverse order.