Table of contents for The Ultimate Linux Handbook 2016 in The Ultimate Linux Handbook

It won’t be long after starting to use Linux that you ask a question and the answer begins with, “Open a terminal and...” At this point, you may be thrown into an alien environment with typed commands instead of cheery-looking icons. But the terminal is not alien, it’s just different. You are used to a GUI now, but you had to learn that, and the same applies to the command line. This raises an obvious question: “I already know how to use a windowed desktop, why must I learn something different?” You don’t have to use the command line, almost anything you need can be done in the GUI, but the terminal has some advantages. It is consistent The commands are generally the same on each distribution while desktops vary.…

If you’re used to Windows, you may be used to each bit of software having its own installer, which gets the appropriate files and puts them in the appropriate places. Linux doesn’t work in the same way (at least, it doesn’t usually). Instead, there is a part of the operating system called the package manager. This is responsible for getting and managing any software you need. It links to a repository of software so it can download all the files for you. Since Linux is built on open source software, almost everything you will need is in the repositories and free. You don’t need to worry about finding the install files, or anything like that – the package manager does it all for you. There are a few different package…

Guess how many files you end up with on your hard drive after a single-CD barebones Debian 8 install. Go on – we’ll wait. Well, the answer is 55,185. That seems almost unbelievable – and impossible to manage – but fortunately the Linux filesystem layout handles this vast number of files well, providing everything with a sensible place to live. You don’t need to know what each individual file does in great detail, but from its location you can determine its overall purpose in the grand scheme of things. Here we’re looking at how the Linux filesystem fits together, partitioning your hard drive and modifying the Grub bootloader’s configuration. We’ll also look at how shared libraries improve security and reduce disk space requirements. As with the other tutorials in this…

As we discovered in the last article, the command line isn’t a crusty, old-fashioned way to interact with a computer, made obsolete by GUIs, but rather a fantastically flexible and powerful way to perform tasks in seconds that would otherwise take hundreds of mouse clicks and may end up in a nasty case of RSI. Additionally, you can’t always rely on the X server functioning properly: Proprietary video drivers in particular have gotten very good at (ahem) rendering it unusable – in which case knowledge of the command line is essential. With said knowledge, a config file can be updated or a module disabled, and your system will be back to normal before you know it. Furthermore, if you’re running Linux as a server OS, you don’t want a hulking…

Your favourite Linux distribution isn’t an individual unit in itself. On the inside, it is made up of various apps, libraries, modules and toolkits. On the outside, it’s part of a much larger and very vibrant ecosystem that sustains several other distros. Also part of this larger Linux ecosystem are very active user communities and various support infrastructures that help nourish the distros and other projects. Over the course of their lifetime, the different elements of the Linux ecosystem interact with each other as well as with their environment. They collaborate, exchange ideas and features, and sometimes swap resources for their mutual benefit as well as for the enhancement of the ecosystem. The Linux ecosystem fosters the development of innovative projects and products. However, since the environment can only sustain…

Your favourite Linux distribution (distro) comes with a set of applications. If it’s one of the top ones, you can be sure that its developers have gone to great lengths to give you an easy to use package manager to swap out the default software that don’t fit your requirements and replace them with the ones that do. But what do you do if you want to alter the distro’s look and feel? After all, the ability to change and alter the default desktop environment, which we’ll call the desktop from here on, is just as important as being able to change and alter the distro’s underlying applications and libraries. The broader Linux ecosystem has multiple desktops, but each Linux distro ships with a single default desktop that goes best…

Sharing files between computers is one of the most fundamental reasons for setting up a local network. It’s something that’s designed to work seamlessly with your day-to-day file browsing, so everything’s handled using shared folders, which are made accessible to other computers connected to the same local network. Users can control exactly who has access to each folder and determine what rights they have over the files within that folder. These rights can usually be boiled down to two basic settings: full read/write access and read-only. The latter makes it possible to share files securely without worrying about your files being overwritten or deleted – other users can copy files from the shared folder as well as view them, but they can’t perform any kind of editing on the folder…

Distributing software and infrastructure as services has established itself in the mainstream with solutions like Google Docs and Amazon Web Services. There’s also Canonical’s Metal as a Service (or MAAS) that is designed to simplify the provisioning of individual server nodes in a cluster. In many ways, MAAS is similar to Infrastructure as a Service or IaaS in that it’s a mechanism for provisioning a new machine. However, the key difference between the two is that while IaaS usually deals with virtual machines, MAAS is designed to provision bare metal. When we say provisioning bare metal, we really mean that. MAAS is designed to bring a server with no operating system installed to a completely working server ready for the user to deploy services on. You can also use MAAS…

In a recent Gartner report on Cloud infrastructure as a service, the Amazon Web Services (AWS) platform was ranked as being in the ‘Magic Quadrant’ (which is Gartner-speak for being the best). While OpenStack remains the most open of cloud ‘operating systems’, AWS is an important platform to know about. While my personal opinion is that a more open ecosystem is preferable, it’s very likely that you will have been, or will be, asked to work with AWS to fire up Linux-based instances and for running all kinds of applications on. I feel it’s only right I cover it to at least at some level. Of course, I want to do this all from the command line if possible and have everything I create stored in version control. I’m going…

While we’re on the subject of online data and its management, let’s take an oh-so-brief look at Redis, a key/value store and part of the ‘NoSQL’ family of software, which seemed incredibly hyped up a few years ago. Actually, calling it a key/value store is a bit unfair; it’s described as “a data structures server, supporting different kind of values”. So as well as handling the traditional string key association with a value type of operation you’d expect, Redis can also handle more complex data structures in the value field (such as lists and hashes). Its biggest selling points are its speed and the number of available clients for it. I’ve seen it used as a very capable session cache, where it has been rock solid and quick to get…

One of the biggest changes that catches Windows users moving to Linux is the way that software is installed. Instead of downloading an executable file from some website or other, running it and hoping it doesn’t clobber your existing library files (DLLs) or install some dubious adware or malware, Linux distributions maintain repositories of software, which are all packaged up for that distro and tested for compatibility with the rest of the distro. In this tutorial, we will look at how this is done by distros that use the Advanced Packaging Tool (apt) software management system, as developed by Debian and used by distros from Ubuntu to Raspbian on the Raspberry Pi. Repositories A repository is a collection of software packages for a distro. Each major release of a distro…

How do you prove just how good your Linux knowledge is? To your friends and family, it’s not so hard – set someone up with a Linux box, let them see you doing a few intricate operations at the command line and they’ll soon be pretty convinced that you’ve got the nous. For companies, however, it’s not so easy. They can’t necessarily tell from a CV, covering letter or interview whether your brain is swelling with useful information on package management, the Linux boot process and so forth. That’s something that has to be confirmed somewhere else. Certified and bonafide Fortunately, here in the Linux world we have an excellent scheme to do just that: LPI Certification. LPI stands for the Linux Professional Institute, a non-profit group that provides exams…

Installing software on Linux – that’s a doddle, right? Just fire up your lovely graphical browser, poke checkboxes next to the apps you fancy and they’ll magically be downloaded from the internet and installed. That’s all well and good for most users, but if you’re looking to be a serious sysadmin some day, you’ll need to know the nitty-gritty of managing packages at the command line, too. (Note: we’ll be covering the command line fully in a later tutorial; we’re just going to focus on a small set of utilities here.) If you’re somewhat new to the world of Linux, it’s worth considering what a package actually is. Ultimately, it’s a single compressed file that expands into multiple files and directories. Many packages contain programs, but some contain artwork and…

We’re coming towards the end of our LPI series of tutorials, with the final instalment just a few scant pages away, so it’s time to look at a few advanced topics that you might come across on your system administrator travels. We’re going to kick off with a look at processes, and how you can manipulate them to your liking. There’s nothing worse than an errant process deleting important files and leaving you feeling helpless, so we’re going to look at solutions to this problem. We’ll also look at filesystems, not in terms of the contents as we covered that earlier, but in how to format partitions as new filesystem types and perform checks on them in case anything goes wrong. Finally, we’ll explore the Vi editor that’s supplied with…

The Debian project has played a significant role in the evolution of Linux and, in many ways, is the first real distribution created for the regular computer user. It was announced in August 1993 and had its first public release later that year, although its first stable release wasn’t available until 1996. The project was even sponsored by the Free Software Foundation from November 1994 to November 1995. A key motivating factor that led Ian Murdock to create a new distro was the perceived poor maintenance and prevalence of bugs in the Softlanding Linux System (SLS) distro. Besides the software itself, Murdock’s release included the Debian Linux Manifesto, which outlined his view for the new project, which prophesied that “distributions are essential to the future of Linux”. In the Manifesto,…

KDE is one of the most configurable desktops and offers a wide range of parameters to help you adapt it to your needs. However, this doesn’t stop many users failing to use it to its full potential because of its elaborate System Settings manager. The latest KDE 5 release has streamlined this crucial component which is now better laid out and easier to navigate. You can extend the options available in the System Settings manager by installing KCMs (KConfig Modules), which power the actual interface for the settings. There are several KCM modules available in your distro’s repos, ie there’s kcm-gtk which can be used to make GTK applications look at home on the desktop. You can install it together with the breeze-gtk theme. Once both packages are installed, head…

Today’s multi-core PCs are built to run multiple tasks simultaneously, and what better way to tap into all that power than through virtualisation? Virtualisation, and in particular hardware virtualisation, is the process of splitting a single physical PC (known as the ‘host’) into multiple virtual PCs (referred to as ‘guests'), each capable of working and acting independently of the other. Virtualisation software allows the host to carve up its memory, processor, storage and other hardware resources in order to share individual parcels with one or more guests. If your PC is powerful enough, you can run multiple virtual machines in parallel, enabling you to effectively split your computer in two to perform different tasks without having to tie up multiple PCs. Virtualisation isn’t simply a means of dividing up computing…

The goal of the CentOS (short for Community ENTerprise Operating System) distribution is to deliver an enterprise-grade operating system without the costs usually associated with such endeavours. It delivers on its Enterprise promise since it’s compiled from open source SRPMs from the Red Hat Enterprise Linux (RHEL) distribution. RHEL is based entirely on open source software and is required by the GPL license to release all of the source code to anyone who has a subscription. Red Hat does one better and makes its source code available to anyone. Note however that while the source code is free, Red Hat still has trademark over the product. That’s where the CentOS developers into the picture. The project takes the freely available RHEL source code (not the binaries), removes trademark and branding…

We’re going to jump back into using Ansible to control Linux-based EC2 instances in the Amazon Web Services cloud, continuing on from our work previously. We’ve already covered some basics and by the end I had a Wordpress blog up and running via the command line. This time, I want to explore AWS further, looking at how I might build up elements of an infrastructure that can be controlled using configuration management via Ansible and one that can cope with a large volume of traffic. Make sure you’ve completed part one, because many of the examples below won’t work without the setup covered there. An updated Ansible config So, previously we ended up with an example site.yml file which created a single EC2 instance. I’ve updated this file since, so…

Since being made the default init system by Fedora 15 in 2011, Systemd has, despite the controversy, seen steady adoption by other distributions. Having made it into both the latest Debian and Ubuntu versions, only Gentoo and Slackware remain as major stalwarts of ye olde SysVinit. There are, of course, a number of smaller and niche distros that do likewise, but the lack of any major exodus of users to any of these distros provides anecdotal evidence that they are at least satisfied with the level of Systemd’s performance and are unswayed by the ideological concerns surrounding it. Indeed, desktop users will typically have witnessed much improved start up times thanks to its parellelisation of startup services and the way it unifies what is a disparate collection of scripts and…

What better way to learn how to use tcpdump than using it to discover how computers communicate over a network. This tutorial combines tcpdump commands and network theory. Using these commands we’ll pull data from a network and demonstrate the key components used for TCP/IP network communication. Data travels through networks, from device to device, in units called protocol data units (PDU), and the composition of a PDU depends on the network protocol. Think of a single PDU as a freight car in a rail yard, which carries goods in the form of data, and each PDU is different depending on the type of data that it’s carrying. Tcpdump is a command line interface utility/tool/program used to capture (ie sniff) the contents of PDUs travelling through a network. The tool…

We’ve looked at various shell commands in the last few tutorials, but they have each been in the context of performing a particular task. It’s time to take an overview of some of the general-purpose commands. There are thousands of terminal commands, from the commonplace to the arcane, but you need only a handful of key commands to get started. Here we will look at some of the core workhorse commands, giving a brief description of what each one is for. As always, the man pages give far more detail on how to use them. Many of these produce more output than can fit in your terminal display, so consider piping them through less. Central to any terminal activity is working with files, creating, removing, listing and otherwise examining them.…

You press the power button on your PC. A bunch of messages scroll by, or perhaps a flashy animation if you’re using a desktop-oriented distro, and finally you arrive at a login prompt. What exactly happens in the mean time? That’s what we’ll be explaining in this instalment of our School of Linux series. Like the last installment, which focused on identifying and managing hardware on your Linux system, this tutorial will help you prepare for Linux Professional Institute (LPI) certification. So it’s useful if you want to get a job in the Linux world, or even if you just want to learn a bit more about your operating system. Linux certification avoids whizz-bang, rapidly updated distros and teaches skills applicable to the more stable, enterprise-friendly flavours such as Red…

Some naysayers would have you believe that the command line is a crusty old relic of the 1970s, a pointless propellerhead playground which real human beings don’t touch. But when it comes to the world of a system administrator, nothing could be further from the truth. The command line, aka shell, is more important than ever – and for good reason: » It’s always there. It exists underneath all the layers of GUI goodness that we see on a typical desktop Linux installation, so even if your window manager is playing up, you can hit Ctrl+Alt+F2 to bring up a prompt and fix it. » It doesn’t require graphics. You can log into a machine remotely (using SSH) from the other side of the planet over a dial-up connection, and…

We’ve come to the final part of this series, and congratulations on making it this far. We really hope you’ve found it useful! Whether you’re an intermediate Linux user looking to find employment with your newfound skills, or you’re a home desktop dabbler who merely wanted to learn more about your operating system, we wish you good luck. It has been really enjoyable to write this too: because Linux is totally open and free, it’s great that we can all delve under the hood and find out precisely what makes it tick. Anyone can become a guru, with the right time, inclination and patience. There are just a few final topics that we haven’t covered yet, so we’ll get them sorted out here. We’ll take a look at filesystem links,…

Gentoo Linux The goal of the Gentoo project was to create a distro without pre-compiled binaries that was tuned to the hardware on which it was installed. Unlike a binary software distribution, the source code is compiled locally according to the user’s preferences and is often optimised. It was initially called Enoch but Gentoo 1.0 was released in 2002. Gentoo has the distinction of being one of the most configurable distros and appeals to Linux users who want full control of the software that’s installed and running on their computer. Gentoo users get to create their system from the ground up: the distro encourages the user to build a Linux kernel tailored to their particular hardware. It allows very fine control of which services are installed and running. Memory usage…

The Debian distribution is a popular choice for creating all kinds of servers. It supports a large number of hardware and is also available in several formats. The recommended installation medium is the minimal netinstall CD that contains just the minimal amount of software to start the installation and fetch the remaining packages over the Internet. One advantage of the netinstall installation is that it only downloads the packages that you selected for installation which saves both time and bandwidth. It also helps cut down the bloat of unnecessary apps and packages. The minimal system is also favourable from a security point of view since it helps avoid installing useless network services that can be used to compromise the server’s integrity. Another advantage of netinstall is that it gets you…

The venerable Apache HTTP server is considered the granddaddy of web servers, although it's only just celebrated its 20th birthday. Recently we’ve extolled the virtues of younger, spryer web servers, (in particular Nginx, but also LiteSpeed and Lighttpd), but Apache has, since 1996, been the most widely used in the world (by any reasonable metric). Sure, if you're just running a simple website then maybe Nginx can serve your pages a few nanoseconds faster, but unless it's a terribly popular website then this is unlikely to trouble you. Indeed compared to Nginx, Apache might look clunky, or even intimidating, with its diverse configuration files and myriad mysteriously monikered modules. But in this tutorial, we'll try to demystify things: Once we've covered the basics, we'll focus on some security and privacy…

There are lots of Linux distributions (distros) out there, each with their own strengths and weaknesses. When you want to try another one you can usually get hold of a live version, or install it in a virtual machine, for a quick test. Both of these are quick and easy but are not the same thing as running an installed distro directly. But perhaps you don’t want to give up on your existing distro or maybe your share a computer with your partner and you prefer Mint but they like Fedora – it would be great to have both. So what are you to do? The term ‘dual booting’ is usually used to refer to having a Linux distro and Windows installed on the same computer and choosing between them…