Table of contents for November/December 2019 in ADMIN Network & Security

A tax-compliant archive, a data warehouse, or an enterprise resource planning (ERP) system largely processes the same data, but unfortunately with a different persistence level. Companies can no longer afford this kind of redundancy. Deepshore provides an alternative concept that draws on blockchain as an important component in its implementation and thus adds often-missed value to the overall system. Superfluity As if the sheer volume of electronic information were not already challenging enough, virtually all large companies in this country allow themselves the luxury of processing data multiple times and then storing the data redundantly. Even today, the areas of archiving, data warehouses, and ERP are still organizationally and technically isolated. IT departments do not like people seeing their hand, and setting the isolation of applications in stone does not…

Soon after C became the first language to be callable from Python, people wanted the same for the millions of lines of proven Fortran code in scientific applications. In the past, combining C and Fortran was dependent on the compiler. The advent of Fortran 2003 [1] brought a standard, compiler-independent C/Fortran interface. The approach for combining C and Fortran for Python is to write a C wrapper for the Fortran functions and subroutines that you want to use in Python. Then, you build the Python function around the C wrapper and use it as a Python module. Calling the C function calls the Fortran routine. Although a little indirect, the concept is straightforward. In the first example, I integrate Cython and Fortran. Fortran/C and Cython The Fortran 90 [2] website,…

Linux has never been the first choice with multimedia users, although the free operating system offers various software for multimedia applications, including tools for collectors of video DVDs and Blu-ray discs that allow you to back up your media to mass storage devices. Backing up is recommended, because optical data carriers do not age well, and mechanical effects such as scratches or cracks can damage discs and even make them unreadable. In this article, I look at the HandBrake and MakeMKV tools for transcoding optical video discs. (See the “Not Considered” box.) Legal If you want to play back self-mastered Blu-ray discs or convert them into other formats, you usually won’t run into any problems. Often however, commercial discs will not play on just any old device because of integrated…

Perhaps one of the most underreported yet regular tasks of a system administrator is to parse logfiles. Scrolling through logfiles isn’t fun, but they are an indispensable resource for helping troubleshoot an issue. Seasoned admins will tell you that they almost always instinctively pull up the logs whenever they face any issues with their installations. Linux, for its part, is ever vigilant and makes careful logs of everything that takes place within your system. Most logfiles under Linux reside under the /var/log/directory. These are all system and service logs, which come in handy when troubleshooting systemwide issues. Different apps also write app-specific logs, which are kept in a configurable location usually under a user’s home directory. You don’t necessarily need to use any special tools to read logfiles, since they…

Managing a network full of computers can sap the energy out of any admin, irrespective of the number of computers in your network. Whether you oversee a lab full of computers at school or the network of a multinational company with offices across the world, you can use the FOG project [1] to manage them with ease. FOG helps take the pain out of timeconsuming and repetitive tasks such as imaging and cloning computers. FOG also makes light of regular administrative tasks such as installing software and can even manage printers on the network. All theatrics aside, FOG’s most essential task is to help you image an installation that can then be deployed to other computers on the network, which is what I’ll cover in this tutorial. Set Up the…

AFICK (another file integrity checker) detects changes to the system and sounds an alert. The tool first creates a unique fingerprint of selected files in the form of a checksum. If a different checksum is computed during a later check, a malicious program, an attacker, or a defect is likely to have modified the files under investigation. In this way, AFICK not only detects manipulation attempts, but also acts as a small intrusion detection system. The tool is licensed under the liberal GNU GPLv3 license, which also allows free use in the enterprise. AFICK only requires Perl v5.10 or newer. Developer Eric Gerbier has tested his tool under all Windows versions from XP upward, various Unix systems (e.g., HPUX and AIX), and numerous Linux distributions (e.g., SUSE, Red Hat, Debian,…

Zuul 3 brings a new approach to the DevOps challenge. (For more on these important DevOps concepts, see the “CI and CD” box.) Whereas other CI/CD tools like Jenkins, Bamboo, or TeamCity focus on constant building and testing, Zuul’s main design goal is to be a gating mechanism. In other words, as soon as a proposed change is verified and approved by a human, it can undergo final tests and be deployed automatically to the production environment. Such a system requires a lot of computing power, but Zuul has been proven reliable and highly scalable by its designers: the Open-Stack community. The general concept behind the Zuul series (Zuul 3 at the time of writing) is to provide developers a system for automatically building, testing, and finally merging new changes…

In the past [1], I discussed how instruction cache misses can completely ruin your CPU’s day when it comes to execution speed by stalling processing for thousands or even millions of cycles while program text is being retrieved from slower caches, or even from (gosh!) RAM [2] [3]. Now the time has come to turn your attention to the data cache: Where large enough datasets are involved, spatial locality and cache-friendly algorithms are among the first optimizations you should employ. Three Times Slower Compare the code presented in Listings 1 and 2. You might use diff [4] to verify that the only difference is in the order of the i and j variables in the array lookup. Listing 1 increments 100 million integers, scanning the array row by row, whereas…

Nonvolatile Memory Express (NVMe)-based storage solutions are currently the fastest way to transfer data from memory to processing, which means you can use the latest generation of multicore CPUs and GPUs that require massively parallel processing capacity. NVMe is capable of accelerating databases and largescale virtual machine (VM) farms to unfold the potential of greater performance density and the advantages of shared storage. The result is significant performance advantages for demanding applications that are slowed down by previous storage technologies. The added speed ensures that website content is loaded before the viewer leaves. In complex financial transactions, a difference of microseconds can determine the gain or loss of multidigit sums. NVMe is gaining importance in all business scenarios where speed of data access directly affects revenue. Fast Interface NVMe is…

One of the disruptive innovations of recent years, which shifted into focus only in the wake of cloud computing and containers, is undoubtedly the microarchitecture approach to computing (see the “Finally Agile” box). Microarchitecture application developers, however, are faced with the task of operating all of the separate microservices and components and maintaining them as part of their application. This is where the Istio developers enter the scene. They promise container application developers a central network interface in which the parts of an application talk. The required services (i.e., load balancing, filtering, discovery of existing services, etc.) are provided by Istio. To be able to use all these features, you only have to integrate Istio into your application. Communication Needed For the individual parts of a microservice application to work…

The Information Technology Infrastructure Library (ITIL) is now more than 30 years old. The currently most widespread standard for managing IT processes and services is renewing itself with an upgrade to the IT philosophy. The primary goal of IT management frameworks is to bring order to the heterogeneity of different systems, processes, and responsibilities that has arisen as a result of the evolved IT landscape. The idea is for processes and procedures to run systematically and comprehensibly, which means careful planning of updates, hardware changes, and software roll-outs; checking the effects of changes in advance; and keeping services continuously available wherever possible. Made for the Cloud and DevOps The restructured ITIL 4 now has two key elements: the Service Value System, which roughly maps the Service Value Chain from ITIL…

Performance values as plain old numbers do not present a visually appealing overview of system performance, but graphical dashboards can help you visualize what would otherwise be boring metrics. A number of free applications visualize metrics in almost any form desired, and one of the most popular open source tools in this family is Grafana [1]. Without much programming knowledge, you can build dashboards to present Internet of Things (IoT) values, stock prices, or the performance data of monitored systems. In this article, I show you how to use Grafana in a convenient GUI to display storage performance values, as well as how to retrieve the desired Simple Network Management Protocol (SNMP) data from InfluxDB. SNMP, the source for the performance information in this example, is supported by all common…

CI and CD are terms that pop up frequently in discussions of modern software development (Figure 1). CI stands for continuous integration, a practice that focuses on making releases easier to prepare. CD can mean either continuous delivery or continuous deployment, and although these practices have a lot in common, they also have significant differences [1]. In the continuous integration software engineering practice, developers integrate code into a shared repository several times a day to obtain rapid feedback on code feasibility. CI enables automated build and testing so that teams can work together rapidly on a single project [2] [3]. In the continuous delivery software engineering practice, teams develop, build, test, and release software in short cycles. Continuous delivery depends on automation at every stage so that cycles can be…

Windows, Linux, and macOS offer a wide range of alternatives for creating software RAID data storage. However, RAID as a software solution does not make sense in some cases because the performance is less than optimal and parity information is not calculated by a dedicated processor. Additionally, it does not support hot swap – that is, the ability to replace hard drives on the fly. Hardware solutions are generally recommended for RAID (redundant array of independent disks) systems, but sometimes admins might find software RAID on their servers useful. Software RAID in Windows In addition to storage pools, legacy disks, spanned volumes, and striped volumes, Windows Server allows you to create a software-based RAID5 system, as well as a mirrored volume that requires two hard drives to which all information…

The Python Clock Has Almost Reached Zero As of January 1, 2020, Python 2 will reach its end of life. Originally an official date wasn’t set, but that has changed to a hard and fast deadline. There’s even a countdown clock (https://pythonclock.org/) to tick away the days, hours, minutes, and seconds before we say goodbye to Python 2. What does this mean? It means a number of critical Python projects will stop supporting version 2 and developers must begin the migration to Python 3 as soon as possible. However, there’s a catch. If you get Python from the Ubuntu repositories (for either 16.04 or 18.04), Python 2 will continue to be supported for the lifetime of those releases (regardless of the upstream status of the Python packages). This is due…

Most computers don’t run nonstop, not least with a view to saving electricity. However, you might still want to run some tasks automatically when you are not sitting in front of your computer: In the long run, launching a backup program manually all the time can be annoying. The classic cron scheduler lets you handle these tasks automatically, but if your PC is not running at the specified time, automatic execution is simply skipped. Therefore, it makes more sense to assign regular tasks to a scheduler that performs routine tasks as soon as the computer is switched back on. I tested the fcron [1] and hcron [2] schedulers on Fedora 29 with the Backup Manager [3] program as the test application. Installation You can pick up RPM and DEB packages…

When talk turns to containers, most people almost automatically think of the well-known Docker engine. For some time now, however, Podman [1] has been a serious alternative. The project was developed together with CRI-O, an implementation of the Kubernetes Container Runtime Interface (CRI). Podman is not limited to use within a Kubernetes environment, however; you also can use it to run individual containers, and you can even adopt the concept of pods, independent of Kubernetes. Look Ma, No Daemon Although the Podman developers have made sure that the Podman command-line tool is almost identical to Docker’s, the two container engines are fundamentally different in terms of architecture. Everything in the Docker world is based on the client-server principle, whereas Podman relies on the fork-exec model. When Docker starts a new…

Classic IDEs usually come with an editor, a compiler, and a debugger. Gitpod Software-as-a-Service (SaaS), on the other hand, provides an integrated development environment with an editor plus a run-time environment as needed, which you call in the browser without having to worry about the technical underpinnings. The Rub Everyone will be familiar with this scenario: A project you recently discovered and find pretty exciting provides detailed documentation; however, for the build process, you need the A, B, and C libraries and some additional tools – all of this, of course, in versions that your distribution choice does not provide. Developers either have to grin and bear the installation overhead, or they can resort to a virtual machine or Docker image. Although this solution might work well for individual developers,…

Since Satya Nadella took over the helm of Microsoft in 2014, Redmond’s aversion to Linux and open source software in general has given way to a new openness. As a result, Linux virtual machines (VMs) now run without complications on the Hyper-V hypervisor and in the Azure cloud. The Windows Subsystem for Linux (WSL) adds a new compatibility layer to Windows 10. As the name implies, WSL is not a hypervisor that runs complete Linux systems in the form of virtual machines, but an interface that directly supports the execution of Linux ELF64 binaries [1]. WSL Architecture The subsystem introduces what are known as pico processes [2], which run in user mode, forming the shell for unmodified executable Linux applications. To do this, they forward their system calls to the…

Several security measures do not protect your systems from compromise, including security by obscurity (i.e., changing ports), intrusion detection (after the fact monitoring and reporting), and poor password policies (allowing non-complex passwords). The better method for protecting your systems is to implement intrusion prevention measures. Fail2ban [1] is one such solution. It scans logs to check for attacks before they occur and blocks the offending host’s access prior to any compromises or break-ins. Malicious attackers know that passwords are the weakest link in the security chain. They also know that with enough time they can break even the best passwords. Rather than allowing attackers to make attempt after attempt, Fail2ban stops them on their first round of attempts. Any password-protected service such as SSH, FTP, IMAP, POP3, Sendmail, and others…

Anyone who has written a fairly complex script has had to deal with interprocess communication (IPC) at one time or another. In this article, I talk about the use and structure of IPC by reading and writing from parent to child and from child to parent processes, and I’ll look at bi-directional communication to achieve both at the same time. A process is a task given to the operating system that has its own execution resources, including its own allocation of CPU time. A process can interact with other processes in a number of different ways, as you’ll see in this article. The support for IPC in Perl is considerable. At its most basic is the use of backticks to perform system commands (e.g., unlink). When you submit a command…