Table of Contents
Classics that work just as well today as they did decades ago
Package managers and third-party plugins can be amazing wonders of productivity. They save time and provide easy ways to extend the functionality of applications and operating systems. But what happens when those tools aren’t available? When you must rely solely on whatever ships with your OS? When your network connection disappears?
In the Linux world, many distributions come with an assortment of fantastic programs and utilities pre-installed. If you’re in a pinch working with an offline machine or are severely limited (by space, policy, etc.) on what software you can install then working with standard tools may be the only option.
In this article, we will explore some default Linux tools at your disposal for manipulating text files. Some of these tools will be actual text editors, and others more generic programs and languages with a wide range of features. Learning these timeless tools will help you be more flexible when it comes to working on limited systems.
1. vi(m)
This is a classic, endlessly useful editor that has been around for over 45 years. The vi editor provides a fast, flexible visual text editor that you can extend and customize in a ton of different ways. The best part is, this editor is also available by default on just about every Linux distribution out there.
Using vi can be daunting at first, but once you become familiar with the basic set of commands you’ll be flying through files in no time. Below are some essential vi commands to get you started:
Although most users are familiar with vim, this isn’t always available by default. Sometimes vim must be installed separately through a package manager. For this reason, it is beneficial to become familiar with basic vi usage, most of which translates directly to vim anyway.
The official man page for vi is available here.
2. sed
The sed (or “stream editor”) utility is even older (1974) than vi. This subtly advanced program is capable of manipulating text in many different and interesting ways. Although there is a high learning curve with sed, once you’ve mastered the custom syntax the possibilities are almost endless.
To process a file, sed loops over each line and applies different manipulations based on the user’s desired conditions. There is a even a little programming language contained within sed for providing complex substitutions and other precise selectors.
Let’s say you had a file with the following text:
# file.txt
foo
bar
baz
foo
bar
baz
What if you wanted to make changes to certain lines and replace them with new text? With sed you can do this easily. Using the in-place editing function sed will replace only the lines matching foo:
sed -i 's/foo/FOO/' file.txt
You can make changes on the command-line through pipes, call sed directly to edit in-place (using -i) or build sed into another program to leverage its text processing ability. There are a ton of ways to use this very handy utility.
The official sed manual is available here.
3. nano
Although not as popular as its more feature-rich counterpart vi, the nano editor is infinitely more simple and easy to get started with. This editor is straightforward and even includes basic usage information at the bottom of the main window just in case you ever need a reminder.
Getting up and running with nano requires very little memorization. It is very good for beginners who simply want to make some quick changes to a file and move on. Nano is best suited for fast edits and not so much for full-fledged development due to its limited features.
Check out the official nano documentation available here.
4. echo
Next up we have the tried, true and available almost everywhere echo utility. This isn’t an editor, but it is a quick way to overwrite or append text to a file. If you want to programmatically add items to a file or you’re just in a hurry you can echo them in.
To overwrite all the contents of a file with new text, you can execute something like:
echo "my cool text" > file.txt
If you want to only append to the file instead you can run:
echo "some more text" >> file.txt
If you’re working on the command-line with the output from other programs and want to save it to a file with some additions, you can do that too:
echo "the current date is: $(date)" > file.txt
While you don’t get the full flexibility of an actual text editor or a parsing language, you do have the ability to save text to a file with minimal fuss.
The very concise man page for echo is available here.
5. AWK
AWK is an absolute classic (1977) and another program that has its own dedicated language. Similar to sed, AWK allows you to process text files and manipulate their contents. The AWK language is a bit more robust than sed’s and allows for more advanced data extraction techniques. With AWK you can easily develop an extensive set of rules for pinpointing extremely specific patterns in a file.
Let’s take the same text file from earlier:
# file.txt
foo
bar
baz
foo
bar
baz
Let’s say we wanted to grab only the lines that contain foo. With AWK we could do this by simply passing a regular expression:
awk '/foo/' file.txt
This would output:
foo
foo
Pretty straightforward right? Where AWK has an advantage is when it comes to tabular data. Let’s say you had a file with multiple columns like this:
# file2.txt
foo foo2
bar bar2
baz baz2
foo foo2
bar bar2
baz baz2
If you wanted to get only the second column, you could do something like this:
awk ' print $2 ' file2.txt
This would output:
foo2
bar2
baz2
foo2
bar2
baz2
While AWK doesn’t come with a default way to edit files in-place like sed, it is very easy to pipe the output to a new file instead. To manipulate the contents of an existing file and save a new one you could use the following snippet:
awk ' print $2 ' file2.txt > results.txt
Now inside of the results.txt file you’ll see only the output of AWK.
Check out The GNU Awk User’s Guide for a ton of documentation on building expressions and getting started with AWK.
