Understanding the linux tr utility

Many command-line program chains include tr near the beginning to restrict input data or for normalization, but the tool itself can take over multiple responsibilities, which are all valuable to understand in their own right.

Simple character translation

The primary function of tr is translating a set of characters into a different set. Think of it as replacing the input text character by character.

The basic form is:

tr <set1> <set2> /path/to/file.txt # read from /path/to/file.txt
echo "Sample" | tr <set1> <set2> # read from standard input

The simplest form is replacing a single character with another one:

echo "aba" | tr "a" "c"

The output will be "cbc", because the tr command replaced every "a" with a "c".

Translation works on sets of characters, so you can supply multiple at once:

echo "aba" | tr "ab" "xy"

This time, the output is "xyx". Note how translation means replacing each character in the first set with the matching position in the second set, so "a" (first character in set 1) is replaced with "x" (first character in set 2), and "b" (second character in set 1) is replaced with "y" (second character in set 2).

This matching also works on entire ranges:

echo "Hello" | tr "[a-z]" "[A-Z]"

The output will show "HELLO" in uppercase letters, since the entire alphabet in lowercase is now mapped to the entire alphabet in uppercase. Characters ranges are expanded, so [a-c] and abc are the same character set.

Character ranges are not regular expressions, but rather part of the POSIX character classes syntax, including support for builtin sets like [:upper:] for uppercase alphabet or [:space:] for all whitespace characters (tabs, space, newlines etc). See man tr or man 7 regex for a complete list of character classes and their contents.

Character classes are just predefined sets of characters, so [a-z] and [:lower:] produce the exact same alphabetical lowercase character set.

Note that posix character classes are locale-dependent, so for a machine with a german locale ä is included in [a-z] and [alpha], whereas an american locale it is not.

Uneven translation sets

The previous examples showed character translation when set 1 and set 2 are equally long, but tr has special handling for cases where their lengths differ.

The first case is mapping multiple characters in set 1 to a single one in set 2:

echo "abc" | tr "ab" "x"

This basically means "replace all characters in set 1 with the one in set 2", resulting in "xxc".

The second case is when set 2 is shorter than set 1, for example:

echo "abcde" | tr "abcd" "xy"

For this edge case, tr implicitly repeats the last character of set 2 internally, using it for all matches from set 1 that exceed the length of set 2. The output of the example above will therefore be "xyyye", since "a" is mapped to "x", "b" is mapped to "y", and "c"/"d" cannot be mapped to set 2, so it reuses the last character "y" for them.

As you may have guessed, mapping a single set 1 character to multiple set 2 characters does not work at all..

If you want to specifically disable the edge case handling and instead force set 1 to be at most as long as set 2, you can use -t to truncate it:

echo "abcd" | tr -t "abc" "x"

The output will show "xbcd", only replacing the first character "a" with "x", as -t has truncated/deleted all other characters in set 1 that could not be mapped directly onto a character in set 2.

Deleting characters

The -d flag is used to delete characters in set 1, and does not support a set 2 argument.

Think of it as "replacing any match in set 1 with nothing".

It supports a list of characters or entire character classes, just like any other mode:

echo "abcd" | tr -d "ac" # => "bd"
echo "aaab01" | tr -d "[a-z]" # => "01"
echo "aaab01" | tr -d "[:digit:]" # => "aaab"

Deletion is really only used in practice to strip expected characters from input text, for example unprintable or control characters.

Inverting translation

The complement mode -c inverts the translation set 1, meaning anything included is not translated.

For normal translation, you can use this mode to normalize characters, for example to replace every non-alphabetic character with simple spaces:

echo "Hello W	o4rld!" | tr -c "[:alpha:]" " " # => "Hello W o rld "

Since inverted translation can only use a single character from set 2, it will always use the last character in set 2:

echo "aabbcc" | tr -c "ab" "xyz" # => "aabbzz"

Note how characters "a" and "b" were matched so not replaced, and "c" was replaced by "z" because it is the last character in set 2.

Deletion commands can also be inverted to create an "allowlist" of characters and remove anything not explicitly allowed:

echo "abc123" | tr -cd "[:digit:]" # => "123"

Inverted deletion is commonly used on unix systems to sanitize source code and configuration files by stripping out any unprintable characters:

tr -cd "[:print:]" sample.conf

These characters can be difficult to spot by humans, but will cause errors with programs parsing the file.

Deduplicating repeated character sequences

The last mode is -s to squeeze a series of characters into a single one. You may think of it as deduplication for repeating characters:

echo "aaabbbcccccc" | tr -s "[a-z]" # => "abc"

Note that squeezing only works for series of consecutive characters, so multiple series are collapsed into only one instance, and already single characters are kept as is:

echo "aaababaaaaaa" | tr -s "[a-z]" # => "ababa"

Squeezing can also be inverted, but this is rarely useful.

You will most commonly see squeezing used in chained linux commands to collapse consecutive spaces into a single one:

echo "one   two three    four" | tr -s " " # => "one two three"

Tools like cut can then use space as a delimiter to split the line into columns/fields and extract one:

echo "one   two three    four" | tr -s " " | cut -d" " -f2

The output will show only "two", no matter how many spaces split the actual input words.

Tr vs sed

The character translation tool tr and the regular expression based "stream editor" sed cover very similar use cases with only minor differences. Almost all kinds of replacements, from stripping unwanted character classes to collapsing whitespace, can also be handled by sed - but it offers more features, like replacing entire words or patterns, extracting subpatterns from input lines and editing files in place.

While they share some common use cases, sed is by no means a replacement for tr. If you need to map many characters to other characters individually (turn all lowercase characters into uppercase), you still have to use tr. Most other cases can arguably be handled by either tool, with many administrators favoring sed because its capabilities are more commonly needed in shell scripting anyway.