Commands to sysadmin duties on GNU/Linux

In this quick entry I want to summarize some commands which I have been using for a while when I have the system administrator role and I found them useful.

Search by size

When you have to release space from a server you can look for bigger files:

$ du -hsx * | sort -rh | head -10
$ for i in G M K; do du -ah | grep [0-9]$i | sort -nr -k 1; done | head -n 11

We could check the package size on Debian GNU/Linux based box using this command:

$ dpkg-query -W --showformat='${Installed-Size} ${Package}\n' | sort -n

Accounts

To verify no accounts have empty passwords, in this case awk uses a condition to do the job:

# awk -F: '($2 == "") {print}' /etc/shadow

To make sure that no non-root accounts have UID Set To 0:

# awk -F: '($3 == "0") {print}' /etc/passwd

Printing content from a text file

Print lines in the n-th /BEGIN/,/END/ range, not inclusive:

# awk -v n=4 '/END/{p=0}; p && c == n; /BEGIN/ && !p {p=1; c++}'

Searching patterns

Look for multiple patterns in files (PX from patternX):

$ grep -E 'P1|P2|P3' foo.txt

Removing content

A practical use of grep command it would be to remove comments and empty lines in a configuration file:

# grep -v -E '(^#|^$)' /full_path/file.conf > /full_path/reduced_file.con

Other options:

$ awk '/P1|P2|P3/' foo.txt
$ awk '/P1/ || /P2/ || /P3/' foo.txt
$ sed '/P1/b; /P2/b; /P3/b; d' foo.txt

Ordered search

$ grep 'P1.*P2.*P3' foo.txt

Unordered search

$ awk '/P1/ && /P2/ && /P3/' file.txt

AWK advanced use

Here’s an awk solution for the inclusive case:

$ awk '!ok && /START/ { ok = 1 }
 ok { p = p sep $0; sep = RS }
 ok && /END/ { print p; p = sep = "" }' foo.txt

When /START/ is seen, start collecting lines. Each time an /END/ is seen (and /START/ was previously seen), print what we have so far, empty the buffer and start collecting lines again, in case we see another /END/ later.

Argument passed to awk:

awk 'BEGIN{print "ARGC is " ARGC; for(i = 0; i < ARGC; i++) print "ARGV["i"] is " ARGV[i]}' foo bar

PS command

The ps is very useful command and it has interesting flags one of them is “–sort” as you expect order metric passed in the second parameter, “rss” in this example:

# ps aux --sort -rss|head -5
 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
 www-data 31771  7.2  1.2 8557572 186180 ?      Sl   Nov25  68:49 /usr/sbin/apache2 -k start
 www-data 31230  2.1  0.8 8494032 122548 ?      Sl   Nov25  20:10 /usr/sbin/apache2 -k start
 www-data  8725  0.8  0.6 8474156 100556 ?      Sl   Nov25   8:33 /usr/sbin/apache2 -k start
 www-data  7222  0.4  0.5 370236 75432 ?        S    08:00   0:02 php-fpm: pool p

If you remove “-” from “rss” param, the output order is reverted:

# ps aux --sort rss|head -5
 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
 root         2  0.0  0.0      0     0 ?        S    Nov25   0:00 [kthreadd]
 root         3  0.0  0.0      0     0 ?        S    Nov25   0:00 [ksoftirqd/0]
 root         5  0.0  0.0      0     0 ?        S    Nov25   0:00 [rcu_sched]

Other pretty handy flag is “-o” with ps command can be useful to calculate median memory for a specific command:

# ps --no-headers -o "rss,cmd" -C COMMAND | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/NR/1024,"Mb") }'

And removing NR from the previous command you can get the total amount of memory:

# ps --no-headers -o "rss,cmd" -C COMMAND | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/1024,"Mb") }'

SSH to copy data remotely

Copying remote data commands from this link:

$ scp -3 -r remote1:/src/dir remote2:/dest/dir

The same using tar:

$ ssh remote1 'tar -C /src/dir/ -cvzf - .' | ssh remote2 'tar -C /dest/dir/ -xzvf -'

(the -t option to SSH is to force a pseudo-tty allocation, otherwise we couldn’t be asked for the password):

$ ssh -t -R10000:remote2:22 remote1 'rsync -e "ssh -l root -p 10000" -avz /src/dir/ localhost:/dest/dir/'

The previous command is the same as:

$ ssh -R10000:remote2:10000 remote1
$ ssh -R10000:remote2:22 remote1
remote1$ rsync -e 'ssh -l root -p 10000' -avz /src/dir/ localhost:/dest/dir/

For loop

I always forget loop syntax on Bash so here a remainder for people like me:

$ for (( i = 0; i < 10; i++ )); do echo $i; done

While loop:

i=0
 while [[ $i -lt 10 ]]; do echo $i; let i++; done

For loop using seq:

$ for i in $(seq 0 9); do echo $i; done

Using brace expansion:

$ for i in {0..19}; do echo $i; done

Backup a file using brace expansions:

$ cp file{,.`date +%Y%m%d`}


“Once you have tasted flight, you will forever walk the earth with your eyes turned skyward, for there you have been, and there you will always long to return”
– Leonardo da Vinci

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s