: > /dev/null

Apr 24, 2017

My favourite commands

I often need to "audit" a GNU/Linux server quickly when dealing with large farm of new servers. After some years, this is what i use:

  • w(1): shorter to type than uptime + who to get load average
# w
 00:05:25 up 51 days,  9:50,  2 users,  load average: 0.01, 0.06, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    localhost        22:03    2.00s  3:05   0.01s w
  • pstree(1): shorter to type than ps with many options, output grouped
# pstree
systemd─┬─acpid
        ├─2*[agetty]
        ├─apache2───7*[apache2]
        ├─atd
        ├─collectdmon───collectd───10*[{collectd}]
        ├─cron
        ├─dbus-daemon
        ├─dmeventd
        ├─haveged
        ├─lvmetad
        ├─master─┬─pickup
        │        └─qmgr
        ├─memcached───5*[{memcached}]
        ├─mysqld_safe───mysqld───24*[{mysqld}]
        ├─nginx───2*[nginx]
        ├─nrpe
        ├─ntpd
        ├─puppet───{ruby-timer-thr}
        ├─rsyslogd─┬─{in:imklog}
        │          ├─{in:imuxsock}
        │          └─{rs:main Q:Reg}
        ├─sshd─┬─sshd───bash───pstree
        │      └─sshd───sshd───bash───vi
        ├─2*[systemd───(sd-pam)]
        ├─systemd-journal
        ├─systemd-logind
        └─systemd-udevd
  • lsblk(8): list block devices 'topology' (from util-linux)
# lsblk 
NAME              MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda               254:0    0   40G  0 disk 
├─vda1            254:1    0  243M  0 part /boot
└─vda2            254:2    0 39.8G  0 part 
  ├─debian-root   253:0    0 38.9G  0 lvm  /
  └─debian-swap_1 253:1    0  872M  0 lvm  [SWAP]
  • lsb_release(1): show full operating system information (may not be installed everywhere, but is configuration managment system installed such as puppet or inventory system, will be)
# lsb_release --all
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.5 (jessie)
Release:        8.5
Codename:       jessie

Some other commands i also use:

  • whatis(1): display on-line manual page description of a binary
  • pgrep(1): shorter to type than his friend ps $your_favourite_options | grep $pattern
# pgrep $pattern
4304
4711

Add --list-name (-l), --list-full (-a) to list name of process and match full command

# pgrep -a mysql
4304 /bin/sh /usr/bin/mysqld_safe
4711 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-error=/var/log/mysql/error.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306
  • namei(1): follow pathname and print all attributes of traversed files, useful to verify rights for a non-privilegied user
# namei --long /etc/hosts                                    
f: /etc/hosts
drwxr-xr-x root root /
drwxr-xr-x root root etc
-rw-r--r-- root root hosts
  • tailf(1): shorter to type than tail -f and don't update access time when not needed
  • httpie(1): http client for humans, with colored output, can replace curl -I when debugging webservers/cache
  • tshark(1): 'tiny wireshark', easier to read for humans than venerable tcpdump, same syntax for bpf filters, dissect protocols. Suppose you want to inspect http transactions with minimum details:

tcpdump -nn -i lo port 80 -A

23:43:33.322766 IP 127.0.0.1.55345 > 127.0.0.1.80: Flags [P.], seq 3338438874:3338438967, ack 2071539399, win 1012, options [nop,nop,TS val 1110051799 ecr 1110049301], length 93
E.....@.@..o.........1.P....{y.............
B*..B*..GET /nginx_status?auto HTTP/1.1
User-Agent: collectd/5.4.1
Host: localhost
Accept: */*


23:43:33.329166 IP 127.0.0.1.80 > 127.0.0.1.55345: Flags [P.], seq 1:249, ack 93, win 342, options [nop,nop,TS val 1110051801 ecr 1110051799], length 248
E.., .@.@............P.1{y.....7...V. .....
B*..B*..HTTP/1.1 200 OK
Date: Tue, 07 Jun 2016 21:43:33 GMT
Content-Type: text/plain
Content-Length: 107
Connection: keep-alive
Server: ohmy

Active connections: 4 
server accepts handled requests
 3139 3139 19247 
Reading: 0 Writing: 3 Waiting: 2

tshark -nn -i lo -f 'port 80'

  1   0.000000    127.0.0.1 -> 127.0.0.1    HTTP 159 GET /nginx_status?auto HTTP/1.1 
  2   0.000132    127.0.0.1 -> 127.0.0.1    HTTP 314 HTTP/1.1 200 OK  (text/plain)

Less details, but more readable at first glance

  • ngrep(8): network grep is easier for humans than venerable tcpdump for pretty printing packets content:

ngrep -q -W byline 'port 80'

T 127.0.0.1:55345 -> 127.0.0.1:80 [AP]
GET /nginx_status?auto HTTP/1.1.
User-Agent: collectd/5.4.1.
Host: localhost.
Accept: */*.
.


T 127.0.0.1:80 -> 127.0.0.1:55345 [AP]
HTTP/1.1 200 OK.
Date: Tue, 07 Jun 2016 21:38:23 GMT.
Content-Type: text/plain.
Content-Length: 107.
Connection: keep-alive.
Server: ohmy.
.
Active connections: 4 
server accepts handled requests
 3125 3125 19165 
Reading: 0 Writing: 3 Waiting: 2

You've maybe noted that HTTP/2 is in binary format, so that ngrep will not display content anymore (but tshark will with dissector)

  • multitail(1): tail but open mutiple files and use colors to tail them; pressing enter add a red mark to see when future update of the file occurs (bye bye pressing Enter multiple times !)
  • jq(1): pretty print JSON. This can replace python -m json.tool because jq has..colors ! It also has great powerful filter to manipulate values
  • watch(1): execute a command periodically, bye bye upwards arrow + Enter to replay commands multiple times !
  • netcat(1): venerable telnet is great, but escape sequence maybe hard on some terminals, this is why i prefer netcat; you can also use option to test layer 4 firewalls in scripts:
$ nc -w 3 -v -z www.iroqwa.org 80; echo $?
Connection to www.iroqwa.org 80 port [tcp/http] succeeded!
0
$ nc -w 3 -v -z www.iroqwa.org 8081; echo $?
nc: connect to www.iroqwa.org port 8081 (tcp) failed: Connection refused
nc: connect to www.iroqwa.org port 8081 (tcp) timed out: Operation now in progress
1

Option -w set a timeout, -z don't send any data (to avoid Netcat escape sequence without -z is ^C; wors with both nc.traditional and nc.openbsd

  • view(1): Open file with vim(1) but read-only, this prevent vi to buffering the file and update time of last access. This can be also achieved by pressing v when opening the file with less(1).

As you see, i generally adopt tools that offers a simple output and even colorized to easier viewing. Life is not only black and white on terminals !

Click to read and post comments