Resource Usage

Disk Usage

Note: If you have a file/files open and your disk space appears full, do a reboot and it will close the open file/s. This is a true indication of disk space then.

To check overall disk usage run:

du -h
df -h
df -h | grep -v 'loop\|nas\|/run/\|tmpfs\|udev'

du -h --max-depth=1 | grep G
du -kh | grep G
du -khs * | grep G

• Using grep looks down one directory and looks for GB disk usage, use M for MB, K for KB.
  
• To dive down more directories change the max depth value.

Use the below command to show number of files in target directory:

ls -1 target|wc -l

For Inodes:

df -i
find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n
cd /directory that is the largest
#run find command again

Disk check

apt-get update
apt-get install smartmontools
smartctl -H /dev/sda
smartctl -A /dev/sda

This will check the health of the hard drive. If you use RAID the software may not be able to pick it up. If you are getting errors, check you know what your HDD type you have by doing:

smartctl --scan

Do not trust df, to check the HDDs use:

fdisk -l

If your server uses SCSI, you may need to the following each time:

smartctl -d scsi /dev/sda

A better alternative is mdadm but be careful, do not install it unless you know what you are doing as it will prompt you with questions via a GUI straight away.

Empty a file

:> /file/path

This always exits true, but produces no output. Good for large old log files.

More complicated options

• Changing the second n option specifies the amount of results you want to show.

du --max-depth=1|sort -nr|head -n10 #this prints out the largest value at the top
du --max-depth=1|sort -n|tail -n5 #this prints out the largest value at the bottom

du --max-depth=20|sort -nr|head -n20>diskusage.txt #this outputs the top 20 results into a file

du --max-depth=1|sort -nr|tail -n10 #this prints out the smallest value at the bottom, good for temp directories
du --max-depth=1|sort -n|head -n10 #this prints out the smallest value at the top

du --max-depth=1 -h * |sort -n -k 1 |egrep 'M|G'
du -sh *

http://www.earthinfo.org/linux-disk-usage-sorted-by-size-and-human-readable/

If the server is Windows, you can use Performance monitor which is built into Windows to run on the server to monitor free space along with disk read/writes. You may wish to install http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx as it will provide a greater overview as to what is writing to the server and when.

If there is a program writing to the temporary directory it may be putting the files in there and then removing them once they are finished being used hence the fluctuation.

NCDU

A useful tool for Unix is called ncdu: "NCurses Disk Usage". https://linux.die.net/man/1/ncdu

This allows you to discover the overall usage, please note this can take a while depending on the amount of data you have.

From root (cd /) run one of the following commands to install and run ncdu:

CentOS (you may need to enable the EPEL repository):

yum search ncdu
yum install ncdu.x86_64
cd / #or the directory you want to check
ncdu

Ubuntu:

apt-get install ncdu
cd / #or the directory you want to check
ncdu

Exlcuding a directory in ncdu:

ncdu --exclude /home/mc/Minecraft/plugins/dynmap/web/tiles/world

Truncate

You can truncate files via SSH:

truncate -s 0 myfile.txt

Progressive truncation just uses different parameters for -s (size). For instance, -s 100G or -s 100M.

Echo is another tool that can truncate a file.

echo > myfile.txt

Useful commands on home NAS' when it hangs when deleting extremely large files via SSH or UI.

General Usage

iotop

apt install sysstat
iotop
iostat | grep -v loop

htop FAQ

On Linux:

top
apt-get install htop
#or
yum install htop
#then
htop

apt-get install goaccess

Also look at http://serverkb.co.uk/wiki/Scripts

On Windows use Task Manager and Performance Monitor.

Munin and Graphite are good for system management.

Memory available

Display the top ten running processes - sorted by memory usage:

ps aux --sort -rss | head

free -m

This shows you the available and currently used memory (RAM) in MB. So 1900-2000 is 1-2GB. On a Shared/VPS environment normally half the Total and you get your Soft Limit (guaranteed), the full amount is your Hard Limit (burstable, not guaranteed).

cat /proc/user_beancounters

You will see something like this:

Version: 2.5

uid        resource         |  held     | maxheld  | barrier               limit                        |      failcnt
328178:    kmemsize         |  4348996  | 4366996  | 16384000              18022400                     |           0
           lockedpages      |  0        |       0  | 4096                  4096                         |           0
           privvmpages      |  68233    |   68262  | 153600                153600                       |       34995

or for example on an 8GB of RAM server:

uid        resource         |  held     | maxheld  barrier               limit                          |    failcnt
506378:    kmemsize         |  62055413 | 62490301 9223372036854775807   9223372036854775807            |         0
           lockedpages      |  0        |       0  9223372036854775807   9223372036854775807            |         0
           privvmpages      |  384054   |  384315  9223372036854775807   9223372036854775807            |         0
           shmpages         |  1687     |    1687  9223372036854775807   9223372036854775807            |         0
           shmpages         |  1687     |    1687  9223372036854775807   9223372036854775807            |         0
           dummy            |  0        |       0  9223372036854775807   9223372036854775807            |         0
           numproc          |  205      |     208  9223372036854775807   9223372036854775807            |         0
           physpages        |  119511   |  119641  9223372036854775807   9223372036854775807            |         0
           vmguarpages      |  0        |       0  9223372036854775807   9223372036854775807            |         0
           oomguarpages     |  119513   |  119643  9223372036854775807   9223372036854775807            |         0
           numtcpsock       |  34       |      36  9223372036854775807   9223372036854775807            |         0
           numflock         |  21       |      23  9223372036854775807   9223372036854775807            |         0
           numpty           |  1        |       1  9223372036854775807   9223372036854775807            |         0
           numsiginfo       |  4        |       5  9223372036854775807   9223372036854775807            |         0
           tcpsndbuf        |  601840   |  648400  9223372036854775807   9223372036854775807            |         0
           tcprcvbuf        |  557056   |  589824  9223372036854775807   9223372036854775807            |         0
           othersockbuf     |  2132376  | 2133680  9223372036854775807   9223372036854775807            |         0
           dgramrcvbuf      |  0        |    2608  9223372036854775807   9223372036854775807            |         0
           numothersock     |  947      |     949  9223372036854775807   9223372036854775807            |         0
           dcachesize       |  3301507  | 3312775  9223372036854775807   9223372036854775807            |         0
           numfile          |  20555    |   20630  9223372036854775807   9223372036854775807            |         0
           numiptent        |  15       |      15  9223372036854775807   9223372036854775807            |         0

Take a look at the numbers left to the first Pipe (|) in the held column: These stand for the actual usage. Right of the last pipe: Failcounts. Having failcounts inidicates that a service is consuming all the ram.