Networking
An important command to check what service is running on a port is (General Guide):
netstat -tulpn
netstat -pant is also good for monitoring network connections. man netstat
To check what is listening on IPv4 or IPv6 use lsof -Pnl +M -i4 (or -i6)
Subnet Cheat Sheet
Block IP addresses by Country
CSF Firewall in cPanel/WHM
GeoIP (resource intensive, hard to configure and not the best tool)
Manual IP table/Firewall blocking
blacklists.linuxadmin.org (offline as of 08/01/13)
Check NIC speed
dmesg | grep -i duplex ethtool eth0 mii-tool
Check number of connections
Get external facing IP address
apt-get install curl curl ifconfig.me
curl ifconfig.me/ip -> IP Address
curl ifconfig.me/host -> Remote Host
curl ifconfig.me/ua -> User Agent
curl ifconfig.me/port -> Port
Credit to User aajjk and http://ifconfig.me
Opening a port to listen on (tmp)
Really you need a service to be listening on the port for it to be open. If when running telnet localhost port (tp quit do CTRL + ] and then quit) or an nmap and get:
telnet localhost 1234 Trying 127.0.0.1... telnet: Unable to connect to remote host: Connection refused nmap serverIP -p1234 PORT STATE SERVICE 1234/tcp closed unknown
What you can do on Ubuntu is apt-get install netcat , then run netcat -l -q-1 -p 1234. If you run this in a screen session it will stay open but if you cancel out of a normal SSH connection it will close it again.
You can check if it is open by doing:
telnet localhost 1234 Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. ^] telnet> quit Connection closed. nmap serverIP -p1234 PORT STATE SERVICE 4430/tcp open unknown
IP address locator
This one seems very accurate: http://www.iplocation.net/index.php
IPtables
To clear all iptables rules use the -F option.
To clear the nat rules do iptables -F -t nat
To clear a single rule in SSH do the following (CHAIN refers to INPUT, OUTPUT or FORWARD) e.g.:
iptables -D INPUT number
and then do iptables-save
For example iptables -D INPUT 18
To check the nat chain/table, do:
iptables -t nat -L
To route traffic from the server straight through to a container for example:
iptables -t nat -A PREROUTING -p tcp --dport PortYouChose -j DNAT --to ContainerIP:22 iptables-save
Allowing or dropping ports
iptables -A INPUT -p tcp --dport 25 -j ACCEPT iptables -A INPUT -p udp --dport 9987 -j ACCEPT iptables-save
Or DROP. These will do source: anywhere and destination: anywhere
For a specific IP to access FTP you would do:
iptables -A INPUT -m tcp -p tcp --src <IPsource> --dst <destinationIP> --dport 21 -j ACCEPT
iptables-save
Block an IP address
iptables -A INPUT -s X.X.X.X/subnet -j DROP
This appends (-A) a new rule to the INPUT chain, which specifies to drop all packets from a source (-s) IP address.
Credit to User Root
Examples
Frequently Used Rules
Opening & Closing Ports
Order of rules
If you plan to have the chains (INPUT, OUTPUT and FORWARD) as ACCEPT by default instead of DROP, and you want to set it so you drop all connections to a port/service and only allow some in, you need to do it in this order:
ACCEPT tcp -- sourceIP destinationIP tcp dpt:ftp ACCEPT tcp -- sourceIP destinationIP tcp dpt:ftp DROP tcp -- anywhere anywhere tcp dpt:ftp
Prevent Plesk Panel from brute-force attacks
Rules needed for Virtuozzo and ICMP (ping) traffic
Please ensure that you have the following rules in your firewall:
iptables -P INPUT ACCEPT iptables -P OUTPUT ACCEPT iptables -P FORWARD ACCEPT iptables-save
Otherwise pinging and apt commands may not work.
IPv6
Address Type | Destination |
Default route | ::/0 or 2000::/3 |
Localhost | ::1 or ::1/128 |
Documentation prefix | 2001:db8::/32 |
Link-local prefix | fe80::/10 |
Multicast prefix | ff00::/8 |
ULA (unique local addresses) | fc00::/7 |
Reverse DNS zone format with BIND | C.0.0.0.B.0.0.0.A.0.0.0.1.0.0.2.ip6.arpa |
IPv6 test | How to |
mtr (more detailed traceroute) | mtr -6 domain.com |
Ping | ping6 on Unix, ping on Windows. |
Reverse DNS/PTR | dig -x 2001:470:0:64::2 PTR |
SSH | ssh -6 domain.com |
tracert/traceroute | traceroute6 (ndisc6) on Unix, tracert on Windows |
wget | wget -6 domain.com/file.txt |
- 16 bytes in an IPv6 address
- 65536 /48 subnets are available in a /32 prefix
- AAAA is an IPv6 equivalent of IPv4 DNS A records
- Native IPv4 and IPv6 at the same time is called Dual stack
- Routers are not allowed to fragment IPv6 packets.
- Proper IPv6 format: http://[2001:470:0:64::2] | with a port: https://[2001:0db8:85a3:08d3:1319:8a2e:0370:7344]:443
- The 6in4 protocol is used for manually configured tunnels.
- 2001:db8:7fa5::/48 is a subnet of 2001:db8::/32
- You request IPv6 glue for your nameservers through the registrar of the domain used by your name servers
- IPv6 glue for nameservers resides on TLD nameservers
- The following query proves working IPv6 glue: dig AAAA ns1.exampledomain.tld @tld.server
- The TLD A.GTLD-SERVERS.net is authoritative for .com & .net IPv6 Glue
- Another name sometimes used for A or AAAA nameserver glue records found in the top level domain zone files is GHOST records
- A registrar is an organization that is able to register domains
- A registry is an organization responsible for operating the authorative nameservers and database for a top level domain
- Getting AAAA records for your nameservers in the corresponding TLD (top level domain) zone matters because it enables entirely native IPv6 DNS queries and makes it possible for IPv6 only hosts to reach the nameservers for your domain, since they can't use glue that is just an A record.
- IPv6 AAAA records have been added for several of the root nameservers
To get IPv6 GLUE for your nameservers you have to get the registrar to request it from the root nameservers, this is so the TLD server can directly answer for the host record (e.g: dig +trace ns $domain to get the TLD server list then dig aaaa $ns @TLD for the glue)
ns1.domain.co.uk 2a02:6e1:9:5241::b327:cd81
ns2.domain.co.uk 2a02:6e1:9:5241::b327:cd81
Enable on Ubuntu 12.04
You will need to edit the network interfaces file.
vim /etc/network/interfaces
And add the following information to it:
iface interface inet6 static #e.g. iface eth0 inet6 static gateway IPv6 address to the fourth octet::1 address IPv6 address netmask 64
Restart networking:
/etc/init.d/networking restart
Then test using the following command:
traceroute -n -w 2 -q 2 -m 30 8.8.8.8 ; traceroute -n -w 2 -q 2 -m 30 2001:4860:4860::8888
Ports and Protocols
15 | netstat |
20 | FTP (data transfer) |
21 | FTP (control) |
22 | SSH (secure logins, file transfers (scp, sftp) and port forwarding) |
23 | Telnet (use Control + ] to exit telnet session) |
24 | Any private mail |
25 | SMTP (Sending mail) |
42 | nameserv, WINS |
43 | whois, nickname |
50 | Remote Mail Checking Protocol |
53 | DNS |
80 | HTTP |
101 | HOSTNAME |
106 | poppassd (for localhost only) |
107 | Remote TELNET Service |
110 | POP3 (Handling and Authenticating mail) |
113 | Authentication |
123 (UDP) | NTP |
139 | NetBIOS |
143 | IMAP |
443 | HTTPS (SSL) (Statistics Port) |
465 | SMTP Secure (SSL for IMAP) |
587 | Alternate SMTP |
990 | FTP Secure (FTP SSL) |
993 | IMAP Secure |
995 | POP3 Secure |
1433 | MSSQL |
2082 / 2083 | cPanel http / https |
2086 / 2087 | WHM http / https |
2500 | Alternate SMTP |
3306 | MySQL |
3389 | RDP Protocol |
4643 | Virtuozzo |
5224 | Plesk License Update (outgoing) |
5666 | Nagios (unauthorized use) |
5938 | TeamViewer |
8306 | Alternate MySQL |
8443 | Plesk Secure |
8880 | Plesk HTTP |
TCP 9101, 9102 and 9103 | Bacula |
UDP 9987 | Team Speak |
TCP 10011 & 30033 | |
TCP 9999 | Java for Red5. Also this, nothing to worry about. |
Full List of TCP and UDP Ports
Preventing remote access lockout
This is for Unix only.
Whenever doing a big iptables change on a system, always check that atd is running, then put an at job for about 10 minutes in the future to take the firewall down, something like
at now + 10 minutes at> service iptables stop at> ^D
That way if you really foul up and lock myself out, in ten minutes' time you'll be able to get back in and fix things. If you finish the work and haven't fouled up, find that job with atq and delete it with atrm before it even runs.
Monitor packet loss
Unix: netstat -s | grep retransmited
netstat -s for a full report, from here
Network Tools
netstat -tulpn apt-get install ethstats
nmap
Check a service is running using its port from outside a VPS. nmap guide, apt-get install nmap
nmap -P0 -vvv IP nmap -P0 -vvv IP | grep port
TCP Scan
nmap 127.0.0.1 -p80 nmap 127.0.0.1 -p T:443
UDP Scan
nmap -sU 127.0.0.1 -p U:9987 -PN
Checking all open ports, using the IP is slightly quicker
nmap destination
If it returns: "Note: Host seems down. If it is really up, but blocking our ping probes, try -PN" run:
nmap destination -PN
If it returns: "All 1000 scanned ports on destination (IP Address) are filtered" run:
nmap -PN -sT -vv -n -p1-1000 -T4 -oNmapTCPConnect.txt IPaddress nmap -PN -sS -f -vv -n -p1001-65535 -T4 -max-rtt-timeout 15 -oNmapSYNScan.txt destination nmap -PN -sU -vv -n -p U:1-65535 -T4 -max-rtt-timeout 15 -oNmapSYNScan.txt destination
List all IPs on a network and put them into a file:
nmap -sP 192.168.1.0/24 | grep "Nmap scan report for"| cut -d' ' -f 5 > ips.txt
NTOP
apt-get install libpcap-dev libgdbm-dev libevent-dev librrd-dev python-dev libgeoip-dev rrdtool rrdtool-dbg rrdtool-tcl rrdcollect rrdweather sw-rrdtool sw-librrd
apt-get install ntop ntop --set-admin-password chmod 755 /var/lib/ntop chown -R nobody:root /var/lib/ntop/ (alternatively try ntop:root, nobody:nogroup or ntop:nogroup) #addressQueue.db can be root:root and interfaces can be nobody:nogroup) ethernet_dev=venet0:0 ntop -i venet0:0
Running the last command will mean if you CTRL + C it will stop the gathering of data, so you will need to open another SSH session. You can also stop ntop via the web interface by going to Admin > Shutdown.
Go to IPAddressOrDomain:3000 to view the stats. The ntop port by default is 3000.
NTOP FAQ
Ubuntu NTOP Docs
Engarde NTOP Guide
Starting NTOP as a Daemon A
Starting NTOP as a Daemon B
If you receive the message "Please enable/make sure that ntop html/ directory is properly installed" upon going to your URL, go to: NTOP Configuration
NTOP on CentOS/Fedora/Other Linux Distrubutions
IP Traffic
yum install iptraf.x86_64 #on CentOS
iftop
apt-get install iftop
yum -y install iftop.x86_64
iftop listens to network traffic on a named interface, or on the first interface it can find which looks like an external interface if none is specified, and displays a table of current bandwidth usage by pairs of hosts. iftop must be run with sufficient permissions to monitor all network traffic on the interface; see pcap(3) for more information, but on most systems this means that it must be run as root.
By default, iftop will look up the hostnames associated with addresses it finds in packets. This can cause substantial traffic of itself, and may result in a confusing display.
You may wish to suppress display of DNS traffic by using filter code such as not port domain, or switch it off entirely, by using the -n option or by pressing R when the program is running.
By default, iftop counts all IP packets that pass through the filter, and the direction of the packet is determined according to the direction the packet is moving across the interface. Using the -F option it is possible to get iftop to show packets entering and leaving a given network. For example, iftop -F 10.0.0.0/255.0.0.0 will analyze packets flowing in and out of the 10.* network
While iftop is running, you can press any one of the following keys to display more output.
S – display source port
D – display destination port
n – show IP instead of host name
1/2/3 – sort by the specified column
< – sort by source name
> – sort by dest name
P – pause display ( else it will be often updated to show the current status )
j/k – scroll display
? – for help
Munin
Documentation
Installation
Wikipedia page
On CentOS 6 take out perl* from /etc/yum.conf and run yum install munin-java-plugins.noarch unbound-munin.x86_64 munin.noarch munin-common.noarch munin-node.noarch
vnstat
apt-get install vnstat vnstati /etc/init.d/vnstat start
Note about vnstat. Virtual and aliased interfaces cannot be monitored because the kernel doesn't provide traffic information for that type of interfaces. Such interfaces are usually named eth0:0, eth0:1, eth0:2, venet0:0 etc. where eth0/venet0 is the actual interface being aliased.
Can edit /etc/vnstat.conf and enter venet0 or 0:0 instead of eth0
# default interface Interface "eth0"
RRD Tool
NTP
To use Stratum 1 or 2 on Ubuntu/Debian you can normally just do:
apt-get install ntp ntpq -p ntpq -pn
You want a result of Stratum 1 or 2.
IPv4 and IPv6
As you may already be aware the internet is moving away from the old IPv4 standard and work is beginning to integrate the new IPv6 standard across the entire internet.
The underpinning of the Internet is “IP” (Internet Protocol). A 32-bit number that every device connecting to the internet receives. This is the old IPv4 standard and the problem is it’s this resource that’s now running short. IPv4 gave us 4.3 billion possible combinations of address, but with the growth of the global internet, and mobile internet means we’ve nearly worked through those numbers. That is why IPv6 is here and already being rolled out.
An IPv4 address looks like this: 212.67.202.5 An IPv6 address will look like this: 001:0db8:85a3::8a2e:0370:7334 or 2a02:4e8:4:1050::212.67.202.5
IPv6 is the next generation of the internet, with improved functionality and more resources (IPs) for adding websites. In fact IPv6 allows for approximately 340 undecillion possible combinations of addresses. To put it in perspective, if the current pool of 4.3 billion addresses were the size of a golf ball, the new 340 undecillion address space would be about the size of the sun. This means more resources and allows the internet to expand way into the future without running out of addresses.
In 2012 the use of IPv6 will become more prominent which is why companies are starting to integrate the new IPv6 standard to their networks.
Understanding traceroutes
Every IP packet has a value called "ttl". It normally starts at 255, then for every router it has to go through the ttl must be reduced by at least 1. If the ttl reaches zero before the packet reaches its destination, the packet goes no further. This is just a mechanism to prevent infinite loops.
If the packet has been dropped, the router handling the packet at the time should send an ICMP (Internet Control Message Protocol) IP packet back to the sender (from the router's IP address) indicating that the TTL expired in transit, so that the sender knows it's pointless to send any more packets for the time being. Since the ICMP packets themselves have ttls, you might expect them to never make it back, but that's usually not the case because the ICMP packets don't backtrack the original route (which probably has a loop in it) but rather they make their own route back.
Example:
Sender | ^ v | | ^ |<--\ | v *->/ \->-/ . . . . Recipient
The circle of lines above represents an unintended router loop; * is the router at which the ttl is zero, and the line coming out to the right is the ICMP packet going back.
For a working route, you can abuse this mechanism to find out which routers you're going through. That's what traceroute is intended to do: it send the same basic packet to the same destination with increasing ttls until it gets all the way through, eg:
Sender v ^ ^ ^ ^ +-/ | | | +->-/ | | +->----/ | +->-------/ v Recipient
When a router is heavily loaded, it needs to start dropping traffic to keep its workload under control, and the lowest-priority traffic of all is ICMP messages, so you can sometimes not get the response at all. There are also some routers (eg. transparent bridges) which either can't respond because they don't actually have an IP address or won't respond as a matter of policy. By sending three packets at each ttl level, traceroute gives you a good chance of working out if you're getting some packets dropped due to high load, or if you're just not going to get anything at that ttl level.
To give an estimation of the logical distance between hops, traceroute compares the send time to the ICMP packet receive time, giving a "round trip time": the time to reach the router and come back. Usually, but not always, this is roughly double the time needed to reach the router.
Reading the traceroute:
If you get several hops in a row which are just three stars (representing dropped or late packets), you've probably hit a dead router, and the one affected is probably the one after the last one you can see in the list, but it MIGHT be any router on the return journey. Of course you can tell if the route works at all just by using ping.
"Dead" routers might be broken in some way, but they might also be rejecting the packet via a firewall. Thus if you've got stars only towards the end (particularly just the last row), it could be that a firewall at the recipient point hates traffic from the sender, or that a firewall at the sending point hates traffic from the recipient.
If you get one or two hops which are all stars, they most likely just can't/won't respond but can forward packets okay.
If you get a very varied set of times (and possibly a star or two) on a line and on all lines after it, that may mean that at the first point where this occurs there is very variable load which might cause some small degree of slowdown. By "very varied", I mean varying by over 100ms.
If you get a one or two reasonably low and very consistent times and one or two stars on a line, then either you happen to have caught a one-off surge in network load (in which case there won't be many stars after that point) or hit a split point with multiple routes of which one has a dead router (in which case there will probably be several stars after that point). In the split-with-dead-router case you might have intermittent but severe latency. But that doesn't happen often.
If you get a consistent set of times on a line which are noticeably higher (again, by at least 100ms) than the previous line, there's probably some network congestion or slow hardware at that point. That's typically what you'd expect slow data transfer to come from.
If you get the same routers appearing repeatedly, you have a router loop.
You might want to note that because you always have to go through all the routers which have already been "tested", odd/intermittent results on/after a particular line are not necessarily caused at that point, but might have been caused at any point before.
Better alternative:
Basically traceroutes can hint at a lot of quite complicated things. In general, it's far better to try a ping first. Send say 20 pings ("ping -c 20" on linux) and then note the statistics at the bottom. Here's an example:
20 packets transmitted, 20 received, 0% packet loss, time 19012ms rtt min/avg/max/mdev = 0.412/0.637/1.994/0.446 ms, pipe 2
1. If you've got 100% packet loss then there may be some kind of routing problem. You could traceroute to try to find out where.
2. If you've got somewhere from 10% to 40% packet loss then there's probably some extreme network congestion at some point (or just slowness on the receiving or sending machine). You could do a traceroute, although see [4-6] below.
3. If you've got over 40% packet loss then I'd expect you're using multiple routes one of which is bad. A traceroute might reveal where the problem is, but I'd expect this to be a rare situation.
4. If all the RTT numbers are low, there's no networking problem, and no point whatsoever in running traceroute.
5. If the RTTs are very varied (again, by over 100ms) then there's highly variable network load which is (because at the server side things tend to be very consistent) very likely to be towards the client side. You could traceroute and then look for the first hop where the variation really starts
6. If all the RTTs are very high (over 100ms?) but not hugely varied, then there's high network load somewhere, most likely at the server side. You should traceroute then look for where the numbers suddenly become higher.
ifconfig
This is used on Unix systems to see the IP addresses assigned to your server and the network interfaces. To see the MAC addresses of the NICs on your system, type ifconfig -a
To start or stop a network interface use ifup or ifdown. Man ifup will show it's options.
To remove an interface you may need to do something like:
ifconfig eth1:1 ifdown
Ubuntu location: /etc/network/ and /etc/network/interfaces
CentOS WHM/cPanel server location: /etc/sysconfig/network-scripts
DOS (Denial of Service)
Under construction.
If it is Outbound, try netstat -tulnap | grep IPaddressBeingAttacked and then ls -lah /proc/XPID
A script to monitor this is below that should be run in screen using watch -n 5 sh netmon.sh:
#!/bin/bash #netmon.sh netstat -tulnap | grep -v LISTEN echo -e "\n`date`" >> netstat.log netstat -tulnap | grep -v LISTEN >> netstat.log
If your website uses mainly static pages, you could use something like Varnish:
Install and Configure on Ubuntu 12.04
Google search for Varnish Install
Manually blocking an inbound DDoS/DOS without a hardware firewall on Linux:
netstat -ant | wc -l netstat -n -p|grep SYN_REC | wc -l
The number for SYN_REC should be pretty low, preferably less than 5. On DoS attack incidents or mail bombs, the number can jump to pretty high. However, the value always depends on system, so a high value may be average on another server.
Check the amount of connections you are getting, then do netstat -ant . Do a WHOIS on the IP and see where it is coming from. If you are being attacked from China for example, do the following in SSH:
cd /root vim ipblockscript.sh
Add the below into the file:
#!/bin/bash ### Block all traffic from CHINA (CN). Use ISO code ### ISO="cn" ### Set PATH ### IPT=/sbin/iptables WGET=/usr/bin/wget EGREP=/bin/egrep ### No editing below ### SPAMLIST="countrydrop" ZONEROOT="/root/iptables" DLROOT="http://www.ipdeny.com/ipblocks/data/countries" cleanOldRules(){ $IPT -F $IPT -X $IPT -t nat -F $IPT -t nat -X $IPT -t mangle -F $IPT -t mangle -X $IPT -P INPUT ACCEPT $IPT -P OUTPUT ACCEPT $IPT -P FORWARD ACCEPT } # create a dir [ ! -d $ZONEROOT ] && /bin/mkdir -p $ZONEROOT # clean old rules cleanOldRules # create a new iptables list $IPT -N $SPAMLIST for c in $ISO do # local zone file tDB=$ZONEROOT/$c.zone # get fresh zone file $WGET -O $tDB $DLROOT/$c.zone # country specific log message SPAMDROPMSG="$c Country Drop" # get BADIPS=$(egrep -v "^#|^$" $tDB) for ipblock in $BADIPS do $IPT -A $SPAMLIST -s $ipblock -j LOG --log-prefix "$SPAMDROPMSG" $IPT -A $SPAMLIST -s $ipblock -j DROP done done # Drop everything $IPT -I INPUT -j $SPAMLIST $IPT -I OUTPUT -j $SPAMLIST $IPT -I FORWARD -j $SPAMLIST # call your other iptable script # /path/to/other/iptables.sh exit 0
chmod +x ipblockscript.sh crontab -e
Add in the below:
@weekly /path/to/country.block.iptables.sh :wq
This will take a while to complete
/root/ipblockscript.sh
Monitor iptables if it is continually adding IP's to the firewall as well as top to see the load of the server
iptables -L top
And keep checking netstat to check how many connections you are getting:
netstat -ant | wc -l netstat -ant
You may get consistently signal exited from the server. If the DOS is just on one port, e.g. Apache on 80, simply kill the process and the attack cannot get through then. This will give you time to set up the rules to block it. Please be aware that the amount of combinations for just two octets (0.0.0.0) is 65000 and so is a lot to manually do, even more if it is over three octets.
To ban an IP by a specific port number, for instance port 80 use this where X.X.X.X = the IP address:
iptables -A INPUT -p tcp -s X.X.X.X --dport 80 -j DROP
You can install ddosdeflate (temporary blocking) to do some of this for you:
wget http://www.inetbase.com/scripts/ddos/install.sh chmod 0700 install.sh ./install.sh
Block outgoing UDP traffic
iptables -A OUTPUT -s 192.168.0.1 -p udp -j DROP iptables -A OUTPUT -s 127.0.0.1 -p udp -j DROP
Display open ports and established UDP connections:
netstat -vaun
Old method:
Before using the drop command please replace 0.0.0.0 with the IP received from netstat.
ps aux netstat -ant netstat -ant|wc -l iptables -L iptables -I INPUT -s 0.0.0.0/0.0.255.255 -j DROP (or individual IP's) iptables-save service iptables restart
Some useful commands:
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr netstat -ntu | grep -v TIME_WAIT | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr netstat -an | grep :80 | awk '{print $5}' | cut -f1 -d":" | sort | uniq -c | sort -n