Opscraft
Linux · Lesson 7 of 10 · 5 min read

Networking from the host's point of view

Interfaces, addresses and routes

ip -br addr                # interfaces and IPs, brief
ip link                    # link state, MTU
ip route                   # routing table; "default via 10.0.0.1" is the gateway
ip route get 8.8.8.8       # which route/interface a destination would use
ip neigh                   # ARP table

(ifconfig, netstat, route are from the deprecated net-tools package; use ip and ss.)

Sockets and ports

ss -tlnp                   # TCP listening sockets with process
ss -ulnp                   # UDP listening
ss -tan state established | wc -l
ss -tan | awk '{print $1}' | sort | uniq -c    # connections by state
ss -s                      # summary

A service listening on 127.0.0.1:8080 is unreachable from other hosts — it must bind 0.0.0.0 (or the specific interface IP). This is a very common "works locally, not remotely" cause, also inside containers.

Enlarge diagramListening on 127.0.0.1:8080Listening on 0.0.0.0:8080Only programs on the same machine can connectOther servers, load balancers and othercontainers get "connection refused"Accepts connections on every network interfaceUse security groups or a firewall to limit who canreach it
Figure 1. Which address a service listens on decides who can reach it. A common "works locally, not remotely" cause, also inside containers.
Swipe sideways to see all of it, or tap to open it full screen.

Ports below 1024 need root or CAP_NET_BIND_SERVICE.

Name resolution on the host

The order is controlled by /etc/nsswitch.conf (hosts: files dns): /etc/hosts first, then DNS servers from /etc/resolv.conf. On systemd-resolved systems resolv.conf points at the stub 127.0.0.53; resolvectl status shows the real upstream servers. getent hosts example.com resolves exactly as applications do (honouring nsswitch), unlike dig which queries DNS directly.

Enlarge diagramProgram calls getaddrinfo("db.internal")/etc/nsswitch.confhosts: files dns → check files first, then DNS/etc/hostsstatic name → IP list; if found, stop here/etc/resolv.confnameserver 127.0.0.53 (systemd-resolved, a local cache)Upstream DNS servere.g. the VPC resolver; resolvectl status shows it
Figure 2. How a program on Linux turns a name into an IP address. getent follows this path; dig skips straight to DNS.
Swipe sideways to see all of it, or tap to open it full screen.

Testing connectivity layer by layer

ping -c3 10.0.2.15                     # L3 reachability (ICMP — often blocked in AWS SGs)
nc -zv db.internal 5432                # can I complete a TCP handshake?
curl -v --connect-timeout 5 https://api.internal/health   # DNS, TCP, TLS, HTTP in one
curl -w '%{time_namelookup} %{time_connect} %{time_appconnect} %{time_total}\n' -o /dev/null -s https://site
traceroute -T -p 443 api.example.com   # TCP traceroute through firewalls
mtr api.example.com                    # continuous per-hop loss/latency
tcpdump -i eth0 -nn port 5432          # see the packets

Interpreting failures:

  • Connection refused — reached the host, nothing listening (RST). Check the service and its bind address.
  • Timeout — packets dropped: security group, NACL, host firewall, routing.
  • No route to host — routing problem or an ICMP reject from a firewall.
Enlarge diagramCONNECTEDCONNECTION REFUSED · FAILS INSTANTLYTIMED OUT · FAILS AFTER A WAITclientserverlistening on :443clientservernothing on :8080clientfirewall / SGdrops the packetSYN → SYN-ACK → ACKSYNRST: "nobody here"SYNno reply: check SG, NACL, routes
Figure 3. The three outcomes of trying to connect, and what each one tells you.
Swipe sideways to see all of it, or tap to open it full screen.

Host firewall

iptables (and its successor nftables) filter packets in chains (INPUT, OUTPUT, FORWARD) and do NAT (PREROUTING/POSTROUTING). Docker and kube-proxy write many iptables rules — that's how published ports and Services work. iptables -L -n -v and iptables -t nat -L -n show them. Higher-level wrappers: ufw (Ubuntu), firewalld (RHEL). On AWS, security groups usually do the filtering and host firewalls are kept simple.

conntrack tracks connections for stateful filtering and NAT. Busy nodes can fill the table (nf_conntrack: table full, dropping packet in dmesg); raise net.netfilter.nf_conntrack_max.

SSH essentials

ssh -i key.pem [email protected]
ssh -J bastion.example.com [email protected]        # jump through a bastion
ssh -L 5432:db.internal:5432 bastion            # local port-forward to a private DB
scp file host:/tmp/ ; rsync -avz --progress dir/ host:/srv/dir/
Enlarge diagramYOUR LAPTOPVPCPUBLIC SUBNETPRIVATE SUBNETpsqllocalhost:5432bastiononly port 22 opendb.internalPostgreSQL, private IPonlySSH :22:5432
Figure 4. ssh -L 5432:db.internal:5432 bastion: your laptop reaches a private database through the bastion. Only port 22 on the bastion is exposed.
Swipe sideways to see all of it, or tap to open it full screen.

~/.ssh/config stores hosts, users, keys and jump hosts. ssh -vvv shows where authentication fails.