Pragmatic Network Scanning

The what and how of scanning and enumeration in the context of a penetration test get defined differently depending on who you ask as well as the objectives and methodologies relevant to the type of testing being done (eg. compliance driven, red team, etc.). Most of the books, blogs, and documented methodologies I've read through treat this phase as too much of a 'point -> click -> find stuff -> get ready to exploit' type process. I have two main issues with this description. Effective discovery in even a moderately complex environment requires a much more tactical application of scanning techniques. Also, in practice, it is a much more cyclical process that happens in bits as an engagement progresses rather than a one and done scan.

The Definition and the Objective

I figured it would be good to start off by trying to nail down the true objective. This needs to be a broad and inclusive definition because the details can change drastically depending on the how and why of each style, purpose, and methodology of the penetration test.

The purpose of network scanning and enumeration in a penetration test is to identify live systems and active network services. 

In reality, the core objective is to leverage the data collected to expand your understanding of the attack surface to setup exploitation opportunities.

The Problem

The issue is that even a brief thought experiment of how this falls out in a real world scenario makes it obvious that applied scanning can run into a complex set of hurdles that require a lot of consideration. Also, as I mentioned above this is likely an activity that happens more than once during an engagement. A quick rundown on just a few of the considerations that can add to this complexity.

Location on the Network

One important consideration when thinking about how you might set up your network scans is the source network segment.

  • Are there likely to be access controls between where you are scanning to other segments?
  • Are there ports that might be open universally between segments regardless of access controls?
  • What types of remote administration channels are open between segments?

Clearly, this becomes more critical as an engagement progresses. Once you have compromised some set of hosts it can be very enlightening to compare the network segmentation differences between segments.

Blind

Occasionally clients require that all network and system discovery is done completely blind during internal penetration test (meaning no IP ranges are provided). Ping and port scan discovery techniques on large networks can be very time-consuming. In these cases, intelligent selection based on observed patterns with a gradually widening scope seems to be a reliable starting point.

Stealthy

This is a topic that could easily be a blog series (or maybe a book) all on its own. Consideration of stealth in the scanning process can be a tricky thing. There has to be consideration given to the various detection methods that might be applied to detecting something as predictable as a network scan. Is the network configured with sensors that can detect a rise in bandwidth usage? What kind of other monitoring and alerting are the NIDS and HIDS configured for? The standard Nmap TCP and SYN packet scans are fairly easy to detect at any reasonable speed. The truth is without a solid knowledge of the monitoring and alerting systems in place on a network that the traditional network scan is going to be a very risky proposition and it makes more sense to utilize alternative methods of host discovery. I'll go over a few that might be useful in the next section.

Example

Imagine you are on a network with 30 or 40 segments.

You run a ping sweep to determine if it might be a reasonable method of host discovery.

nmap -sn -PE -iL target-subnet-list.txt -oA target.pingsweep

Starting Nmap 7.40 ( https://nmap.org ) at 2017-04-04 20:56 PDT
Nmap scan report for cisco-router (192.168.0.1)
Host is up (0.0040s latency).
Nmap scan report for 192.168.0.103
Host is up (0.0060s latency).
Nmap scan report for 192.168.0.104
Host is up (0.0084s latency).
Nmap scan report for 192.168.0.115
...
grep Up target.pingsweep.gnmap | wc -l
      2458

The results come back with 2458 systems as being Up.

You think "great, we know what systems are live." Then you take a closer look at the results and notice that several segments where you expected to find hosts returned no responses to the ping sweeps. Now you have to decide how you want to go about your host discovery. Do you want to use the TCP ping capability of Nmap? You run this scan with a set of ports you think will catch the most hosts.

nmap -sn -PA22,80,443,8080 -PS22,80,443,445,139 -oA target.hostdisc

Starting Nmap 7.40 ( https://nmap.org ) at 2017-04-04 20:56 PDT
Nmap scan report for cisco-router (192.168.0.1)
Host is up (0.0040s latency).
Nmap scan report for 192.168.0.10
Host is up (0.0060s latency).
Nmap scan report for 192.168.0.11
Host is up (0.0084s latency).
Nmap scan report for 192.168.0.43
Host is up (0.0085s latency).
Nmap scan report for 192.168.0.42
...
grep Up target.hostdisc.gnmap | wc -l
      2622

This scan discovered 2622 hosts, 164 more than just the ping sweep. Now you have to decide if you want to expand this port list further and try again. At what point do we start enumerating services on those open ports? If the lists of hosts and services are big enough you have to narrow down what you want to spend your time trying to enumerate. Of course, you have to balance this all very carefully with the time allowed for the engagement so we might have to operate under the assumption that there might be hosts and network services we never look at. There are a lot of other tools and techniques at your disposal that needs to be considered too.

So network environments can be complex animals with lots of potential pitfalls. In the example above we only tackle the idea of effective host discovery. There are lots of other things to take into account like firewalls that reply to every syn with a syn-ack. How do we define what ports and services we need to target in a fluid testing situation? I think this makes the point that as complexity increases so does the need to apply intelligent selection to your scanning and enumeration techniques to make sure you cover as much ground as possible in a limited amount of time. Any one technique applied too generally will likely miss stuff we want to find.

Pragmatic Scanning Processes

None of the techniques I will go over in this section are groundbreaking. Like I mentioned in the intro my only objective is to address some applied methods for host and service enumeration. The real trick is likely going to be finding the correct combination of techniques and applying them as you learn more about the environment.

Balancing Time, Scope, and Effectiveness

I've covered this a bit already, but it's worth reiterating the idea that doing a good job here might come down to a balance of the time allotted for the engagement, the size of the network you are testing, and how thorough you want to be. Keep this in mind as you set up and monitor your first scans. For example, with Nmap, you might need to utilize the -vv and --reason flags to get a feel for how long a scan is going to take and if the results are what you expect them to be. In those cases where you have a need to be broad utilize the wide range of timing options Nmap gives you such as --min-hostgroup, --max-retries, --host-timeout, and --min-rate.

Low Hanging Fruit

A few ways to speed up the first tasks in host discovery might be to leverage some fast and effective alternatives to the regular old port scan.

Start by reviewing local broadcast traffic. I always like to fire up a packet capture as soon as I connect to see what interesting things come in. With that capture file in hand it's easy to parse out a list of host IPs:

tshark -r capture1.pcap -T fields -e ip.dst ip.src | sort -u
10.20.12.23
10.50.2.21
10.50.235.243
10.50.235.252
10.50.235.253
10.50.237.239

ARP is a fast a reliable way to see who is willing to communicate on the local subnet. My goto tool for ARP scans is arp-scan:

arp-scan --localnet
Interface: en0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.9 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)
192.168.0.1    2c:21:24:3e:d3:2c    (Unknown)
192.168.0.103    7c:c4:7b:af:53:9b    (Unknown)
192.168.0.110    4c:29:38:d4:b1:c6    (Unknown)
192.168.0.132    ac:c4:74:ab:20:f0    (Unknown)
192.168.0.102    10:ed:2b:5a:cf:d2    (Unknown)

Keep in mind as you run through these tasks that some of the magic will be in how you store and process the data you capture. An often telling data point is what one type of scans shows that another doesn't. Think carefully about your storage and analysis practices and how they lend themselves to review.

Continue Building a Host List

When it comes to IPs I like to work with lists. When possible I like to generate a list of live hosts on the network so I can be more targeted in my service enumeration. When I approach it like this I always keep in mind that there are likely groups of hosts I might miss due to users with laptops that might be walking around or a host that is getting a reboot when I run my discovery.

When I want to run a scan across a large number of IP addresses with little knowledge of the network I've found that the Nmap default host discovery scan is pretty useful.

The default host discovery done with -sn consists of an ICMP echo request, TCP SYN to port 443, TCP ACK to port 80, and an ICMP timestamp request by default.

nmap -sn -iL scope-network-ranges

Starting Nmap 7.40 ( https://nmap.org ) at 2017-04-06 13:01 PDT
Nmap scan report for cisco (192.168.0.1)
Host is up (0.0056s latency).
Nmap scan report for 192.168.0.103
Host is up (0.0061s latency).
Nmap scan report for 192.168.0.104

If these results seem wonky for any reason or you learn during testing that a more defined set of ports will produce more info it is easy to extend the type of ping used and the associated ports probed with the -PA, -PS, or -PU flags followed by the target ports.

The larger the groups of hosts/size of the network the more it makes sense to be more targeted with service discovery. Keeping in mind that as you become more specific you increases the likelihood of missing stuff.

Masscan for Port Sweeps

One situation I encounter during engagement is a compliance requirement to verify segmentation. This generally involves scanning large subnet blocks as quickly and efficiently as possible. I've found that Masscan does a better job of covering large swaths of service discovery than any other tool I've tried. The ability to define a configuration file and pause/resume scans is very handy too.

$ cat masscan-cde.conf
range=192.168.75.0/24
range=192.168.0.0/24
range=192.168.76.0/24
range=192.168.113.0/24
ping=false
ports=8000,8443,10000,8888,900,993,995,111,8080,3306,135,53,143,139,445,110,389,3389,25,22,21,443,23,80
rate=250

Execution works very much like Nmap.

masscan -c masscan.conf -oL masscan.txt

Starting masscan 1.0.3 (http://bit.ly/14GZzcT) at 2017-01-26 16:27:24 GMT
 -- forced options: -sS -Pn -n --randomize-hosts -v --send-eth
Initiating SYN Stealth Scan
Scanning 256 hosts [25 ports/host]

The text based output option (-oL) is handy because now we can awk out sets of ports and run service version discovery and enumeration if we want to.

awk '$3 == "445" { print $4 }' masscan.txt > masscan-port445.txt
tail masscan-port445.txt
192.168.82.59
192.168.82.60
192.168.82.61

A Few Non-Scanning Options

There are a lot of ways you can perform host discovery and enumeration outside of standard scanning techniques. This is a topic that deserves a bit more depth but I want to touch on a few easy ones just to give a starting point.

DNS Queries

DNS can be a great source of information during a penetration test. One of the first things I like to do when connecting to a network (assuming I know all the network ranges in the testing scope) is look at all the system names. Sometimes we can enumerate usernames, system purposes, domain controllers, and much more just by looking at the conventions used. If we are on a network that is using DHCP and we don't have to go find the DNS server we can just take a look at the resolv.conf.

cat /etc/resolv.conf
domain targetdomain
search targetdomain
nameserver 10.10.11.12
nameserver 10.10.11.13

Net View

Depending on how an engagement shakes out this might be veering into the post-exploitation territory since you will likely need AD credentials but from a Windows machine, you can use the 'net view' command to generate a list of systems.

Z:\USERS\Administrator>net view /domain

Server Name            Remark

—————————————————————

\\WOOKIE1.targetdomain

\\WOOKIE2.targetdomain

\\WOOKIE3.targetdomain

\\LUKE

\\HAN

...

Post-Exploitation

Keep in mind that as a penetration test progresses the opportunities to enumerate target data will expand. Activities such as dumping a list of domain computer accounts, running 'netstat' on compromised hosts, and reviewing host routing tables all can give valuable information about what we might need to add to our list of potential targets.