Nmap Security Scanner Ref Guide Security Lists Security Tools Site News Advertising About/Contact Credits Sponsors |
Intro | Reference Guide | Book | Install Guide
Download | Changelog | Zenmap GUI | Docs
Bug Reports | OS Detection | Propaganda | Related Projects
In the Movies | In the News
Technique Described
| Chapter 7. Service and Application Version Detection |
## Technique Described Nmap version scanning is actually rather straightforward. It was designed to be as simple as possible while still being scalable, fast, and accurate. The truly nitty-gritty details are best discovered by downloading and reviewing the source code, but a synopsis of the techniques used follows. Nmap first does a port scan as per your instructions, and then passes all the open TCP and/or UDP ports to the service scanning module. Those ports are then interrogated in parallel, although a single port is described here for simplicity. Nmap checks to see if the port is one of the ports to be excluded, as specified by the `Exclude` directive in `nmap-service-probes`. If it is, Nmap will not scan this port for reasons mentioned in the section called “`nmap-service-probes` File Format” If the port is TCP, Nmap starts by connecting to it. Once the TCP connection is made, Nmap listens for roughly 5 seconds. Many common services, including most FTP, SSH, SMTP, telnet, POP33, and IMAP servers, identify themselves in an initial welcome banner. Nmap refers to this as the “NULL probe”, because Nmap just listens for responses without sending any probe data. If any data is received, Nmap compares it to hundreds of signature regular expressions in its `nmap-service-probes` file (described in the section called “`nmap-service-probes` File Format”). If the service is fully identified, we are done with that port! The regular expression includes substrings that can be used to pick version numbers out of the response. In some cases, Nmap gets a “soft match” on the service type, but no version info. In that case, Nmap continues but only sends probes that are known to recognize the soft-matched service type. At this point, Nmap UDP probes start, and TCP connections end up here if the NULL probe above fails or soft-matches. Since the reality is that most ports are used by the service they are registered to in `nmap-services`, every probe has a list of port numbers that are considered to be most effective. For example, the probe called GetRequest that recognizes web servers (among other services) lists 80-85, 8000-8010, and 8080-8085 as probable ports. Nmap sequentially executes the probe(s) that match the port number being scanned. Each probe includes a probe string (which can be arbitrary ASCII text or \xHH escaped binary), which is sent to the port. Responses that come back are compared to a list of regular expressions of the same type as discussed in the NULL probe description above. As with the NULL probe, these tests can either result in a full match (ends processing for the remote service), a soft match (limits future probes to those which match a certain service), or no match at all. The exact list of regular expressions that Nmap uses to test for a match depends on the probe fallback configuration. For instance, the data returned from the X11Probe is very unlikely to match any regular expressions crafted for the GetRequest probe. On the other hand, it is likely that results returned from a Probe such as RTSPRequest might match a regular expression crafted for GetRequest since the 2 protocols being tested for are closely related. So the RTSPRequest probe has a fallback to GetRequest matches. For a more comprehensive explanation, see the section called “Cheats and Fallbacks” In most cases, the NULL probe or the probable port probe(s) (there is usually only one) described above matches the service. Since the NULL probe shares its connection with the probable port probe, this allows service detection to be done with only one brief connection in most cases. With UDP only one packet is usually required. But should the NULL probe and probable port probe(s) fail, Nmap goes through all of the existing probes sequentially. In the case of TCP, Nmap must make a new connection for each probe to avoid having previous probes corrupt the results. This worst-case scenario can take a bit of time, especially since Nmap must wait about five seconds for the results from each probe because of slow network connections and otherwise slowly responding services. Fortunately, Nmap utilizes several automatic techniques to speed up scans: Nmap makes most probes generic enough to match many services. For example, the GenericLines probe sends two blank lines (“`\r\n\r\n`”) to the service. This matches daemons of many diverse service types, including FTP, ident, pop3, uucp, postgres, and whois. The GetRequest probe matches even more service types. Other examples include “`help\r\n`” and generic RPC and MS SMB probes. If a service matches a softmatch directive, Nmap only needs to try probes that can potentially match that service. All probes were not created equal! Some match many more services than others. Because of this, Nmap uses the rarity metric to avoid trying probes that are extremely unlikely to match. Experienced Nmap users can force all probes to be tried regardless or limit probe attempts even further than the default by using the `--version-intensity`, `--version-all`, and `--version-light` options discussed in the section called “Probe Selection and Rarity” One of the probes tests whether the target port is running SSL. If so (and if OpenSSL is available), Nmap connects back via SSL and restart the service scan to determine what is listening behind the encryption. A special directive allows different probable ports for normal and SSL tunneled connections. For example, Nmap should start against port 443 (https) with an SSL probe. But after SSL is detected and enabled, Nmap should try the GetRequest probe against port 443 because that port usually has a web server listening behind SSL encryption. Another generic probe identifies RPC-based services. When these are found, the Nmap RPC grinder (discussed later) is initiated to brute force the RPC program number/name and supported version numbers. Similarly, an SMB post-processor for fingerprinting Windows services may be added eventually. If at least one of the probes elicits some sort of response, yet Nmap is unable to recognize the service, the response content is printed to the user in the form of a fingerprint that can be submitted at a provided URL for the next version of Nmap. ### Cheats and Fallbacks Even though Nmap waits a generous amount of time for services to reply, sometimes an application is slow to respond to the NULL probe. This can occur for a number of reasons, including slow reverse DNS lookups performed by some services. Because of this, Nmap can sometimes match the results from a subsequent probe to a match line designed for the NULL probe. For example, suppose we scan port 25 (SMTP) on a server to determine what is listening. As soon as we connect, that service may conduct a bunch of DNS blacklist lookups to determine whether we should be treated as spammers and denied service. Before it finishes that, Nmap gives up waiting for a NULL probe response and sends the next probe with port 25 registered, which is “`HELP\r\n`”. When the service finally completes its anti-spam checks, it prints a greeting banner, reads the Help probe, and responds as shown in Example 7.4, “NULL Probe Cheat Example Output” Example 7.4. NULL Probe Cheat Example Output
    220 hcsw.org ESMTP Sendmail 8.12.3/8.12.3/Debian-7.1; Tue, [cut]
    214-2.0.0 This is sendmail version 8.12.3
    214-2.0.0 Topics:
    214-2.0.0       HELO    EHLO    MAIL    RCPT    DATA
    214-2.0.0       RSET    NOOP    QUIT    HELP    VRFY
    214-2.0.0       EXPN    VERB    ETRN    DSN     AUTH
    214-2.0.0       STARTTLS
    214-2.0.0 For more info use "HELP <topic>".
    214-2.0.0 To report bugs in the implementation send email to
    214-2.0.0       sendmail-bugs@sendmail.org.
    214-2.0.0 For local information send email to Postmaster at your site.
    214 2.0.0 End of HELP info
Nmap reads this data from the socket and finds that no regular expressions from the Help probe match the data returned. This is because Nmap normally expects to receive the ESMTP banner during the NULL probe and match it there. Because this is a relatively common scenario, Nmap “cheats” by trying to match responses to any of the NULL Probe match lines if none of the probe-specific lines match. In this case, a null match line exists which reports that the program is Sendmail, the version is 8.12.3/8.12.3/Debian-7.1, and the hostname is hcsw.org. The NULL probe cheat is actually just a specific example of a more general Nmap feature: fallbacks. The fallback directive is described in detail in the section called “`nmap-service-probes` File Format” Essentially, any probe that is likely to encounter results that can be matched by regular expressions in other probes has a fallback directive that specifies these other probes. For example, in some configurations of the popular Apache web server, Apache won't respond to the GetRequest (“`GET / HTTP/1.0\r\n\r\n`”) probe because no virtual host namehas been specified. Nmap is still able to correctly identify these servers because those servers usually respond to the HTTPOptions probe. That probe has a fallback to the GetRequest regular expressions, which are sufficiently general to recognize Apache's responses to the HTTPOptions probes. ### Probe Selection and Rarity In determining what probes to use, Nmap considers their `rarity`. This is an indication of how likely the probe is to return useful data. If a probe has a high rarity, it is considered less common and is less likely to be tried. Nmap users can specify which probes are tried by changing the intensity level of the version scan, as described below. The precise algorithm Nmap uses when determining which probes to use follows: For TCP, the NULL probe is always tried first. All probes that have the port being scanned listed as a probable port (see the section called “`nmap-service-probes` File Format”) are tried in the order they appear in `nmap-service-probes`. All other probes that have a rarity value less than or equal to the current intensity value of the scan are tried, also in the order they appear in `nmap-service-probes`. Once a probe is found to match, the algorithm terminates and results are reported. Because all of Nmap's probes (other than the NULL probe) have a rarity value associated with them, it is relatively easy to control how many of them are tried when performing a version scan. Simply choose an intensity level appropriate for a scan. The higher an intensity level, the more probes will be tried. So if a very comprehensive scan is desired, a high intensity level is appropriate—even though it may take longer than a scan conducted at a lower intensity level. Nmap's default intensity level is 7 but Nmap provides the following switches for different scanning needs: #### `--version-intensity` Syntax: `--version-intensity` <`intensity level between 0 and 9`> Sets the intensity level of a version scan to the specified value. If 0 is specified, only the NULL probe (for TCP) and probes that list the port as a probable port are tried. Example:
    nmap -sV --version-intensity 3 scanme.nmap.org
#### `--version-light` Syntax: `--version-light` Sets the intensity level to 2. Example:
    nmap -sV --version-light host.com
#### `--version-all` Syntax: `--version-all` Sets the intensity level to 9. Since all probes have a rarity level between 1 and 9, this tries all of the probes. Example:
    nmap -sV --version-all host.com
Usage and Examples |  | Technique Demonstrated
| [ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]