Nmap Security Scanner
Ref Guide
Security Lists
Security Tools
Site News
Advertising
About/Contact
Credits
Sponsors
|

 |
Intro |
Reference Guide |
Book |
Install Guide |
Download |
Changelog |
Zenmap GUI |
Docs |
Bug Reports |
OS Detection |
Propaganda |
Related Projects |
In the Movies |
In the News |
|
 |
In addition to the significant built-in capabilities of
Lua, we have written or integrated several extensions to make
NSE scripts more powerful and convenient to write. These
modules are compiled and installed along with
Nmap. They have their own directory, nselib , which
is installed in the configured datadir. Scripts need only require the default modules in order to use them. The default modules are described in the following sections.
Bitwise Logical Operations
Lua does not provide bitwise logical operations. Since they
are often useful for low-level network communication, Reuben
Thomas' bitwise operation library for Lua has been
integrated into NSE. The arguments to the bitwise operation
functions should be integers. The number of bits available
for logical operations depends on the data type used to
represent Lua numbers—this is typically 8-byte IEEE
floats, which give 53 bits (the size of the mantissa).
This implies that the bitwise operations won't work (as expected)
for numbers larger than 1014. You
can use them with 32-bit wide numbers without any problems. Operations
involving 64-bit wide numbers, however, may not return the expected
result.
The logical operations start with “b” (for bit ) to avoid
clashing with reserved words; although xor isn't a
reserved word, it seemed better to use bxor for
consistency. In NSE the bitwise functions are in the bit
namespace.
bit.bnot(a)
Returns the one's complement of a.
bit.band(w1,...)
Returns the bitwise and of the
w's.
bit.bor(w1,...)
Returns the bitwise or of the w's.
bit.bxor(w1,...)
Returns the bitwise xor of the
w's.
bit.lshift(a,b)
Returns a shifted left b places—padded with zeros.
bit.rshift(a,b)
Returns a shifted logically right b places.
bit.arshift(a,b)
Returns a shifted arithmetically right b places.
bit.mod(a,b)
Returns the integer remainder of a divided by b.
Perl Compatible Regular Expressions
One of Lua's quirks is its string patterns. While they have
great performance and are tightly integrated into the Lua
interpreter, they are very different in syntax and not as
powerful as standard regular expressions. So we have
integrated Perl compatible regular expressions into Lua
using libPCRE and a modified version of the Lua PCRE library
written by Reuben Thomas and Shmuel Zeigerman. These are
the same sort of regular expressions used by Nmap version
detection. The main modification to their library is that
the NSE version only supports PCRExpressions instead of both
PCRE and POSIX patterns. In order to maintain a high script
execution speed, the library interfacing with libPCRE is
kept very thin. It is not integrated as seamlessly as the
Lua string pattern API. This allows script authors to decide
when to use PCRE expressions versus Lua patterns. PCRE
involves a separate pattern compilation step, which saves
execution time when patterns are reused. Compiled patterns
can be cached in the NSE registry and reused by other
scripts. The PCRE functions reside inside the pcre
namespace.
![[Warning]](warning.png) | Warning |
---|
LibPCRE has a history of security vulnerabilities
allowing attackers who are able to compile arbitrary regular
expressions to execute arbitrary code. More such
vulnerabilities may be discovered in the future. These have
never affected Nmap because it doesn't give attackers any
control over the regular expressions it uses. Similarly, NSE
scripts should never build regular expressions with untrusted
network input. Matching hardcoded regular expressions
against the untrusted input is
fine. |
The following documentation is derived from that supplied by
the PCRE Lua lib. pcre.new(pattern, flags, locale)
Returns a compiled regular expression. The first
argument is a string describing the pattern, such as
^foo$ . The second
argument is a number describing which compilation
flags are set. The compilation flags are set
bitwise. If you want to set the 3rd (corresponding to
the number 4) and the 1st (corresponding to 1) bit
for example you would pass the number 5 as a second
argument. The compilation flags accepted are those
of the PCRE C library. These include flags for case
insensitive matching (1), matching line beginnings (^)
and endings ($) even in multiline strings (i.e. strings
containing “\n”) (2) and a flag for matching across
line boundaries (4). No compilation flags yield a default
value of 0. The third (optional) argument is a string
describing the locale which should be used to compile the
regular expression. The variable is a string which is
passed to the C standard library function
setlocale . For more
information on this argument refer to the
documentation of setlocale . The
resulting compiled regular expression is ready to be
matched against strings. Compiled regular
expressions are subject to Lua's garbage collection.
Generally speaking, my_regex = pcre.new("pcre-pattern ",0,"C")
should do the job most of the time.
pcre.flags()
Returns a table of the available PCRE option flags
(numbers) keyed by their names (strings). Possible
names of the available strings can be retrieved from
the documentation of the PCRE library used to link
against Nmap. The key is the option name in the
manual minus the PCRE
prefix. PCRE_CASELESS becomes
CASELESS for example.
pcre.version()
Returns the version of the PCRE library in use as a
string. For example 6.4 05-Sep-2005 .
pcre_obj:match(string, start, flags)
Returns the start point and the end point point of
the first match of the compiled regular expression
pcre_obj in the string. A third
returned value is a table which contains
false in the positions where the
pattern did not match. If named sub-patterns were
used the table also contains substring matches keyed
by their sub-pattern name. Should no match be found the
function returns nil .
The second and third arguments are optional. The second
argument is a number specifying where the engine should
start trying to apply the pattern. The third argument
specifies execution flags for the pattern.
If you want to see if a given string matches a certain expression
you could use: s = pcre_obj:match("string to be searched", 0,0);
if(s) code_to_be_done_on_match end
pcre_obj:exec(string, start, flags)
This function is like match() except that a table returned as
a third result contains offsets of substring matches rather
than substring matches themselves. That table will not
contain string keys, even if named sub-patterns are used. For
example, if the whole match is at offsets 10, 20 and substring
matches are at offsets 12, 14 and 16, 19 then the function
returns the following: 10, 20, {12,14,16,19}
pcre_obj:gmatch(string, func, n, ef)
Tries to match the regular expression pcre_obj against string
up to n times (or as many as possible if n is either
not given or is not a positive number), subject to
execution flags ef. Each time there is a match, func
is called as func(m, t) , where m is the matched
string and t is a table of substring matches. This
table contains false in the
positions where the corresponding sub-pattern did
not match. If named sub-patterns are used then the
table also contains substring matches keyed by their
correspondent sub-pattern names (strings). If func
returns a true value, then gmatch
immediately returns; gmatch returns the number of
matches made.
The ipOps module provides some functions for
manipulating IPv4 addresses. The functions reside inside the
ipOps namespace.
bool = ipOps.isPrivate("ip-string")
checks whether an IP address, provided as a string in
dotted-quad notation, is part of the non-routed private IP address
space, as described in RFC 1918 These addresses are the well known
10.0.0.0/8 ,192.168.0.0/16 and
172.16.0.0/12 networks.
DWORD = ipOps.todword("ip-string")
returns the IP address as DWORD value (i.e. the IP a.b.c.d becomes
(((a*256+b)*256+c)*256+d) )
a,b,c,d = ipOps.get_parts_as_number("ip-string")
returns 4 numbers corresponding to the fields in dotted-quad notation.
For example, ipOps.get_parts_as_number("192.168.1.1")
returns 192,168,1,1 .
Since portrules are mostly the same for many scripts, the
shortport module provides functions for the most common tests.
The arguments in brackets ([] ) are optional. If no
proto is provided, tcp is used. The default
state is open
shortport.portnumber(port,[proto],[state])
The port argument is either a number or a table of numbers which are
interpreted as port numbers, against which the script should run.
shortport.service(service,[proto],[state])
The service argument is either a string or a table
of strings which are interpreted as service names
(e.g. "http" , "https" , "smtp" or "ftp" ) against which the
script should run. These service names are
determined by Nmap's version scan or (if no version
scan information is available) the service assigned
to the port in nmap-services
(i.e. "http" for TCP port 80).
shortport.port_or_service(port,service,[proto],[state])
This is a combination of the above functions, since many scripts
explicitly try to run against the well known ports, but want
also to run against any other port which was discovered to run the
named service. A typical example for this function is:
portrule = shortport.port_or_service(22,"ssh") .
Functional Programming Style List Operations
People used to programming in functional languages, such as Lisp or
Haskell appreciate their handling of lists very much. The listop module tries to bring much of the functionality from
functional languages to Lua using Lua's central data structure, the table,
as a base for its list operations. Highlights include a map
function applying a given function to each element of a list.
bool = listop.is_empty(list)
Returns true if the given list is empty.
bool = listop.is_list(value)
Returns true if the given value is a list (or rather a table).
list = listop.map(function, list)
The provided function is applied to each element of the list
separately. The returned list contains the results of each
function call. For example listop.map(tostring,{1,2,true})
returns {"1","2","true"} .
value = listop.apply(function, list)
All of the elements in the list are passed to a call of
function . The result is then returned. For example
listop.apply(math.max,{1,5,6,7,50000})
yields 50000 .
list = listop.filter(predicate, list)
Returns a list containing only those elements for which the predicate
returns true. The predicate has to be a function, which takes an
element of the list as argument and the result of which
is interpreted as boolean value. If it returns true (or rather
anything besides false and nil )
the argument is appended to the return value of filter .
For example: listop.filter(isnumber,{1,2,3,"foo",4,"bar"}) returns
{1,2,3,4} .
list = listop.flatten(list)
Since a list can itself contain lists as elements,
flatten returns a list which
only contains values that are not themselves
lists. For example:
listop.flatten({1,2,3,"foo",{4,5,{"bar"}}}) returns
{1,2,3,"foo",4,5,"bar"} .
list = listop.append(list1, list2)
Returns a list containing all elements of list1 appended by all
elements of list2 .
list = listop.cons(value1, value2)
Returns a list containing value1 appended by value2 , which may be
of any type.
list = listop.reverse(list)
Returns a list containing all elements of the given list in inverted
order.
value = listop.car(list)
Returns the first element of the given list.
value = listop.ncar(list,n)
Returns the nth (or first if n is omitted) element of the given list.
value = listop.cdr(list)
Returns a list containing all elements but the first of the
given list.
value = listop.ncdr(list, n)
Returns a list containing all elements but the first n of the
given list, where n is 2 if it is omitted.
Lua's string operations are very flexible and offer an easy-to-use way
to manipulate strings. Concatenation using the ..
operator is such an operation. The drawback of the built-in API however is the way it handles
concatenation of many string values. Since strings in Lua are
immutable values, each time you concatenate two strings both get copied
into the result string. The strbuf module offers a
workaround for this problem, while maintaining the nice syntax. This
is accomplished by overloading the concatenation operator (.. ) the equality operator (== ) and the
tostring operator. By overloading
these operators, we reduce the overhead of using a string buffer instead
of a plain string to wrap the first literal string assigned to a
variable inside a strbuf.new() call. Afterwards you can append to the string buffer, or compare
two string buffers for equality just as you would do with normal strings.
When looking at the details there are some more restrictions/oddities:
The concatenation operator requires its left-hand value to be a
string buffer. Therefore, if you want to prepend a string to a given
string buffer you have to create a new string buffer out of the string
you want to prepend.
The string buffer's tostring operator concatenates the
strings inside the buffer using newlines by default, since this appears to
be the separator used most often.
buffer = strbuf.new("first-string")
Creates a new string buffer. The argument is optional and is the
first string to be added to the buffer.
buffer = strbuf.concat(strbuf1, value)
Concatenates the value (which has to be either
a string or a string buffer) to strbuf1 . This
is also the function serving as the string buffer's concatenation operator.
The above function call can thus also be expressed as:
buffer = strbuf1 .. value
bool = strbuf.eqbuf(strbuf1, strbuf2)
Compares strbuf1 and strbuf2
for equality. For the function to return true , both values must be
string buffers containing exactly the same strings. The eqbuf function is called to compare two strings for equality.
strbuf.clear(strbuf)
Deletes all strings in strbuf .
string = strbuf.dump(strbuf, "delimiter")
Dumps strbuf 's contents as string. The second
parameter is used as a delimiter between the strings stored inside
strbuf . dump(strbuf, "\n") is
used as the tostring function of string buffers.
URL Manipulation FunctionsURL manipulation functions have obvious uses. Fortunately
there is already an implementation of URL generation functions
inside the Lua-socket package, which is fairly complete and
well
documented For NSE, Lua-socket's URL module was
extended with two functions: table = url.parse_query("query-string")
This function takes a query-string of the form name1=value1&name2=value2... and returns a table
containing the name-value pairs, with the name
as the key and the value as its associated value.
The table corresponding to the above query-string would have two
entries: table["name1"]="value1" and
table["name2"]="value2" . query_string = url.build_query(table)
This is the inverse function to parse_query() .
Buffered Network I/O Helper Functions
The match module was written to provide
functions which can be used for delimiting data received by the
receive_buf() function from the Network I/O API:
start,end = match.regex("regexpattern")
This is actually a wrapper around NSE' PCRE library exec function (see the section called “Perl Compatible Regular Expressions”, thus
giving script developers the possibility to use regular expressions
for delimiting instead of Lua's string patterns. If you want to get
the data in chunks separated by regex (which has to be a valid
regular expression), you would write status, val =
sockobj:receive_buf(match.lua("regex")) .
start,end = match.numbytes(number)
Takes a number as argument and returns that
many bytes. It can be used to get a buffered
version of
sockobj:receive_bytes(n) in
case a script requires more than one
fixed-size chunk, as the unbuffered version
may return more bytes than requested and thus
would require you to do the parsing on your
own.
The http module provides functions for dealing with the client side of the http protocol.
The functions reside inside the http namespace.
The return value of each function in this module is a table with the following keys:
status , header and body .
status is the status code of the http request
In case of an error status is nil . header
is a table with the headers received from the server. The header names are
lower-cased and multiple headers of the same name are concatenated with comma.
body holds a string with the request body.
table = http.get(host,port,path,[options])
Fetches a resource with a GET request.
The first argument is either a string with the hostname or a
table like the host table passed by nmap. The second argument
is either the port number or a table like the port table passed
by nmap. The third argument is the path of the resource. The fourth
argument is a table for further options. The table may have 2 keys:
timeout and header .
timeout is the timeout used for the socket
operations. header is a table with additional
headers to be used for the request.
The function builds the request and calls http.request
table = http.request(host,port,request,[options])
Sends request to host :port
and parses the answer.
The first argument is either a string with the hostname or a
table like the host table passed by nmap. The second argument
is either the port number or a table like the port table passed
by nmap. SSL is used for the request if either port.service
equals "https" or port.version.service_tunnel
equals "ssl" . The third argument is the request. The fourth
argument is a table for further options. You can specify a timeout
for the socket operations with the timeout key.
table = http.get_url(url,[options])
Parses url and calls http.get
with the result.
The second argument is a table for further options. The table may have 2 keys:
timeout and header .
timeout is the timeout used for the socket
operations. header is a table with additional
headers to be used for the request.
Data File Parsing Functions
The datafiles module provides functions for reading and parsing
Nmap's data files (e.g. nmap-protocol , nmap-rpc ,
etc.). These functions' return values are setup for use with exception handling via
nmap.new_try() .
bool, table = datafiles.parse_protocols()
This function reads and parses Nmap's nmap-protocols
file. bool is a boolen value indicating success.
If bool is true, then the second returned
value is a table with protocol numbers indexing the protocol
names. If bool is false, an error message
is returned as the second value instead of the table.
bool, table = datafiles.parse_rpc()
This function reads and parses Nmap's nmap-rpc
file. bool is a boolen value indicating success.
If bool is true, then the second returned
value is a table with RPC numbers indexing the RPC names. If
bool is false, an error message is returned
as the second value instead of the table.
bool, table = datafiles.parse_services([protocol])
This function reads and parses Nmap's nmap-services
file. bool is a boolen value indicating success.
If bool is true, then the second returned
value is a table containing two other tables:
tcp{} and udp{} .
tcp{} contains services indexed by TCP port
numbers. udp{} is the same, but for UDP.
You can pass "tcp" or "udp" as an argument to
parse_services() to only get the corresponding
table. If bool is false, an error message is
returned as the second value instead of the table.
Various Utility Functions
The stdnse library contains various handy
functions which are too small to justify modules of their own:
stdnse.print_debug(...)
Wrapper function around print_debug_unformatted()
in the nmap namespace. The first argument, if numeric, is
used as the necessary debug level to print the message (it defaults
to 1 if omitted). All remaining arguments are processed with
Lua's string.format() function, which provides a
C-style printf interface.
table = stdnse.strsplit("delimiter","text")
This function will certainly be appreciated by Perl programmers.
It takes two strings as arguments and splits the second one around
all occurrences of the first one, returning a table, which contains
the substrings without the delimiting string.
string = stdnse.strjoin("delimiter", table)
Inverse function to strsplit() . Basically this is
Lua's table.concat() function with the parameters
swapped for coherence.
|
|