Nmap Security Scanner
Ref Guide
Security Lists
Security Tools
Site News
Advertising
About/Contact
Credits
Sponsors
|

 |
Intro |
Reference Guide |
Book |
Install Guide |
Download |
Changelog |
Zenmap GUI |
Docs |
Bug Reports |
OS Detection |
Propaganda |
Related Projects |
In the Movies |
In the News |
|
 |
Now how does all this work? The following section describes
some interesting aspects of the NSE. While the focus primarily lies on
giving script writers a better feeling of what happens with scripts, it
should also provide a starting point for understanding (and extending) the
NSE sources.
During its initialization stage, Nmap loads the Lua interpreter and its provided
libraries get loaded. These libraries are: The package library (namespace:
package )—Lua's
package-lib provides (among others) the require function, used to load modules from the
nselib.
The table library (namespace:
table )—The
table manipulation library contains many functions used
to operate on tables —Lua's central data
structure.
The I/O library (namespace:
io )—The
Input/Output library offers functions such as reading files and reading the output from programs you execute.
The OS library (namespace:
os )—The
Operating System lib provides facilities of the operating
system, including filesystem operations (renaming/removing files,
creating of temporary filenames) and access to the environment.
The string library (namespace:
string )—The
string library helps you with functions used to manipulate
strings inside Lua. Functions include: printf-style
string formating, pattern matching using Lua-style patterns,
substring extraction, etc.
The math library (namespace:
math )—Since usually numbers in Lua correspond
to the double C-type, the
math library gives you access to rounding functions,
trigonometric functions, random number generation, and many more.
The debug library (namespace:
debug )—The
debug library provides you with a somewhat lower level API
to the Lua-interpreter. Through it you can access functions along
the execution stack, get function closures and object metatables,
etc.
In addition to loading the libraries provided with Lua, the functions
in the nmap namespace also get loaded. and search
path for modules is set to the default one prepended by the nselib
directory (which is searched in the locations Nmap searches for its
data files and scripts). In this step the provided script arguments
also get stored inside the registry .
The next phase of NSE initialization is loading the chosen
scripts, which are the arguments provided to the
--script option or safe,intrusive , in
case of a default script scan. The string version
is appended, if version detection was enabled.
The arguments afterwards are tried to be
interpreted as script categories. This is done via a short Lua function
hard-coded into nse_init.cc called Entry . If you take a look into the script.db you'll see that the Entry lines inside
it are Lua function calls with a table as argument.
The arguments that didn't produce any filenames are then interpreted
as file or directory names themselves. If this also fails, the script scan is aborted.
In the next stage the found files are loaded as chunks, each with
its own environment, having read but not write access to the global
name space and saved inside two globally accessible Lua tables:
hosttests and porttests
depending on the type of script. Because scripts only get loaded once, values stored inside variables during a script's execution against one host or port can be accessed when the same script runs against another target. This can be used to save computation time when a script is run
against multiple targets. See Example 9.3, “Using local variables to save data.”
During this stage scripts are
also are also provided with a default runlevel (1.0), if they
don't specify one themselves and a check is performed whether they
contain an action and a description field.
Example 9.3. Using local variables to save data. id="persistent locals example"
description="This sample script shows how data can be stored across \
several invocations of a script against multiple targets"
author="Stoiko Ivanov"
categories = {"safe"}
require "shortport"
portrule = shortport.portnumber(80)
-- we have to declare the variable in the script's global scope
-- because if we declare it inside the action it would get redefined
-- with each call to the action
local filecontent = nil
require "strbuf"
action= function(host, port)
if(filecontent == nil) then
filecontent = strbuf.new()
for line in io.lines("a_filename_we_want_to_read_from")
filecontent = filecontent .. line
end
end
--rest of the script doing something with the filecontent, we just
--read
end
Matching of Scripts to Targets
After the initialization is finished the hostrules and
portrules are evaluated for each host in the current
target group. At this check a list is built which contains the combinations of scripts and the hosts they will run against.
It should be noted that the rules of all chosen scripts are
checked against all hosts and their open and open|filtered ports.
Therefore it is advisable to leave the rules as simple as possible and
to do all the computation inside the action , as a script will only be
executed if it is run against a specific target. After the check those script-target combinations get their own Lua-thread which is anchored in Lua's C-API registry to prevent their garbage collection. These thread_records are afterwards sorted by run level and all script-target combinations of one run level are stored in a list, in order to ensure that scripts with a higher run level are run after those with a lower one.
Now to the actual script scanning, and the way NSE accomplishes
parallelization. Lua, through its concept of
coroutines
offers collaborative multi-threading, which means that scripts
can suspend themselves, at defined points, and let other coroutines
execute. Since network I/O, especially waiting for responses from
remote host, is the part of scripts which would consume most time with
waiting, this is the point where scripts suspend themselves and let
others execute. Each call to some of the functions of the Nsock wrapper
causes the calling script to yield (pause). Once the request is processed by the Nsock library, the
callback causes the script to be pushed from the waiting queue to the
running queue, which will eventually let it resume its operation.
Adding C Modules to Nselib
This section tries to give a short walk-through to adding
nselib modules written in C (or C++) to Nmap's build system, since
this has shown to be sometimes tedious. Writing C modules is
described at length in Programming
in Lua, Second Edition Basically C modules consist of the
functions they provide to Lua, which have to be of type lua_CFunction Additionally they have to contain a function
which is used to actually open the module. By convention these function names are luaopen_modulename .
A good starting point for writing such modules is provided with
bit.c and pcre.c inside
the nselib/ subdirectory of Nmap's source tree,
which are two C modules already provided by the nselib. C modules
basically are shared libraries which get loaded at runtime by Lua.
The Unix build system uses libtool for
compilation in a platform independent way.
As long as the new module
does not depend on foreign libraries, you should only need to add
modulename .so to the
all and clean targets in
Makefile.in
and copy and adapt the lines from bit.so .
If your module does have dependencies you will most probably have to
add checks for those libraries to configure.ac
and put the dependencies inside the libtool
commands in Makefile.in —here you should
take a look at how pcre.so handles this.
So much for the way it should work. Now for some pitfalls we've
come across so far: These are results from building shared libraries in
general and not really specific to nselib. Linking with
static libraries (e.g. libnbase ) sometimes leads
to problems with exporting symbols on some platforms (in our case
this was a x86_64-linux platform). To our knowledge no such
problems occur when linking against already existing shared
libraries.
The Windows build system requires C module developers to create a
MS Visual Studio Project file for their module
(<modulename>.vcproj ) inside the
nselib subdirectory. On Windows you have to
include the liblua/ subdirectory as
an additional include path as well as a library search path. In addition
you have to tell the project to link against the
liblua.lib static library provided with Nmap.
Other properties of the project should be the same as for other
nselib C modules (e.g. see nse_bitlib.vcproj ).
Afterwards you have to include the newly created project file in
Nmap's Visual Studio solution file
(mswin32\nmap.sln ) and make sure that
nse_bitlib.vcproj depends on your project,
because it is there nselib-modules get copied to their final destinations and are included in Nmap.
|
|