Abstract
This chapter explores system
services under DOS, Windows 3.x, Windows 95/98,
and Windows NT. The authors discuss the need for
hooking these system services.
THIS CHAPTER DISCUSSES hooking
Windows NT system services. Before we begin, let’s
first review what we mean by a system
service. A system service refers to a set of
functions (primitive or elaborate) provided by the
operating system. Application programming
interfaces (APIs) enable developers to call
several system services, directly or indirectly.
The operating system provides APIs in the form of
a dynamic link library (DLL) or a static compiler
library. These APIs are often based on system
services provided by the operating system. Some of
the API calls are directly based on a
corresponding system service, and some depend on
making multiple system service calls. Also, some
of the API calls may not make any calls to system
services. In short, you do not need a one-to-one
mapping between API functions and system services.
Figure
6-1 demonstrates this in context of Windows
NT.
SYSTEM SERVICES: THE LONG
VIEW
System services and the APIs
calling these system services have come a long way
from DOS to Windows NT.
System Services under
DOS
Under DOS, system services comprise
part of the MS-DOS kernel (including MSDOS.SYS and
IO.SYS). These system services are available to
users in the form of Interrupt Service Routines
(ISRs). ISRs can be invoked by calling the
appropriate interrupt handlers using the INT
instruction. API functions, provided by compiler
libraries, call the interrupt handler for system
services (the INT 21h interrupt). For example, to
open a file, MS-DOS provides a system service for
which you have to specify the function number 0x3D
in the AH register, attribute mask in the CL
register, filename in the DS:DX register, as well
as issue the INT 21h instruction. Compilers
typically provide wrappers around this and provide
a nice API function for this purpose.
System Services under Windows
3.x and Windows 95/98
Under Windows
3.x or Windows 95/98, the core system
services take the form of VXDs and DLLs and some
real-mode DOS code. The APIs are provided in the
form of dynamic link libraries. These dynamic link
libraries call the system services to implement
the APIs. For example, to open a file,
applications call an API function from
KERNEL32.DLL such as OpenFile() or CreateFile().
These APIs, in turn, call a system
service.
System
Services under Windows NT
Under Windows
NT, the NT executive (part of NTOSKRNL.EXE)
provides core system services. These services are
rather generic and primitive. Various APIs such as
Win32, OS/2, and POSIX are provided in the form of
DLLs. These APIs, in turn, call services provided
by the NT executive. The name of the API function
to call differs for users calling from different
subsystems even though the same system service is
invoked. For example, to open a file from the
Win32 API, applications call CreateFile() and to
open a file from the POSIX API, applications call
the open() function. Both of these applications
ultimately call the NtCreateFile() system service
from the NT executive.
| Note: Under Windows NT
3.51, the system services are provided by a
kernel-mode component called NTOSKRNL.EXE. Most
of the KERNEL32.DLL calls—such as those related
to memory management and kernel objects
management—are handled by these system services.
The USER32 and GDI32 calls are handled by a
separate subsystem process called CSRSS.
Starting with Windows NT 4.0, Microsoft moved
most of the functionality of CSRSS into a
kernel-mode driver called WIN32K.SYS. The
functionality moved into WIN32K.SYS is made
available to the applications in the form of
system services. These system services are not
truly part of native system services since they
are specific to the user interface and not used
by all subsystems. This chapter and the next
chapter focus only on the system services
provided by
NTOSKRNL.EXE. |
NEED FOR HOOKING SYSTEM
SERVICES
Hooking represents a very
common mechanism of intercepting a particular
section of executing code. Hooking provides a
useful way of modifying the behavior of the
operating system. Hooking can help the developer
in several ways. Often developers are concerned
more with how to hook a system service or an API
call rather than why to hook. Nevertheless, we
examine the various possible situations in which
the need to hook a system service arises. How
hooking can help the developer is explained in the
following sections.
Trapping Events at
Occurrence
Developers trap events such
as the creation of a file (CreateFile()), creation
of a mutex (CreateMutex()), or Registry accesses
(RegCreateKey()) for specific purposes. Hooking a
particular event-related API or system service
call, synchronously, can help trap those events.
Applications doing system monitoring will find
these kinds of hooking invaluable. These hooks
could act as interrupts triggered by the
occurrence of these events. A developer could
write a routine to handle the occurrence of these
events and take appropriate action.
Modifying System Behavior to
Suit User Needs
Diverting the normal
flow of control by introducing the hooks can
modify operating system behavior. This enables the
developer to change data structures and context at
the time of hooking–enough to induce new behavior.
For example, you can protect the opening of a
sensitive file by hooking the NtCreateFile()
system service. Although NTFS provides user-level
security for files, this security is not available
on FAT partitions. You should ensure that hooking
does not have any undesirable side effects on the
operating system. Protecting modifications to
Registry keys is something easily doable when you
hook the Registry system services. This has
several applications, since little protection is
provided for Registry settings created by
applications.
Studying the Behavior of the
System
In order to get a better idea of
the internal workings of the operating system,
studying the behavior of the system is something
most debuggers or system hackers will relate to.
Understanding of undocumented operating system
functionality requires a lot of hacking, which
goes hand in hand with hooking.
Debugging
Complex
programs could make use of system-service hooking
to debug the stickiest problems. For example, a
few days back, we had a problem with the
installation of a piece of software. We had
difficulty creating folders and shortcuts for this
application. Using a systemwide hook, we quickly
figured that the installation program was looking
for a Registry value that indicated where to
install the folders (which happened to be the
Start menu). We hooked the NtQueryValueKey() call,
then obtained the value the installation program
was looking for. We created that value and solved
our problem.
Getting Performance Data for
Specific Tasks and Generating
Statistics
These tasks can prove very
useful to those writing benchmarks and
applications to critically measure system
performance under specific conditions. Even
measuring the frequency of certain system services
becomes very easy with this type of hooking.
Measuring file system performance by hooking the
file system-related system services exemplify this
procedure.
Life without hooking is
unthinkable for most Windows developers in today’s
Microsoft-dominated world of operating systems.
Windows NT system services lie at the center of
the NT universe, and having the ability to hook
these can prove extremely handy.
TYPES OF
HOOKS
The following sections explore
two types of hooking.
Kernel-Level
Hooking
You can achieve kernel-level
hooking by writing a VXD or device driver. In this
method, essential functions provided by the kernel
are hooked. The advantage of this type of hooking
is that you get one central place from which you
can monitor the events occurring as a result of a
user-mode call or a kernel-mode
call. The disadvantage of this method is that you
need to decipher the parameters of the call passed
from kernel mode, since many times these services
are undocumented. Also, the data passed to the
kernel-mode call might differ from the data passed
in a user-mode call. Also, a user-level API call
might be implemented using multiple calls to the
kernel. In this case, hooking becomes far more
difficult. In general, this type of hooking is
more difficult to achieve, but it can produce more
rewarding results.
User-Level Hooking
You
can perform this type of hooking with some help
from a VXD or device driver. In this method, the
functions provided by the user-mode DLLs are
hooked. The advantage of this method is that these
functions are usually well documented. Therefore,
you know the parameters to expect. This makes it
easy to write the hook function. This type of
hooking limits your field of vision to user mode
only and does not extend to kernel
mode.
IMPLEMENTATIONS OF
HOOKS
The following sections detail
the implementation of hooks under various
Microsoft platforms.
DOS
In the DOS world,
system services are implemented as an interrupt
handler routine (INT 21h). The compiler library
routines typically call this interrupt handler to
provide an API function to the programmer. It is
trivial to hook this handler using the GetVect
(INT 21h, AX=25h) and SetVect (Int 21h, AX=35h)
services. Hence, hooking system services are
fairly straightforward. DOS does not contain
separate user and kernel modes.
Windows 3.x
In the
Windows 3.x world, system services are
implemented in DLLs. The compiler library routines
represent stubs that jump to the DLL code (this is
called dynamic linking of DLLs). Also, because the
address space is common to all applications,
hooking amounts to getting the address of that
particular system service and changing a few bytes
at that address. Changing of these bytes sometimes
requires the simple aliasing of
selectors.
| XREF: Refer to the MSDN
article in Microsoft Systems Journal
(Vol. 9, No. 1) entitled, “Hook and Monitor Any
16-bit Windows(tm) Function With Our ProcHook
DLL,” by James
Finnegan. |
Windows
95 and 98
In the Windows 95/98 world,
system services are implemented in a DLL as in
Windows 3.1. However, under Windows 95/98, all
32-bit applications run in separate address
spaces. Because of this, you cannot easily hook
any unshared DLL. It is fairly easy to hook a
shared DLL such as KERNEL32.DLL. You simply modify
a few code bytes at the start of the system
service you want to hook and write your hook
function in a DLL that is loaded in shared memory.
Modifying the code bytes may involve writing a
VXD, because KERNEL32.DLL is loaded in the upper
2GB of the address space and protected by the
operating system.
Windows NT
In the
Windows NT world, system services are implemented
in the kernel component of NT (NTOSKRNL.EXE). The
APIs supported by various subsystems (Win32, OS/2,
and POSIX) are implemented by using these system
services. There is no documented way of hooking
these system services from kernel mode. There
are several documented ways for hooking user-level
API calls.
XREF: Refer to the MSDN
articles in Microsoft Systems Journal
entitled, “Learn System-Level Win32(r) Coding
Techniques by Writing and API Spy Program,” by
Matt Pietrek (Vol.9, No.12), and “Load Your
32-bit DLL into Another Process’s Address Space
Using INJLIB,” by Jeffrey Richter (Vol.9,
No.5).
Refer to CyberSensor on
http://www.cybermedia.co.in |
We will present one way of
achieving hooking of NT system services in kernel
mode in this chapter. We also provide the code for
this on the CD-ROM accompanying this
book.
|
Network Home 
.gif){kind=tracking}
.gif){kind=tracking}
•