Skip to main content

Command Palette

Search for a command to run...

Introduction To The Windows API

Updated
12 min readView as Markdown
Introduction To The Windows API
F

Hey, I'm a Full Stack Web Developer. My weapon of choice is React with Tailwind, Express and PostgreSQL or MongoDB for the back end. I do a lot of Cyber Security, particularly Red Teaming. I like to probe web apps and networks for vulnerabilities. I also enjoy Malware Dev and Revers Engineering. I am constantly learning new concepts and technologies. So if you're a nerd like me, head over to my blog where you will find posts on my latest adventures.

The Windows API can be intimidating at first. With it’s weird naming conventions and data types it can be hard to work with. In this blog post, I will cover everything you need to know to get up and running with it.

Note that all the code samples will be in C, this is because it makes it easier for us to work with the Windows API and not have to worry much about the syntax. With that being said, the API can be used in any language so you should be able to follow along just fine.

What is the Windows API

An API (Application Programming Interface) is a way for two pieces of software to communicate with each other, in our case we want our code to talk with the Windows OS.

The Windows API or Win32 API acts as an intermediary by providing functions that we can call to interact with the OS. This makes it easier because developers don’t have to understand how Windows works under the hood to be able to interact with it.

Taking a look at the Windows API Documentation we can see that the functions are categorized by functionality making it simple to find the function you are looking for. The Win API is very well documented and has many code examples. It is the number one resource if you don’t understand anything.

But before we can start calling functions, there are a few concepts we have to understand so let’s take a look at them.

Data Types

One of the first things you will realize when you take a look at the documentation for any function is the unusual data types they take as arguments.

Let's take the CreateProcessA function for example. As it’s name suggests it is used to create a new process.

BOOL CreateProcessA(
  [in, optional]      LPCSTR                lpApplicationName,
  [in, out, optional] LPSTR                 lpCommandLine,
  [in, optional]      LPSECURITY_ATTRIBUTES lpProcessAttributes,
  [in, optional]      LPSECURITY_ATTRIBUTES lpThreadAttributes,
  [in]                BOOL                  bInheritHandles,
  [in]                DWORD                 dwCreationFlags,
  [in, optional]      LPVOID                lpEnvironment,
  [in, optional]      LPCSTR                lpCurrentDirectory,
  [in]                LPSTARTUPINFOA        lpStartupInfo,
  [out]               LPPROCESS_INFORMATION lpProcessInformation
);

We can see that it takes in arguments of types LPCSTR LPSTR DWORD LPVOID and some other weird ones. And what’s up with the names of the arguments? lpApplicationName bInheritHandles dwCreationFlags

The Win32 API uses the Hungarian notation which is just a naming convention in which the name of a variable is preceded by a letter to indicate it’s kind, or in our case it’s type. You might name a variable bToggle to indicate that it is of type bool or pAge to show that it’s a pointer. You will see this everywhere in the docs and you will get used to it very quickly.

Some of the confusion is also a result of Windows maintaining backwards compatibility for the same API over such a long time. For example the ‘L’ you see on most of the variables stands for ‘long’ which is derived from the 16-bit Windows programming period but is no longer required. But the naming convention still exists.

This clarifies the confusion on the variable names but what about the unusual types?

Well they are just typedefs of data types you are already familiar with. They have just been given new fancy names to make reading the documentation easier. You can find a list of all the types documented Here

Here is a list of some common ones

Data Type Description
DWORD A 32-bit unsigned integer. The range is 0 through 4294967295 decimal.
This type is declared in IntSafe.h as follows:
typedef unsigned long DWORD;
SIZE_T Used to represent the size of an object. It's a 32-bit unsigned integer on 32-bit systems and 64-bit on 64-bit systems This type is declared in BaseTsd.h as follows:
typedef ULONG_PTR SIZE_T;
PVOID A pointer to any type.
This type is declared in WinNT.h as follows:
typedef void *PVOID;
HANDEL A handle to an object.
This type is declared in WinNT.h as follows:
typedef PVOID HANDLE;
HMODULE A pointer to a null-terminated string of 8-bit Windows (ANSI) characters. As stated earlier, the “l” which stands for long does not affect the data type anymore and this these types are effectively the same.
This type is declared in WinNT.h as follows:
typedef CHAR *PSTR;
LPCSTR\PCSTR A pointer to a null-terminated string of 8-bit Windows (ANSI) characters. As stated earlier, the “L” which stands for long does not affect the data type anymore and this these types are effectively the same.
This type is declared in WinNT.h as follows:
typedef CHAR *PSTR;
LPCWSTR\PCWSTR A pointer to a constant null-terminated string of 16-bit Unicode characters. Same thing here, ‘L’ doesn’t make a difference.
This type is declared in WinNT.h as follows:
typedef CONST WCHAR *LPCWSTR;

Pointers to data types

Some functions expect the actual datatype to be passed as an argument while others expect you to pass a pointer to the data type. Arguments that expect a pointer will be preceded by a ‘P’ to indicate it’s a pointer.

Here are some examples

PDWORD is same as DWORD*

PWCHAR is same as WCHAR*

PHANDEL is same as HANDEL*

ANSI, Unicode and Extended Functions

You might have noticed that the CreateProcessA function we looked at earlier had an “A” at the end. This means that it’s an ANSI function.

When you look through the Win API docs you will notice that there are variants of the same function. Most functions have have a generic variant, an ANSI variant which ends with an “A” and a Unicode or Wide variant which ends with a “W”.

The main difference between the ANSI and Unicode functions is the data type they take as parameters. ANSI functions will take ANSI data types and Unicode will take Unicode data types where applicable. The generic functions are compatible with both.

For example, there is a Unicode version of the CreateProcess function CreateProcessW. This function takes a LPCWSTR (pointer to wchar) for the lpApplicationName instead of LPCSTR (pointer to char).

Here’s how these would be declared in C.

ANSI:

LPCSTR = pApplicationName[] = "c:\\program.exe"; // LPCSTR provided by win.h
char *pApplicationName[] = L"c:\\program.exe"; //Same thing

Unicode:

LPCWSTR pApplicationName[] = L"c:\\program.exe"; // LPCWSTR provided by win.h
wchar *pApplicationName[] = L"c:\\program.exe"; //Same thing

You can read more on ANSI and Unicode and what their differences are Here.

Extended Functions

Another type of function you will find is an extended function. This is basically a variant of the function with more options and parameters. These functions end with “Ex”.

Calling Win32 API Functions

Now that we have gone through all the common causes of confusion, let’s see how the functions are actually called.

In C we gain access to the Win API functions by including windows.h . The first function we will be calling is the MessageBox function. This function displays a simple modal dialog box that contains a system icon, a set of buttons, and a brief application-specific message. Consider it as the Hello World of the Windows API

Here is the function signature from the documentation:

int MessageBox(
  [in, optional] HWND    hWnd,
  [in, optional] LPCTSTR lpText,
  [in, optional] LPCTSTR lpCaption,
  [in]           UINT    uType
);

Let's break down the parameters:

  • hWnd - A handle to the owner window of the message box. If this parameter is NULL, the message box has no owner window.

  • lpText - The message to be displayed in the message box.

  • lpCaption - The dialog box title.

  • uType - The contents and behavior of the dialog box. This can be combinations of flags like MB_OK, MB_ICONINFORMATION, etc. A complete list and description of all flags can be found in the documentation. The MB_OK flag adds an OK button, and MB_ICONINFORMATION adds an information icon.

Here's a simple example of calling this function:

#include <windows.h>

int main() {
    MessageBox(
        NULL,
        L"Hello from the Windows API!",
        L"My First Win32 Program",
        MB_OK | MB_ICONINFORMATION
    );
    
    return 0;
}

And there you have it, your first program using the Windows API. See! It wasn’t that hard.

More Complex Example

Okay let’s try that again but this time with a slightly more complex example using the CreateProcessA function we discussed earlier. This function creates a new process that runs independently of the creating process.

Here is the function signature.

BOOL CreateProcessA(
  [in, optional]      LPCSTR                lpApplicationName,
  [in, out, optional] LPSTR                 lpCommandLine,
  [in, optional]      LPSECURITY_ATTRIBUTES lpProcessAttributes,
  [in, optional]      LPSECURITY_ATTRIBUTES lpThreadAttributes,
  [in]                BOOL                  bInheritHandles,
  [in]                DWORD                 dwCreationFlags,
  [in, optional]      LPVOID                lpEnvironment,
  [in, optional]      LPCSTR                lpCurrentDirectory,
  [in]                LPSTARTUPINFOA        lpStartupInfo,
  [out]               LPPROCESS_INFORMATION lpProcessInformation
);

A detailed explanation of the function parameters can be found in the documentation.

#include <windows.h>
#include <stdio.h>

int main() {
    STARTUPINFO si;
    PROCESS_INFORMATION pi;
    
    // Zero out the structures
    ZeroMemory(&si, sizeof(si));
    si.cb = sizeof(si);
    ZeroMemory(&pi, sizeof(pi));
    
    // Create the process
    if (!CreateProcessA(
        "C:\\\\Windows\\\\System32\\\\notepad.exe",  // Application name
        NULL,                                    // Command line
        NULL,                                    // Process security attributes
        NULL,                                    // Thread security attributes
        FALSE,                                   // Inherit handles
        0,                                       // Creation flags
        NULL,                                    // Environment
        NULL,                                    // Current directory
        &si,                                     // Startup info
        &pi                                      // Process information
    )) {
        printf("CreateProcess failed (%d).\\n", GetLastError());
        return 1;
    }
    
    printf("Process created successfully!\\n");
    printf("Process ID: %d\\n", pi.dwProcessId);
    
    // Wait for the process to complete
    WaitForSingleObject(pi.hProcess, INFINITE);
    
    // Don't forget to close process and thread handles
    CloseHandle(pi.hProcess);
    CloseHandle(pi.hThread);
    
    return 0;
}

This program creates a new instance of Notepad. Notice how we:

  • Initialize the STARTUPINFO and PROCESS_INFORMATION structures

  • Call CreateProcessA with the appropriate parameters

  • Check if the function succeeded

  • Clean up by closing the handles with CloseHandle

A Note on Handles

One term you will see a lot in Windows API documentation is Handle. As stated earlier a handle is really just a type definition for a pointer.

typedef void* HANDLE;

Conceptually a handle is a reference to a resource managed by the operating system, such as a file, process, window, or memory block. Think of a handle as an ID card that gives you access to a specific resource. When you create or open a resource, Windows gives you a handle to it. You then use this handle in subsequent API calls to work with that resource (Just like how you would use a pointer).

So why not just call it a pointer? Well using a handle allows Microsoft to encapsulate and abstract away implementation details. And that means the can change the underlying implementation without breaking existing code. Handles also sometime store metadata and properties such as whether the handle can be inherited by a child process.

And because handles are just pointers, it means you must treat them like one

  • Always close handles: When you're done with a resource, close its handle using the appropriate function (usually CloseHandle()). Failing to do so causes resource leaks.

  • Check for validity: Always check if a handle is valid after creation. Invalid handles are typically NULL or INVALID_HANDLE_VALUE.

  • Don't share handles carelessly: Handles are specific to the process that created them, though some can be inherited by child processes.

Error Handling

Proper error handling is crucial when working with the Windows API. Most functions return a value indicating success or failure. Functions returning BOOL return TRUE (non-zero) on success and FALSE (0) on failure. Functions returning HANDLE return a valid handle on success and NULL or INVALID_HANDLE_VALUE on failure.

These errors are often non-verbose. For example, if CreateFileW fails it returns INVALID_HANDLE_VALUE which indicates that a file could not be created. To gain more insight as to why the file couldn't be created, the error code must be retrieved using the GetLastError function. Once the code is retrieved, it needs to be looked up in Windows's System Error Codes List.

Here's an example:

HANDLE hFile = CreateFileA(
    "nonexistent.txt",
    GENERIC_READ,
    0,
    NULL,
    OPEN_EXISTING,
    FILE_ATTRIBUTE_NORMAL,
    NULL
);

if (hFile == INVALID_HANDLE_VALUE) {
    DWORD error = GetLastError();
    printf("Failed to open file. Error code: %d\\n", error);
    return 1;
}

// Use the file handle...
CloseHandle(hFile);

You can also use FormatMessage() to convert error codes into human-readable strings.

More Windows API Functions

The only way to really get comfortable with the Windows API is to keep practicing with different functions. Here are a list of some cool functions for you to play with.

  • SetCursorPos / GetCursorPos : Functions to get and set the cursor position. You could easily make a game with this one.

  • SetWindowPos / MoveWindow : Functions to change the position and dimensions of the specified window. Including moving it behind other windows. Imagine making your windows shake or move around frantically. That would make for a nasty prank.

  • ShellExecute : Opens files, documents, or URLs using the appropriate application, useful for programmatically launching other things

  • Playsound: plays a sound specified by the given file name, resource, or system event.

Conclusion

The Windows API is a powerful tool for interacting with the Windows operating system. While it may seem daunting at first, once you understand the fundamentals covered in this post, you'll find it becomes much more approachable.

Thank you for reading.

Stay s3cur3 lads 😎