MESSAGE_TRACE_HEADER

The MESSAGE_TRACE_HEADER is one of several types of fixed-size header that introduce variable-size data for events that are logged through Event Tracing for Windows (ETW). As with other types of event, those that begin with a MESSAGE_TRACE_HEADER accumulate first in trace buffers. To have these events persist in this raw form for ready inspection, configure the event tracing session to flush the trace buffers to an Event Trace Log (ETL) file.

Usage

The events that get a MESSAGE_TRACE_HEADER, rather than one of the other types of trace header, are generated through a handful of interfaces that are mostly used for Windows Pre-Processor (WPP) software tracing. The kernel-mode interface is the pair of documented kernel exports WmiTraceMessage and WmiTraceMessageVa. The kernel exposes the functionality to user mode through the undocumented NtTraceEvent function. In user mode, this path to the kernel-mode implementation goes through the the undocumented NTDLL functions EtwTraceMessage and EtwTraceMessageVa, which are in turn accessed as forwards from the documented ADVAPI32 functions TraceMessage and TraceMessageVa. Much of the magic of WPP is that it hides all this behind what looks to the programmer to be a programmer-defined function in the style of printf from the C Run-Time library.

Documentation Status

The MESSAGE_TRACE_HEADER structure is not documented. However, Microsoft has published a C-language definition in the NTWMI.H from the Enterprise edition of the Windows Driver Kit (WDK) for Windows 10 version 1511.

Were it not for this relatively recent and possibly unintended disclosure, much would anyway be known from type information in symbol files. Curiously though, type information for this structure has never appeared in any public symbol files for the kernel or for the obvious low-level user-mode DLLs. In the whole of Microsoft’s packages of public symbol files, relevant type information is unknown before Windows 8 and appears in symbol files only for appxdeploymentclient.dll, certenroll.dll (before Windows 10) and windows.storage.applicationdata.dll.

Layout

The MESSAGE_TRACE_HEADER is 8 bytes in both 32-bit and 64-bit Windows in all known versions.

Offset Definition
0x00
union {
    ULONG Marker;
    struct {
        USHORT Size;
        UCHAR Reserved;
        UCHAR Version;
    };
};
0x04
union {
    ULONG Header;
    WMI_TRACE_MESSAGE_PACKET Packet;
};

What distinguishes the MESSAGE_TRACE_HEADER from other Trace Headers is a particular combination of high bits in the Marker. NTWMI.H defines them as:

Value Name Remarks
0x80000000 TRACE_HEADER_FLAG set in all trace headers
0x40000000 TRACE_HEADER_EVENT_TRACE clear, but set in other trace headers
0x10000000 TRACE_MESSAGE set, but clear in other trace headers

While the WMI_TRACE_MESSAGE_PACKET is not known to have any other use, it is as well presented here:

Offset Definition
0x00
USHORT MessageNumber;
0x02
USHORT OptionFlags;

Both are vital for interpreting the event-specific data that follows the header. The MessageNumber is the primary identifier of what event occurred. It is what the kernel received as the MessageNumber argument via the documented API functions. The OptionFlags tell what data follow the header. There can be any selection of the following items in the following order:

The Size in the low word of the Marker is the total size in bytes of the header and all these items. Note that although the MESSAGE_TRACE_HEADER always has 8-byte alignment in a trace buffer, the 8-byte time stamp need not have (and typically hasn’t, since a sequence number is usual).

The first six OptionFlags come directly from the MessageFlags argument of the relevant API functions and indicate which items are present between the header and the arbitrary data:

Value Name Versions Remarks
0x0001 TRACE_MESSAGE_SEQUENCE 5.1 and higher 32-bit sequence number included from logger
0x0002 TRACE_MESSAGE_GUID 5.1 and higher GUID included from provider via MessageGuid argument
0x0004 TRACE_MESSAGE_COMPONENT_ID 5.1 and higher component ID included from provider via MessageGuid argument
0x0008 TRACE_MESSAGE_TIMESTAMP 5.1 and higher time stamp included from logger
0x0010 TRACE_MESSAGE_PERFORMANCE_TIMESTAMP 5.1 and higher time stamp uses performance counter (but see below)
0x0020 TRACE_MESSAGE_SYSTEMINFO 5.1 and higher thread ID and process ID included
0x0040 TRACE_MESSAGE_POINTER32 6.0 and higher message traced for 32-bit provider
0x0080 TRACE_MESSAGE_POINTER64 6.0 and higher message traced for 64-bit provider

In all versions, TRACE_MESSAGE_COMPONENT_ID has precedence over TRACE_MESSAGE_GUID. The MessageGuid argument can have supplied only one or the other, not both. If both flags are set, what’s present is the component ID, not the GUID. Note that Microsoft somehow documents TRACE_MESSAGE_COMPONENTID with TraceMessage but not with TraceMessageVa, and seems never to have documented it for kernel-mode use.

Interpretation of the time-stamping flags is version-dependent. All versions calculate space for a time stamp if either flag is set, but in no known version does an event actually receive a time stamp unless TRACE_MESSAGE_TIMESTAMP is set. The intention seems to be that TRACE_MESSAGE_TIMESTAMP indicates that a time stamp is present and then TRACE_MESSAGE_PERFORMANCE_TIMESTAMP is a refinement to tell which type, but only in version 5.1 does a set TRACE_MESSAGE_PERFORMANCE_TIMESTAMP indicate certainly that the time stamp is from the performance counter (in contrast to the system time). In later versions, the event’s time stamp comes from whatever clock the logger ordinarily uses and TRACE_MESSAGE_PERFORMANCE_TIMESTAMP is meaningless. Note that Microsoft already had TRACE_MESSAGE_PERFORMANCE_TIMESTAMP documented as obsolete in the Windows XP Device Driver Kit (DDK).

The arbitrary data are copied from the addresses and sizes that are given to the API functions as variable arguments or via the MessageArgList argument. See that no types or sizes are recorded. All data from the argument list is simply aggregated as one blob. Its useful interpretation as placeholders for resolving arguments must be the shared knowledge of provider and consumer, presumably varying according to event-specific identifiers such as the GUID or component ID, and the MessageNumber—and, where pointer and size_t types are expected in the data, on the last two of the OptionFlags.