1===================================== 2The PDB DBI (Debug Info) Stream 3===================================== 4 5.. contents:: 6 :local: 7 8.. _dbi_intro: 9 10Introduction 11============ 12 13The PDB DBI Stream (Index 3) is one of the largest and most important streams 14in a PDB file. It contains information about how the program was compiled, 15(e.g. compilation flags, etc), the compilands (e.g. object files) that 16were used to link together the program, the source files which were used 17to build the program, as well as references to other streams that contain more 18detailed information about each compiland, such as the CodeView symbol records 19contained within each compiland and the source and line information for 20functions and other symbols within each compiland. 21 22 23.. _dbi_header: 24 25Stream Header 26============= 27At offset 0 of the DBI Stream is a header with the following layout: 28 29 30.. code-block:: c++ 31 32 struct DbiStreamHeader { 33 int32_t VersionSignature; 34 uint32_t VersionHeader; 35 uint32_t Age; 36 uint16_t GlobalStreamIndex; 37 uint16_t BuildNumber; 38 uint16_t PublicStreamIndex; 39 uint16_t PdbDllVersion; 40 uint16_t SymRecordStream; 41 uint16_t PdbDllRbld; 42 int32_t ModInfoSize; 43 int32_t SectionContributionSize; 44 int32_t SectionMapSize; 45 int32_t SourceInfoSize; 46 int32_t TypeServerMapSize; 47 uint32_t MFCTypeServerIndex; 48 int32_t OptionalDbgHeaderSize; 49 int32_t ECSubstreamSize; 50 uint16_t Flags; 51 uint16_t Machine; 52 uint32_t Padding; 53 }; 54 55- **VersionSignature** - Unknown meaning. Appears to always be ``-1``. 56 57- **VersionHeader** - A value from the following enum. 58 59.. code-block:: c++ 60 61 enum class DbiStreamVersion : uint32_t { 62 VC41 = 930803, 63 V50 = 19960307, 64 V60 = 19970606, 65 V70 = 19990903, 66 V110 = 20091201 67 }; 68 69Similar to the :doc:`PDB Stream <PdbStream>`, this value always appears to be 70``V70``, and it is not clear what the other values are for. 71 72- **Age** - The number of times the PDB has been written. Equal to the same 73 field from the :ref:`PDB Stream header <pdb_stream_header>`. 74 75- **GlobalStreamIndex** - The index of the :doc:`Global Symbol Stream <GlobalStream>`, 76 which contains CodeView symbol records for all global symbols. Actual records 77 are stored in the symbol record stream, and are referenced from this stream. 78 79- **BuildNumber** - A bitfield containing values representing the major and minor 80 version number of the toolchain (e.g. 12.0 for MSVC 2013) used to build the 81 program, with the following layout: 82 83.. code-block:: c++ 84 85 uint16_t MinorVersion : 8; 86 uint16_t MajorVersion : 7; 87 uint16_t NewVersionFormat : 1; 88 89For the purposes of LLVM, we assume ``NewVersionFormat`` to be always ``true``. 90If it is ``false``, the layout above does not apply and the reader should consult 91the `Microsoft Source Code <https://github.com/Microsoft/microsoft-pdb>`__ for 92further guidance. 93 94- **PublicStreamIndex** - The index of the :doc:`Public Symbol Stream <PublicStream>`, 95 which contains CodeView symbol records for all public symbols. Actual records 96 are stored in the symbol record stream, and are referenced from this stream. 97 98- **PdbDllVersion** - The version number of ``mspdbXXXX.dll`` used to produce this 99 PDB. Note this obviously does not apply for LLVM as LLVM does not use ``mspdb.dll``. 100 101- **SymRecordStream** - The stream containing all CodeView symbol records used 102 by the program. This is used for deduplication, so that many different 103 compilands can refer to the same symbols without having to include the full record 104 content inside of each module stream. 105 106- **PdbDllRbld** - Unknown 107 108- **MFCTypeServerIndex** - The index of the MFC type server in the 109 :ref:`dbi_type_server_map_substream`. 110 111- **Flags** - A bitfield with the following layout, containing various 112 information about how the program was built: 113 114.. code-block:: c++ 115 116 uint16_t WasIncrementallyLinked : 1; 117 uint16_t ArePrivateSymbolsStripped : 1; 118 uint16_t HasConflictingTypes : 1; 119 uint16_t Reserved : 13; 120 121The only one of these that is not self-explanatory is ``HasConflictingTypes``. 122Although undocumented, ``link.exe`` contains a hidden flag ``/DEBUG:CTYPES``. 123If it is passed to ``link.exe``, this field will be set. Otherwise it will 124not be set. It is unclear what this flag does, although it seems to have 125subtle implications on the algorithm used to look up type records. 126 127- **Machine** - A value from the `CV_CPU_TYPE_e <https://msdn.microsoft.com/en-us/library/b2fc64ek.aspx>`__ 128 enumeration. Common values are ``0x8664`` (x86-64) and ``0x14C`` (x86). 129 130Immediately after the fixed-size DBI Stream header are ``7`` variable-length 131`substreams`. The following ``7`` fields of the DBI Stream header specify the 132number of bytes of the corresponding substream. Each substream's contents will 133be described in detail :ref:`below <dbi_substreams>`. The length of the entire 134DBI Stream should equal ``64`` (the length of the header above) plus the value 135of each of the following ``7`` fields. 136 137- **ModInfoSize** - The length of the :ref:`dbi_mod_info_substream`. 138 139- **SectionContributionSize** - The length of the :ref:`dbi_sec_contr_substream`. 140 141- **SectionMapSize** - The length of the :ref:`dbi_section_map_substream`. 142 143- **SourceInfoSize** - The length of the :ref:`dbi_file_info_substream`. 144 145- **TypeServerMapSize** - The length of the :ref:`dbi_type_server_map_substream`. 146 147- **OptionalDbgHeaderSize** - The length of the :ref:`dbi_optional_dbg_stream`. 148 149- **ECSubstreamSize** - The length of the :ref:`dbi_ec_substream`. 150 151.. _dbi_substreams: 152 153Substreams 154========== 155 156.. _dbi_mod_info_substream: 157 158Module Info Substream 159^^^^^^^^^^^^^^^^^^^^^ 160 161Begins at offset ``0`` immediately after the :ref:`header <dbi_header>`. The 162module info substream is an array of variable-length records, each one 163describing a single module (e.g. object file) linked into the program. Each 164record in the array has the format: 165 166.. code-block:: c++ 167 168 struct ModInfo { 169 uint32_t Unused1; 170 struct SectionContribEntry { 171 uint16_t Section; 172 char Padding1[2]; 173 int32_t Offset; 174 int32_t Size; 175 uint32_t Characteristics; 176 uint16_t ModuleIndex; 177 char Padding2[2]; 178 uint32_t DataCrc; 179 uint32_t RelocCrc; 180 } SectionContr; 181 uint16_t Flags; 182 uint16_t ModuleSymStream; 183 uint32_t SymByteSize; 184 uint32_t C11ByteSize; 185 uint32_t C13ByteSize; 186 uint16_t SourceFileCount; 187 char Padding[2]; 188 uint32_t Unused2; 189 uint32_t SourceFileNameIndex; 190 uint32_t PdbFilePathNameIndex; 191 char ModuleName[]; 192 char ObjFileName[]; 193 }; 194 195- **SectionContr** - Describes the properties of the section in the final binary 196 which contain the code and data from this module. 197 198 ``SectionContr.Characteristics`` corresponds to the ``Characteristics`` field 199 of the `IMAGE_SECTION_HEADER <https://msdn.microsoft.com/en-us/library/windows/desktop/ms680341(v=vs.85).aspx>`__ 200 structure. 201 202 203- **Flags** - A bitfield with the following format: 204 205.. code-block:: c++ 206 207 // ``true`` if this ModInfo has been written since reading the PDB. This is 208 // likely used to support incremental linking, so that the linker can decide 209 // if it needs to commit changes to disk. 210 uint16_t Dirty : 1; 211 // ``true`` if EC information is present for this module. EC is presumed to 212 // stand for "Edit & Continue", which LLVM does not support. So this flag 213 // will always be be false. 214 uint16_t EC : 1; 215 uint16_t Unused : 6; 216 // Type Server Index for this module. This is assumed to be related to /Zi, 217 // but as LLVM treats /Zi as /Z7, this field will always be invalid for LLVM 218 // generated PDBs. 219 uint16_t TSM : 8; 220 221 222- **ModuleSymStream** - The index of the stream that contains symbol information 223 for this module. This includes CodeView symbol information as well as source 224 and line information. If this field is -1, then no additional debug info will 225 be present for this module (for example, this is what happens when you strip 226 private symbols from a PDB). 227 228- **SymByteSize** - The number of bytes of data from the stream identified by 229 ``ModuleSymStream`` that represent CodeView symbol records. 230 231- **C11ByteSize** - The number of bytes of data from the stream identified by 232 ``ModuleSymStream`` that represent C11-style CodeView line information. 233 234- **C13ByteSize** - The number of bytes of data from the stream identified by 235 ``ModuleSymStream`` that represent C13-style CodeView line information. At 236 most one of ``C11ByteSize`` and ``C13ByteSize`` will be non-zero. Modern PDBs 237 always use C13 instead of C11. 238 239- **SourceFileCount** - The number of source files that contributed to this 240 module during compilation. 241 242- **SourceFileNameIndex** - The offset in the names buffer of the primary 243 translation unit used to build this module. All PDB files observed to date 244 always have this value equal to 0. 245 246- **PdbFilePathNameIndex** - The offset in the names buffer of the PDB file 247 containing this module's symbol information. This has only been observed 248 to be non-zero for the special ``* Linker *`` module. 249 250- **ModuleName** - The module name. This is usually either a full path to an 251 object file (either directly passed to ``link.exe`` or from an archive) or 252 a string of the form ``Import:<dll name>``. 253 254- **ObjFileName** - The object file name. In the case of an module that is 255 linked directly passed to ``link.exe``, this is the same as **ModuleName**. 256 In the case of a module that comes from an archive, this is usually the full 257 path to the archive. 258 259.. _dbi_sec_contr_substream: 260 261Section Contribution Substream 262^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 263Begins at offset ``0`` immediately after the :ref:`dbi_mod_info_substream` ends, 264and consumes ``Header->SectionContributionSize`` bytes. This substream begins 265with a single ``uint32_t`` which will be one of the following values: 266 267.. code-block:: c++ 268 269 enum class SectionContrSubstreamVersion : uint32_t { 270 Ver60 = 0xeffe0000 + 19970605, 271 V2 = 0xeffe0000 + 20140516 272 }; 273 274``Ver60`` is the only value which has been observed in a PDB so far. Following 275this is an array of fixed-length structures. If the version is ``Ver60``, 276it is an array of ``SectionContribEntry`` structures (this is the nested structure 277from the ``ModInfo`` type. If the version is ``V2``, it is an array of 278``SectionContribEntry2`` structures, defined as follows: 279 280.. code-block:: c++ 281 282 struct SectionContribEntry2 { 283 SectionContribEntry SC; 284 uint32_t ISectCoff; 285 }; 286 287The purpose of the second field is not well understood. The name implies that 288is the index of the COFF section, but this also describes the existing field 289``SectionContribEntry::Section``. 290 291 292.. _dbi_section_map_substream: 293 294Section Map Substream 295^^^^^^^^^^^^^^^^^^^^^ 296Begins at offset ``0`` immediately after the :ref:`dbi_sec_contr_substream` ends, 297and consumes ``Header->SectionMapSize`` bytes. This substream begins with an ``4`` 298byte header followed by an array of fixed-length records. The header and records 299have the following layout: 300 301.. code-block:: c++ 302 303 struct SectionMapHeader { 304 uint16_t Count; // Number of segment descriptors 305 uint16_t LogCount; // Number of logical segment descriptors 306 }; 307 308 struct SectionMapEntry { 309 uint16_t Flags; // See the SectionMapEntryFlags enum below. 310 uint16_t Ovl; // Logical overlay number 311 uint16_t Group; // Group index into descriptor array. 312 uint16_t Frame; 313 uint16_t SectionName; // Byte index of segment / group name in string table, or 0xFFFF. 314 uint16_t ClassName; // Byte index of class in string table, or 0xFFFF. 315 uint32_t Offset; // Byte offset of the logical segment within physical segment. If group is set in flags, this is the offset of the group. 316 uint32_t SectionLength; // Byte count of the segment or group. 317 }; 318 319 enum class SectionMapEntryFlags : uint16_t { 320 Read = 1 << 0, // Segment is readable. 321 Write = 1 << 1, // Segment is writable. 322 Execute = 1 << 2, // Segment is executable. 323 AddressIs32Bit = 1 << 3, // Descriptor describes a 32-bit linear address. 324 IsSelector = 1 << 8, // Frame represents a selector. 325 IsAbsoluteAddress = 1 << 9, // Frame represents an absolute address. 326 IsGroup = 1 << 10 // If set, descriptor represents a group. 327 }; 328 329Many of these fields are not well understood, so will not be discussed further. 330 331.. _dbi_file_info_substream: 332 333File Info Substream 334^^^^^^^^^^^^^^^^^^^ 335Begins at offset ``0`` immediately after the :ref:`dbi_section_map_substream` ends, 336and consumes ``Header->SourceInfoSize`` bytes. This substream defines the mapping 337from module to the source files that contribute to that module. Since multiple 338modules can use the same source file (for example, a header file), this substream 339uses a string table to store each unique file name only once, and then have each 340module use offsets into the string table rather than embedding the string's value 341directly. The format of this substream is as follows: 342 343.. code-block:: c++ 344 345 struct FileInfoSubstream { 346 uint16_t NumModules; 347 uint16_t NumSourceFiles; 348 349 uint16_t ModIndices[NumModules]; 350 uint16_t ModFileCounts[NumModules]; 351 uint32_t FileNameOffsets[NumSourceFiles]; 352 char NamesBuffer[][NumSourceFiles]; 353 }; 354 355**NumModules** - The number of modules for which source file information is 356contained within this substream. Should match the corresponding value from the 357ref:`dbi_header`. 358 359**NumSourceFiles**: In theory this is supposed to contain the number of source 360files for which this substream contains information. But that would present a 361problem in that the width of this field being ``16``-bits would prevent one from 362having more than 64K source files in a program. In early versions of the file 363format, this seems to have been the case. In order to support more than this, this 364field of the is simply ignored, and computed dynamically by summing up the values of 365the ``ModFileCounts`` array (discussed below). In short, this value should be 366ignored. 367 368**ModIndices** - This array is present, but does not appear to be useful. 369 370**ModFileCountArray** - An array of ``NumModules`` integers, each one containing 371the number of source files which contribute to the module at the specified index. 372While each individual module is limited to 64K contributing source files, the 373union of all modules' source files may be greater than 64K. The real number of 374source files is thus computed by summing this array. Note that summing this array 375does not give the number of `unique` source files, only the total number of source 376file contributions to modules. 377 378**FileNameOffsets** - An array of **NumSourceFiles** integers (where **NumSourceFiles** 379here refers to the 32-bit value obtained from summing **ModFileCountArray**), where 380each integer is an offset into **NamesBuffer** pointing to a null terminated string. 381 382**NamesBuffer** - An array of null terminated strings containing the actual source 383file names. 384 385.. _dbi_type_server_map_substream: 386 387Type Server Map Substream 388^^^^^^^^^^^^^^^^^^^^^^^^^ 389Begins at offset ``0`` immediately after the :ref:`dbi_file_info_substream` 390ends, and consumes ``Header->TypeServerMapSize`` bytes. Neither the purpose 391nor the layout of this substream is understood, although it is assumed to 392related somehow to the usage of ``/Zi`` and ``mspdbsrv.exe``. This substream 393will not be discussed further. 394 395.. _dbi_ec_substream: 396 397EC Substream 398^^^^^^^^^^^^ 399Begins at offset ``0`` immediately after the 400:ref:`dbi_type_server_map_substream` ends, and consumes 401``Header->ECSubstreamSize`` bytes. This is presumed to be related to Edit & 402Continue support in MSVC. LLVM does not support Edit & Continue, so this 403stream will not be discussed further. 404 405.. _dbi_optional_dbg_stream: 406 407Optional Debug Header Stream 408^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 409Begins at offset ``0`` immediately after the :ref:`dbi_ec_substream` ends, and 410consumes ``Header->OptionalDbgHeaderSize`` bytes. This field is an array of 411stream indices (e.g. ``uint16_t``'s), each of which identifies a stream 412index in the larger MSF file which contains some additional debug information. 413Each position of this array has a special meaning, allowing one to determine 414what kind of debug information is at the referenced stream. ``11`` indices 415are currently understood, although it's possible there may be more. The 416layout of each stream generally corresponds exactly to a particular type 417of debug data directory from the PE/COFF file. The format of these fields 418can be found in the `Microsoft PE/COFF Specification <https://www.microsoft.com/en-us/download/details.aspx?id=19509>`__. 419If any of these fields is -1, it means the corresponding type of debug info is 420not present in the PDB. 421 422**FPO Data** - ``DbgStreamArray[0]``. The data in the referenced stream is an 423array of ``FPO_DATA`` structures. This contains the relocated contents of 424any ``.debug$F`` section from any of the linker inputs. 425 426**Exception Data** - ``DbgStreamArray[1]``. The data in the referenced stream 427is a debug data directory of type ``IMAGE_DEBUG_TYPE_EXCEPTION``. 428 429**Fixup Data** - ``DbgStreamArray[2]``. The data in the referenced stream is a 430debug data directory of type ``IMAGE_DEBUG_TYPE_FIXUP``. 431 432**Omap To Src Data** - ``DbgStreamArray[3]``. The data in the referenced stream 433is a debug data directory of type ``IMAGE_DEBUG_TYPE_OMAP_TO_SRC``. This 434is used for mapping addresses between instrumented and uninstrumented code. 435 436**Omap From Src Data** - ``DbgStreamArray[4]``. The data in the referenced stream 437is a debug data directory of type ``IMAGE_DEBUG_TYPE_OMAP_FROM_SRC``. This 438is used for mapping addresses between instrumented and uninstrumented code. 439 440**Section Header Data** - ``DbgStreamArray[5]``. A dump of all section headers from 441the original executable. 442 443**Token / RID Map** - ``DbgStreamArray[6]``. The layout of this stream is not 444understood, but it is assumed to be a mapping from ``CLR Token`` to 445``CLR Record ID``. Refer to `ECMA 335 <http://www.ecma-international.org/publications/standards/Ecma-335.htm>`__ 446for more information. 447 448**Xdata** - ``DbgStreamArray[7]``. A copy of the ``.xdata`` section from the 449executable. 450 451**Pdata** - ``DbgStreamArray[8]``. This is assumed to be a copy of the ``.pdata`` 452section from the executable, but that would make it identical to 453``DbgStreamArray[1]``. The difference between these two indices is not well 454understood. 455 456**New FPO Data** - ``DbgStreamArray[9]``. The data in the referenced stream is a 457debug data directory of type ``IMAGE_DEBUG_TYPE_FPO``. Note that this is different 458from ``DbgStreamArray[0]`` in that ``.debug$F`` sections are only emitted by MASM. 459Thus, it is possible for both to appear in the same PDB if both MASM object files 460and cl object files are linked into the same program. 461 462**Original Section Header Data** - ``DbgStreamArray[10]``. Similar to 463``DbgStreamArray[5]``, but contains the section headers before any binary translation 464has been performed. This can be used in conjunction with ``DebugStreamArray[3]`` 465and ``DbgStreamArray[4]`` to map instrumented and uninstrumented addresses. 466