1Use in C++ {#flatbuffers_guide_use_cpp} 2========== 3 4## Before you get started 5 6Before diving into the FlatBuffers usage in C++, it should be noted that 7the [Tutorial](@ref flatbuffers_guide_tutorial) page has a complete guide 8to general FlatBuffers usage in all of the supported languages (including C++). 9This page is designed to cover the nuances of FlatBuffers usage, specific to 10C++. 11 12#### Prerequisites 13 14This page assumes you have written a FlatBuffers schema and compiled it 15with the Schema Compiler. If you have not, please see 16[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler) 17and [Writing a schema](@ref flatbuffers_guide_writing_schema). 18 19Assuming you wrote a schema, say `mygame.fbs` (though the extension doesn't 20matter), you've generated a C++ header called `mygame_generated.h` using the 21compiler (e.g. `flatc -c mygame.fbs`), you can now start using this in 22your program by including the header. As noted, this header relies on 23`flatbuffers/flatbuffers.h`, which should be in your include path. 24 25## FlatBuffers C++ library code location 26 27The code for the FlatBuffers C++ library can be found at 28`flatbuffers/include/flatbuffers`. You can browse the library code on the 29[FlatBuffers GitHub page](https://github.com/google/flatbuffers/tree/master/include/flatbuffers). 30 31## Testing the FlatBuffers C++ library 32 33The code to test the C++ library can be found at `flatbuffers/tests`. 34The test code itself is located in 35[test.cpp](https://github.com/google/flatbuffers/blob/master/tests/test.cpp). 36 37This test file is built alongside `flatc`. To review how to build the project, 38please read the [Building](@ref flatbuffers_guide_building) documenation. 39 40To run the tests, execute `flattests` from the root `flatbuffers/` directory. 41For example, on [Linux](https://en.wikipedia.org/wiki/Linux), you would simply 42run: `./flattests`. 43 44## Using the FlatBuffers C++ library 45 46*Note: See [Tutorial](@ref flatbuffers_guide_tutorial) for a more in-depth 47example of how to use FlatBuffers in C++.* 48 49FlatBuffers supports both reading and writing FlatBuffers in C++. 50 51To use FlatBuffers in your code, first generate the C++ classes from your 52schema with the `--cpp` option to `flatc`. Then you can include both FlatBuffers 53and the generated code to read or write FlatBuffers. 54 55For example, here is how you would read a FlatBuffer binary file in C++: 56First, include the library and generated code. Then read the file into 57a `char *` array, which you pass to `GetMonster()`. 58 59~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 60 #include "flatbuffers/flatbuffers.h" 61 #include "monster_test_generate.h" 62 #include <cstdio> // For printing and file access. 63 64 FILE* file = fopen("monsterdata_test.mon", "rb"); 65 fseek(file, 0L, SEEK_END); 66 int length = ftell(file); 67 fseek(file, 0L, SEEK_SET); 68 char *data = new char[length]; 69 fread(data, sizeof(char), length, file); 70 fclose(file); 71 72 auto monster = GetMonster(data); 73~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 74 75`monster` is of type `Monster *`, and points to somewhere *inside* your 76buffer (root object pointers are not the same as `buffer_pointer` !). 77If you look in your generated header, you'll see it has 78convenient accessors for all fields, e.g. `hp()`, `mana()`, etc: 79 80~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 81 printf("%d\n", monster->hp()); // `80` 82 printf("%d\n", monster->mana()); // default value of `150` 83 printf("%s\n", monster->name()->c_str()); // "MyMonster" 84~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 85 86*Note: That we never stored a `mana` value, so it will return the default.* 87 88## Object based API. {#flatbuffers_cpp_object_based_api} 89 90FlatBuffers is all about memory efficiency, which is why its base API is written 91around using as little as possible of it. This does make the API clumsier 92(requiring pre-order construction of all data, and making mutation harder). 93 94For times when efficiency is less important a more convenient object based API 95can be used (through `--gen-object-api`) that is able to unpack & pack a 96FlatBuffer into objects and standard STL containers, allowing for convenient 97construction, access and mutation. 98 99To use: 100 101~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 102 // Autogenerated class from table Monster. 103 MonsterT monsterobj; 104 105 // Deserialize from buffer into object. 106 UnPackTo(&monsterobj, flatbuffer); 107 108 // Update object directly like a C++ class instance. 109 cout << monsterobj->name; // This is now a std::string! 110 monsterobj->name = "Bob"; // Change the name. 111 112 // Serialize into new flatbuffer. 113 FlatBufferBuilder fbb; 114 Pack(fbb, &monsterobj); 115~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 116 117The following attributes are specific to the object-based API code generation: 118 119- `native_inline` (on a field): Because FlatBuffer tables and structs are 120 optionally present in a given buffer, they are best represented as pointers 121 (specifically std::unique_ptrs) in the native class since they can be null. 122 This attribute changes the member declaration to use the type directly 123 rather than wrapped in a unique_ptr. 124 125- `native_default`: "value" (on a field): For members that are declared 126 "native_inline", the value specified with this attribute will be included 127 verbatim in the class constructor initializer list for this member. 128 129- `native_custom_alloc`:"custom_allocator" (on a table or struct): When using the 130 object-based API all generated NativeTables that are allocated when unpacking 131 your flatbuffer will use "custom allocator". The allocator is also used by 132 any std::vector that appears in a table defined with `native_custom_alloc`. 133 This can be used to provide allocation from a pool for example, for faster 134 unpacking when using the object-based API. 135 136 Minimal Example: 137 138 schema: 139 140 table mytable(native_custom_alloc:"custom_allocator") { 141 ... 142 } 143 144 with custom_allocator defined before flatbuffers.h is included, as: 145 146 template <typename T> struct custom_allocator : public std::allocator<T> { 147 148 typedef T *pointer; 149 150 template <class U> 151 struct rebind { 152 typedef custom_allocator<U> other; 153 }; 154 155 pointer allocate(const std::size_t n) { 156 return std::allocator<T>::allocate(n); 157 } 158 159 void deallocate(T* ptr, std::size_t n) { 160 return std::allocator<T>::deallocate(ptr,n); 161 } 162 163 custom_allocator() throw() {} 164 template <class U> 165 custom_allocator(const custom_allocator<U>&) throw() {} 166 }; 167 168- `native_type`' "type" (on a struct): In some cases, a more optimal C++ data 169 type exists for a given struct. For example, the following schema: 170 171 struct Vec2 { 172 x: float; 173 y: float; 174 } 175 176 generates the following Object-Based API class: 177 178 struct Vec2T : flatbuffers::NativeTable { 179 float x; 180 float y; 181 }; 182 183 However, it can be useful to instead use a user-defined C++ type since it 184 can provide more functionality, eg. 185 186 struct vector2 { 187 float x = 0, y = 0; 188 vector2 operator+(vector2 rhs) const { ... } 189 vector2 operator-(vector2 rhs) const { ... } 190 float length() const { ... } 191 // etc. 192 }; 193 194 The `native_type` attribute will replace the usage of the generated class 195 with the given type. So, continuing with the example, the generated 196 code would use |vector2| in place of |Vec2T| for all generated code. 197 198 However, becuase the native_type is unknown to flatbuffers, the user must 199 provide the following functions to aide in the serialization process: 200 201 namespace flatbuffers { 202 FlatbufferStruct Pack(const native_type& obj); 203 native_type UnPack(const FlatbufferStruct& obj); 204 } 205 206Finally, the following top-level attribute 207 208- native_include: "path" (at file level): Because the `native_type` attribute 209 can be used to introduce types that are unknown to flatbuffers, it may be 210 necessary to include "external" header files in the generated code. This 211 attribute can be used to directly add an #include directive to the top of 212 the generated code that includes the specified path directly. 213 214# External references. 215 216An additional feature of the object API is the ability to allow you to load 217multiple independent FlatBuffers, and have them refer to eachothers objects 218using hashes which are then represented as typed pointers in the object API. 219 220To make this work have a field in the objects you want to referred to which is 221using the string hashing feature (see `hash` attribute in the 222[schema](@ref flatbuffers_guide_writing_schema) documentation). Then you have 223a similar hash in the field referring to it, along with a `cpp_type` 224attribute specifying the C++ type this will refer to (this can be any C++ 225type, and will get a `*` added). 226 227Then, in JSON or however you create these buffers, make sure they use the 228same string (or hash). 229 230When you call `UnPack` (or `Create`), you'll need a function that maps from 231hash to the object (see `resolver_function_t` for details). 232 233# Using different pointer types. 234 235By default the object tree is built out of `std::unique_ptr`, but you can 236influence this either globally (using the `--cpp-ptr-type` argument to 237`flatc`) or per field (using the `cpp_ptr_type` attribute) to by any smart 238pointer type (`my_ptr<T>`), or by specifying `naked` as the type to get `T *` 239pointers. Unlike the smart pointers, naked pointers do not manage memory for 240you, so you'll have to manage their lifecycles manually. 241 242 243# Using different string type. 244 245By default the object tree is built out of `std::string`, but you can 246influence this either globally (using the `--cpp-str-type` argument to 247`flatc`) or per field using the `cpp_str_type` attribute. 248 249The type must support T::c_str() and T::length() as member functions. 250 251## Reflection (& Resizing) 252 253There is experimental support for reflection in FlatBuffers, allowing you to 254read and write data even if you don't know the exact format of a buffer, and 255even allows you to change sizes of strings and vectors in-place. 256 257The way this works is very elegant; there is actually a FlatBuffer schema that 258describes schemas (!) which you can find in `reflection/reflection.fbs`. 259The compiler, `flatc`, can write out any schemas it has just parsed as a binary 260FlatBuffer, corresponding to this meta-schema. 261 262Loading in one of these binary schemas at runtime allows you traverse any 263FlatBuffer data that corresponds to it without knowing the exact format. You 264can query what fields are present, and then read/write them after. 265 266For convenient field manipulation, you can include the header 267`flatbuffers/reflection.h` which includes both the generated code from the meta 268schema, as well as a lot of helper functions. 269 270And example of usage, for the time being, can be found in 271`test.cpp/ReflectionTest()`. 272 273## Mini Reflection 274 275A more limited form of reflection is available for direct inclusion in 276generated code, which doesn't any (binary) schema access at all. It was designed 277to keep the overhead of reflection as low as possible (on the order of 2-6 278bytes per field added to your executable), but doesn't contain all the 279information the (binary) schema contains. 280 281You add this information to your generated code by specifying `--reflect-types` 282(or instead `--reflect-names` if you also want field / enum names). 283 284You can now use this information, for example to print a FlatBuffer to text: 285 286 auto s = flatbuffers::FlatBufferToString(flatbuf, MonsterTypeTable()); 287 288`MonsterTypeTable()` is declared in the generated code for each type. The 289string produced is very similar to the JSON produced by the `Parser` based 290text generator. 291 292You'll need `flatbuffers/minireflect.h` for this functionality. In there is also 293a convenient visitor/iterator so you can write your own output / functionality 294based on the mini reflection tables without having to know the FlatBuffers or 295reflection encoding. 296 297## Storing maps / dictionaries in a FlatBuffer 298 299FlatBuffers doesn't support maps natively, but there is support to 300emulate their behavior with vectors and binary search, which means you 301can have fast lookups directly from a FlatBuffer without having to unpack 302your data into a `std::map` or similar. 303 304To use it: 305- Designate one of the fields in a table as they "key" field. You do this 306 by setting the `key` attribute on this field, e.g. 307 `name:string (key)`. 308 You may only have one key field, and it must be of string or scalar type. 309- Write out tables of this type as usual, collect their offsets in an 310 array or vector. 311- Instead of `CreateVector`, call `CreateVectorOfSortedTables`, 312 which will first sort all offsets such that the tables they refer to 313 are sorted by the key field, then serialize it. 314- Now when you're accessing the FlatBuffer, you can use `Vector::LookupByKey` 315 instead of just `Vector::Get` to access elements of the vector, e.g.: 316 `myvector->LookupByKey("Fred")`, which returns a pointer to the 317 corresponding table type, or `nullptr` if not found. 318 `LookupByKey` performs a binary search, so should have a similar speed to 319 `std::map`, though may be faster because of better caching. `LookupByKey` 320 only works if the vector has been sorted, it will likely not find elements 321 if it hasn't been sorted. 322 323## Direct memory access 324 325As you can see from the above examples, all elements in a buffer are 326accessed through generated accessors. This is because everything is 327stored in little endian format on all platforms (the accessor 328performs a swap operation on big endian machines), and also because 329the layout of things is generally not known to the user. 330 331For structs, layout is deterministic and guaranteed to be the same 332accross platforms (scalars are aligned to their 333own size, and structs themselves to their largest member), and you 334are allowed to access this memory directly by using `sizeof()` and 335`memcpy` on the pointer to a struct, or even an array of structs. 336 337To compute offsets to sub-elements of a struct, make sure they 338are a structs themselves, as then you can use the pointers to 339figure out the offset without having to hardcode it. This is 340handy for use of arrays of structs with calls like `glVertexAttribPointer` 341in OpenGL or similar APIs. 342 343It is important to note is that structs are still little endian on all 344machines, so only use tricks like this if you can guarantee you're not 345shipping on a big endian machine (an `assert(FLATBUFFERS_LITTLEENDIAN)` 346would be wise). 347 348## Access of untrusted buffers 349 350The generated accessor functions access fields over offsets, which is 351very quick. These offsets are not verified at run-time, so a malformed 352buffer could cause a program to crash by accessing random memory. 353 354When you're processing large amounts of data from a source you know (e.g. 355your own generated data on disk), this is acceptable, but when reading 356data from the network that can potentially have been modified by an 357attacker, this is undesirable. 358 359For this reason, you can optionally use a buffer verifier before you 360access the data. This verifier will check all offsets, all sizes of 361fields, and null termination of strings to ensure that when a buffer 362is accessed, all reads will end up inside the buffer. 363 364Each root type will have a verification function generated for it, 365e.g. for `Monster`, you can call: 366 367~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 368 bool ok = VerifyMonsterBuffer(Verifier(buf, len)); 369~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 370 371if `ok` is true, the buffer is safe to read. 372 373Besides untrusted data, this function may be useful to call in debug 374mode, as extra insurance against data being corrupted somewhere along 375the way. 376 377While verifying a buffer isn't "free", it is typically faster than 378a full traversal (since any scalar data is not actually touched), 379and since it may cause the buffer to be brought into cache before 380reading, the actual overhead may be even lower than expected. 381 382In specialized cases where a denial of service attack is possible, 383the verifier has two additional constructor arguments that allow 384you to limit the nesting depth and total amount of tables the 385verifier may encounter before declaring the buffer malformed. The default is 386`Verifier(buf, len, 64 /* max depth */, 1000000, /* max tables */)` which 387should be sufficient for most uses. 388 389## Text & schema parsing 390 391Using binary buffers with the generated header provides a super low 392overhead use of FlatBuffer data. There are, however, times when you want 393to use text formats, for example because it interacts better with source 394control, or you want to give your users easy access to data. 395 396Another reason might be that you already have a lot of data in JSON 397format, or a tool that generates JSON, and if you can write a schema for 398it, this will provide you an easy way to use that data directly. 399 400(see the schema documentation for some specifics on the JSON format 401accepted). 402 403There are two ways to use text formats: 404 405#### Using the compiler as a conversion tool 406 407This is the preferred path, as it doesn't require you to add any new 408code to your program, and is maximally efficient since you can ship with 409binary data. The disadvantage is that it is an extra step for your 410users/developers to perform, though you might be able to automate it. 411 412 flatc -b myschema.fbs mydata.json 413 414This will generate the binary file `mydata_wire.bin` which can be loaded 415as before. 416 417#### Making your program capable of loading text directly 418 419This gives you maximum flexibility. You could even opt to support both, 420i.e. check for both files, and regenerate the binary from text when 421required, otherwise just load the binary. 422 423This option is currently only available for C++, or Java through JNI. 424 425As mentioned in the section "Building" above, this technique requires 426you to link a few more files into your program, and you'll want to include 427`flatbuffers/idl.h`. 428 429Load text (either a schema or json) into an in-memory buffer (there is a 430convenient `LoadFile()` utility function in `flatbuffers/util.h` if you 431wish). Construct a parser: 432 433~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 434 flatbuffers::Parser parser; 435~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 436 437Now you can parse any number of text files in sequence: 438 439~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp} 440 parser.Parse(text_file.c_str()); 441~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 442 443This works similarly to how the command-line compiler works: a sequence 444of files parsed by the same `Parser` object allow later files to 445reference definitions in earlier files. Typically this means you first 446load a schema file (which populates `Parser` with definitions), followed 447by one or more JSON files. 448 449As optional argument to `Parse`, you may specify a null-terminated list of 450include paths. If not specified, any include statements try to resolve from 451the current directory. 452 453If there were any parsing errors, `Parse` will return `false`, and 454`Parser::err` contains a human readable error string with a line number 455etc, which you should present to the creator of that file. 456 457After each JSON file, the `Parser::fbb` member variable is the 458`FlatBufferBuilder` that contains the binary buffer version of that 459file, that you can access as described above. 460 461`samples/sample_text.cpp` is a code sample showing the above operations. 462 463## Threading 464 465Reading a FlatBuffer does not touch any memory outside the original buffer, 466and is entirely read-only (all const), so is safe to access from multiple 467threads even without synchronisation primitives. 468 469Creating a FlatBuffer is not thread safe. All state related to building 470a FlatBuffer is contained in a FlatBufferBuilder instance, and no memory 471outside of it is touched. To make this thread safe, either do not 472share instances of FlatBufferBuilder between threads (recommended), or 473manually wrap it in synchronisation primites. There's no automatic way to 474accomplish this, by design, as we feel multithreaded construction 475of a single buffer will be rare, and synchronisation overhead would be costly. 476 477## Advanced union features 478 479The C++ implementation currently supports vectors of unions (i.e. you can 480declare a field as `[T]` where `T` is a union type instead of a table type). It 481also supports structs and strings in unions, besides tables. 482 483For an example of these features, see `tests/union_vector`, and 484`UnionVectorTest` in `test.cpp`. 485 486Since these features haven't been ported to other languages yet, if you 487choose to use them, you won't be able to use these buffers in other languages 488(`flatc` will refuse to compile a schema that uses these features). 489 490These features reduce the amount of "table wrapping" that was previously 491needed to use unions. 492 493To use scalars, simply wrap them in a struct. 494 495<br> 496