1# Perfetto key concepts and architecture
2
3Producer <> Service <> Consumer model
4-------------------------------------
5![Perfetto Stack](https://storage.googleapis.com/perfetto/markdown_img/producer-service-consumer.png)
6
7**Service**
8The tracing service is a long-lived entity (a system daemon on Linux/Android,
9a service in Chrome) that has the following responsibilities:
10- Maintains a registry of active producers and their data sources.
11- Owns the trace buffers.
12- Handles multiplexing of several tracing sessions.
13- Routes the trace config from the consumers to the corresponding producers.
14- Tells the Producers when and what to trace.
15- Moves data from the Producer's shared memory buffer to the central non-shared
16  trace buffers.
17
18**Producer**
19A producer is an untrusted entity that offers the ability to contribute to the
20trace. In a multiprocess model, a producer almost always corresponds to a client
21process of the tracing service. It advertises its ability to contribute to the trace with one or more data sources.
22Each producer has exactly:
23- One shared memory buffer, shared exclusively with the tracing service.
24- One IPC channel with the tracing service.
25
26A producer is completely decoupled (both technically and conceptually) from
27consumer(s). A producer knows nothing about:
28- How many consumer(s) are connected to the service.
29- How many tracing sessions are active.
30- How many other producer(s) are registered or active.
31- Trace data written by other producer(s).
32
33*** aside
34In rare circumstances a process can host more than one producer and hence more
35than one shared memory buffer. This can be the case for a process bundling
36third-party libraries that in turn include the Perfetto client library.
37Concrete example: at some point in the future Chrome might expose one Producer for tracing within the main project, one for V8 and one for Skia (for each child
38process).
39***
40
41**Consumer**
42A consumer is a trusted entity (a cmdline client on Linux/Android, an interface
43of the Browser process in Chrome) that controls (non-exclusively) the tracing service and reads back (destructively) the trace buffers.
44A consumer has the ability to:
45- Send a [trace config](trace-config.md) to the service, determining:
46 - How many trace buffers to create.
47 - How big the trace buffers should be.
48 - The policy for each buffer (*ring-buffer* or *stop-when-full*).
49 - Which data sources to enable.
50 - The configuration for each data source.
51 - The target buffer for the data produced by each data source configured.
52- Enable and disable tracing.
53- Read back the trace buffers:
54  - Streaming data over the IPC channel.
55  - Passing a file descriptor to the service and instructing it to periodically
56    save the trace buffers into the file.
57
58**Data source**
59A data source is a capability, exposed by a Producer, of providing some tracing
60data. A data source almost always defines its own schema (a protobuf) consisting
61of:
62- At most one `DataSourceConfig` sub-message
63  ([example](/protos/perfetto/config/ftrace/ftrace_config.proto))
64- One or more `TracePacket` sub-messages
65  ([example](/protos/perfetto/trace/ps/process_tree.proto))
66
67Different producers may expose the same data source. Concrete example:
68*** aside
69At some point in the near future we might offer, as part of Perfetto, a library
70for in-process heap profiling. In such case more than one producer, linking
71against the updated Perfetto library, will expose the heap profiler data source,
72for its own process.
73***
74
75**IPC channel**
76In a multiprocess scenario, each producer and each consumer interact with the
77service using an IPC channel. IPC is used only in non-fast-path interactions,
78mostly handshakes such as enabling/disabling trace (consumer), (un)registering
79and starting/stopping data sources (producer). The IPC is typically NOT employed
80to transport the protobufs for the trace.
81Perfetto provides a POSIX-friendly IPC implementation, based on protobufs over a
82UNIX socket (see [ipc.md](ipc.md)). That IPC implementation is not mandated.
83Perfetto allows the embedder:
84- Wrap its own IPC subsystem (e.g., Perfetto in Chromium uses Mojo)
85- Not use an IPC mechanism at all and just short circuit the
86  Producer <> Service <> Consumer interaction via `PostTask(s)`.
87
88See [embedder-guide.md](embedder-guide.md) for more details.
89
90
91**Shared memory buffer**
92Producer(s) write tracing data, in the form of protobuf-encoded binary blobs,
93directly into its shared memory buffer, using a special library called
94[ProtoZero](protozero.md). The shared memory buffer:
95- Has a fixed and typically small size (configurable, default: 128 KB).
96- Is an ABI and must maintain backwards compatibility.
97- Is shared by all data sources of the producer.
98- Is independent of the number and the size of the trace buffers.
99- Is independent of the number of Consumer(s).
100- Is partitioned in *chunks* of variable size.
101
102Each chunk:
103- Is owned exclusively by one Producer thread (or shared through a mutex).
104- Contains a linear sequence of [`TracePacket(s)`](trace-format.md), or
105  fragments of that. A `TracePacket` can span across several chunks, the
106  fragmentation is not exposed to the consumers (consumers always see whole
107  packets as if they were never fragmented).
108- Can be owned and written by exactly one `TraceWriter`.
109- Is part of a reliable and ordered sequence, identified by the `WriterID`:
110  packets in a sequence are guaranteed to be read back in order, without gaps
111  and without repetitions (see [trace-format.md](trace-format.md) for more).
112
113See the comments in
114[shared_memory_abi.h](/include/perfetto/tracing/core/shared_memory_abi.h)
115for more details about the binary format of this buffer.
116
117Other resources
118---------------
119* [Life of a tracing session](life-of-a-tracing-session.md)
120* [Trace config](trace-config.md)
121* [Trace format](trace-format.md)
122