1# CHRE API Parser + Code Generator for CHPP
2
3Since one of the main use cases of CHPP is to support remoting a CHRE PAL
4implementation to a peripheral component, we leverage the CHRE PAL API
5definitions for the platform-specific implementation of those services, such
6as the WWAN service. However, the structures used in the CHRE PAL APIs are the
7same as those used in the CHRE API for nanoapps, which are not inherently
8serializable, as they include things like pointers to other structures. So we
9must create slightly updated versions of these to send over the wire with CHPP.
10
11The `chre_api_to_chpp.py` script in this directory parses CHRE APIs according
12to instructions given in `chre_api_annotations.json` and generates structures
13and conversion code used for serialization and deserialization.
14
15## Serialization format
16
17_TL;DR: The CHPP format is just a packed and flattened version of the CHRE
18structures, replacing pointers with offsets._
19
20The serialized output is meant to match the source CHRE structure as closely as
21possible, however some special handling is needed to be able to send them over
22the wire to a different processor in the system. The general process for
23 generating a serializable CHPP version of a CHRE structure is as follows:
24
25 1. Rewrite the name to start with "Chpp" instead of "chre" and mark it packed,
26    to ensure there's no padding between fields (which is compiler-dependent)
27 1. Flatten the structure by replacing pointers to nested structures/arrays with
28    a ChppOffset which describes the location in the payload where the nested
29    data resides.
30 1. A CHPP application layer header is allocated for convenience when used in
31    CHPP. The header values need to be set by CHPP and they are not zeroed out.
32    The header length is not included in the offset calculation.
33
34ChppOffset is a collection of a 16-bit offset from the beginning of the payload,
35and a length. While we could implicitly derive the offset, the length cannot
36exceed 16 bits due to limitations in the encapsulating CHPP protocol
37(and we don't expect to bump up against this limit anyways), and including the
38offset field helps simplify decoding.
39
40This approach allows for a highly optimized, in-place encoding and decoding that
41only requires converting between pointer and ChppOffset types, if the following
42conditions are met:
43
44 1. The size of the CHRE and CHPP structures are the same, and the offsets to
45    all contained fields are the same, and the same property holds for all
46    nested structures/unions
47 1. (Encoding only) All nested structures appear in contiguous memory - for
48    example, if a structure has a pointer to an array, then the array appears
49    immediately after the structure in memory, with no padding
50 1. (Decoding only) Processor alignment constraints are met for all fields, such
51    that they can be safely accessed through normal means at the location in
52    memory at which they reside, when accounting for any headers, etc.
53
54If conditions 2 or 3 are not met, then optimized encode/decode is still possible
55after copying the memory to a separate allocation. So in-place conversion can't
56be done, but the only translation needed is between pointer and ChppOffset. If
57the first condition is not met, then conversion is done using field-wise
58assignment, which allows the compiler to generate the necessary instructions to
59handle differences in alignment and packing.
60
61All generated code currently assumes that it's running on a little endian CPU,
62as the wire format requires little endian byte order.
63
64TODO: The script currently only generates code for the pessimistic case, where
65none of the constraints are met. By generating code that checks sizeof/offsetof
66on all fields and placing the optimized path in an if/else condition, the
67compiler can evaluate condition 1 and prune away code for the less-optimal path.
68
69## Annotations
70
71As the C API does not have enough information for the script to know how to
72handle things like variable-length arrays (VLAs), we use the
73`chre_api_annotations.json` file to provide this detail. The supported fields
74 are explained in the illustrated example below:
75
76```json
77[// A list of entries for each input file
78{
79  // Path starting at <android_root>/system/chre to the input file
80  "filename": "chre_api/include/chre_api/chre/wwan.h",
81  // List of #includes that also need to be parsed, e.g. because they provide
82  // struct definitions that are used in the file
83  "includes": [
84    "chre_api/include/chre_api/chre/common.h"
85  ],
86  // Which files the output header should pull in to satisfy dependencies
87  "output_includes": [
88    "chpp/common/common_types.h",
89    "chre_api/chre/wwan.h"
90  ],
91  // A list of entries providing additional information for structs/unions that
92  // appear in the input
93  "struct_info": [
94    {
95      "name": "chreWwanCellInfoResult",
96      // List of annotations for fields within the struct
97      "annotations": [
98        // Instead of copying the input, always force setting a specific value
99        // for a field. The value is whatever should appear in the code for the
100        // assignment statement, or the value to memset it to for an array type.
101        {
102          // Which field within the struct this annotation applies to
103          "field": "version",
104          // The type of annotation we're supplying
105          "annotation": "fixed_value",
106          // Additional information is dependent upon the annotation type
107          "value": "CHRE_WWAN_CELL_INFO_RESULT_VERSION"
108        },
109        // Since the 'cookie' field here is a void*, we're rewriting to a
110        // uint32_t to keep the same structure size + field offsets (on 32-bit
111        // architectures) - in practice we'll also force the value to 0
112        {
113          "field": "cookie",
114          "annotation": "rewrite_type",
115          "type_override": "uint32_t"
116        },
117        // Indicates a variable length array field, with the number of elements
118        // given by another field in the same structure called cellInfoCount
119        {
120          "field": "cells",
121          "annotation": "var_len_array",
122          "length_field": "cellInfoCount"
123        }
124      ]
125    },
126    {
127      "name": "chreWwanCellInfo",
128      "annotations": [
129        // In this case, we have a union field, where only one of the members is
130        // used, according to the provided mapping on the adjacent discriminator
131        // field
132        {
133          "field": "CellInfo",
134          "annotation": "union_variant",
135          "discriminator": "cellInfoType",
136          "mapping": [
137            ["CHRE_WWAN_CELL_INFO_TYPE_GSM", "gsm"],
138            ["CHRE_WWAN_CELL_INFO_TYPE_CDMA", "cdma"],
139            ["CHRE_WWAN_CELL_INFO_TYPE_LTE", "lte"],
140            ["CHRE_WWAN_CELL_INFO_TYPE_WCDMA", "wcdma"],
141            ["CHRE_WWAN_CELL_INFO_TYPE_TD_SCDMA", "tdscdma"],
142            ["CHRE_WWAN_CELL_INFO_TYPE_NR", "nr"]
143          ]
144        }
145      ]
146    }
147  ],
148  // The list of top-level structures appearing in the input that we should
149  // create conversion routines for
150  "root_structs": [
151    "chreWwanCellInfoResult"
152  ]
153}]
154```
155
156## Requirements
157
158Tested with Python 3.7, but most versions of Python 3 should work.
159
160Requires pyclibrary - see requirements.txt.
161