1# Introduction
2
3The `sym_upload` tool is able to operate in `sym-upload-v2` protocol mode, in
4addition to the legacy protocol (which will be referred to as `sym-upload-v1`
5for the rest of this document). For now `sym-upload-v2` is HTTP/REST-based but
6it could be extended to operate over gRPC instead, in the future.
7
8# Table of Contents
9* [Why](#why)
10* [How](#how)
11   * [Uploading](#uploading)
12     * [Uploading with `sym_upload`](#uploading-with-sym_upload)
13     * [Uploading with curl](#uploading-with-curl)
14     * [Serving the `sym-upload-v2` protocol](#serving-the-sym-upload-v2-protocol)
15       * [Authenticate using `key`](#authenticate-using-key)
16       * [Symbol `checkStatus`](#symbol-checkstatus)
17       * [Upload `create`](#upload-create)
18       * [Uploading the symbol file](#uploading-the-symbol-file)
19       * [Upload complete](#upload-complete)
20
21
22# Why
23
24Using `sym_upload` in `sym-upload-v2` protocol mode has the following features
25beyond `sym-upload-v1`:
26  * Authentication via `key` (arbitrary secret).
27  * Symbol identifier (product of `debug_file` and `debug_id`, as recorded in
28output from `dump_syms`) can be checked against existing symbol information on
29server. If it's present, then the upload is skipped entirely.
30
31# How
32
33## Uploading
34
35### Uploading with `sym_upload`
36
37Uploading in `sym-upload-v2` protocol mode is easy. Invoke `sym_upload` like
38```
39$ ./sym_upload -p sym-upload-v2 [-k <API-key>] <symbol-file> <API-URL>
40```
41
42Where `symbol-file` is a symbol file created by `dump_syms`, `API-URL` is the
43URL of your `sym-upload-v2` API service (see next section for details), and
44`API-key` is a secret known to your uploader and server.
45
46For more options see `sym_upload --help`.
47
48### Uploading with curl
49
50As an example, if:
51  * Your API's URL was "https://sym-upload-api".
52  * Your service has assigned you `key` "myfancysecret123".
53  * You wanted to upload the symbol file at "path/to/file_name", with
54`debug_file` being "file_name" and `debug_id` being
55"123123123123123123123123123". Normally you would read these values from
56"path/to/file_name", which in turn was generated by `dump_syms`.
57
58Then you might run:
59```
60$ curl https://sym-upload-api/symbols/file_name/123123123123123123123123123:checkStatus?key=myfancysecret123
61```
62
63And, upon seeing that this `debug_file`/`debug_id` combo is missing from symbol
64storage then you could run:
65```
66$ curl --request POST https://sym-upload-api/uploads:create?key=myfancysecret123
67```
68
69Which returns `upload_url` "https://upload-server/42?creds=shhhhh" and
70`upload_key` "42". Next you upload the file directly like:
71```
72$ curl -T path/to/file_name "https://upload-server/42?creds=shhhhh"
73```
74
75Once the HTTP PUT is complete, run:
76```
77$ curl --header "Content-Type: application/json" \
78    --request POST \
79    --data '{symbol_id:{"debugFile":"file_name",'\
80        '"debugId":"123123123123123123123123123"}}' \
81    https://sym-upload-api/uploads/42:complete?key=myfancysecret123
82```
83
84### Serving the `sym-upload-v2` Protocol
85
86The protocol is currently defined only in HTTP/REST. There are three necessary
87REST operations to implement in your service:
88* `/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>`
89* `/uploads:create?key=<key>`
90* `/uploads/<upload_key>:complete?key=<key>`
91
92#### Authenticate Using `key`
93
94The query string arg `key` contains some secret that both the uploader and
95server understand. It is up to the service implementer to decide on what
96constitutes a valid `key`, how the uploader acquires one, and how to handle
97requests made with invalid ones.
98
99#### Symbol `checkStatus`
100
101```
102/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>
103```
104
105This operation expects an empty (or no) JSON payload in the request.
106
107This operation should return the status of the symbol file uniquely identified
108by the given `debug_file` and `debug_id`. JSON schema:
109```
110{
111    "type": object",
112    "properties": {
113        "status": {
114            "type": "string",
115            "enum": ["STATUS_UNSPECIFIED", "MISING", "FOUND"],
116            "required": true
117        }
118    }
119}
120```
121
122Where `MISSING` denotes that the symbol file does not exist on the server and
123`FOUND` denotes that the symbol file exists on the server.
124
125#### Upload `create`
126
127```
128/uploads:create?key=<key>
129```
130
131This operation expects an empty (or no) JSON payload in the request.
132
133This operation should return a URL that uploader can HTTP PUT their symbol file
134to, along with an "upload key" that can be used to notify the service once the
135file upload is completed. JSON schema:
136```
137{
138    "type": "object",
139    "properties": {
140        "upload_url": {
141            "type: "string",
142            "required": true
143        },
144        "upload_key": {
145            "type": "string",
146            "required": true
147        }
148    }
149}
150```
151
152Since this REST API operation can be authenticated via the `key` query string
153arg, the service can return a URL that encodes permission delegation to the
154upload endpoint resource and thereby constrain the ability to upload to those
155with valid `key`s.
156
157#### Uploading the Symbol File
158
159Note that the actual symbol upload step is _not_ part of the REST API. The
160upload URL obtained in the above operation is meant to be used as the endpoint
161for a normal HTTP PUT request for the contents of the symbol file. Once that
162HTTP PUT request is completed use the upload `complete` operation.
163
164#### Upload `complete`
165
166```
167/uploads/<upload_key>:complete?key=<key>
168```
169
170This operation expects a JSON payload in the HTTP request body with the
171following schema:
172```
173{
174    "type": "object",
175    "properties": {
176        "symbol_id": {
177            "type": "object",
178            "properties": {
179                "debug_file": {
180                    "type": "string",
181                    "required": true
182                },
183                "debug_id": {
184                    "type": "string",
185                    "required": true
186                }
187            }
188        }
189    }
190}
191```
192
193This operation should cause the symbol storage back-end (however implemented)
194to consume the symbol file identified by `upload_key`. It is up to the service
195implementation to decide how uploads are assigned `upload_key`s and how to
196retrieve a completed upload by its `upload_key`. If the symbol file cannot be
197found, is malformed, or the operation cannot be completed for any other reason
198then an HTTP error will be returned. JSON schema of non-error responses:
199```
200{
201    "type": "object",
202    "properties": {
203        "result": {
204            "type": string,
205            "enum": ["RESULT_UNSPECIFIED", "OK", "DUPLICATE_DATA"],
206            "required": true
207        }
208    }
209}
210```
211
212Where `OK` denotes that the symbol storage was updated with the new symbol file
213and `DUPLICATE_DATA` denotes that the symbol file data was identical to data
214already in symbol storage and therefore nothing changed.
215