1Updating Platform Toolchains
2============================
3
4Updating Toolchain Source
5-------------------------
6
7The following process is done in the Android LLVM tree. To fetch the sources:
8
9    repo init -u https://android.googlesource.com/platform/manifest -b llvm
10
11    # Googlers, use
12    repo init -u \
13        persistent-https://android.git.corp.google.com/platform/manifest -b llvm
14
15Loop over llvm, clang, compiler-rt (in this order):
16
171. We are working from a separate untracked/merged branch called *aosp/dev*.
18
19        git branch -D working_dev
20        repo start working_dev .
21
222. **OPTIONAL FIXUPS**.
23   These aren't really necessary if you remember to always keep *aosp/dev* and
24   *aosp/master* synchronized otherwise, but very often someone will forget to
25   merge back a change.
26
27   1. Grab the squashed commit that went into *aosp/master* and mark it
28      committed to *aosp/dev* too.
29
30      **Note**: If there were changes to *aosp/master* before the squashed
31      commit, grab those changes (using step 2), before applying this step,
32      and finally repeat step 2 for changes after the squashed commit.
33
34          git branch -D clean_master
35          git checkout -b clean_master <SHA_FOR_SQUASH>
36          git checkout working_dev
37          git merge -s ours clean_master
38          git push aosp refs/heads/working_dev:refs/heads/dev
39          git branch -D clean_master
40
41   2. Grab all outstanding changes that went into *aosp/master* and put them
42      into *aosp/dev* too.
43
44          git branch -D clean_master
45          git checkout -b clean_master aosp/master
46          git checkout working_dev
47          git merge clean_master
48          git push aosp refs/heads/working_dev:refs/heads/dev
49          git branch -D clean_master
50
513. Merge the upstream branch.
52   Use `git log aosp/upsteam-master` to browse upstream commits and find a SHA.
53
54       git merge <upstream_sha>
55
564. Fix conflicts.
57
585. Update build rules and commit that patch on top.
59
606. Test everything before pushing.
61
627. Submit your work to *aosp/dev*.
63
64       git push aosp refs/heads/working_dev:refs/heads/dev
65
668. Squash your work for *aosp/master*.
67
68       repo start update_38 .
69       git merge --squash working_dev
70       git commit -a
71       repo upload .
72
739. Test everything before submitting the patch from the previous step.
74
7510. Grab the squashed commit and replay it in *aosp/dev*.
76
77        repo sync .
78        git remote update
79        git branch -D clean_master
80        git checkout -b clean_master aosp/master
81        git checkout working_dev
82
83    Use `-s ours` to ensure that we skip the squashed set of changes.
84    If/when we forget this, we have to do it later.
85
86        git merge -s ours clean_master
87        git push aosp refs/heads/working_dev:refs/heads/dev
88        git branch -D clean_master
89
9011. Clean up after our working branch.
91
92        git checkout --detach
93        git branch -D working_dev
94
95This works better because we can keep full history in *aosp/dev*, while
96maintaining easy reverts/commits through *aosp/master*.
97
98
99Generating New Prebuilts
100------------------------
101
1021. Run the toolchain build script. This will perform a two stage build and
103   create a tarball of the final toolchain.
104
105        python external/clang/build.py
106
1072. The just built toolchain can be tested in an existing AOSP tree by invoking
108   make with:
109
110        make \
111            LLVM_PREBUILTS_VERSION=clang-dev \
112            LLVM_PREBUILTS_BASE=/path/to/llvm/out/install
113
114   This will use the just built toolchain rather than the one in **prebuilts/**.
115   If you used something other than the default for `--build-name`, use
116   `clang-$BUILD_NAME` instead of `clang-dev`.
117
1183. Once the updates have been verified, upload to gerrit, review, submit. The
119   build server will pick up the changes and build them. The LLVM build page is
120   http://go/clang-build. Sorry, Googlers only (for now) :(
121
1224. To update the platform compiler, download the selected package from the build
123   server and extract them to the appropriate prebuilts directory. The new
124   directory will be named "clang-BUILD\_NUMBER".
125
1265. Update `LLVM\_PREBUILTS\_VERSION` in `build/core/clang/config.mk` to match
127   the new prebuilt directory. We typically keep around two versions of the
128   toolchain in prebuilts so we can easily switch between them in the build
129   system rather than needing to revert prebuilts. This also allows developers
130   that need new toolchain features to take advantage of them locally while
131   validation for the new compiler is still in progress.
132
1336. Rebuild/test everything one more time to ensure correctness.
134
135   Make sure you check *goog/master* as well as *aosp/master*.
136
137   There may be necessary fixups here, to handle .ll reading or other projects
138   where new warnings/errors are firing.
139
140        m -j48 checkbuild
141
1426. Upload the changes produced in **prebuilts/clang/host**.
143   This may entail more than a simple `git commit -a`, so look at `git status`
144   before finally uploading/committing.
145
146       repo start updated_toolchain .
147       git add clang-BUILD_NUMBER
148       git commit
149       repo upload --cbr .
150
1517. Submit CLs.
152
153
154Testing Checklist
155-----------------
156
1571. Do a checkbuild.
1582. Go to **external/llvm** and run `./android_test.sh` (no known failures as of
159   2015-10-08).
1603. Ensure successful build for all architectures: 32- and 64- bit ARM, x86 and
161   Mips.
1624. Run ART host tests.
163   This was broken by a rebase once, and worth testing after every rebase.
164
165       croot && cd art && mma -j40 test-art-host
166
1675. Run ART device tests.
168
169       croot && cd art && mma -j4 test-art-device
170
171
172Checklist for CLs
173-----------------
174
175The following projects will almost always have CLs as a part of the rebase.
176Depending on the changes in LLVM, there might be updates to other projects as
177well.
178
179* External projects
180
181  * **external/clang**
182  * **external/compiler-rt**
183  * **external/llvm**
184
185* Prebuilts
186
187  * **prebuilts/clang/host/darwin-x86/**
188  * **prebuilts/clang/host/linux-x86/**
189  * **prebuilts/clang/host/windows-x86/**
190