Introduction to Apollo Release Build¶
Background¶
For quite some time, binary distribution was unavailable for CyberRT-based Apollo. Users have to build the whole project themselves inside Apollo Dev container before they can run various modules and tools. This incapability of deployment causes trouble in certain situations. For example, people from SVL Simulator find that the total size of all Docker images, volumes and Bazel caches and build artifacts sums up to 40+ GB, which was rather inconvenient for their use case.
Principle of Release Build Implementation¶
The root cause of this incapability was Bazel’s lack of out-of-box installation
support similar to those provided in other build systems (e.g., make install
).
To resolve this issue, we borrowed ideas from the
Drake project, and implemented the
install
extension in the Starlark language, which can be used to install
binaries, shared libraries, resource files (config, data, launch files, dags,
etc) and documents for the Apollo project.
Installation for standalone binaries was rather straightforward. As you may
already know, the core concept of the CyberRT framework was to load each module
(e.g, Perception, Prediction, Planning, etc) dynamically as a component in the
form of libX_component.so
. Both the mainboard
binary program and
libX_component.so
links thousands of other shared libraries themselves. For
example, running ldd
on the Planning module with the following command
ldd bazel-bin/modules/planning/libplanning_component.so
will show the following output:
linux-vdso.so.1 (0x00007ffc8a77c000)
libmodules_Splanning_Slibplanning_Ucomponent_Ulib.so => /apollo/bazel-bin/modules/planning/../../_solib_local/libmodules_Splanning_Slibplanning_Ucomponent_Ulib.so (0x00007fe8a7f9f000)
libmodules_Splanning_Slibnavi_Uplanning.so => /apollo/bazel-bin/modules/planning/../../_solib_local/libmodules_Splanning_Slibnavi_Uplanning.so (0x00007fe8a7d81000)
libmodules_Splanning_Slibon_Ulane_Uplanning.so => /apollo/bazel-bin/modules/planning/../../_solib_local/libmodules_Splanning_Slibon_Ulane_Uplanning.so (0x00007fe8a7b53000)
libmodules_Splanning_Slibplanning_Ubase.so => /apollo/bazel-bin/modules/planning/../../_solib_local/libmodules_Splanning_Slibplanning_Ubase.so (0x00007fe8a7945000)
libmodules_Splanning_Scommon_Ssmoothers_Slibsmoother.so => /apollo/bazel-bin/modules/planning/../../_solib_local/libmodules_Splanning_Scommon_Ssmoothers_Slibsmoother.so (0x00007fe8a7739000)
libmodules_Splanning_Splanner_Slibplanner_Udispatcher.so => /apollo/bazel-bin/modules/planning/../../_solib_local/libmodules_Splanning_Splanner_Slibplanner_Udispatcher.so (0x00007fe8a752e000)
...
How to install libplanning_component.so
with all the shared objects (i.e.,
“*.so” files) it links becomes the hardest issue to solve to implement the
install
rule.
patchelf
comes to rescue. All the shared objects can be retrieved from
runfiles_data
, while patchelf --force-rpath --set-rpath
can be used to
change RPATH settings.
Refer to tools/install/install.bzl for a thorough understanding.
How to Run Release Build for Apollo¶
To generate binary outputs for Apollo, simply run:
./apollo.sh release -c
where -c
is an optional argument used for pre-cleaning. The output was located
at /apollo/output
inside Apollo Dev container.
The command above is roughly equivalent to the following:
bazel run --config=opt --config=gpu //:install \
-- --pre_clean /apollo/output
Please type ./apollo.sh release -h
for more usage of apollo.sh release
.
How to Run Apollo with Release Build Output¶
In order to run Apollo Runtime Docker with release build output, type the following command from the root of release build output directory outside Apollo Dev Docker:
bash docker/scripts/runtime_start.sh
For users in China, use -g cn
to speed up pulling of Docker images/volumes.
bash docker/scripts/runtime_start.sh -g cn
Log into Apollo Runtime Docker by running:
bash docker/scripts/runtime_into.sh
Start Dreamview by running:
./scripts/bootstrap.sh
from inside Apollo Runtime container.
How to Use install
Rule to Install a Custom Module¶
To install a custom module, you can follow examples for installing other modules from the Apollo repository.
Take the Planning module as an example.
This is part of the topmost BUILD file:
install(
name = "install",
deps = [
"//cyber:install",
# ...
"//modules/planning:install",
# ...
],
)
This is a snippet from modules/planning/BUILD:
filegroup(
name = "planning_conf",
srcs = glob([
"conf/**",
]),
)
filegroup(
name = "runtime_data",
srcs = glob([
"dag/*.dag",
"launch/*.launch",
]) + [":planning_conf"],
)
install(
name = "install",
data = [
":runtime_data",
],
targets = [
":libplanning_component.so",
],
deps = [
"//cyber:install",
],
)
Arguments of the install
Bazel Rule¶
From tools/install/install.bzl:
install = rule(
attrs = {
"deps": attr.label_list(providers = [InstallInfo]),
"data": attr.label_list(allow_files = True),
"data_dest": attr.string(default = "@PACKAGE@"),
"data_strip_prefix": attr.string_list(),
"targets": attr.label_list(),
"library_dest": attr.string(default = "@PACKAGE@"),
"library_strip_prefix": attr.string_list(),
"mangled_library_dest": attr.string(default = "lib"),
"mangled_library_strip_prefix": attr.string_list(),
"runtime_dest": attr.string(default = "bin"),
"runtime_strip_prefix": attr.string_list(),
"rename": attr.string_dict(),
"install_script_template": attr.label(
allow_files = True,
executable = True,
cfg = "target",
default = Label("//tools/install:install.py.in"),
),
},
executable = True,
implementation = _install_impl,
)
The detailed attributes of the install
rule was listed below:
Argument |
Notes |
---|---|
deps |
List of other install rules that this rule should include. |
data |
List of (platform-independent) resource files to install |
data_dest |
Destination for resource files (default = “@PACKAGE@”) |
data_strip_prefix |
List of prefixes to remove from resource paths |
targets |
List of targets to install |
runtime_dest |
Destination for executable targets (default = “bin”) |
runtime_strip_prefix |
List of prefixes to remove from executable paths |
rename |
Mapping of install paths to alternate file names, used to rename files upon installation |
Limitations of Current Release Build Implementation¶
C++ support only. Installation for Python was not ready.
x86_64 support only. ARM64 support was not ready at the moment.