Source tree organization¶
This page is intended to give you a quick look of the overall organization of the CLIP OS source tree as managed by the repo tool.
Source tree contents
About the nomenclature of the underlying Git repositories
The Git repositories behind each sub-directory of the CLIP OS source tree have a specific nomenclature intended to reflect the location of the Git repository “check-out” in the CLIP OS source tree.
However, due to a limitation of GitHub-hosted repositories, we could not use
forward slashes (
/) in the Git repository names. Folder separators are
therefore representated by underscores (
_) in those repository names. If
a sub-directory path already contains an underscore character, (e.g., for a
third-party project under
src/external/), then the underlying Git
repository name should have the problematic underscores stripped.
|Sub-directory path||Underlying Git repository name|
.repo directory holds the internal objects and files of the repo
Do not mess with the contents of this directory
.repo directory should not be messed with as it holds all the
.git directories (i.e., the internal Git working directories) of all the
sub-projects of this source tree managed by repo.
The sub-directories of this folder enclose all kinds of binaries or archive files that are required either to build CLIP OS or to make the CLIP OS toolkit work on a development environment that meets the minimal environment requirements.
Providing those assets in the source tree also allows the usage of the CLIP OS toolkit on an offline development setup (e.g., a build environment in a security constrained infrastructure) and thus the lack of dependency on any remote resource.
Git LFS-backed repositories
As most of the underlying directories store large (and often binary) files, most of them rely on Git LFS for the revision of those large files to avoid cluttering their Git repository internal objects.
This directory holds Rust crates from crates.io that
serve to bootstrap and compile
just (handy CLI tool to launch and abstract
sequences of shell commands).
This directory holds all the Debian packages used by the alternative SDK based on a Debian environment.
This directory holds the distfiles from Gentoo which serve for the Gentoo ebuild files.
This directory contains the Gentoo stage3 image that serves to bootstrap the CLIP OS SDK.
This directory holds all the Python packages that are dependencies of the CLIP OS toolkit and need to be installed in the CLIP OS toolkit virtualenv.
This directory encloses all the build by-products that are not required to build CLIP OS from scratch. This directory and its contents can be safely erased from a building machine. In this case, the toolkit will then build entirely from scratch.
The subdirectories in this folder will reflect the
The binary packages produced by Portage/emerge during each build step and that serve to build the target root images populated only with runtime-dependencies (since the only way to deploy Portage packages without any build-dependency is to emerge them only from binary packages, otherwise emerge may need to emerge build-dependencies that may not be required or wanted in the target root).
Another purpose of this directory is to provide emerge with a complete set of
the packages already compiled to speed up builds and image construction by
avoiding pointless identical package rebuilds (
emerge is configured to
properly manage the comparison between the compilation settings, the USE flag
sets of the binary packages and the packages to be deployed in a specified
target root and will trigger a rebuild if those do not match). Usually, on
developers workstations, the contents of this directory are meant to be
populated with the binary packages archive set produced by the continuous
integration server during the nightly build. This will prevent annoying and
time-consuming package builds on the developers’ machines.
This directory encloses all the repo manifest files in charge of constructing the whole CLIP OS source tree.
This is not the manifest check-out directory on which repo is working on.
repo does not work with the manifest files present in that specific
directory but in the manifest files checked out in its internal directory
.repo/manifests/). Do not expect repo to take into account the
changes you can introduce in that directory.
The rationale behind this repository check-out is to serve only as a working
directory for the developer to bring changes (before committing them) in the
manifests files since it seems to be a bad practise to modify directly the
manifests on which
repo operates in
manifest/ directory serves also as a working directory for the
cosmk snapshot feature.
This directory encloses all the build results from commands run by the toolkit.
Watch out to the mount options of your current partition
Implementation detail: since the rootfs of the CLIP OS jails and targets are
built within this directory, it must not be located on a filesystem mount
point with options restricting filesystem features such as the creation of
device nodes or the usage of executable files. In other words, options such
nosuid MUST NOT be set on the underlying
mount point of this directory.
The sub-directories in this folder will reflect the
tree with separation with the version number.
The output results for a given
<recipe> of a specific
About the subdirectory
root/ below directories for the
build, image and configure recipe actions
root directory can be found under the directories dedicated
to the build, image and configure recipe actions. This
directory is the location where the respective recipe actions are working to
build the rootfs.
You need to be careful not to change any file or folder (including modes or
ownerships) under that directory because those changes may end up in the
final built image for the corresponding recipe. It might not be a good idea
to apply changes directly in these
root directories except for
experimenting tweaks while debugging.
This directory holds the project main build and configuration steps in the
Each downstream project based on CLIP OS must create a directory here and mirror part of the CLIP OS directory hierarchy.
How to add a custom product?
Instructions on how to derive this project for your specific use case are available in the How to derive this project guide.
The recipes files in charge of spawning SDK containers making use of the scripts below to build the sub-parts of CLIP OS and bundling them together in a final image or set of installable images.
This directory encloses all the documentation related to the CLIP OS project (i.e., CLIP OS toolkit excluded).
Runtime working directory for the CLIP OS toolkit utilities (i.e. the Python virtual environment location, some tools part of the toolkit working directory, etc.).
Anything under this directory is source code of third-party or in-house projects.
This directory encloses all the repositories of external upstream sources that may receive patches specific to CLIP OS.
This directory encloses all the repositories of the in-house sub-projects which are part of CLIP OS.
This directory encloses all the Portage tree overlays exposed in the SDK.
Third-party Portage tree overlays
Any potential third-party Portage tree overlays must be added here.
The CLIP OS toolkit.
repo_root.justfile: The source tree root (symlinked in the source tree root) justfile.
setup.sh: The script to call to setup the Python virtualenv.
activate: The script to source to activate the CLIP OS toolkit environment/virtualenv.
The Python package/module containing all the source code for the CLIP OS build and source tree management tools developed in Python.
This directory contains all the helpers scripts that are of particular interest for the project development common tasks. These tasks include (but are not limited to) the ability to build the documentation or to process the annotations in the repo manifest files (e.g., for fetching or synchronizing the upstream branches of downstream projects that are part of the CLIP OS source tree or for verifying the OpenPGP signatures of the Git commits).
Scripts under this directory are available in the
When the CLIP OS toolkit environment is activated (see the
script), the scripts under this directory are exposed in the environment
PATH thanks to symbolic links in the CLIP OS toolkit virtualenv.
This directory contains wrapper scripts and resources (e.g. configuration files for common Python checker tools such as mypy or pylint) used for quality assurance of the CLIP OS toolkit.
Justfile with the often used commands and recipes to work on the
project and interact easily with the Git repositories.
Origin of this file
This file is symlinked at the root of the source tree by the toolkit
activation script to source (
toolkit/activate). It won’t show up here
until you have
source’d that specific file.