Age | Commit message (Collapse) | Author | |
---|---|---|---|
2023-04-04 | local API: fall back to git api, if available | Klaus Aehlig | |
2023-04-04 | clean up targets files | Klaus Aehlig | |
- deduplicate dependencies - remove unused dependency | |||
2023-03-23 | targets: Fix deps structure | Paul Cristian Sarbu | |
2023-03-13 | Storage: Reworked storage and garbage collection | Oliver Reiche | |
The improved GC implementation uses refactored storage classes instead of directly accessing "unknown" file paths. The required storage class refactoring is quite substantial and outlined in the following paragraphs. The module `buildtool/file_system` was extended by: - `ObjectCAS`: a plain CAS implementation for reading/writing blobs and computing digests for a given `ObjectType`. Depending on that type, files written to the file system may have different properties (e.g., the x-bit set) or the digest may be computed differently (e.g., tree digests in non-compatible mode). A new module `buildtool/storage` was introduced containing: - `LocalCAS`: provides a common interface for the "logical CAS", which internally combines three `ObjectCAS`s, one for each `ObjectType` (file, executable, tree). - `LocalAC`: implements the action cache, which needs the `LocalCAS` for storing cache values. - `TargetCache`: implements the high-level target cache, which also needs the `LocalCAS` for storing cache values. - `LocalStorage`: combines the storage classes `LocalCAS`, `LocalAC`, and `TargetCache`. Those are initialized with settings from `StorageConfig`, such as the build root base path or number of generations for the garbage collector. `LocalStorage` is templated with a Boolean parameter `kDoGlobalUplink`, which indicates that, on every read/write access, the garbage collector should be used for uplinking across all generations (global). - `GarbageCollector`: responsible for garbage collection and the global uplinking across all generations. To do so, it employs instances of `LocalStorage` with `kDoGlobalUplink` set to false, in order to avoid endless recursion. The actual (local) uplinking within two single generations is performed by the corresponding storage class (e.g., `TargetCache` implements uplinking of target cache entries between two target cache generations etc.). Thereby, the actual knowledge how data should be uplinked is implemented by the instance that is responsible for creating the data in the first place. | |||
2023-01-20 | Add local garbage collection | Sascha Roloff | |
2023-01-20 | Move execution-backend-id calculation from target-level cache to local config | Sascha Roloff | |
This code movement is required to break a cyclic dependency coming with the introduction of the garbage collector. target_cache depends on garbage_collector and garbage_collector would depend on target_cache to determine the target-level-cache directory. After moving this calculation to a more general location, the cycle is broken. | |||
2023-01-20 | Introduce multiple cache generations | Sascha Roloff | |
2023-01-20 | Move file_storage.hpp to file_system subdirectory | Sascha Roloff | |
2022-11-24 | Fix root directory upload to honor tree invariant. | Sascha Roloff | |
2022-11-21 | Use the newly-added concept of private-deps | Klaus Aehlig | |
While there, also add all direct dependencies explicitly; using directly dependencies that are pulled in only indireclty causes problems from a maintainability point of view. | |||
2022-08-05 | LocalExecution: Use Git tree format | Oliver Reiche | |
- LocalStorage Add tree CAS and support reading Git trees - LocalAction: Create Git tree for output directory - LocalApi: Support availability and upload of Git trees - LocalStorage: Support dumping tree to stream in native mode | |||
2022-04-21 | add compatibility with original remote build protocol | Alberto Sartori | |
2022-03-16 | Format: Apply compact JSON formatting | Oliver Reiche | |
2022-03-08 | SystemCommand: Move to new module "src/buildtool/system" | Oliver Reiche | |
2022-02-22 | Initial self-hosting commit | Klaus Aehlig | |
This is the initial version of our tool that is able to build itself. In can be bootstrapped by ./bin/bootstrap.py Co-authored-by: Oliver Reiche <oliver.reiche@huawei.com> Co-authored-by: Victor Moreno <victor.moreno1@huawei.com> |