summaryrefslogtreecommitdiff
path: root/test/buildtool/storage
AgeCommit message (Collapse)Author
2025-06-13Test: Fix flaky LargeObjectCAS test.Maksim Denisov
Due to a random nature of the LargeObjectUtils generator, it may generate 2 identical files in a row. To prevent the test from failing, check that a newly generated file doesn't collide with any already added to the CAS.
2025-06-12GarbageCollector: Support removal of all generations at onceMaksim Denisov
...that ignores compactification.
2025-03-24TmpDir: minor refactoringMaksim Denisov
2025-02-28Include ArtifactDigestFactory into "common" libraryMaksim Denisov
2025-02-20Separate off id generation to a separate libraryKlaus Aehlig
... and rename appropriately to reflect contents more precisely than the generic "common". This separation also disentangles dependencies a bit.
2024-12-09Move garbage_collector to a separate libraryMaksim Denisov
2024-11-14tests: Implement IWYU suggestionsMaksim Denisov
2024-10-25Add dependencies explicitly that are included directlyKlaus Aehlig
... instead of relying on those dependencies being pulled in indirectly.
2024-10-08Name value template parameters using kCamelCase.Maksim Denisov
2024-10-08Name static constants using kCamelCase.Maksim Denisov
2024-10-07Enable google-* checks.Maksim Denisov
2024-10-07Enable bugprone-implicit-widening-of-multiplication-result check.Maksim Denisov
2024-09-23Reorder dependencies and remove duplicates in OSSMaksim Denisov
2024-09-13Use TestHashType in testsMaksim Denisov
...instead of calling ProtocolTraits::IsCompatible
2024-09-13Rename Compatibility class to ProtocolTraitsMaksim Denisov
...and move it to the common stage.
2024-09-09Use ArtifactDigestFactory casts in StorageMaksim Denisov
2024-09-09Replace ArtifactDigest::CreateMaksim Denisov
...with ArtifactDigestFactory::HashDataAs
2024-09-09Replace ArtifactDigest::CreateFromFileMaksim Denisov
...with ArtifactDigestFactory::HashFileAs
2024-08-30Replace bazel_re::Digest in BazelMsgFactory (local trees)Maksim Denisov
...with ArtifactDigest.
2024-08-30Replace bazel_re::Digest in LocalACMaksim Denisov
...with ArtifactDigest.
2024-08-30Replace bazel_re::Digest in LocalCAS::Split/SpliceMaksim Denisov
...with ArtifactDigest.
2024-08-30Replace bazel_re::Digest in LocalCAS::{...}PathMaksim Denisov
...with ArtifactDigest.
2024-08-30Remove blob_creator lib from testsMaksim Denisov
...and move this functionality to bazel_msg_factory_test, where it is actually used. For local_cas.test the regular hashing is used, since blob_creator is redundant there.
2024-08-27Reformat code to comply with clang-format 18Klaus Aehlig
... while keeping our .clang-format file.
2024-08-21tests: use newly defined test suiteKlaus Aehlig
... so that linting information gets propagated properly.
2024-07-22Store HashFunction in StorageConfigMaksim Denisov
2024-07-22Pass HashFunction to ArtifactDigest::CreateMaksim Denisov
2024-07-16Check compatibility in the test of large_object_casMaksim Denisov
2024-07-05Convert StorageConfig to a general classMaksim Denisov
2024-07-05Remove HermeticLocalTestFixtureMaksim Denisov
...and create StorageConfig and Storage in place if needed.
2024-07-05Convert Storage to a general classMaksim Denisov
2024-07-05Call uplinking via UplinkerMaksim Denisov
... instead of static calls to GarbageCollector
2024-07-05Use StorageConfig with generation for initialization of Storage's generationsMaksim Denisov
...instead of std::filesystem::path. StorageConfig is extended to return paths of Storage's parts.
2024-07-05Pass StorageConfig to GarbageCollectionMaksim Denisov
2024-07-05Use StorageConfig functionality via Instance()Maksim Denisov
...to track changes during refactoring easier.
2024-06-28Use (un)expected for CAS accessOliver Reiche
2024-06-07Remove unused codeMaksim Denisov
2024-05-28Rename BlobContainer to BazelBlobContainerMaksim Denisov
2024-05-28Move CreateBlobFromPath since it is only used in tests.Maksim Denisov
2024-05-27Fix headers in local_cas and bazel_network.Maksim Denisov
2024-04-22Compactification: Remove invalid entries from the storage.Maksim Denisov
During compactification, invalid entries must be deleted.
2024-04-17Compactification: Split large entries.Maksim Denisov
During garbage collection split and remove from the storage every entry that is larger than a threshold.
2024-04-17Compactification: Remove spliced entries.Maksim Denisov
During garbage collection remove from the storage every entry that has the large entry.
2024-04-15LargeBlobs: Skip splicing of dependent objects during uplinking of AC, TC ↵Maksim Denisov
and trees.
2024-04-15LargeBlobs: Make LocalCAS::BlobPathNoSync public to fix synchronization of ↵Maksim Denisov
executable files during splitting.
2024-04-12file chunker: increase chunk sizesKlaus Aehlig
As we use chunking also for reducing storage, we have to consider the overhead of block devices which is in the order of kB per file. So our target chunk size should be at least 2 orders of magnitude above this. This suggests to minimally aim for a chunk size of 128kB, a target size that also has the advantage the that maximal chunk size associated with this size is 1MB which is still well below the maximal transmission size of grpc allowing us to avoid the streaming API. As we're scaling everything up by a factor of 16, we also have to increase the number of bits in the involved masks by 4. We use this to also extend the window size by using the 2 most significant octets. Following the advice of the paper proposing FastCDC to spread out the ones roughly equally suggests 0x4444 as a suitable value for the two most significant octets. We also change the suggested extension of the remote-execution API accordingly. As the precise parameters for FastCDC when announced over the remote-execution APIs are still under discussion upstream, we simplify the name to not mention the target size.
2024-04-08test: Add missing includes and fix depsPaul Cristian Sarbu
2024-04-02LargeBlobs: Splice large objects from external sources.Maksim Denisov
For splicing of large objects from external sources additional checks are performed: * The digest of the spliced result must be equal to the expected digest; * The parts of a spliced tree must be in the storage. Tested: * Regular splicing of large objects; * If the result is unexpected, splicing fails; * If some parts of a tree are missing, splicing fails.
2024-04-02LargeBlobs: Uplink large objects.Maksim Denisov
* Uplink parts of the large entry before entry itself; * Uplink large entries in LargeObjectCAS::GetEntryPath to not split things two times; * Promote spliced tree during uplinking of a large tree entry to properly promote parts of the tree; * Uplink large entries in LocalUplink{Blob, Tree} to support proper uplinking in Action Cache and Target Cache; Tested: * Uplink large blobs and trees; * Uplink a large object that depends on other large objects.
2024-04-02LargeBlobs: Splice large objects implicitly.Maksim Denisov
Implicitly reconstruct objects during regular uplinking of Blobs/Trees.
generated by cgit v1.2.3 (git 2.39.1) at 2025-07-03 16:39:42 +0000