Implement atomic hashing and chunking in PulpExport #7220
+352
−50
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vycius
force-pushed
the
atomic-chunked-export
branch
from
83fc336 to
6226daa
Compare
vycius
force-pushed
the
atomic-chunked-export
branch
5 times, most recently
from
6e91cce to
0579e8b
Compare
Replace the external 'split' subprocess and secondary hashing pass with a custom HashingFileWriter. - Resolves a race condition where files moved/deleted between the write and hash phases caused export failures. - Eliminates redundant I/O by calculating checksums while streaming data to disk. - Removes OS dependency on the 'split' binary. - Ensures the Table of Contents (TOC) is generated from in-memory metadata for 100% consistency. closes pulp#7218
vycius
force-pushed
the
atomic-chunked-export
branch
from
0579e8b to
aa02e28
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace the external 'split' subprocess and secondary hashing pass with a custom HashingFileWriter.
closes #7218