wasip3: Refactor the readdir stream iterators #11615
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Generalize this into a `FallibleStreamProducer` structure * Don't read the entire iterator on the first call to `poll_produce` * Do a blocking read of `dir.entries()` in the original function call to avoid handling state in the iterator itself.
* Start the reading task before iteration starts to move the spawn out of the `poll_*` method. * Rely on fusing behavior of mpsc/tasks to avoid extra state structure. * Specifically handle 0-length reads. Mostly try to refactor the state representation to be more struct-like rather than enum like which is a little easier to follow.
Have at least one test looking at the readdir behavior.
alexcrichton
requested review from
fitzgen
and removed request for
a team and
fitzgen
rvolosatovs
reviewed
Sep 5, 2025
rvolosatovs
approved these changes
Sep 5, 2025
alexcrichton
added a commit
to alexcrichton/wasmtime
that referenced
this pull request
Sep 5, 2025
* wasip3: Refactor `BlockingDirectoryStreamProducer` * Generalize this into a `FallibleStreamProducer` structure * Don't read the entire iterator on the first call to `poll_produce` * Do a blocking read of `dir.entries()` in the original function call to avoid handling state in the iterator itself. * wasip3: Refactor `NonblockingDirectoryStreamProducer` * Start the reading task before iteration starts to move the spawn out of the `poll_*` method. * Rely on fusing behavior of mpsc/tasks to avoid extra state structure. * Specifically handle 0-length reads. Mostly try to refactor the state representation to be more struct-like rather than enum like which is a little easier to follow. * wasip3: Port `preview1_fd_readdir` to WASIp3 Have at least one test looking at the readdir behavior. * Fix `finish` handling in `FallibleIteratorProducer` * Fix a typo
alexcrichton
added a commit
that referenced
this pull request
Sep 5, 2025
* make `Destination::as_direct` work for both host and guest readers (#11612) In order to reduce code duplication (and code paths to test) in `wasmtime-wasi` and custom host embeddings, I've made `Destination::as_direct` (formerly known as `as_direct_destination`) work for host readers as well as guest ones. In the process, I noticed and fixed a couple of related issues: - I had forgotten to implement or test host reader support in `DirectSource` 🤦 - The code to support host-to-host pipes failed to account for partial reads I've also simplified the `StreamConsumer` and `StreamProducer` APIs slightly by having them take their `Source` and `Destination` parameters by value rather than by reference, respectively. Note that, per WebAssembly/component-model#561, I've tweaked the documentation for `StreamProducer` to indicate that implementations might reasonably opt to "pretend" they're ready without buffering any items when handling zero-length reads given that buffering has its own hazards. Likewise, I've updated the `wasi-filesystem` and `wasi-cli` implementations to "pretend" instead of buffering. Signed-off-by: Joel Dice <joel.dice@fermyon.com> * wasip3: Refine 0-length behavior in cli/sockets (#11614) Add some comments for issues and `if` blocks in a few places. I'd like to add some tests but I ran into #11611 so I'll defer tests to later. * wasip3: Refactor the readdir stream iterators (#11615) * wasip3: Refactor `BlockingDirectoryStreamProducer` * Generalize this into a `FallibleStreamProducer` structure * Don't read the entire iterator on the first call to `poll_produce` * Do a blocking read of `dir.entries()` in the original function call to avoid handling state in the iterator itself. * wasip3: Refactor `NonblockingDirectoryStreamProducer` * Start the reading task before iteration starts to move the spawn out of the `poll_*` method. * Rely on fusing behavior of mpsc/tasks to avoid extra state structure. * Specifically handle 0-length reads. Mostly try to refactor the state representation to be more struct-like rather than enum like which is a little easier to follow. * wasip3: Port `preview1_fd_readdir` to WASIp3 Have at least one test looking at the readdir behavior. * Fix `finish` handling in `FallibleIteratorProducer` * Fix a typo --------- Signed-off-by: Joel Dice <joel.dice@fermyon.com> Co-authored-by: Joel Dice <joel.dice@fermyon.com>
bongjunj
pushed a commit
to prosyslab/wasmtime
that referenced
this pull request
Oct 20, 2025
* wasip3: Refactor `BlockingDirectoryStreamProducer` * Generalize this into a `FallibleStreamProducer` structure * Don't read the entire iterator on the first call to `poll_produce` * Do a blocking read of `dir.entries()` in the original function call to avoid handling state in the iterator itself. * wasip3: Refactor `NonblockingDirectoryStreamProducer` * Start the reading task before iteration starts to move the spawn out of the `poll_*` method. * Rely on fusing behavior of mpsc/tasks to avoid extra state structure. * Specifically handle 0-length reads. Mostly try to refactor the state representation to be more struct-like rather than enum like which is a little easier to follow. * wasip3: Port `preview1_fd_readdir` to WASIp3 Have at least one test looking at the readdir behavior. * Fix `finish` handling in `FallibleIteratorProducer` * Fix a typo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-up from #11515, part stylistic part tweaking behavior. Includes a test now too!