Clarification on Aynchronous programming

[wl-spaceline]

Hi,
In my application I need to use send objects to a S3 bucket asynchronously, to don't block the main thread. I went through this tutorial to know how to use the PutObjectAsynch properly, and there is in particular a sentence in the tutorial that I would like to understand better:

The application itself cannot terminate until the asynchronous operation completes.

Assuming that I can grant that the S3Clent and the PutObjectRequest are in scope for the entire duration of the transfer (until the application is running), my questions are the following:

• What are the implications of not doing that (a part that obviously I risk to don't have all the objects in the bucket)?
• Is this a matter of resource handling?
• What if it is acceptable to miss some objects, while I want to prioritize to stop the application (I might have multiple running transfers and I don't want to wait to be completed).
• Is there any downside of defining the callback handler as lambda function? Something like the following:

void put_async(const S3Client& client, const PutObjectRequest& request) {
   ...
   s3Client.PutObjectAsync(
        request,
        [](const Aws::S3::S3Client*,
                  const Aws::S3::Model::PutObjectRequest&,
                  const Aws::S3::Model::PutObjectOutcome& outcome,
                  const std::shared_ptr<const Aws::Client::AsyncCallerContext>&)
        {
             //Callback logic here.
        });
  ...
}

Thank you in advance for any input!

[sbaluja]

Hi. Thanks for your question.

What are the implications of not doing that (apart that obviously I risk to don't have all the objects in the bucket)?

There are a couple things here:

• Calling Aws::ShutdownAPI() during an async operation would likely result in the in-flight operations to return errors about client initialization - this will be a graceful exit. However, it's important to note that Aws::ShutdownAPI() will still wait for threads to finish completion in order to properly cleanup resources.
• I know you mentioned that this is not your scenario but if the client or request go out of scope and are destructed during an operation, you'll likely see far worse issues, likely even a crash as the SDK expects these objects to exist during operations. This is the main intent behind the sentence in the tutorial you've quoted.

Is this a matter of resource handling?

Yes, It's primarily about resource handling.

What if it is acceptable to miss some objects, while I want to prioritize to stop the application (I might have multiple running transfers and I don't want to wait to be completed).

As noted earlier, you can use Aws::ShutdownAPI(), which will still wait for threads to finish completion in order to properly cleanup resources, and should error out any in-progress operations. There isn't really a way to immediately terminate the application while maintaining proper resource collection. If this isn't enough for your use case - I would suggest looking into SetContinueRequestHandler. This can be used to 'cancel' requests early.

Is there any downside of defining the callback handler as lambda function?

Lambdas are fine, the only thing to be careful about is captures (potential for lifetime issues). Consider the following example:

// DANGEROUS: Capturing by reference
void bad_example() {
    Aws::String localData = "some data";
    PutObjectRequest request;
    
    client.PutObjectAsync(request, 
        [&localData](/* params */) {  // BAD: localData may be destroyed
            std::cout << localData << std::endl;  // Undefined behavior
        });
}  // localData destroyed here, but callback hasn't run yet

// SAFE: Capture by value
void good_example() {
    Aws::String localData = "some data";
    PutObjectRequest request;
    
    client.PutObjectAsync(request, 
        [localData](/* params */) {  // GOOD: Copy is made
            std::cout << localData << std::endl;
        });
}

// SAFE: Using shared_ptr for complex objects
void good_example_object() {
    auto sharedData = Aws::MakeShared<LargeObject>("LargeObjectLoggingTag");
    PutObjectRequest request;
    
    client.PutObjectAsync(request, 
        [sharedData](/* params */) {  // GOOD: shared_ptr keeps object alive
            sharedData->process();
        });
}

[wl-spaceline]

Thank you @sbaluja for your answer! It was very helpful to have a better understanding. I can guarantee that the request and context doesn't go out of scope to avoid UB, but I did not considered the Aws::ShutdownAPI(). I can focus on that now, thanks!