-
Notifications
You must be signed in to change notification settings - Fork 1k
Opensearch transport query sanitization #15634
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Opensearch transport query sanitization #15634
Conversation
| if (request.getBody() == null) { | ||
| return request.getMethod() + " " + request.getOperation(); | ||
| } else { | ||
| return request.getBody(); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if (request.getBody() == null) { | |
| return request.getMethod() + " " + request.getOperation(); | |
| } else { | |
| return request.getBody(); | |
| } | |
| if (request.getBody() == null) { | |
| return request.getMethod() + " " + request.getOperation(); | |
| } | |
| return request.getBody(); |
| import org.opensearch.client.json.NdJsonpSerializable; | ||
| import org.opensearch.client.transport.GenericSerializable; | ||
|
|
||
| public class OpenSearchBodyExtractor { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Public classes should be declared final where possible (ref)
| public class OpenSearchBodyExtractor { | |
| public final class OpenSearchBodyExtractor { |
| import java.util.Map; | ||
| import java.util.logging.Logger; | ||
|
|
||
| public class OpenSearchBodySanitizer { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| public class OpenSearchBodySanitizer { | |
| public final class OpenSearchBodySanitizer { |
| /** | ||
| * Splits multiple queries separated by semicolons. Single Responsibility: Only responsible for | ||
| * query separation logic. | ||
| */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i dont think is right?
| /** | |
| * Splits multiple queries separated by semicolons. Single Responsibility: Only responsible for | |
| * query separation logic. | |
| */ | |
| /** | |
| * Splits and joins queries for newline-delimited JSON (nd-json) format. | |
| * Splits input by newlines and joins output with semicolons for display. | |
| */ |
| testImplementation("software.amazon.awssdk:identity-spi:2.26.0") | ||
| testImplementation("software.amazon.awssdk:apache-client:2.26.0") | ||
| testImplementation("software.amazon.awssdk:netty-nio-client:2.26.0") | ||
| testImplementation("software.amazon.awssdk:netty-nio-client:2.26.0") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicate
| testImplementation("software.amazon.awssdk:netty-nio-client:2.26.0") |
| public static OpenSearchBodySanitizer create() { | ||
| return new OpenSearchBodySanitizer(DEFAULT_OBJECT_MAPPER); | ||
| } | ||
|
|
||
| public static OpenSearchBodySanitizer create(ObjectMapper objectMapper) { | ||
| return new OpenSearchBodySanitizer(objectMapper); | ||
| } | ||
|
|
||
| public static OpenSearchBodySanitizer getDefault() { | ||
| return DEFAULT_INSTANCE; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this all looks to be unused, is it needed?
| public static OpenSearchBodySanitizer create() { | |
| return new OpenSearchBodySanitizer(DEFAULT_OBJECT_MAPPER); | |
| } | |
| public static OpenSearchBodySanitizer create(ObjectMapper objectMapper) { | |
| return new OpenSearchBodySanitizer(objectMapper); | |
| } | |
| public static OpenSearchBodySanitizer getDefault() { | |
| return DEFAULT_INSTANCE; | |
| } |
| if (request instanceof NdJsonpSerializable) { | ||
| writeNdJson(mapper, (NdJsonpSerializable) request, baos); | ||
| } else if (request instanceof GenericSerializable) { | ||
| ContentType.parse(((GenericSerializable) request).serialize(baos)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is the ContentType.parse needed here? could it just be ((GenericSerializable) request).serialize(baos); ?
| server.enqueue(HttpResponse.of(HttpStatus.OK, MediaType.JSON_UTF_8, searchResponseJson)); | ||
| } | ||
|
|
||
| void setupForMsearchResponse() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is unused
| | System property | Type | Default | Description | | ||
| |-----------------------------------------------------------------|---------| ------- |------------------------------------------------------| | ||
| | `otel.instrumentation.opensearch.experimental-span-attributes` | Boolean | `false` | Enable the capture of experimental span attributes. | | ||
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if it's worth mentioning that this will likely impact performance in some way due to the serialization?
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. | | |
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. **Note**: Enabling this feature adds overhead for JSON serialization and parsing on search requests. | |
Since extracting query body can impact application performance, I made it optional with
otel.instrumentation.elasticsearch.capture-search-query. Not sure if adding this config is okay though.close #15466