Impossible to access `default_headers` from `ClientOptions` for custom `HttpConnector`

[kylebarron]

object_store: Impossible to access default_headers from ClientOptions; thus impossible to pass on to a custom HTTPService. We should make sure that all values from ClientOptions are accessible for custom http implementations.

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

When trying to implement a custom HttpConnector, the connect method takes in a &ClientOptions to be used for creating an HttpClient.

This implies that you should be able access all properties of ClientOptions in order to define the creation of your custom HttpClient.

However the only public API to access data from a ClientOptions is get_config_value, which is only defined for values of ClientConfigKey. This doesn't include (at least) default_headers. There may be other properties not included.

Describe the solution you'd like

Perhaps we should create accessors for all struct fields inside ClientOptions?

Describe alternatives you've considered

Not forward on all configuration options to user-defined HTTP connectors.

Additional context

I'm prototyping integrating the HttpConnector into obstore — Python bindings to object_store — so that someone can define an arbitrary Python-based HTTP client to make the requests initiated by object_store.