Releases: aws-powertools/powertools-lambda-python
v.1.7.0
Changes
Event source data classes
This release adds support for Cognito Authentication Challenge Lambda Triggers for create, define and verify trigger sources. It also enhances get_header_value method by optionally supporting case insensitive headers.
Credits: Thanks to @michaelbrewer from Gyft for both enhancements.
Parser utility
Parser is a new utility that provides parsing and deep data validation using Pydantic Models - It requires an optional dependency (Pydantic) to work.
It uses Pydantic Model classes, similar to dataclasses, to model the shape of your data, enforce type hints at runtime, serialize your models to JSON, JSON Schema, and with third party tools you can also auto-generate Model classes from JSON, YAML, OpenAPI, etc.
from aws_lambda_powertools.utilities.parser import event_parser, BaseModel, ValidationError
from aws_lambda_powertools.utilities.typing import LambdaContext
import json
class OrderItem(BaseModel):
id: int
quantity: int
description: str
class Order(BaseModel):
id: int
description: str
items: List[OrderItem] # nesting models are supported
optional_field: Optional[str] # this field may or may not be available when parsing
@event_parser(model=Order)
def handler(event: Order, context: LambdaContext):
assert event.id == 10876546789
assert event.description == "My order"
assert len(event.items) == 1
order_items = [items for item in event.items]
...
payload = {
"id": 10876546789,
"description": "My order",
"items": [
{
"id": 1015938732,
"quantity": 1,
"description": "item xpto"
}
]
}
handler(event=payload, context=LambdaContext())
# also works if event is a JSON string
handler(event=json.dumps(payload), context=LambdaContext()) With this release, we provide a few built-in models to start with such as Amazon EventBridge, Amazon DynamoDB, and Amazon SQS - You can extend them by inheriting and overriding their properties to plug-in your models.
from aws_lambda_powertools.utilities.parser import parse, BaseModel
from aws_lambda_powertools.utilities.parser.models import EventBridgeModel
from typing import List, Optional
class OrderItem(BaseModel):
id: int
quantity: int
description: str
class Order(BaseModel):
id: int
description: str
items: List[OrderItem]
# Override `detail` key of a custom event in EventBridge from str to Order
class OrderEventModel(EventBridgeModel):
detail: Order
payload = {...} # EventBridge event dict with Order inside detail as JSON
order = parse(model=OrderEventModel, event=payload) # parse input event into OrderEventModel
assert order.source == "OrderService"
assert order.detail.description == "My order"
assert order.detail_type == "OrderPurchased" # we rename it to snake_case since detail-type is an invalid name
# We can access our Order just as fine now
for order_item in order.detail.items:
...
# We can also serialize any property of our parsed model into JSON, JSON Schema, or as a Dict
order_dict = order.dict()
order_json = order.json()
order_json_schema_as_dict = order.schema()
order_json_schema_as_json = order.schema_json(indent=2)Similar to Validator utility, it provides an envelope feature to parse known structures that wrap your event. It's useful when you you want to parse both the structure and your model but only return your actual data from the envelope.
Example using one of the built-in envelopes provided from day one:
from aws_lambda_powertools.utilities.parser import event_parser, parse, BaseModel, envelopes
from aws_lambda_powertools.utilities.typing import LambdaContext
class UserModel(BaseModel):
username: str
password1: str
password2: str
payload = {
"version": "0",
"id": "6a7e8feb-b491-4cf7-a9f1-bf3703467718",
"detail-type": "CustomerSignedUp",
"source": "CustomerService",
"account": "111122223333",
"time": "2020-10-22T18:43:48Z",
"region": "us-west-1",
"resources": ["some_additional_"],
"detail": {
"username": "universe",
"password1": "myp@ssword",
"password2": "repeat password"
}
}
ret = parse(model=UserModel, envelope=envelopes.EventBridgeModel, event=payload)
# Parsed model only contains our actual model, not the entire EventBridge + Payload parsed
assert ret.password1 == ret.password2
# Same behaviour but using our decorator
@event_parser(model=UserModel, envelope=envelopes.EventBridgeModel)
def handler(event: UserModel, context: LambdaContext):
assert event.password1 == event.password2Credits: Thanks to @risenberg-cyberark from CyberArk for the idea, implementation, and guidance on how to best support Pydantic to provide both parsing and deep data validation. Also, special thanks to @koxudaxi for helping review with his extensive Pydantic experience.
🌟New features and non-breaking changes
- feat: Advanced parser utility (pydantic) (#118) by @risenberg-cyberark
🌟 Minor Changes
- feat(data_classes): case insensitive header lookup (#186) by @michaelbrewer
- test(parser): Add missing test coverage and lint changes (#188) by @michaelbrewer
- feat(data_classes): Cognito custom auth triggers (#178) by @michaelbrewer
📜 Documentation updates
- docs: add more info on conditional keys #195 (#199) by @heitorlessa
- docs: new parser utility (#192) by @heitorlessa
- improv: Update README.md (#190) by @bmicklea
🐛 Bug and hot fixes
- improv: keeps Lambda root logger handler intact, and add log filter instead to prevent child log records duplication (#198) by @heitorlessa
🔧 Internal
- fix: move from npm to yarn to fix peer deps vuln with resolutions feature (#197) by @heitorlessa
- build(deps): bump object-path from 0.11.4 to 0.11.5 in /docs (#196) by @dependabot
- chore: docs npm security node-forge (#181) by @heitorlessa
- chore: remove kitchen sink example (#180) by @heitorlessa
- chore: ease maintenance of upcoming parser #118 (#189) by @heitorlessa
- chore: bump to 1.7.0 (#200) by @heitorlessa
This release was made possible by the following contributors:
@bmicklea, @dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer, @risenberg-cyberark, @Nr18, and @koxudaxi
v1.6.1
v1.6.0
Changes
Data_classes utility
New utility to easily describe event schema of popular event sources, including helper methods to access common objects (s3 bucket key) and data deserialization (records from Kinesis, CloudWatch Logs, etc).
Huge prop to @michaelbrewer for the contribution, and @cakepietoast for the comprehensive docs with examples.
JSON Schema validation utility
New utility to quickly validate inbound events and responses using JSON Schema. It also supports unwrapping events using JMESPath expressions, so you can validate only the payload or key that interests you.
Oh, before I forget! This also includes custom JMESPath functions for de-serializing JSON Strings, base64, and ZIP compressed data before applying validation too 🥰
Metrics with multiple values
Metrics utility now support adding multiple values to the same metric - This was updated in CloudWatch EMF, and Powertools happily support that too ;) - Thanks to @Dunedan for spotting that
Minor doc changes
We added a Testing your code section for Logger and Metrics for customers like @patrickwerz who had difficulties to do unit testing their code with Powertools - Pytest fixture and examples are now provided!
We also increased the content width to ease reading more elaborate sections, and gives us room to start tinkering with a Tutorial/Guide section in coming releases \ o /
🌟New features and non-breaking changes
- improv: disable tracer when using the Chalice CLI (#172) by @jamesls
- feat(trigger): data class and helper functions for lambda trigger events (#159) by @michaelbrewer
- feat: emf multiple metric values (#167) by @cakepietoast
- feat: simple JSON Schema validator utility (#153) by @heitorlessa
📜 Documentation updates
- docs: Data Classes Utility (#171) by @cakepietoast
- docs: add testing tips, increase content width, and improve log sampling wording (#174) by @heitorlessa
🐛 Bug and hot fixes
- fix: remove DeleteMessageBatch call to SQS api if there are no messages to delete (#170) by @cakepietoast
- fix: add missing tests, correct calls to DictWrapper constructor and improve metrics type hints (#168) by @michaelbrewer
This release was made possible by the following contributors:
@cakepietoast, @heitorlessa, @jamesls and @michaelbrewer
v1.5.0
Changes
SQS batch processing utility
Add a new utility to handle partial failures when processing batches of SQS messages in Lambda. The default behaviour with Lambda - SQS is to return all messages to the queue when there is a failure during processing. This utility provides functionality to handle failures individually at the message level, and avoid re-processing messages. Thanks to @gmcrocetti who contributed this utility.
Integration with CloudWatch ServiceLens
The xray_trace_id key is now added to log output when tracing is active. This enables the log correlation functionality in ServiceLens to work with applications using powertools.
Static types for Lambda context object
You can now import a static type for the Lambda context object from this library. Thanks to @Nr18 for the implementation.
Control order of logging output
You can now change the order of the fields output by the logger. Thanks to @michaelbrewer for the implementation.
Automatically deserialize parameters
Thanks to @michaelbrewer, the parameters utility can now automatically decide how to deserialize parameter values (json/base64) based on the key name.
Documentation improvements
Lots of improvements made to the documentation. Thanks to the community contributors: @Nr18 for adding a troubleshooting section, @michaelbrewer and @bls20AWS for several housekeeping contributions.
🌟 Minor Changes
- fix(logging): Don't include
json_defaultin logs (#132) by @michaelbrewer - chore(batch): Housekeeping for recent changes (#157) by @michaelbrewer
📜 Documentation updates
- chore: fix typos, docstrings and type hints (#154) by @michaelbrewer
- docs: add description where to find the layer arn (#145) by @am29d
- docs: new section "Migrating from other Loggers" (#148) by @heitorlessa
- docs: minor edit to letter case part 2 (#142) by @michaelbrewer
- docs: minor edit to letter case (#141) by @bls20AWS
- docs: fix log sampling percentage wording (#135) by @pankajagrawal16
🐛 Bug and hot fixes
- fix: batch processing util (#155) by @cakepietoast
This release was made possible by the following contributors:
@Nr18, @am29d, @bls20AWS, @cakepietoast, @gmcrocetti, @heitorlessa, @michaelbrewer and @pankajagrawal16
v1.4.0
814062e
heitorlessa
Heitor Lessa
Changes
Official Lambda Layer
We now provide an official Lambda Layer via AWS Serverless Application Repository (SAR) App. SAR App follows semantic versioning, and it is synchronized with what's published on PyPi.
| SAR App | ARN |
|---|---|
| aws-lambda-powertools-python-layer | arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer |
A big thanks to @am29d for the implementation, and to Keith Rosario, a long standing contributor, and beta customer, who created the first Layers through his KLayers project - Lambda Layers for Python on GitHub before.
Auto-complete for SSMProvider on Parameters utility
Thanks to @michaelbrewer, SSMProvider within Parameters utility now have decrypt and recursive parameters correctly defined to support autocompletion.
Tracer support to not capture response as metadata
For customers returning sensitive information from their methods and Lambda handlers, Tracer decorators capture_method and capture_lambda_handler now support capture_response=False parameter to override this behaviour.
Cold start metric bugfix
This release ensures Metrics utility creates a Cold Start metric with dedicated CloudWatch dimensions function_name and service to explicitly separate Application metrics from System metrics.
Previously, when capturing cold start metric via capture_cold_start_metric parameter, we would add ColdStart metric to an existing metric set along with function_name as a dimension. This caused the problem of Application metrics having data points in two separate metrics with the same name, since one of them would have an additional function_name dimension in the event of a cold start.
Minor improvements to docs
Content in Tracer and Logger have been reordered to match what customers are looking for. Tracer docs have less main sections to improve navigation, a new section named Patching Modules, and a note for customers capturing sensitive information they might not want Tracer to capture it.
🌟 Minor Changes
- feat: add workflow to trigger external pipeline for publishing lambda layers (#129) by @am29d
- feat: option to disallow tracer to capture response as metadata (#128) by @heitorlessa
📜 Documentation updates
- docs: readability improvements (#130) by @heitorlessa
🐛 Bug and hot fixes
- fix: split cold start metric from application metrics to prevent duplicate app metrics (#126) by @heitorlessa
- fix(ssm): Make decrypt an explicit option and refactoring (#123) by @michaelbrewer
This release was made possible by the following contributors:
v1.3.1
Changes
Fixed issue with capture_method returning not working properly when the decorated function made us of context managers during its execution.
Patch
- fix(capture_method): should yield inside with for generator method using a context manager (#124) by @michaelbrewer
Chore
- chore(deps): bump elliptic from 6.5.2 to 6.5.3 in /docs (#120) by @dependabot
- chore(deps): bump prismjs from 1.20.0 to 1.21.0 in /docs (#121) by @dependabot
This release was made possible by the following contributors:
@cakepietoast, @dependabot, @heitorlessa and @michaelbrewer
v1.3.0
Changes
Add a new utility to fetch and cache parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager or Amazon DynamoDB. It also provides a base class to create your parameter provider implementation.
Retrieve values from Systems Manager Parameter Store:
from aws_lambda_powertools.utilities import parameters
def handler(event, context):
# Retrieve a single parameter
value = parameters.get_parameter("/my/parameter")
# Retrieve multiple parameters from a path prefix recursively
# This returns a dict with the parameter name as key
values = parameters.get_parameters("/my/path/prefix")
for k, v in values.items():
print(f"{k}: {v}")Retrieve secrets from AWS Secrets Managers:
from aws_lambda_powertools.utilities import parameters
def handler(event, context):
# Retrieve a single secret
value = parameters.get_secret("my-secret")- chore: bump version to 1.3.0 (#122) by @nmoutschen
🌟 Minor Changes
- feat: add parameter utility (#96) by @nmoutschen
This release was made possible by the following contributors:
v1.2.0
Changes
The Tracer capture_method decorator can now be used to capture execution of generator functions, including context managers.
- chore: bump version to 1.2.0 (#119) by @cakepietoast
🌟 Minor Changes
- feat: add support for tracing of generators using capture_method decorator (#113) by @cakepietoast
This release was made possible by the following contributors:
@cakepietoast
v1.1.3
Changes
Fix logged messages being emitted twice, once structured and once unstructured - We now remove the root logger handler set by Lambda during initialization.
Patch
- fix: remove root logger handler set by Lambda #115 (#116) by @heitorlessa
Chore
- chore: bump to 1.1.3 (#117) by @heitorlessa
This release was made possible by the following contributors:
v1.1.2
Changes
Minor patch to improve Tracer documentation on reusability, and ensures PyCharm/VSCode Jedi Language Server can autocomplete log statements for Logger.
- chore: bump version to 1.1.2 (#111) by @heitorlessa
- improv: logger autocomplete for PyCharm (#110) by @heitorlessa
📜 Documentation updates
- docs: clarify Tracer auto_patch as per #108 (#109) by @heitorlessa