Skip to content

Conversation

@lapenna-bjss
Copy link
Collaborator

@lapenna-bjss lapenna-bjss commented Nov 11, 2025

Description

This PR introduces two components that together implement the full workflow for pulling and downloading requests submitted to the MESH mailbox.

The first component, mesh-poll, runs every five minutes, reads messages from the MESH inbox, extracts metadata from the message headers, including meshMessageId and messageReference (the MESH local_id), validates the sender by comparing the message’s mailbox ID against sender configurations stored in SSM, and resolves the corresponding senderId. Once validated, mesh-poll publishes a MeshInboxMessageReceived CloudEvent containing senderId, meshMessageId, and messageReference.

The second component, mesh-download, consumes these events via SQS, retrieves the MESH message using the meshMessageId, stores the document in the PII S3 bucket under the document-reference/{senderId}_{messageReference} key, acknowledges the message in MESH (removing it from the inbox), and finally publishes a MeshInboxMessageDownloaded CloudEvent.

Note:
In development environments, the pipeline uses mock MESH

Utils Added

  • event-publisher-py: Python implementation of the EventPublisher (mirrors the TypeScript version).
  • Pydantic CloudEvent Models: Pydantic schemas used to validate all inbound and outbound CloudEvents (MeshInboxMessageReceived, MeshInboxMessageDownloaded).
  • metric-publishers
  • PyMockMesh: Mock implementation of the MESH client used in tests.

Type of changes

  • Refactoring (non-breaking change)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would change existing functionality)
  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • I am familiar with the contributing guidelines
  • I have followed the code style of the project
  • I have added tests to cover my changes
  • I have updated the documentation accordingly
  • This PR is a result of pair or mob programming

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

  • I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

@lapenna-bjss lapenna-bjss requested review from a team as code owners November 11, 2025 09:55
@lapenna-bjss lapenna-bjss marked this pull request as draft November 11, 2025 15:19
@lapenna-bjss lapenna-bjss force-pushed the feature/CCM-12616_mesh_poll_retrieve branch from d1e3058 to 331d6c4 Compare November 12, 2025 11:15
@lapenna-bjss lapenna-bjss marked this pull request as ready for review November 28, 2025 11:50
@lapenna-bjss lapenna-bjss requested a review from a team as a code owner November 28, 2025 11:50
@gareth-allan gareth-allan self-assigned this Dec 17, 2025
gareth-allan
gareth-allan previously approved these changes Dec 17, 2025
Copy link
Contributor

@gareth-allan gareth-allan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just a couple of minor points.

scripts/**/*.*, \
docs/**/*.*, \
utils/py-mock-mesh/**, \
utils/event-publisher-py/event_publisher/mesh_config.py, \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should mesh_config.py not have some test coverage?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have any tests for the config files in core, so I didn't bother. I can add them if we think they're needed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does quite a lot, so it might be nice, but I won't block the PR for this. We could always put a TD ticket in perhaps?

@Ian-Hodges Ian-Hodges self-assigned this Dec 17, 2025
@Ian-Hodges Ian-Hodges changed the title Feature/ccm 12616 mesh poll retrieve CCM-12616 mesh poll retrieve Dec 17, 2025
gareth-allan
gareth-allan previously approved these changes Dec 17, 2025
tdroza-nhs
tdroza-nhs previously approved these changes Dec 17, 2025
Copy link
Contributor

@tdroza-nhs tdroza-nhs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous comments addressed

from pydantic import BaseModel, ConfigDict, Field, field_validator


class CloudEvent(BaseModel):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll need to update the schema otherwise the consuming components will fail when validating them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should now be done.

@lapenna-bjss lapenna-bjss dismissed stale reviews from tdroza-nhs and gareth-allan via ca56d6e December 17, 2025 16:16
gareth-allan
gareth-allan previously approved these changes Dec 18, 2025
tdroza-nhs
tdroza-nhs previously approved these changes Dec 18, 2025
@gareth-allan gareth-allan dismissed stale reviews from aidenvaines-cgi, tdroza-nhs, and themself via c7076d1 December 19, 2025 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants