Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ jobs:
arize_tracing \
bedrock_agentcore/strands_agent \
lambda_worker \
nexus_serialization_context \
openai_agents/mcp_v2
do
if grep -q "^${project}/" <<< "$changed_files"; then
Expand All @@ -58,6 +59,9 @@ jobs:
# temporalio 1.36.0, which arize_tracing needs, requires Python 3.11+
- project: arize_tracing
python: "3.10"
# Nexus serialization context requires SDK 1.36.0 and Python 3.11+.
- project: nexus_serialization_context
python: "3.10"
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ Some examples require extra dependencies. See each sample's directory for specif
* [nexus_messaging_temporal_operation](nexus_messaging_temporal_operation): Demonstrates how to send signal, update and query messages through Nexus.
This version uses `@nexus.temporal_operation` to allow Nexus operation handlers to either return a synchronous result or start a Workflow as the async backing operation.
* [nexus_multiple_args](nexus_multiple_args) - Map a Nexus operation to a handler workflow that takes multiple arguments.
* [nexus_serialization_context](nexus_serialization_context) - Select encryption keys and compression by endpoint for synchronous and workflow-backed asynchronous Nexus operations.
* [nexus_standalone_activity](nexus_standalone_activity) - Back a Nexus operation with a standalone Activity.
* [nexus_standalone_operations](nexus_standalone_operations) - Execute Nexus operations directly from client code,
without wrapping them in a workflow.
Expand Down
193 changes: 193 additions & 0 deletions nexus_serialization_context/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
# Nexus serialization context

This sample uses `NexusSerializationContext` to select an encryption key for each Nexus
endpoint. The caller invokes synchronous and workflow-backed asynchronous operations
through four endpoints. The `compressed-encrypted` worker compresses and then encrypts
Nexus payloads. The `encrypted` worker encrypts without compression. Each handler selects
between its own key 1 and key 2 using the endpoint in the Nexus context.
The caller and each handler run in separate namespaces.

| Handler worker | Endpoint | Key ID | Nexus payload encoding |
|---|---|---|---|
| Compressed/encrypted | `nexus-serialization-compressed-encrypted-1` | `compressed-encrypted-1` | zlib, then AES-GCM |
| Compressed/encrypted | `nexus-serialization-compressed-encrypted-2` | `compressed-encrypted-2` | zlib, then AES-GCM |
| Encrypted | `nexus-serialization-encrypted-1` | `encrypted-1` | AES-GCM |
| Encrypted | `nexus-serialization-encrypted-2` | `encrypted-2` | AES-GCM |

Each endpoint uses a different 128-bit AES key. The codec uses `cryptography` for
AES-GCM and Python's `zlib` module for compression.

`NexusCodec` uses the endpoint from the Nexus context to select a codec. It uses a default
codec for ordinary workflow payloads. Use `NexusCodec` as the `payload_codec` on a
`DataConverter`, then pass that converter as `data_converter` to `Client.connect` for
the caller and handler clients.

Because AES-GCM uses symmetric encryption, the caller and handler share a secret key
for each endpoint. This sample uses that key for both requests and results.

The caller uses a separate key for its workflow input and final result. Each handler
reuses its key 1 for backing workflow inputs.

| Ordinary workflow payload | Default key ID |
|---|---|
| Caller workflow input and final result | `caller-workflow` |
| Compressed/encrypted handler workflow inputs | `compressed-encrypted-1` |
| Encrypted handler workflow inputs | `encrypted-1` |

These workflow payloads use AES-GCM without compression or endpoint metadata.
An async Nexus result uses the key and compression selected by its endpoint,
even when that differs from the backing workflow's input encryption.

The SDK supplies Nexus context when encoding and decoding operation requests and
results. For workflow-backed asynchronous operations, it preserves that context in
workflow history and restores it when encoding the result. A call through endpoint 2
can therefore return a result encrypted with key 2, as the caller expects, even though
the backing workflow's input uses the handler's default key 1.

The caller starts all eight operations before waiting for their results. Each operation
keeps its own endpoint context, even when operations for different keys are in flight.
`NexusCodec.with_context` returns a copy so binding one operation does not change another.
No custom interceptor or workflow-header propagation is required.

A workflow execution produces one serialized result, shared by all Nexus operations
attached to it. Operations that need differently encrypted results therefore need
separate workflow executions. This sample starts a backing workflow for each async
operation, using its Nexus request ID as the workflow ID suffix. Retrying the same
request uses the same workflow ID.

The outer encrypted payload includes `encoding`, `encryption-key-id`, and, for Nexus
payloads, `nexus-endpoint-name` metadata. For example:

```text
encoding: binary/nexus-aes-gcm
encryption-key-id: compressed-encrypted-1
nexus-endpoint-name: nexus-serialization-compressed-encrypted-1
```

The metadata identifies the key and endpoint. The key material stays in the codec
configuration.
Ordinary workflow payloads have a key ID but no endpoint metadata or compression.

The AES-GCM codec and hard-coded keys are for this example. For production, use an
established encryption implementation and secure key storage.

## Prerequisites

Use Python 3.11 or newer, [uv](https://docs.astral.sh/uv/), and Temporal Python SDK
1.36.0 or later.
This is a separate uv project because the required SDK needs Python 3.11+, while the
repository's other samples support Python 3.10.

Use a Temporal Server that supports Nexus serialization-context propagation for
workflow-backed asynchronous operations. For local development, download
[Temporal CLI `v1.9.3-nexus-serialization-context`](https://github.com/temporalio/cli/releases/tag/v1.9.3-nexus-serialization-context).
Extract it and place the `temporal` executable in the repository root. The commands below
use `../temporal` to run this build separately from your installed CLI.

## Run locally

From the repository root, enter the sample directory and install its dependencies:

```shell
cd nexus_serialization_context
uv sync
../temporal --version
```

The CLI should report `1.9.3-nexus-serialization-context`. Run the remaining commands
from `nexus_serialization_context`, including commands in new terminals.

The caller and each handler use separate namespaces. Each worker polls a task queue
with the same name as its namespace. Both endpoints for a handler target that handler's
namespace and task queue.

| Worker | Namespace and task queue |
|---|---|
| Caller | `nexus-serialization-caller` |
| Compressed/encrypted handler | `nexus-serialization-compressed-encrypted-handler` |
| Encrypted handler | `nexus-serialization-encrypted-handler` |

Start the dev server in its own terminal and leave it running:

```shell
../temporal server start-dev \
--namespace nexus-serialization-caller \
--namespace nexus-serialization-compressed-encrypted-handler \
--namespace nexus-serialization-encrypted-handler
```

In another terminal, create two endpoints for each handler worker:

```shell
../temporal operator nexus endpoint create \
--name nexus-serialization-compressed-encrypted-1 \
--target-namespace nexus-serialization-compressed-encrypted-handler \
--target-task-queue nexus-serialization-compressed-encrypted-handler
../temporal operator nexus endpoint create \
--name nexus-serialization-compressed-encrypted-2 \
--target-namespace nexus-serialization-compressed-encrypted-handler \
--target-task-queue nexus-serialization-compressed-encrypted-handler
../temporal operator nexus endpoint create \
--name nexus-serialization-encrypted-1 \
--target-namespace nexus-serialization-encrypted-handler \
--target-task-queue nexus-serialization-encrypted-handler
../temporal operator nexus endpoint create \
--name nexus-serialization-encrypted-2 \
--target-namespace nexus-serialization-encrypted-handler \
--target-task-queue nexus-serialization-encrypted-handler
```

Run each worker in its own terminal and leave all three running alongside the server:

```shell
uv run worker.py compressed-encrypted
```

```shell
uv run worker.py encrypted
```

```shell
uv run worker.py caller
```

Start the caller workflow in another terminal:

```shell
uv run starter.py
```

### Output

The starter prints the workflow and run IDs, followed by a sync result and an async result
for each endpoint. Replacing the generated IDs, the output is:

```text
Started workflow: ID=nexus-serialization-<workflow UUID> RunID=<run UUID>
nexus-serialization-compressed-encrypted-1: sync="Hello from Nexus" async="Hello from Nexus"
nexus-serialization-compressed-encrypted-2: sync="Hello from Nexus" async="Hello from Nexus"
nexus-serialization-encrypted-1: sync="Hello from Nexus" async="Hello from Nexus"
nexus-serialization-encrypted-2: sync="Hello from Nexus" async="Hello from Nexus"
```

All eight results should be `Hello from Nexus`.

To see what Temporal stored, open the printed workflow ID in the
[Temporal UI](http://localhost:8233) under the `nexus-serialization-caller` namespace.
In **History**, check **Input** on a **Nexus Operation Scheduled** event and **Result**
on its **Nexus Operation Completed** event. The payload data is encrypted and shown in
Base64. Look for `encryption-key-id` in the payload metadata to compare endpoints.

The run creates five workflow executions: one caller and four async backing workflows.
The four sync operations return directly from their handlers.

## Test

Run the codec and workflow tests:

```shell
uv run poe test
```

The workflow test starts its own server and checks all eight results, the stored Nexus
context, and the backing workflows' input and result encryption metadata.
109 changes: 109 additions & 0 deletions nexus_serialization_context/codec.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
import copy
import os
import zlib
from collections.abc import Mapping, Sequence
from types import MappingProxyType
from typing import Self

from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from temporalio.api.common.v1 import Payload
from temporalio.converter import (
NexusSerializationContext,
PayloadCodec,
SerializationContext,
WithSerializationContext,
)


class NexusCodec(PayloadCodec, WithSerializationContext):
"""Select an endpoint's codec, with a default for ordinary workflow payloads."""

def __init__(
self, endpoint_codecs: Mapping[str, PayloadCodec], workflow_codec: PayloadCodec
) -> None:
self._endpoint_codecs = MappingProxyType(dict(endpoint_codecs))
self._workflow_codec = workflow_codec
self._selected_codec = workflow_codec

def with_context(self, context: SerializationContext) -> Self:
selected = self._workflow_codec
if isinstance(context, NexusSerializationContext):
try:
selected = self._endpoint_codecs[context.endpoint]
except KeyError:
raise ValueError(
f"Unknown Nexus endpoint: {context.endpoint}"
) from None
if isinstance(selected, WithSerializationContext):
selected = selected.with_context(context)
# Keep each operation's selection separate; share the read-only configuration.
bound = copy.copy(self)
bound._selected_codec = selected
return bound

async def encode(self, payloads: Sequence[Payload]) -> list[Payload]:
return await self._selected_codec.encode(payloads)

async def decode(self, payloads: Sequence[Payload]) -> list[Payload]:
return await self._selected_codec.decode(payloads)


class EncryptionCodec(PayloadCodec, WithSerializationContext):
"""Encrypt the whole payload so its encoding metadata is restored on decode."""

def __init__(self, key_id: str, key: bytes, *, compress: bool = False) -> None:
self._key_id = key_id
self._encryptor = AESGCM(key)
self._compress = compress
self._endpoint: str | None = None

def with_context(self, context: SerializationContext) -> Self:
bound = copy.copy(self)
bound._endpoint = (
context.endpoint if isinstance(context, NexusSerializationContext) else None
)
return bound

async def encode(self, payloads: Sequence[Payload]) -> list[Payload]:
encoded = []
for payload in payloads:
if self._compress:
payload = Payload(
metadata={"encoding": b"binary/nexus-zlib"},
data=zlib.compress(payload.SerializeToString()),
)
# Store a fresh public 12-byte nonce before the ciphertext and 16-byte tag.
nonce = os.urandom(12)
metadata = {
"encoding": b"binary/nexus-aes-gcm",
"encryption-key-id": self._key_id.encode(),
}
if self._endpoint:
metadata["nexus-endpoint-name"] = self._endpoint.encode()
encoded.append(
Payload(
metadata=metadata,
data=nonce
+ self._encryptor.encrypt(nonce, payload.SerializeToString(), None),
)
)
return encoded

async def decode(self, payloads: Sequence[Payload]) -> list[Payload]:
decoded = []
for payload in payloads:
if payload.metadata.get("encoding") != b"binary/nexus-aes-gcm":
raise ValueError("Expected an AES-GCM payload")
if payload.metadata.get("encryption-key-id") != self._key_id.encode():
raise ValueError("Unexpected encryption key ID")
if len(payload.data) < 28:
raise ValueError("AES-GCM payload is too short")
plaintext = Payload.FromString(
self._encryptor.decrypt(payload.data[:12], payload.data[12:], None)
)
if self._compress:
if plaintext.metadata.get("encoding") != b"binary/nexus-zlib":
raise ValueError("Expected a Nexus zlib payload")
plaintext = Payload.FromString(zlib.decompress(plaintext.data))
decoded.append(plaintext)
return decoded
60 changes: 60 additions & 0 deletions nexus_serialization_context/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
from temporalio.converter import DataConverter

from codec import EncryptionCodec, NexusCodec
from service import (
COMPRESSED_ENCRYPTED_1_ENDPOINT,
COMPRESSED_ENCRYPTED_2_ENDPOINT,
ENCRYPTED_1_ENDPOINT,
ENCRYPTED_2_ENDPOINT,
)

# These keys are for this local example. Load production keys from secure storage.
COMPRESSED_ENCRYPTED_KEY_1 = b"handler-a-key-01"
COMPRESSED_ENCRYPTED_KEY_2 = b"handler-a-key-02"
ENCRYPTED_KEY_1 = b"handler-b-key-01"
ENCRYPTED_KEY_2 = b"handler-b-key-02"
CALLER_WORKFLOW_KEY = b"sample-key-C-123"


def _compressed_encrypted_codecs() -> dict[str, EncryptionCodec]:
return {
COMPRESSED_ENCRYPTED_1_ENDPOINT: EncryptionCodec(
"compressed-encrypted-1", COMPRESSED_ENCRYPTED_KEY_1, compress=True
),
COMPRESSED_ENCRYPTED_2_ENDPOINT: EncryptionCodec(
"compressed-encrypted-2", COMPRESSED_ENCRYPTED_KEY_2, compress=True
),
}


def _encrypted_codecs() -> dict[str, EncryptionCodec]:
return {
ENCRYPTED_1_ENDPOINT: EncryptionCodec("encrypted-1", ENCRYPTED_KEY_1),
ENCRYPTED_2_ENDPOINT: EncryptionCodec("encrypted-2", ENCRYPTED_KEY_2),
}


def caller_data_converter() -> DataConverter:
return DataConverter(
payload_codec=NexusCodec(
{**_compressed_encrypted_codecs(), **_encrypted_codecs()},
EncryptionCodec("caller-workflow", CALLER_WORKFLOW_KEY),
)
)


def compressed_encrypted_data_converter() -> DataConverter:
return DataConverter(
payload_codec=NexusCodec(
_compressed_encrypted_codecs(),
EncryptionCodec("compressed-encrypted-1", COMPRESSED_ENCRYPTED_KEY_1),
)
)


def encrypted_data_converter() -> DataConverter:
return DataConverter(
payload_codec=NexusCodec(
_encrypted_codecs(), EncryptionCodec("encrypted-1", ENCRYPTED_KEY_1)
)
)
Loading
Loading