Skip to content

[Bug] value_to_type fails with custom string types as dictionary keys #1032

Closed

Description

@avalatea

What are you really trying to do?

I'm trying to use Temporal's value_to_type function to convert JSON payloads to dataclasses that contain dictionary fields with custom string types as keys (either string subclasses or NewType string aliases). This is a common pattern when working with typed identifiers like MongoDB ObjectIds or UUIDs.

Note: I am not calling the function directly; the bug is caused when triggering a workflow, which, under the hood will use value_to_type to convert types.

Describe the bug

The value_to_type function has two bugs when handling dictionary keys with custom string types:

  1. String subclasses are converted to character lists: When the key type is a string subclass (like class CustomStr(str)), the converter treats the string as an iterable and converts it to a list of characters, making it unhashable and unusable as a dictionary key.

  2. NewType fails isinstance check: When the key type is a NewType alias (like MyNewTypeStr = NewType("MyNewTypeStr", str)), the converter fails with "isinstance() arg 2 must be a type" because NewType creates a callable, not a proper type for isinstance checks.

Minimal Reproduction

import traceback
from dataclasses import dataclass
from typing import NewType

from temporalio.converter import value_to_type


class CustomStr(str):
    pass


MyNewTypeStr = NewType("MyNewTypeStr", str)


@dataclass
class SimpleMessage:
    data: dict[CustomStr, str]


@dataclass 
class NewTypeMessage:
    data: dict[MyNewTypeStr, str]


def test_key_conversion():
    print("=" * 50)
    print("Testing direct key conversion with CustomStr...")
    print("=" * 50)
    key = "key1"

    converted_key = value_to_type(CustomStr, key)
    print(f"Original key: {repr(key)} (type: {type(key)})")
    print(f"Converted key: {repr(converted_key)} (type: {type(converted_key)})")


def test_newtype_key_conversion():
    print("=" * 50)
    print("Testing direct key conversion with NewType...")
    print("=" * 50)
    key = "key1"

    converted_key = value_to_type(MyNewTypeStr, key)
    print(f"Original key: {repr(key)} (type: {type(key)})")
    print(f"Converted key: {repr(converted_key)} (type: {type(converted_key)})")


def test_full_conversion():
    print("=" * 50)
    print("Testing full message conversion with CustomStr...")
    print("=" * 50)
    payload = {"data": {"key1": "value1"}}

    try:
        result = value_to_type(SimpleMessage, payload)
        print(f"Success: {result}")
    except Exception as e:
        print(f"Full conversion failed: {e}")
        traceback.print_exc()


def test_newtype_full_conversion():
    print("=" * 50)
    print("Testing full message conversion with NewType...")
    print("=" * 50)
    payload = {"data": {"key1": "value1"}}

    try:
        result = value_to_type(NewTypeMessage, payload)
        print(f"Success: {result}")
    except Exception as e:
        print(f"Full conversion failed: {e}")
        traceback.print_exc()


if __name__ == "__main__":
    test_key_conversion()
    test_newtype_key_conversion()
    test_full_conversion()
    test_newtype_full_conversion()

Activity

  1. tconley1428 commented on Aug 19, 2025

    @tconley1428
    Contributor

    Thanks @avalatea. The default data converter is known not to be the most fully featured serialization library there is. We'll look into this particular issue, but I can't promise it will be very high priority. As an alternative, we recommend the pydantic data converter for a more fully featured serialization story.

  2. avalatea commented on Aug 19, 2025

    @avalatea
    Author

    @tconley1428 Thanks for the response. We're exploring several options at the moment.

    We did find that this behavior did not happen with temporal 1.10.0 vs 1.15.0. Do you happen to know at which version major changes happened with converter behavior? If not, that's fine. I am just curious what change made NewType suddenly no longer supported.

  3. tconley1428 commented on Aug 19, 2025

    @tconley1428
    Contributor

    My guess based on the error message, though a stack would be make it more certain:
    4933dc5

    Supporting non-string key types added this line:

                        if not isinstance(key, key_type):
    

    Potentially that is failing on the newtype. The fact that it used to work will make it more likely that we will address it. I'll bring it up in triage.

  4. tconley1428 commented on Aug 27, 2025

    @tconley1428
    Contributor

    Created a PR here: #1059. Am I right in understanding that the CustomStr method is not known to have worked in the past?

  5. avalatea commented on Aug 27, 2025

    @avalatea
    Author

    I am unsure if the CustomStr ever worked in the past, but I was attempting a fix by using a CustomStr class, which did not work as well.

    I figured I would throw it in. It would make sense that a custom class be serializable, but perhaps I am mistaken? In any case, our systems currently only care about the NewType case as far as I can tell.

  6. tconley1428 commented on Aug 27, 2025

    @tconley1428
    Contributor

    It does make sense, but we aren't prioritizing additional coverage in the default data converter. I don't think the implementation would be entirely straightforward either since we'd have to ensure that the custom class doesn't have additional data and is just a string.

  7. tconley1428 commented on Aug 29, 2025

    @tconley1428
    Contributor

    Completed the PR to fix newtype. I'm going to close the issue, if you think custom str subtypes is a feature you would like, please open a feature request issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions