feat(prebuilt): native structured output support w/ all sorts of models (#5961)

* Adds support for `NativeOutput` via a new `NativeOutput` dataclass
* Adds support for structured output specification via the following
(pydantic models already supported)
  * dataclasses
  * typed dicts
  * json schemas 
* Adds mocking support to support native strategies with
`FakeToolCallingModel`
* Add new default tool message when `tool_message_content` not provided
* Smart "selection" of native vs tool output based on provider support,
necessitates profiles down the line
  
Considered questions
* do we want to enforce docstrings? -- decided on no for now
* do we want to enforce names (titles) on json schemas? -- decided no
for now, defaulting to `structured_output`
* do we want to validate that json schemas coming in are valid? --
decided no for now
* do we want to validate model results against a given json schema? we
validate against all other types (typed dict, dataclass, etc) w/
pydantic -- decided no for now

TODO in future PRs:
* Figure out retry policy
* Add standard testing (handed off to @casparb)
* Further privatize certain structures (like the bindings) -- this is
low prio

---------

Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
This commit is contained in:
Caspar Broekhuizen
2025-08-21 10:43:50 -04:00
committed by GitHub
co-authored by Sydney Runkle Sydney Runkle
parent 1cd1373788
commit f4cdeea6ad
8 changed files with 1097 additions and 159 deletions
+3 -3
View File
@@ -447,7 +447,7 @@ def test_react_agent_with_structured_response() -> None:
"ai", # "What's the weather?"
"tool", # "The weather is sunny and 75°F."
"ai", # structured response
"tool", # ok!
"tool", # artificial tool message
]
assert [m.content for m in response["messages"]] == [
@@ -455,7 +455,7 @@ def test_react_agent_with_structured_response() -> None:
"What's the weather?",
"The weather is sunny and 75°F.",
"What's the weather?-What's the weather?-The weather is sunny and 75°F.",
"ok!",
"Returning structured response: {'temperature': 75.0}",
]
@@ -1608,7 +1608,7 @@ def test_post_model_hook_with_structured_output() -> None:
],
),
_AnyIdToolMessage(
content="ok!",
content="Returning structured response: {'temperature': 75.0}",
name="WeatherResponse",
tool_call_id="2",
),