Commit Graph
6 Commits
Author SHA1 Message Date
haelyra f4f8beba01 fix: require explicit reasoning format before stripping content 2026-10-11 02:26:58 -04:00
FenjuFu 586d4b91cc fix(llm): keep answers too deeply nested to parse whole in prefilled mode
Signed-off-by: FenjuFu <fufenjupku@gmail.com>
2026-10-05 01:19:14 +08:00
FenjuFu d351fc5502 fix(llm): fall back when a prefilled answer is too deeply nested to parse
json.loads raises RecursionError on deeply nested input, which escaped the
complete-JSON check and failed the request. Treat it like any other input
that is not a complete JSON document.

Signed-off-by: FenjuFu <fufenjupku@gmail.com>
2026-10-04 16:31:47 +08:00
FenjuFu af325fbb85 fix(llm): keep complete JSON answers whole in prefilled reasoning mode
With prefilled reasoning enabled, a structured answer such as
{"closing":"</think>"} was cut at the tag inside its string value.
Reasoning followed by a closing tag is never a valid JSON document, so a
reply that parses as one complete object or array is now returned
unchanged. Also annotate the tests and helpers this PR adds.

Signed-off-by: FenjuFu <fufenjupku@gmail.com>
2026-10-03 01:44:22 +08:00
FenjuFu 2861dee176 fix(llm): only strip close-only reasoning when the deployment declares it
Strip a balanced <think>...</think> block that opens the reply by default and
leave every other reply untouched, so answers that mention the tags (a lone
or reversed closing tag, JSON string values) are never truncated.

The close-only form produced by chat templates that prefill <think> is now
handled only when the OpenAI or Ollama provider is configured for it, via
prefilled_reasoning=True or OPENAI_PREFILLED_REASONING /
OLLAMA_PREFILLED_REASONING. Hosted providers keep the default.

Signed-off-by: FenjuFu <92919259+FenjuFu@users.noreply.github.com>
2026-10-02 11:03:17 +08:00
FenjuFu d6f20191de fix(llm): strip leaked reasoning from self-hosted model output
Self-hosted reasoning models served through vLLM or llama.cpp without a
reasoning parser, or through Ollama without a thinking template, return
their chain of thought inside message content. The OpenAI-compatible
providers and the Ollama provider passed it through verbatim, so
LLMOutput.content and ReActAgent final answers began with the reasoning.

Models whose chat template prefills <think> in the generation prompt
(Spark-X2.5, Qwen3 thinking variants) only emit the closing tag, so the
content arrives as "...</think>answer" with no opening tag.

Add strip_reasoning() and apply it in the OpenAI, Astraflow, Atlas and
Ollama providers. A <think> that does not open the output is kept as a
literal, and unterminated reasoning is left unchanged.

Signed-off-by: FenjuFu <fufenjupku@gmail.com>
2026-09-30 18:41:24 +08:00