Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
75 changes: 73 additions & 2 deletions assets/js/mermaid.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,80 @@
{{ if .enable }}
(function($) {
var needMermaid = false;
$('.language-mermaid').parent().replaceWith(function() {
function preprocessMermaid(text) {
var fm = text.match(/^\s*---\s*\n([\s\S]*?)\n---\s*\n?/);
var directive = '';
var body = text;
if (fm) {
var yaml = fm[1];
var obj = {};
var current = null;
yaml.split('\n').forEach(function(line) {
var l = line.trim();
if (!l) return;
if (/^[\w-]+\s*:\s*$/.test(l)) {
current = l.replace(/:\s*$/, '').trim();
obj[current] = obj[current] || {};
return;
}
var m = l.match(/^(\w[\w-]*)\s*:\s*(.*)$/);
if (m) {
var k = m[1];
var v = m[2].trim();
v = v.replace(/^['"]|['"]$/g, '');
if (current) {
obj[current][k] = v;
} else {
obj[k] = v;
}
}
});
var cfg = obj.config || obj;
directive = '%%{init: ' + JSON.stringify(cfg) + '}%%\n';
body = text.replace(fm[0], '');
}
var isV10 = typeof mermaid !== 'undefined' && mermaid.version && parseInt(mermaid.version.split('.')[0], 10) >= 10;
body = body.replace(/<br\s*\/?>(?![^`]*`)/gi, '\\n');
if (!isV10) {
body = body.split('\n').map(function(line) {
var m = line.match(/^\s*([A-Za-z0-9_]+)\s*@\{\s*([^}]*)\s*\}\s*$/);
if (m) {
var id = m[1];
var props = m[2];
var lm = props.match(/label\s*:\s*("[^"]*"|'[^']*'|[^,]+)/);
var label = lm ? lm[1].replace(/^['"]|['"]$/g, '') : id;
return id + '["' + label + '"]';
}
return line;
}).join('\n');
}
return directive + body;
}

function isMermaidLike(text) {
var t = text.trim();
if (/^%%\{init:/.test(t) || /^---\s*/.test(t)) return true;
var firstLine = t.split('\n')[0].trim();
if (/^(flowchart|sequenceDiagram|classDiagram|stateDiagram|gantt|pie)\b/.test(firstLine)) return true;
// Only treat as mermaid when 'graph' starts with a valid direction token
if (/^graph\s+(TD|TB|LR|RL)\b/.test(firstLine)) return true;
return false;
}

var toReplace = [];
$('pre > code.language-mermaid').each(function() { toReplace.push($(this)); });

toReplace.forEach(function($code) {
needMermaid = true;
return $('<pre class="mermaid">').text($(this).text());
var raw = $code.text();
var processed = preprocessMermaid(raw);
var $wrapper = $code.closest('div[class*=language-]');
var $new = $('<pre class="mermaid">').text(processed);
if ($wrapper.length) {
$wrapper.replaceWith($new);
} else {
$code.parent().replaceWith($new);
}
});

if (!needMermaid) {
Expand Down
2 changes: 1 addition & 1 deletion config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ prism_syntax_highlighting = true
[params.mermaid]
enable = true
theme = "default"
securitylevel = "strict"
securitylevel = "loose"

# User interface configuration
[params.ui]
Expand Down
153 changes: 153 additions & 0 deletions content/en/docs/eino/FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
---
Description: ""
date: "2025-12-11"
lastmod: ""
tags: []
title: FAQ
weight: 6
---

# Q: cannot use openapi3.TypeObject (untyped string constant "object") as *openapi3.Types value in struct literal; cannot use types (variable of type string) as *openapi3.Types value in struct literal

Ensure the `github.com/getkin/kin-openapi` dependency version does not exceed `v0.118.0`. Starting from Eino `v0.6.0`, Eino no longer depends on the `kin-openapi` library.

# Q: During agent streaming, it never reaches ToolsNode, or streaming is lost and appears non-streaming.

- First, update Eino to the latest version.

Different models output tool calls differently in streaming mode. Some models (e.g., OpenAI) emit tool calls directly; others (e.g., Claude) might emit text first and then the tool call. You therefore need different logic to determine whether a tool call is present in a streamed message.

The ReAct Agent `Config` has a `StreamToolCallChecker`. If omitted, the agent uses a default checker that determines a tool call by inspecting whether any non-empty early chunk contains tool calls:

```go
func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()

for {
msg, err := sr.Recv()
if err == io.EOF {
return false, nil
}
if err != nil {
return false, err
}

if len(msg.ToolCalls) > 0 {
return true, nil
}

if len(msg.Content) == 0 { // skip empty chunks at the front
continue
}

return false, nil
}
}
```

This default works when the Tool Call message contains only a tool call.

It does not fit cases where a non-empty content chunk appears before the tool call. In such cases, provide a custom checker like this:

```go
toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()
for {
msg, err := sr.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
// finish
break
}
return false, err
}
if len(msg.ToolCalls) > 0 {
return true, nil
}
}
return false, nil
}
```

Note: this custom `StreamToolCallChecker` checks all chunks for tool calls. When the model is outputting a normal answer, this may reduce “early streaming detection”, because it waits until all chunks are inspected. To preserve streaming responsiveness, try guiding the model with prompts:

> 💡
> Add prompt constraints such as: “If a tool is required, output only the tool call; do not output text.”
>
> Models vary in how much they adhere to such prompts. Tune and validate for your chosen model.

# Q: [github.com/bytedance/sonic/loader](http://github.com/bytedance/sonic/loader): invalid reference to runtime.lastmoduledatap

Older versions of `sonic` are incompatible with `go1.24`. Upgrade to `v1.13.2` or higher.

# Q: Tool input deserialization failed: `failed to invoke tool call {tool_call_id}: unmarshal input fail`

Models typically do not produce invalid JSON. Investigate the specific reason for deserialization failure; in most cases this is due to output truncation when the model’s response exceeds limits.

# Q: How can I implement batch processing nodes in Eino (like Coze’s batch nodes)?

Eino currently does not support batch processing. Two options:

1. Dynamically build the graph per request — the overhead is low. Note that `Chain Parallel` requires the number of parallel nodes to be greater than one.
2. Implement a custom batch node and handle batching inside the node.

# Q: Panic occurs in Fornax SDK or panic stack mentions Fornax SDK

Upgrade both the Fornax SDK and Eino to the latest versions and retry.

# Q: Does Eino support structured model outputs?

Yes, in two steps:

1. Ensure the model outputs structured data. Options:
- Some models support configuration for structured output (e.g., OpenAI response format).
- Use tool calls to obtain structured results.
- Prompt the model explicitly to output structured data.
2. After obtaining a structured message, use `schema.NewMessageJSONParser` to parse the message into your target struct.

# Q: In image recognition scenarios, error: `One or more parameters specified in the request are not valid`

Check whether the model supports image input (for Doubao models, only variants with `vision` support it).

# Q: How to access Reasoning Content / “thinking” output from a chat model?

If the model implementation supports Reasoning Content, it is stored in the `ReasoningContent` field of the output `Message`.

# Q: Errors include `context deadline exceeded`, `timeout`, or `context canceled`

Cases:

1. `context.canceled`: While executing a graph or agent, the user code passed a cancelable context and triggered cancellation. Investigate your application’s context-cancel logic. This is unrelated to the Eino framework.
2. `context deadline exceeded`: Two common possibilities:
1. During graph or agent execution, the user code passed a context with a timeout, which was reached.
2. A `ChatModel` or other external resource has its own timeout configured (or its HTTP client does), which was reached.

Inspect the thrown error for `node path: [node name x]`. If the node name is not a `ChatModel` or any node that performs external calls, it is likely case 2-a; otherwise, it is likely case 2-b.

If you suspect 2-a, trace upstream to find where a timeout was set on the context (common sources include FaaS platforms, gateways, etc.).

If you suspect 2-b, check whether the node itself configures a timeout (e.g., Ark ChatModel `Timeout`, or OpenAI ChatModel via an `HttpClient` with `Timeout`). If none are configured but timeouts still occur, it may be the SDK’s default timeout. Known defaults: Ark SDK 10 minutes; Deepseek SDK 5 minutes.

# Q: How to access parent graph `State` within a subgraph?

If the subgraph and parent graph have different `State` types, use `ProcessState[ParentStateType]()` to process the parent’s state. If they share the same `State` type, make the types distinct (for example, with a type alias: `type NewParentStateType StateType`).

# Q: How does `eino-ext` adapt multimodal input/output for supported models?

For multimodal support, see [https://www.cloudwego.io/docs/eino/ecosystem_integration/chat_model](https://www.cloudwego.io/docs/eino/ecosystem_integration/chat_model) and the corresponding examples for each model.

# Q: Using `UserInputMultiContent` for multimodal input, but the model side seems to miss the data or cannot read `multicontent`

Recent versions of Eino introduce `UserInputMultiContent` and `AssistantGenMultiContent` for multimodal user input and model output respectively. All `eino-ext` chat model implementations have been adapted. If the model does not receive the multimodal payload, upgrade the provider package to the latest version and try again.

# Q: After upgrading to `0.6.x`, there are breaking changes

Per the migration plan [Migration from OpenAPI 3.0 Schema Object to JSONSchema in Eino · Discussion #397](https://github.com/cloudwego/eino/discussions/397), Eino `v0.6.1` removed the dependency on `getkin/kin-openapi` and all OpenAPI 3.0-related code.

If `eino-ext` modules error with `undefined: schema.NewParamsOneOfByOpenAPIV3`, upgrade those modules to the latest versions.

If schema migration is complex, use the helper tooling in the [JSONSchema conversion guide](https://bytedance.larkoffice.com/wiki/ZMaawoQC4iIjNykzahwc6YOknXf).

# Q: Which ChatModels in `eino-ext` support the Responses API form?

By default, ChatModels generated by `eino-ext` do not support the Responses API; they support only the Chat Completions API. A special case is the Ark Chat Model, which implicitly supports the Responses API when you set `Cache.APIType = ResponsesAPI`.
64 changes: 3 additions & 61 deletions content/en/docs/eino/_index.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,14 @@
---
Description: Eino is an AI application development framework based on Go
date: "2025-02-21"
Description: Eino is a Golang-based AI application development framework
date: "2025-12-09"
lastmod: ""
linktitle: Eino
menu:
main:
parent: Documentation
weight: 6
tags: []
title: 'Eino: User Manual'
title: Eino User Manual
weight: 6
---

> Eino pronunciation: US / 'aino /, approximate sound: i know, with the hope that the application can achieve the vision of "i know"

## What is Eino

> 💡
> Eino:An AI Application Development Framework Built with Go

Eino aims to provide an AI application development framework built with Go. Eino refers to many excellent AI application development frameworks in the open-source community, such as LangChain, LangGraph, LlamaIndex, etc., and provides an AI application development framework that is more in line with the programming habits of Go.

Eino provides rich capabilities such as **atomic components**, **integrated components**, **component orchestration**, and **aspect extension** that assist in AI application development, which can help developers more simply and conveniently develop AI applications with a clear architecture, easy maintenance, and high availability.

Take ReactAgent as an example:

- It provides common components such as ChatModel, ToolNode, and PromptTemplate, and the business can be customized and extended.
- Flexible orchestration can be performed based on existing components to generate integrated components for use in business services.

<a href="/img/eino/en_eino_graph_2.png" target="_blank"><img src="/img/eino/en_eino_graph_2.png" width="100%" /></a>

## Eino Components

> One of Eino's goals is: to collect and improve the component system in the context of AI applications, so that the business can easily find some common AI components, facilitating the iteration of the business.
>
> Eino will provide components with a relatively good abstraction around the scenarios of AI applications, and provide some common implementations around this abstraction.

- The abstract definition of Eino components is in: [eino/components](https://github.com/cloudwego/eino/tree/main/components)
- The implementation of Eino components is in: [eino-ext/components](https://github.com/cloudwego/eino-ext/tree/main/components)

## Eino application scenarios

Thanks to the lightweight and in-field affinity properties of Eino, users can introduce powerful large model capabilities to their existing microservices with just a few lines of code, allowing traditional microservices to evolve with AI genes.

When people hear the term "Graph Orchestration", their first reaction might be to segment and layer the implementation logic of the entire application interface, and convert it into an orchestratable Node. The biggest problem encountered in this process is the issue of **long-distance context passing (variable passing across Node nodes)**, whether using the State of Graph/Chain or using Options for transparent passing, the entire orchestration process is extremely complex, far from being as simple as directly making function calls.

Based on the current Graph orchestration capabilities, the scenarios suitable for orchestration have the following characteristics:

- The overall focus is on the semantic processing-related capabilities of the model. Here, semantics are not limited to modalities.
- In orchestration output, there are very few nodes related to Session. Overall, the vast majority of nodes do not have processing logic at the granularity of unenumerable business entities such as users/devices.
- Whether through the State of Graph/Chain or through CallOptions, the methods for reading, writing, or transparently passing user/device granularity information are not convenient.
- Require common aspect capabilities, and build horizontal governance capabilities such as observation, rate limiting, and evaluation based on this.

What is the significance of orchestration: To aggregate, control, and present the context of long-distance orchestration elements in a fixed paradigm.

- Context of long-distance orchestration elements: Generally, orchestration elements are scattered throughout the system of the entire orchestration product, making it difficult for people to have a macroscopic and overall cognition. The role of orchestration is to gather and present this macroscopic information.
- Aggregation control and presentation: It is to easily organize and control the relationship between orchestration elements, facilitating adjustment and display.

Overall, the scenarios where "Graph Orchestration" is applicable are: business-customized AI integration components. That is, to flexibly orchestrate AI-related atomic capabilities and provide simple and easy-to-use scenario-based AI components. Moreover, in this AI component, there is a unified and complete horizontal governance capability.

- When generating the corresponding AI component, the core is to provide common cross-cutting capabilities.
- The logic inside a service interface: Generally speaking, the responsibilities are relatively single, the distribution is relatively concentrated, and the context is relatively cohesive. It does not match the applicable scenarios of "orchestration". The AI integration component can be used as an SDK in the business interface.
- Recommended usage methods

<a href="/img/eino/en_eino_recommend_using_graph.png" target="_blank"><img src="/img/eino/en_eino_recommend_using_graph.png" width="100%" /></a>

- A more challenging approach -- 【Node orchestration of the entire business process】
- Biz Handler typically focuses on business logic and has a relatively light focus on data flow, and is more suitable for development in a function stack call manner.
- If the logical division and combination are carried out in a graph orchestration manner, it will increase the difficulty of business logic development.

<a href="/img/eino/en_eino_not_recommend_of_biz.png" target="_blank"><img src="/img/eino/en_eino_not_recommend_of_biz.png" width="100%" /></a>
Loading