Commit Graph

15 Commits

Author SHA1 Message Date
QuantumGhost
349c3cf7b8 feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2025-04-30 17:28:02 +08:00
ZalterCitty
a1158cc946 fix: Update prompt message content types to use Literal and add union type for content (#17136)
Co-authored-by: 朱庆超 <zhuqingchao@xiaomi.com>
Co-authored-by: crazywoola <427733928@qq.com>
2025-04-22 16:17:55 +08:00
kenwoodjw
3306228840 fix: workflow file add related-id in iteration node (#16255)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
2025-03-21 20:57:02 +08:00
yihong
56e15d09a9 feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
-LAN-
996a9135f6 feat(llm_node): support order in text and files (#11837)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2024-12-20 14:12:50 +08:00
非法操作
c9b4029ce7 chore: the consistency of MultiModalPromptMessageContent (#11721) 2024-12-17 15:01:38 +08:00
非法操作
74fdc16bd1 feat: enhance gemini models (#11497) 2024-12-17 12:05:13 +08:00
非法操作
9e23c3d625 chore: LOCAL_FILE also try to use remote_url as Prompt message (#11443) 2024-12-10 10:56:49 +08:00
-LAN-
c5f7d650b5 feat: Allow using file variables directly in the LLM node and support more file types. (#10679)
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-11-22 16:30:22 +08:00
crazywoola
5656f81bde Revert "fix the error of unable to retrieve url from file" (#10511) 2024-11-11 08:56:19 +08:00
Xiao Ley
5eb27afd63 fix the error of unable to retrieve url from file (#10498) 2024-11-11 08:47:47 +08:00
-LAN-
25ca0278dd refactor(core): Remove extra_config from File. (#10203) 2024-11-08 18:13:24 +08:00
非法操作
033ab5490b feat: support LLM understand video (#9828) 2024-11-08 13:22:52 +08:00
非法操作
ddb960ddfb feat: support Vectorizer can be used in workflow (#9932) 2024-10-28 16:52:57 +08:00
-LAN-
e61752bd3a feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00