Model Runtime (#1858)

Co-authored-by: StyleZhang <jasonapring2015@outlook.com> Co-authored-by: Garfield Dai <dai.hai@foxmail.com> Co-authored-by: chenhe <guchenhe@gmail.com> Co-authored-by: jyong <jyong@dify.ai> Co-authored-by: Joel <iamjoel007@gmail.com> Co-authored-by: Yeuoly <admin@srmxy.cn>
2024-01-02 23:42:00 +08:00
parent e91dd28a76
commit d069c668f8
807 changed files with 171310 additions and 23806 deletions
--- a/api/core/model_runtime/README.md
+++ b/api/core/model_runtime/README.md
@@ -0,0 +1,69 @@
+# Model Runtime
+
+This module provides the interface for invoking and authenticating various models, and offers Dify a unified information and credentials form rule for model providers.
+
+- On one hand, it decouples models from upstream and downstream processes, facilitating horizontal expansion for developers,
+- On the other hand, it allows for direct display of providers and models in the frontend interface by simply defining them in the backend, eliminating the need to modify frontend logic.
+
+## Features
+
+- Supports capability invocation for 5 types of models
+
+  - `LLM` - LLM text completion, dialogue, pre-computed tokens capability
+  - `Text Embedding Model` - Text Embedding, pre-computed tokens capability
+  - `Rerank Model` - Segment Rerank capability
+  - `Speech-to-text Model` - Speech to text capability
+  - `Moderation` - Moderation capability
+
+- Model provider display
+
+  ![image-20231210143654461](./docs/en_US/images/index/image-20231210143654461.png)
+
+  Displays a list of all supported providers, including provider names, icons, supported model types list, predefined model list, configuration method, and credentials form rules, etc. For detailed rule design, see: [Schema](./schema.md).
+
+- Selectable model list display
+
+  ![image-20231210144229650](./docs/en_US/images/index/image-20231210144229650.png)
+
+  After configuring provider/model credentials, the dropdown (application orchestration interface/default model) allows viewing of the available LLM list. Greyed out items represent predefined model lists from providers without configured credentials, facilitating user review of supported models.
+
+  In addition, this list also returns configurable parameter information and rules for LLM, as shown below:
+
+  ![image-20231210144814617](./docs/en_US/images/index/image-20231210144814617.png)	
+
+  These parameters are all defined in the backend, allowing different settings for various parameters supported by different models, as detailed in: [Schema](./docs/en_US/schema.md#ParameterRule).
+
+- Provider/model credential authentication
+
+  ![image-20231210151548521](./docs/en_US/images/index/image-20231210151548521.png)
+
+  ![image-20231210151628992](./docs/en_US/images/index/image-20231210151628992.png)
+
+  The provider list returns configuration information for the credentials form, which can be authenticated through Runtime's interface. The first image above is a provider credential DEMO, and the second is a model credential DEMO.
+
+## Structure
+
+![](./docs/en_US/images/index/image-20231210165243632.png)
+
+Model Runtime is divided into three layers:
+
+- The outermost layer is the factory method
+
+  It provides methods for obtaining all providers, all model lists, getting provider instances, and authenticating provider/model credentials.
+
+- The second layer is the provider layer
+
+  It provides the current provider's model list, model instance obtaining, provider credential authentication, and provider configuration rule information, **allowing horizontal expansion** to support different providers.
+
+- The bottom layer is the model layer
+
+  It offers direct invocation of various model types, predefined model configuration information, getting predefined/remote model lists, model credential authentication methods. Different models provide additional special methods, like LLM's pre-computed tokens method, cost information obtaining method, etc., **allowing horizontal expansion** for different models under the same provider (within supported model types).
+
+
+
+## Next Steps
+
+- Add new provider configuration: [Link](./docs/en_US/provider_scale_out.md)
+- Add new models for existing providers: [Link](./docs/en_US/provider_scale_out.md#AddModel)
+- View YAML configuration rules: [Link](./docs/en_US/schema.md)
+- Implement interface methods: [Link](./docs/en_US/interfaces.md)