手把手教你搭建免费的大模型应用

myzbx 2025-01-04 21:51 15 浏览

许多用户因为大模型高昂的调用费用和技术门槛而倍感困扰，即便有众多创新的想法，也往往因为无法顺利实现而受阻。

本文将指导您亲手搭建属于自己的本地大模型服务，让您无拘无束地实现创意，并享受低成本的便利。更重要的是，您还将能够拓展更多开源大模型，为生活和工作效率带来质的飞跃。

一、使用效果

本地部署 LLaMA2-7B，模型大小：3.8GB，效果如下：

1、示例一：who are you

>>> who are you
I'm LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner. I'm here to help you with any
questions or topics you'd like to discuss! How can I assist you today?

2、示例二：Start work on Monday, a week full of vitality and advantages.

>>> Start work on Monday, a week full of vitality and advantages.
Sounds great! I'm glad to hear that you have a productive week ahead of you. Here are some positive affirmations to help you start your week on a high note:


* I am excited and ready to tackle the challenges of the week ahead.
* I am confident in my abilities and know that I can accomplish great things.
* I embrace this week with an open mind and heart, ready to learn and grow.
* I am grateful for the opportunities that lie ahead and look forward to making the most of them.
* I am energized and motivated to make this week a successful and fulfilling one.


Remember, the power to make your week great lies within you. Stay positive, stay focused, and always believe in yourself!

3、示例三：你会中文吗

>>> 你会中文吗
我們可以進行中文交流，但是我的語言能力 limited，因為我的訓練只包括一些基本的中文句子和phrases。如果你有任何特定的問題或話題想要談論，我將try my best to help you。

二、仓库地址

使用 Llama 2、Mistral、Gemma 和其他大型语言模型。

https://github.com/jmorganca/ollama

三、模型库

1、模型库地址

https://ollama.com/library/llama2

Model	Parameters	Size
Llama 2	7B	3.8GB
Mistral	7B	4.1GB
Dolphin Phi	2.7B	1.6GB
Phi-2	2.7B	1.7GB
Neural Chat	7B	4.1GB
Starling	7B	4.1GB
Code Llama	7B	3.8GB
Llama 2 Uncensored	7B	3.8GB
Llama 2 13B	13B	7.3GB
Llama 2 70B	70B	39GB
Orca Mini	3B	1.9GB
Vicuna	7B	3.8GB
LLaVA	7B	4.5GB

2、内存要求

7b 模型通常需要至少 8GB 的 RAM；
13b 模型通常需要至少 16GB 的 RAM；
70b 模型通常需要至少 64GB 的 RAM；

四、代码示例

1、下载源码

按照你的操作系统选择不同的下载地址。

# macOS
https://ollama.com/download/Ollama-darwin.zip

# Windows preview
https://ollama.com/download/OllamaSetup.exe

# Linux
curl -fsSL https://ollama.com/install.sh | sh

2、启动服务

ollama run llama2

3、本地请求

本地启动服务后，可通过 API 请求数据：

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
 }'

4、API 数据

API 数据按照流式返回：

{"model":"llama2","created_at":"2023-12-25T02:23:41.333728Z","response":"\n","done":false}
{"model":"llama2","created_at":"2023-12-25T02:23:41.587704Z","response":"The","done":false}
{"model":"llama2","created_at":"2023-12-25T02:23:41.787339Z","response":" sky","done":false}
{"model":"llama2","created_at":"2023-12-25T02:23:41.97912Z","response":" appears","done":false}
.....
{"model":"llama2","created_at":"2023-12-25T02:24:23.541595Z","response":".","done":false}
{"model":"llama2","created_at":"2023-12-25T02:24:23.755337Z","response":"","done":true,"context":[518,25580,29962,3532,14816,29903,29958,5299,829,14816,29903,6778,13,13,11008,338,278,14744,7254,29973,518,29914,25580,29962,13,13,1576,14744,5692,7254,1363,310,263,27791,265,2000,9596,280,1141,14801,292,29889,1932,6575,4366,24395,11563,29915,29879,25005,29892,372,2094,1309,2153,21577,13206,21337,310,330,2129,1316,408,21767,307,1885,322,288,28596,29889,4525,13206,21337,14801,278,3578,297,599,18112,29892,541,896,14801,20511,313,9539,29897,281,6447,1477,29879,901,1135,5520,313,1127,29897,281,6447,1477,29879,29889,910,338,2998,408,9596,280,1141,14801,292,29889,13,13,2887,263,1121,310,445,14801,292,29892,278,7254,3578,338,29106,287,10106,278,25005,29892,6820,278,14744,967,17443,7254,2927,29889,450,2654,3578,29892,373,278,916,1361,29892,14517,7812,1549,278,25005,411,2217,14801,292,29892,607,338,2020,278,28205,5692,901,2654,1135,278,14744,29889,13,13,1576,7254,2927,310,278,14744,508,884,367,15201,491,916,13879,1316,408,21180,918,29892,19786,29892,322,4094,325,26191,297,278,25005,29892,607,508,14801,278,3578,297,1422,5837,322,7738,263,3464,310,11955,29889,2398,29892,278,1667,4556,310,278,7254,2927,310,278,14744,338,9596,280,1141,14801,292,29889],"total_duration":56899446480,"load_duration":10037867844,"prompt_eval_count":26,"prompt_eval_duration":4662218000,"eval_count":210,"eval_duration":42167419000}

5、流式输出

前端拿到数据后通过 SSE 协议进行流式输出：

SSE：Server-Sent Events (SSE) 是一种基于 HTTP 协议的服务器推送技术，它允许服务器向客户端发送数据和信息。与 WebSocket 不同，SSE 是一种单向通信方式，只有服务器可以向客户端推送消息。SSE 是 HTML5 规范的一部分，使用非常简单，主要由服务端与浏览器端的通讯协议（HTTP协议）和 EventSource 接口来处理 Server-sent events 组成，服务器端的响应的内容类型是“text/event-stream”。

6、监听数据变化

前端实现非常简单，创建一个 SSE 的实例然后监听数据的状态即可：

let reply = ''
    const evtSource = new EventSource(`${baseUrl}/chat?content=你好`);
    evtSource.onmessage = (event) => {
      //  流式数据最后一条的状态为[DONE]代表结束返回
      if(event.data === '[DONE]') {
        console.log(`回答完毕！：${reply}`)
      } else {
        const res = JSON.parse(event.data)
        reply += res.choices[0].delta.content || ''
      }
    }