ubuntu 22.04 安裝Text Generation WebUI

設定固定IP選1或2

1.sudo nano /etc/netplan/00-installer-config.yaml

設定固定IP

network:

  version: 2

  ethernets:

    ens32:

      dhcp4: false

      addresses: [192.168.0.50/24]

      routes:

      - to: default

        via: 192.168.0.1

      nameservers:

        addresses: [1.1.1.1,8.8.8.8]

重啟網路設定

sudo netplan apply

sudo apt update

sudo apt upgrade

sudo apt install python3-pip python3-numpy python3-requests python3-matplotlib

wget https://github.com/oobabooga/text-generation-webui/releases/download/installers/oobabooga_linux.zip

sudo apt install unzip

unzip oobabooga_linux.zip

cd oobabooga_linux/

chmod +x *.sh

./start_linux.sh

出現 

What is your GPU

A) NVIDIA

B) AMD (Linux/MacOS only. Requires ROCm SDK 5.4.2/5.4.3 on Linux)

C) Apple M Series

D) None (I want to run in CPU mode)

依電腦現狀選擇。

最後出現，表示沒有安裝model

*******************************************************************

* WARNING: You haven't downloaded any model yet.

* Once the web UI launches, head over to the bottom of the "Model" tab and download one.

*******************************************************************

用winscp將Nous-Hermes-13b-Chinese.ggmlv3.q5_K_M.bin 複製到 /xxx/oobabooga_linux/text-generation-webui/models

cd oobabooga_linux/

修改webui.py，修改如下

sudo nano webui.py

填入--auto-devices引數自動分配計算任務給CPU和GPU，gpu-memory設定最多分配2GB VRAM（防止OOM，數值視您的顯卡VRAM而定)--model指定要載入的模型。--chat以聊天模式啟動。--listen則是可從外部網路存取。

於是該段就會變成這樣：

#CMD_FLAGS = '--chat --listen --auto-devices --gpu-memory 2 --model Nous-Hermes-13b-Chinese.ggmlv3.q5_K_M.bin

CMD_FLAGS = '--chat --listen --cpu --model Nous-Hermes-13b-Chinese.ggmlv3.q5_K_M.bin'

./start_linux.sh

開啟瀏覽器大Http://IP:7860

選上面Model看Nous-Hermes-13b-Chinese.ggmlv3.q5_K_M.bin看是否有進入

選上面Chat settings後選Instruction template，在Change this according to the model/LoRA that you are using. Used in instruct and chat-instruct modes.下選ChatGLM 或Chinese-Vicuna-Chat，然後選Text generation在Input即可鍵入想要問的問題