finetune.md

微调大语言模型 LLMs

QLoRA 微调 InternLM

• InternLM-7B, oasst1

  xtuner train internlm_7b_qlora_oasst1_e3

• InternLM-7B, Arxiv Gentitle

  xtuner train internlm_7b_qlora_arxiv_gentitle_e3

• InternLM-7B, Colorist

  xtuner train internlm_7b_qlora_colorist_e5

• InternLM-7B, Coder

  xtuner train internlm_7b_qlora_code_alpaca_e3

• InternLM-7B, SQL

  xtuner train internlm_7b_qlora_sql_e3

• InternLM-7B, Lawyer

  xtuner train internlm_7b_qlora_lawyer_e3

• InternLM-7B, Open-Platypus

  xtuner train internlm_7b_qlora_open_platypus_e3

• InternLM-7B, Alpaca-enzh

  xtuner train internlm_7b_qlora_alpaca_enzh_e3

QLoRA 微调 Llama2

在使用 Llama2 之前，请先使用 `huggingface-cli login`` 输入你的访问令牌（access token）！查看这里了解如何获取访问令牌（access token）。

• Llama2-7B, MOSS-003-SFT (插件！)

  NPROC_PER_NODE=8 xtuner train llama2_7b_qlora_moss_sft_all_e2_gpu8  # Recommended!
  xtuner train llama2_7b_qlora_moss_sft_all_e1

• Llama2-7B, Arxiv Gentitle

  xtuner train llama2_7b_qlora_arxiv_gentitle_e3

• Llama2-7B, Colorist

  xtuner train llama2_7b_qlora_colorist_e5

QLoRA 微调 Qwen

• Qwen-7B, MOSS-003-SFT (插件！)

  NPROC_PER_NODE=8 xtuner train qwen_7b_qlora_moss_sft_all_e2_gpu8  # Recommended!
  xtuner train qwen_7b_qlora_moss_sft_all_e1

• Qwen-7B, oasst1

  xtuner train qwen_7b_qlora_oasst1_e3

• Qwen-7B, Arxiv Gentitle

  xtuner train qwen_7b_qlora_arxiv_gentitle_e3

• Qwen-7B, Alpaca-enzh

  xtuner train qwen_7b_qlora_alpaca_enzh_e3

QLoRA 微调 Baichuan

• Baichuan-7B, oasst1

  xtuner train baichuan_7b_qlora_oasst1_e3

• Baichuan-7B, Arxiv Gentitle

  xtuner train baichuan_7b_qlora_arxiv_gentitle_e3

• Baichuan-7B, Alpaca-enzh

  xtuner train baichuan_7b_qlora_alpaca_enzh_e3