From 32792bdd02765426e3887be405d4033ff074c716 Mon Sep 17 00:00:00 2001 From: Roo Ryu <34956359+deepbig@users.noreply.github.com> Date: Sat, 21 Sep 2024 18:56:36 +0900 Subject: [PATCH] =?UTF-8?q?[27=EC=A3=BC=EC=B0=A8=201]=20Local=20LLM?= =?UTF-8?q?=EA=B3=BC=20=ED=95=A8=EA=BB=98=20=EC=9D=BC=ED=95=B4=EB=B3=B4?= =?UTF-8?q?=EA=B8=B0=20(#246)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Create Local LLM과 함께 일해보기.md * Update Local LLM과 함께 일해보기.md (#247) --- ...74\355\225\264\353\263\264\352\270\260.md" | 302 ++++++++++++++++++ 1 file changed, 302 insertions(+) create mode 100644 "hong/machine-learning/Local LLM\352\263\274 \355\225\250\352\273\230 \354\235\274\355\225\264\353\263\264\352\270\260.md" diff --git "a/hong/machine-learning/Local LLM\352\263\274 \355\225\250\352\273\230 \354\235\274\355\225\264\353\263\264\352\270\260.md" "b/hong/machine-learning/Local LLM\352\263\274 \355\225\250\352\273\230 \354\235\274\355\225\264\353\263\264\352\270\260.md" new file mode 100644 index 0000000..3aa9c6c --- /dev/null +++ "b/hong/machine-learning/Local LLM\352\263\274 \355\225\250\352\273\230 \354\235\274\355\225\264\353\263\264\352\270\260.md" @@ -0,0 +1,302 @@ +# Local LLM과 함께 일해보기 + +## 배경 +우아한테크코스에서 "로컬 LLM과 함께 일해보기" 발표를 보고 흥미로워서 개인적으로 실험해본 결과를 공유합니다. + +출처: +- [우아한테크세미나: 생성AI로 똑똑하게 일하는 법](https://www.youtube.com/watch?v=v2icwh-nyl4) +- [10분 만에 RAG 이해하기](https://brunch.co.kr/@ywkim36/146) + +## 목표 +- 로컬 장비를 사용해서 LLM을 실행시켜본다. +- RAG(Retrieval-Augmented Generation)을 로컬 LLM에 적용하여 대외비(비공개 또는 독점 데이터 소스) 정보에 대한 챗봇을 만든다. +- 위 기능을 편리하게 사용할 수 있는 플러그인 또는 extension을 개발해본다. + +## 사전조사 + +### Core +- [Ollama](https://ollama.com/) + - 오픈소스 LLM을 로컬 PC에서 쉽게 실행할 수 있게 해주는 도구 + - 모델 가중치, 설정, 데이터셋을 하나의 패키지로 묶어서 Modelfile로 관리 + - 다양한 LLM을 로컬 PC에서 구동 가능 + - 지원 LLM 모델(https://ollama.com/library) + - 사용자 정의 프롬프트로 모델 커스터마이징 가능 + - REST API 제공 + - 기본설정으로 실행하는 방법 ([Ollama란? Ollama 사용법: 내 PC에서 무료로 LLM 실행하기](https://www.developerfastlane.com/blog/ollama-usage-guide#지원-모델)) + +### Web +- [Open WebUI](https://docs.openwebui.com/) + - ChatGPT와 유사한 UI를 제공하는 템플릿 + - Ollama와의 연결성이 좋음 + - Ollama에서 제공하는 커스텀 기능들을 컨트롤하는 GUI 제공(예: documents, defaut propmt 등) + +### RAG(Custom Database 연동) +- RAG(Retrieval-Augmented Generation): 검색 증강 생성 +- R(Retrieval): "어디선가 가져오는 것, 집어오는 것". 즉 어딘가에 가서 요청된 무엇인가를 집어온다는 뜻 +- A(Augmented): "증강되었다". 즉 원래 것에 뭔가 덧붙이거나 보태어 더 충실하게 좋아졌다는 뜻 +- G(Generation): "생성". 즉 프롬프트라고 하는 사용자의 질문/질의에 대한 응답을 텍스트로 생성하는 것을 뜻. +- RAG를 사용하면, LLM이 맘대로 대답하기 전에 먼저 지식 콘텐츠 저장소에 들려서 "사용자 잘의와 관련된 정보를 검색"하라는 순서가 추가되는 것. 이것이 바로 찾아오기(Retrieval), 그리고 그 대답이 풍부한 컨텍스트로 증강(Augmented)되는 것. +- +- [LangChain](https://www.langchain.com/) + - LangChain은 LL을 활용한 애플리케이션 개발에 특화된 오픈소스 프레임워크 + - 기존의 언어 모델이 주로 텍스트 생성에 중점을 둔 반면, LangChain은 다양한 외부 데이터 소스와 통합하여 보다 복잡하고 유용한 애플리케이션을 만들 수 있도록 설계됨. + - [Build a Retrieval Augmented Generation (RAG) App](https://python.langchain.com/v0.2/docs/tutorials/rag/) + - Ollama에 적용 방법: [ChatOllama | LangChain](https://python.langchain.com/v0.2/docs/integrations/chat/ollama/) + +## 적용 + +### Ollama 설치 및 실행 +- Ollama 다운받기: https://ollama.com/download +- 설치 후 터미널 열어서 아래 커맨드 실행 + ``` + ollama run llama3.1 + ``` + + - llama3.1 기준 약 5GB 사용됨. +- 테스트 실행 결과 + + ``` + roo-mac:~ roo$ ollama run llama3.1 + pulling manifest + pulling 8eeb52dfb3bb... 100% ▕████████████████████████████████████████████████████████████████████▏ 4.7 GB + pulling 73b313b5552d... 100% ▕████████████████████████████████████████████████████████████████████▏ 1.4 KB + pulling 0ba8f0e314b4... 100% ▕████████████████████████████████████████████████████████████████████▏ 12 KB + pulling 56bb8bd477a5... 100% ▕████████████████████████████████████████████████████████████████████▏ 96 B + pulling 1a4c3c319823... 100% ▕████████████████████████████████████████████████████████████████████▏ 485 B + verifying sha256 digest + writing manifest + success + >>> /? + Available Commands: + /set Set session variables + /show Show model information + /load Load a session or model + /save Save your current session + /clear Clear session context + /bye Exit + /?, /help Help for a command + /? shortcuts Help for keyboard shortcuts + + Use """ to begin a multi-line message. + ``` + +- 동화책을 쓰는데 필요한 아이디어 질문해보기 + ``` + >>> I need to write a fairytale. What is the most important things that I need to consider? + Writing a fairytale can be a delightful and imaginative experience. Here are some essential considerations to help you craft a + magical story: + + 1. **Know your genre**: Fairytale is a specific type of storytelling, often featuring fantastical elements, moral lessons, and + happy endings. Understand the characteristics that define this genre. + 2. **Identify your target audience**: Who will be reading or listening to your fairytale? Children? Adults? Both? Tailor your + story's tone, language, and themes accordingly. + 3. **Develop a compelling narrative structure**: + * Establish a clear three-act structure: Setup, Confrontation, and Resolution. + * Include a clear inciting incident that sets the story in motion. + * Ensure a satisfying conclusion, often with a moral lesson or message. + 4. **Create memorable characters**: + * Give your protagonist (hero) relatable goals, motivations, and flaws to make them more human-like. + * Develop interesting antagonists or challenges that create tension and conflict. + * Consider adding secondary characters to enrich the story. + 5. **Magic and fantasy elements**: Incorporate magical or fantastical aspects that fit within the fairytale genre. Be mindful + of how these elements will be perceived by your target audience. + 6. **Setting**: Choose a rich, immersive setting that complements the narrative and enhances the overall atmosphere. This can + include enchanted forests, mythical lands, or even ordinary worlds with extraordinary twists. + 7. **Themes and symbolism**: Incorporate universal themes, such as: + * Good vs. Evil + * Love and friendship + * Courage and perseverance + * Self-discovery and growth + * Emphasize the importance of these themes through your character's journey. + 8. **Language and style**: + * Use language that is evocative, lyrical, and engaging. + * Employ a narrative voice that is suitable for your target audience (e.g., formal or informal). + * Consider using metaphors, allegories, or allusions to add depth and richness to the story. + 9. **Cultural sensitivity**: Be aware of cultural influences and traditions that may be associated with fairytale tropes. + Avoid stereotypes and ensure respectful representation. + 10. **Edit and refine**: + * Revise your story based on feedback from peers, mentors, or writing communities. + * Pay attention to pacing, tension, and plot inconsistencies. + * Make sure the story flows logically and is cohesive. + + Remember, these guidelines are not exhaustive, and you can always experiment and innovate within the fairytale genre. Most + importantly, have fun crafting your magical tale! + + Which aspect of writing a fairytale do you want to explore further? + + >>> Do you know virtuoso of fairytale in the world? Please share three options you know. + There are many renowned storytellers and authors who have made significant contributions to the world of fairytales. Here are + three virtuosos of fairytale that I'd like to share: + + 1. **The Brothers Grimm**: Jacob (1785-1863) and Wilhelm Grimm (1786-1859) were German academics, linguists, and cultural + researchers who collected and published folktales from across Europe. Their book "Kinder- und Hausmärchen" (1812), translated + as "Children's and Household Tales," contains 200 classic fairytales that are still widely known today, including: + * Cinderella + * Snow White + * Sleeping Beauty + * Little Red Riding Hood + * Rumpelstiltskin + * Hansel and Gretel + + The Brothers Grimm's work helped preserve the rich cultural heritage of European folktales, making them an essential part of + our shared storytelling legacy. + + 2. **Hans Christian Andersen** (1805-1875): A Danish author and poet, Andersen is best known for his beautiful and poignant + fairytales that have become beloved classics worldwide. Some of his most famous works include: + * The Little Mermaid + * The Emperor's New Clothes + * Thumbelina + * The Ugly Duckling + * The Steadfast Tin Soldier + + Andersen's stories often explored themes of love, identity, and the human condition, making them relatable to readers across + cultures. + + 3. **Walt Disney**: While not a traditional writer or storyteller, Walt Disney (1901-1966) revolutionized the way fairytales + were told and shared with the world through his iconic animated films. His adaptations of classic stories, such as: + * Snow White (1937) + * Pinocchio (1940) + * Cinderella (1950) + * Sleeping Beauty (1959) + + Disney's work helped popularize fairytales among a broader audience and paved the way for new generations of storytellers. + + These three virtuosos have contributed significantly to the world of fairytale, shaping our shared cultural heritage and + inspiring countless adaptations, interpretations, and retellings. + + >>> You are Walt Disney, the virtuos of fairytales. Can you suggest a theme and lesson of my fairytale as Walt Disney? + Partner! I'd be delighted to help you with that. Let me put on my thinking cap and imagine a tale that's just right for ya. + + **Theme: The Power of Courage and Self-Discovery** + + My suggestion is to create a story around the theme of courage and self-discovery. You see, kiddo, every individual faces their own + unique challenges in life. But it's how we respond to those challenges that matters. I want your fairytale to inspire young readers (and + not-so-young ones too!) to be brave, take risks, and trust themselves to find their way through the ups and downs of life. + + **Lesson: Believe in Yourself** + + The lesson is simple yet profound: believe in yourself! Your protagonist should learn that they have the power within themselves to + overcome obstacles, make wise decisions, and discover their true potential. It's not about being perfect; it's about being brave enough + to try, to fail, and to keep trying again. + + **Story Idea: "The Luminous Leaf"** + + Here's a tale idea to get you started: + + In a mystical forest, there lives a young girl named Luna who possesses a rare gift – the ability to communicate with plants. A wise old + tree, known as the Guardian of the Forest, tells Luna that she is the chosen one, destined to restore balance to the natural world by + finding the fabled Luminous Leaf. + + Luna embarks on an epic journey, facing fearsome creatures and overcoming treacherous obstacles along the way. As she draws closer to her + goal, she discovers that the true power of courage lies not in external validation but in trusting her own instincts and inner wisdom. + + **Supporting Characters:** + + 1. **Aphid**: A tiny, quick-witted insect who becomes Luna's trusted friend and confidant. + 2. **Bramble**: A gentle, yet fiercely protective hedgehog who teaches Luna about the importance of self-care and boundaries. + 3. **Gnarlgold**: A wise, ancient badger who shares tales of the forest's history and helps Luna understand her role in restoring balance + to nature. + + **The Luminous Leaf:** + + This magical artifact represents the power of inner light within each individual. It symbolizes that we all have a unique gift or talent + waiting to be discovered and shared with the world. When Luna finally finds the Luminous Leaf, she realizes that its true purpose is not + just for herself but also for inspiring others to believe in their own potential. + + Now, my friend, I want you to remember that this is your story to tell. Feel free to add, modify, or discard any elements as you see fit! + What matters most is the heart and soul you put into crafting a tale that will captivate and inspire young readers. + + So, are ya ready to get started on "The Luminous Leaf"? + ``` +- 한줄평: + - 적용하기 너무 쉬워서 살짝 당황쓰.. + - 생각보다 결과 값이 잘 나온다.. 👍 + +### Open WebUI 기본 설정으로 연동하기 +- 아래 커맨드로 docker로 실행하기 + ``` + docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main + ``` +- [http://localhost:3000](http://localhost:3000/)로 접속하면 끝. +- 결과: +Screenshot 2024-08-29 at 7 38 27 PM + +- 한줄평: + - 이것도 적용하기 너무 쉬워서 당.. 당황쓰.. + - 생각보다 UI가 이쁘게 잘나왔다.. 👍 + +### Ollama & Open WebUI 연동 테스트 +- 위의 로컬에서 띄운 Open WebUI (http://localhost:3000)에서 model(llama3.1)을 선택한 후 채팅하면 끝. +- 결과 (위의 CLI에서 질문한 내용과 동일하게 실행함): +Screenshot 2024-08-29 at 7 39 13 PM + +- 한줄평: + - 이렇게 쉽게 적용할 수 있는 줄 알았으면 진즉 해볼걸.. 👍👍 + +### Custom Dataset 수집 및 Ollama에 연동 +- Open WebUI에서 Ollama에서 기본적으로 제공하는 System Prompt, Documents, Tools, Filters, Actions, Tags, Functions을 쉽게 사용할 수 있는 GUI를 제공한다. +- 학습시킬 데이터가 잘 변하지 않고, 수량이 많지 않다면 해당 기능을 활용하면 LangChain 없이도 RAG가 가능! +Screenshot 2024-08-29 at 7 30 38 PM + +## Action Items +- 한국어 최적화된 모델을 찾아서 적용하기 +- 사내 자료에 대한 질의응답이 가능하도록 모델 개선하기 + +### 사전 리서치 +- [Ollama와 Llama3 모델로 나만의 챗GPT 만들기](https://www.youtube.com/watch?v=GGPTbZ3uG2U) + - 한국어 튜닝된 모델 + - Google Colab + - [Google Colab features you may have missed](https://www.youtube.com/watch?v=rNgswRZ2C1Y) + - Colaboratory(Colab)을 통해 브라우저 내에서 Python 스크립트를 작성하고 실행할 수 있음. + - 구성이 필요 없음 + - 무료로 GPU 사용 + - 간편한 공유 + - Open WebUI 세부 설정 확인(Google Colab에서 띄워둔 LLM 연동 가능) + - [외부 문서 사용으로 정확도 높이기 (Ollama RAG)](https://youtube.com/watch?v=_0M7nInr4Us) + - 채팅창에 질문할 때 외부 문서를 업로드하고 질문을 하면 정확한 답을 얻을 수 있음. + - LLM을 대할 때는 “자나치게 열정적인 신입사원을 대하듯이” + - pdf 문서로 업로드 함 (txt, pdf 다 가능 ← 외부 자료가 잘 파싱되지 않았는지 확인 필요) + - 평소에 자주 업로드하는 부분은 workspace > documents에 업로드해두기 + - 채팅창에서 `#` 를 입력하면 해당 문서를 참조할 수 있음. + - [Ollama 한국어 모델 사용하기 (Open WebUI](https://www.youtube.com/watch?v=VrVTWX0H44I&list=PLVjIhc0am1qIbIri-OtwAQRFmc-4AXQ_v&index=2) + - Ollama에 없는 모델을 import 해서 사용하기 + - https://github.com/ollama/ollama/blob/main/docs/import.md + - GGUF 파일의 경우, Modelfile을 만들어서 import하기 + - Hugging Face: 다양한 LLM을 찾을 수 있음. + - [huggingface Models](https://huggingface.co/models?library=gguf&sort=trending&search=korean) + - [yanolja](https://huggingface.co/yanolja)/[**EEVE-Korean-Instruct-10.8B-v1.0](https://huggingface.co/yanolja/EEVE-Korean-Instruct-10.8B-v1.0) upstage/SOLAR-10.7B 모델을 야놀자에서 Finetuning한 모델 사용. + - https://github.com/ollama/ollama/blob/main/docs/modelfile.md + - custom 설정 가능. + - 예시 Modelfile: + + ``` + FROM ./ggml-model-Q5_K_M.gguf + + PARAMETER temperature 0.6 + PARAMETER top_p 0.9 + + TEMPLATE """{{ if .System }}<|im_start|>system + {{ .System }}<|im_end|> + {{ end }}{{ if .Prompt }}<|im_start|>user + {{ .Prompt }}<|im_end|> + {{ end }}<|im_start|>assistant + """ + + PARAMETER stop <|im_start|> + PARAMETER stop <|im_end|> + + SYSTEM """당신은 유용한 AI 어시스턴트입니다. 사용자의 질의에 대해 친절하고 정확하게 답변해야 합니다. + You are a helpful AI assistant, you'll need to answer users' queries in a friendly and accurate manner.""" + ``` + + - Modelfile로 모델 생성하기 + + ``` + ollama create EEVE-Ko -f Modelfile + ``` + + - Open WebUI에서 모델 튜닝하기 + + +Screenshot 2024-09-01 at 6 59 30 PM +