Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ollama support #6

Open
ImVexed opened this issue May 20, 2024 · 1 comment
Open

Ollama support #6

ImVexed opened this issue May 20, 2024 · 1 comment

Comments

@ImVexed
Copy link

ImVexed commented May 20, 2024

Would be nice to support self hosted LLMs. It doesn't have to be ollama, but it seems to be fairly easy to interface with.

@jank
Copy link

jank commented Jul 3, 2024

I have quite limited knowledge on TS or yarn. By poking around I came to the following minimal changes required to talk to a local LLM hosted with Ollama (in my case a Gemma 2B).

main...jank:omniplex:main

Please note that you need a an OpenAI key and Bing key in your environment. OpenAI Key is required as GPT-3 is used for the 'tool' selection process (e.g. if to use search or stocks or weather). Replacing that by Ollama will likely require a fine-tuned prompt.

Also note that I had to comment out the Firebase Authentication. Otherwise the web app would not correctly start.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants