ALTERNATIVE TO OPEN AI!!!

[AXIOM-XD]

Hey folks! I really would like to see an alternative llm in langflow. I would like to be able to use at least the HuggingfaceHub models, better would be even custom model on my local machine. I think this is for all beneficial and could free us developers from private costs and code blindness. :(((

[mtremoulet]

Have you tried the llamacpp LLM? I'm still debugging the flows but I think I'm able to use that against 7B and 13B models I have running locally. (I think. Still working out the rough spots.)

[123385918]

Have you use llamacpp successfuly? I want use RWKV-14B localy.

[rodgermoore]

I am able to run models using the CTransformers wrapper but I can´t seem to be able to use GPU inference. So for now it isn´t workable yet.

[ogabrielluiz]

What would be other steps to take to make it work on GPU? It seems strange that it wouldn't work because at the moment, we instantiate the object just as you would in LangChain.

[carlosrcoelho]

Hey!

Thanks for sharing your question here. I'm sorry for not getting back to you sooner.
Langflow has gone through some major changes since then. I recommend trying out the new version for the best experience. If you have any issues, please open a new one; we'll be here to help.