The patterns are not followed

[ParthipPR]

I have installed fabric as per the readme file
And I use ollama with the model llama2

yt --transcript (youtube link) | fabric -sp extract_wisdom 

I have set llama2 as default model

The output just provides a summary of the transcript and doesn't follow the pattern extract_wisdom
Can somebody help me with this issue?

[HeyZeus1232]

I am having this exact problem. i have tried many different local models, and they all do this. It might be this needs a huge model to work. like 70b or something. I've tried 13b - 20+ ones as well but same thing.

[HeyZeus1232]

I figured out the problem, its 'context length' The context is set too low, even using a model with a big context, you have to set context length to a higher amount. I don't know how to do this with Fabric.

I tried this with Web UI, i used a 7b model with a huge context, i manually set the context length higher in the web ui settings, copy and pasted the pattern i wanted to use, and it worked perfectly. it was slow, since higher context = more vram.

[timrohrbaugh]

If you use local models hosted by lets say ollama you create a model file in ollama and set num_ctx to the amount the model supports. Qwen27b supports 131k for instance. The new modified model us created through the ollama create statement so now make that new model name the default model or call it direct in fabric with the --model switch

[Cyber-Kaeh]

This is to piggyback on what @timrohrbaugh said. Thank you for the input @timrohrbaugh but I couldn't quite get my head around what you were talking about, it did help lead me in the right direction for what to even look for as a solution to the problem. Here is an article another user posted in the issues section that walks through creating a custom model:
https://medium.com/@celobusana/solving-fabric-and-local-ollama-context-issues-a-step-by-step-guide-1d67e443e27e
As a side note the 4096 context length still wasn't enough, I had to increase it to 8192 for it to use the extract_wisdom pattern on a 36 minute YouTube video.

[timrohrbaugh]

Sorry it took so long to see this. Make sure you do a WC (word count) of any context and then keep track of session tokens that are consumed prior. if in doubt do a session restart but make sure you count any SYSTEM, or MESSAGE data you insert in the model file. When you extend he context limit you probably have found out by now that each model architecture can produce different linear increases in VRAM consumption. The best way to test is run a model then in session extent the context with /set... ask a question then check memory footprint. keep extending as far as you can within the model limits. then take what you learned and place it in the model file so each time it instantiates. its ready for the pattern and the total context you submit.