Ollama

[werruww]

How fun it with ollama? or huggingface model free without api

[rejojer]

Yes! You can run PageIndex with local models — e.g., via Ollama or Hugging Face — no paid API needed.

[Justin-12138]

@werruww It seems we need to modify the utils.py,is that right? @rejojer

[werruww]

how??????

[werruww]

attempts

https://github.com/werruww/suc-PageIndex-ollama/blob/main/suc_PageIndex.ipynb

[electroheadfx]

how you connect to ollama ? is it mean its openai compatible, I dont see where to add openai url for exemple : http://locahost:8000/v1
thanjs

[thekarmakazi]

I would love to get this working with ollama or vllm. Is this being considered or does anyone have a working example?

[Abdelilah04116]

I have a question: would it be better to use Ollama or Transformers? I'm looking for strong inference performance and well-structured responses

[KylinMountain]

Ollama works out of the box now — no utils.py edits needed. Just set the model name:

python run_pageindex.py --pdf_path doc.pdf --model ollama/llama3

LiteLLM defaults to http://localhost:11434; override with OLLAMA_API_BASE in .env.

@electroheadfx for an OpenAI-compatible URL, use --model openai/<name> with OPENAI_API_BASE=http://localhost:8000/v1 and OPENAI_API_KEY=any-placeholder. Full provider list: https://docs.litellm.ai/docs/providers

Heads-up: small local models can struggle with the structured JSON the pipeline expects — a capable instruct model works best (we're hardening this in #333).

Closing as answered!