Edit history

Deleted

I’m coding a Kiwix mcp (I assume you’re using kiwix as your wikipedia source?) that might be of interest to you. I’ll post about it on !localLlama when done.

I don’t like any of the extant Kiwix MCP tools that I found because they leave formulation, ranking, search and interpretation up to the LLM.

I’m looking at a different way to get exact spans with re-ranking via TinyBERT right in front of the llm so that it doesn’t have to do 3 jobs and hallucinate itself into stupid, which is what small LLMs tend to do when given multi-hop tools.

I’m testing it with Ling Tiny 3, miniCPM5-2b, Granite 4-3b and Qwen 3.5-2b and 4b, with the goal of making sure that it actually generalizes. It’s been a massive pain in the ass and I’ve basically proven RAG from first principles…but when I’m done, you won’t need to vectorize the source into any kind of database nor “just trust me bro” the llm is searching Kiwix properly.

I’m coding a Kiwix mcp (I assume you’re using kiwix as your wikipedia source?) that might be of interest to you. I’ll post about it on !localLlama when done.

I don’t like any of the extant Kiwix MCP tools that I found because they leave formulation, ranking, search and interpretation up to the LLM.

I’m looking at a different way to get exact spans with re-ranking via TinyBERT right in front of the llm so that it doesn’t have to do 3 jobs and hallucinate itself into stupid, which is what small LLMs tend to do when given multi-hop tools.

I’m testing it with Ling Tiny 3, miniCPM5-2b, Granite 4-3b and Qwen 3.5-2b and 4b, with the goal of making sure that it actually generalizes. It’s been a massive pain in the ass and I’ve basically proven RAG from first principles…but when I’m done, you won’t need to vectorize the source into any kind of database nor “just trust me bro” the llm.

I’m coding a Kiwix mcp (I assume you’re using kiwix as your wikipedia source?) that might be of interest to you. I’ll post about it on !localLlama when done.

I don’t like any of the extant Kiwix MCP tools that I found because they leave formulation, ranking, search and interpretation up to the LLM.

I’m looking at a different way to get exact spans with re-ranking via TinyBERT right in front of the llm so that it doesn’t have to do 3 jobs and hallucinate itself into stupid.

I’m testing it with Ling Tiny 3, miniCPM5-2b, Granite 4-3b and Qwen 3.5-2b and 4b, with the goal of making sure that it actually generalizes. It’s been a massive pain in the ass and I’ve basically proven RAG from first principles…but when I’m done, you won’t need to vectorize the source into any kind of database.

I’m coding a Kiwix mcp (I assume you’re using kiwix as your wikipedia source?) that might be of interest to you. I’ll post about it on !localLlama when done.

I don’t like any of the extant Kiwix MCP tools that I found because they leave interpretation up to the LLM. I’m looking at a different way to get exact spans with re-ranking right in front of the llm so that it doesn’t have to do 3 jobs and hallucinate itself into stupid.