Core system for BioASQ

Now I think most knowledge is gained. All that is left is to learn sparse and dense searches and FAISS but I will do that later. For now I am building the core skeleton to interact with a model. That brings us to the first question - what free model is out there to use?

Generation: Decided to use something other than openai API for a change. Gemini-flash free tier with google’s own API it is.
It turned out to be pretty easy to implement, honestly.

The core skeleton is working fine. I ask a question, my retriever uses “the selected” search method (today I only have bm25) to retrieve 10 abstracts from the corpus, and then creates a bigger prompt with all content. System instruction is written for the RAG setting - basically use the corpus results and give citation for each paragraph.

All working fine now. I can ask questions, it gets answers and gives citations. I tried to verify these citations and they seem to be in order. All good at the moment. Gonna chill out now. Next step is to learn the Sparse and Dense search algorithms, and then implementing them.