Ahmed Soulmani

ResearchAgent

A paper QA system I built end to end: ingest PDFs, retrieve passages, answer with citations, and abstain when the sources do not support the claim. Retrieval and generation are scored separately so a fluent reply cannot hide a weak search.

Query

Does spatially coherent token selection change robustness under pruning?

↓ retrieved evidence

Passage 1

high relevance

Superpixel-based selection preserves neighboring tokens that independently sampled scores discard.

Passage 2

medium relevance

Accuracy at high keep-rates is close; the gap appears in dense prediction and under perturbation.

Passage 3

low relevance

A FLOPs reduction is reported, but latency is measured separately.

↓ generated answer

Yes, when the evaluation includes dense prediction and robustness, not only classification accuracy. [1][2]

01System

A local system that answers questions over research PDFs and plain text. The code is kept flat so the full pipeline is easy to follow.

Ingest

Load PDFs and plain text, extract the content, then split it into overlapping chunks that follow paragraph breaks instead of fixed character cuts.

Index

Embed the chunks and store them in Chroma. A re-index rebuilds the store from scratch so it stays easy to inspect.

Retrieve

Embed the question and return the closest passages. This stage has its own metrics.

Answer

The model returns JSON with supporting quotes before the written answer. If the sources do not back the claim, the system abstains.

02Failure mode

Early chunking often cut paragraphs in the wrong place. Retrieval then returned weakly related fragments. The model could still answer with confidence. A fluent final sentence was not proof that the answer was grounded.

03Observability

I scored retrieval and generation on their own, improved the chunking, rebuilt the index, and made the system refuse when the papers do not contain enough evidence. A small supervisor flags off-topic inputs and clear output drift. It is basic, not a production safety layer.

04Architecture

Ingest

  1. PDF / TXT
  2. Extract
  3. Chunks

Index

  1. Embeddings
  2. Chroma

QA

  1. Query
  2. Top-k
  3. JSON + quotes

Retrieval and generation are measured on their own. If either is weak, the system can abstain instead of guessing.

05Evaluation

Evaluated on 20 hand-labelled development questions. This is a development set, not a production benchmark.

Hit@5

1.00

MRR

0.97

Development set

20

06Grounding

Every answer should point back to a source passage. The UI supports yes or no answers when the text is clear, and short definition answers when it is not. Both still need evidence in the JSON response.

07Repository

Ingest, chunking, indexing, retrieval eval, and the Streamlit UI all live in one repo.

GitHub