Vector search is fast because it cheats. Each document was embedded once, alone, long before anyone asked a question, so matching is just distance between two numbers. That is why it scales to millions, and why it stays approximate: the query and the document were never actually read together.
Reranking reads them together. A cross-encoder scores the query against each candidate as a pair, which is far more accurate and far too expensive to run across a million documents.
So the pipeline splits the job. Retrieve the top 50 from millions in milliseconds, then rerank those 50 in seconds and keep the top 5.
The difference shows up immediately. For "how do I change my address", vector search puts account security guidelines first and buries the address-change page at four. After reranking it leads, because something finally read the question and the page at the same time.
Screening and deciding are different jobs. Using the fast filter to make the final call is where quality quietly dies.