Vireon

The gap is evaluability. When a general-purpose assistant reports that oil surged and equities rotated defensive, the user has no way to assess whether that came from a wire service this morning or a content farm last quarter. The answer arrives with the same confident cadence either way. In most domains that is an annoyance. In financial research, where someone may size a position against it, it is a defect in the product's core promise.

So the thesis was deliberately narrow. The differentiator is not answer quality, it is verifiability. Every claim traces to a source, every source carries a known trust weight, and the system publishes its own confidence with the arithmetic visible.

That reframes nearly every decision downstream of it. Retrieval stops being a means to an answer and becomes the product surface itself. Confidence stops being a tone and becomes a specification. And accuracy stops being a quality metric that improves over time, and becomes a gate that blocks a response from ever reaching a user.

Three surfaces came out of it. Two answer a question you asked. The third never stops running.

Source tiering is an editorial decision, not a technical one

The foundation is a registry of 378 domains, hand classified into four trust tiers plus an exclusion list. It is the least glamorous part of the system and the part everything else rests on.

The column sums to 382 against the 378 above. Four domains are currently counted twice because they sit in two tiers at once. That is a live defect, not a rounding convention, and it is the subject of the next three paragraphs.

The anchor property is the part that matters. A and B sources can carry a claim on their own. C and D sources can corroborate but never establish. That is a judgment call about what counts as evidence, and it belongs with product rather than sitting implicit inside a ranking function where nobody has to defend it.

The exclusion list was the harder call. Dropping forums and PR wires removes real signal, and sometimes the earliest signal. We took that cost on purpose. On a product whose entire promise is trustworthiness, a fast wrong answer is more expensive than a slow right one.

A registry is also a thing that rots. Auditing it while writing this page turned up four domains sitting in two tiers at once, which means their effective weight is decided by which array gets checked first rather than by anyone's decision. Between tier B and tier C that is a swing of 0.35 in the weight, on outlets that appear in results often.

Writing the fix took ten minutes. Deciding it took longer, because the question was never technical: Business Insider is an aggregator that also does original reporting, and Investopedia is a reference site that occasionally breaks news. The resolution moves Business Insider to C, Investopedia and TheStreet to D, and leaves Fortune at B, on the same standard the rest of the registry uses. Three of the four are downgrades, which lowers published confidence on answers that lean on those outlets. That is the correct direction: a score should reflect what a source is, not which array happened to be checked first.

The durable part is not the fix, it is the test. The registry now fails its own suite if any domain appears in two tiers, if any tier repeats a domain, or if the distinct count drifts from the 378 published here. A second suite, on the backend copy, asserts that the two registries classify every domain identically and that no subdomain is ranked above its own parent publication. Nothing about a duplicated entry announces itself at runtime, so the editorial layer needed the same regression discipline as the code.

Writing those tests surfaced eight stale failures already sitting in the existing suite. One of them the fix itself turned green, because the duplicate had been masking the tier the original editors intended. The other seven were assertions that later, deliberate decisions had quietly overtaken and nobody had reconciled. That is the more useful finding: the tests existed, and not running them cost more than not having them.

At the time of writing this the change is green in the test suite and staged, not shipped. It reclassifies sources on four edge functions and five backend modules at once, so it waits for a review gate rather than riding out alone. The numbers in the table above are what production returns today, duplicates and all. Publishing the corrected figures before the code that produces them would be the same category of error as the defect itself.

The formula, and what it is actually measuring

Three components, weighted. Each answers a different question a careful reader would ask.

• Publisher quality asks how good these sources are. The mean tier weight across the source set.
• Outlet corroboration asks how many independent credible outlets converged on this. Distinct A and B publishers, log scaled.
• Tier diversity asks whether this is an echo chamber. How many distinct tiers appear at all.

The metric I shipped, and then killed

Version one weighted source agreement at thirty percent. The system tagged each source as aligned, neutral, or divergent, then scored the consensus.

In production it did nothing. Stance tagging was never reliable enough to act on, so almost every source landed on neutral, agreement collapsed toward zero, and the component became a near constant subtracted from every answer. Scores clustered in the twenties and thirties no matter how strong the sourcing actually was.

The number was not exactly wrong. It was uninformative, which on a trust surface is worse. It was quietly teaching users that Ori was unsure of everything, including the things it had every reason to be sure about.

The replacement measures something the system can actually observe: how many distinct credible outlets independently converged on the story. Count unique A and B publishers, then scale logarithmically.

Distinct credible outlets, mapped to a corroboration score.

Two details in that change carried more weight than the curve itself.

Deduplicate by hostname, not by name. Sources arrive with a name field that is sometimes a publisher and sometimes an article headline. Eight articles from two outlets were counting as eight distinct publishers, inflating confidence by roughly fifteen points on exactly the queries where a single story had been syndicated everywhere.

Give the curve a floor. A pure logarithm scores a single source at zero, so one exclusive wire story rendered as thin evidence. That is not what a reader means by unreliable. Mapping one outlet to 32 instead of 0 keeps the honest signal, that a single source is genuinely less corroborated, without the dishonest one, that a wire exclusive is worthless.

Ori answers a question you asked. LiveFeed never stops running, clustering incoming market news on a thirty minute cycle. Same verification thesis, different physics: nobody is waiting on the result, so the system can afford to go looking for evidence rather than only scoring whatever it happened to retrieve.

The unit of truth is the event, not the article

The naive version of a news feed lists articles. That surfaces the same event five times whenever a story gets picked up widely, and it quietly converts syndication into the appearance of significance. It is the same distortion as counting eight articles from two outlets as eight publishers, arriving through a different door.

So the unit is the event. A first pass groups incoming articles by the real world event they describe, and a second pass synthesizes each group into one story. The clustering instruction is explicit on two points: every article lands in exactly one cluster, and multi source clusters should be maximized, because a cluster that pulls five outlets onto one event is the thing actually worth reading.

Two guardrails sit around that model call, and they are the part I would point at.

That second guardrail is the same principle as Ori discarding model emitted sources, reached independently on a different surface by a different route. The model is genuinely good at judging which articles describe the same event. It has no business minting a database key.

Cross examination: going and getting the second source

Here is the case a confidence score alone handles badly. One outlet breaks a story and publishes it three times, as a wire piece, an analysis, and a live blog. Count articles and the event looks well covered. Count publishers and it scores as thin, which is correct, but the user is still left holding a thin story and a low number.

So when a cluster holds at least two scraped articles and every one of them shares a root domain, the system reads that as a signal rather than a verdict. This event mattered enough for one outlet to cover it repeatedly, and we have not yet heard it from anybody else.

1. 01

   Search the web for the same event, using the cluster's own event summary as the query.

2. 02

   Discard any result from a domain already in the cluster, and any domain outside a whitelist of 21 established outlets.

3. 03

   Scrape up to three new publishers and fold them into the cluster, tagged as search discovered rather than natively retrieved.

That cap of three is not arbitrary. It is matched to the three source display cap of the drawer this feeds, which is called Cross Examined Consensus. The backend does not spend money fetching evidence the interface has no room to show.

The whole path is failure tolerant by construction. It never raises. On any error, or any cluster that does not qualify, it returns nothing and leaves the original single source cluster intact and honestly labeled as single source. Corroboration is an upgrade the system attempts, never a claim it depends on.

What this surface still gets wrong

Two things, and both are the same shape as the failures in the next act. First, the tier. When cross examination folds a discovered article into a cluster, it stamps the source as tier A rather than classifying it, on the assumption that a whitelisted outlet is A or B. Four of the twenty one are not: one is on the exclusion list outright, and the scoring path prefers a supplied tier over a computed one, so the stamp is what gets scored. A string inherits a weight it did not earn, which is Failure 02 arriving through a third door on the surface this act is about.

Second, the formula. Everything in act two describes Ori Search. LiveFeed and the macro surfaces still run an older confidence formula on different weights, with a floor of thirty rather than zero, and its consensus term is a flat count of publishers rather than the log scaled count of distinct credible outlets. That is the metric act two describes replacing, still running here. I found both while writing this page. Neither is hard to fix and neither is fixed yet, which is the honest version of a corroboration moat: the idea is right, one of the two surfaces implements it, and the older one has been quietly scoring on the thing I said I removed.

Untrusted content, one layer down

Scraped article bodies are the primary material the synthesis model reads, and they are also arbitrary text from the open web. They are never placed inside the structured JSON the model is told to trust. Each body is quoted between explicit untrusted markers, and the marker tokens are stripped from the body first, so that a crafted page cannot close its own fence and begin issuing instructions.

Same adversary as Ori's fabricated sources, different mechanism, because the attack surface differs. There the risk was a planted URL inheriting a trusted domain's tier. Here it is planted text inheriting the authority of source material.

Most of that cost is not the synthesis model. It is breadth. A deep run searches up to 40 sources, then runs a separate extraction pass on every one of them before synthesis begins. Cost scales close to linearly with depth, and depth is precisely what makes the feature worth paying for. No clever optimization escapes that. The only real lever is deciding who gets how much.

Depth as a product boundary

Setting depth per tier does two jobs with one parameter. It caps the cost of the cheaper plan, and it makes the expensive plan visibly better along the exact axis the feature is sold on. Pro users are not handed a degraded deep research, they get a shallower one, and the difference surfaces in the interface as the number of sources scanned. The paywall and the cost control are the same line of configuration.

Trial users get full Edge depth. That is a deliberate acquisition cost. A trial has to sell the thing being sold, so it runs at the expensive setting and gets capped on frequency instead of quality.

Spend caps, and why there are two layers

Requests per day.

Per-user caps bound a single account. The global counter exists because trial signup is free and self-serve, so someone creating accounts in bulk could scale spend arbitrarily without any one account exceeding its limit. Capping total deep research generations at 300 per day holds worst-case spend near $85 regardless of how many accounts are involved, which is sized against a launch roster rather than against current traffic.

The limiter is specified to fail closed. If its backing store errors, requests should be briefly blocked rather than waved through uncapped, because on the most expensive path in the product a short outage is cheaper than an unbounded one. That is a product decision rather than an implementation detail: a worse experience during a rare failure, chosen over an unbounded bill during that same failure.

Auditing this page is how I found that the deployed code does the opposite. Both Ori paths pass the fail-closed flag as false, so a limiter outage today is an uncapped one, and the branch that handles the blocked case is unreachable. The comment directly above the flag argues for failing closed. The argument was written, agreed with, and then not wired up, which is a more ordinary way for a decision to go missing than getting it wrong.

Latency, and the constraint I stopped fighting

The dominant cost is the synthesis model writing roughly 1,500 output tokens. That is physics for the answer depth we chose, not an inefficiency to engineer away. Sub-fifteen-second fresh answers are not reachable without cutting depth or dropping to a smaller model, and both trade directly against what the product is for. Saying that plainly, rather than treating it as a bug to keep grinding on, freed the effort to go where it actually paid.

What was reachable were the costs that bought nothing.

1. A token ceiling that was buying a second model call

   The output cap sat at 1,800 tokens. Full-length answers hit it, produced truncated JSON, and triggered a complete retry at a higher limit, adding 30 to 40 seconds. Raising the ceiling to 2,500 removed the retry outright. Because the cap is a ceiling rather than a target, shorter answers were never slowed by the higher value. The lower limit was not saving anything. It was purchasing a second inference.

2. Paying the model to write a field we discard

   The output schema asked the model for its source list. The server rebuilds that list from its own retrieval set and throws the model's version away, for reasons in the next section. So we were spending 150 to 400 output tokens, three to five seconds, generating data that could never reach a user. Deleting the field from the schema deleted the cost.

3. Work on the critical path that did not belong there

   Session persistence was awaited before the answer was released to the user. Moving it to a background task cut that tail from six seconds to three milliseconds. Nothing about it needed to happen before the reader saw the response.