The Journal

How We Score Stays (Show Your Work Edition)

Theo VanceJun 8, 20266 min read
ratingsmanifestohow-to

Every travel site gives you a number. Almost none will tell you where it came from. An 8.4 on the big platforms is a black box — you do not know what was measured, how it was weighted, or whether the property's ad spend leaned on the scale. We think a rating you cannot audit is closer to a mood than a measurement. So here is ours, fully disassembled.

The five categories

Every stay, experience, and eatery on this site is scored 1–10 across the same five axes:

  • Quality — the fabric of the thing. Is the bed good, the shower hot, the room as promised? For a tour: is the thing itself excellent?
  • Location — not "is it central" but "is it in the right place for what it is." A beach shack scores on the beach, not on proximity to the opera.
  • Service — did humans make it better or worse. Includes the quiet stuff: how problems got handled, whether the answer to a weird request was yes.
  • Value — the score most sites are afraid of. Judged against the all-in price, never the advertised one. A $40 guesthouse can beat a $400 resort here, and frequently does.
  • Vibe — the honest name for the thing you actually remember. Atmosphere, character, the feeling at 7pm. Subjective? Completely. Pretending it does not exist would be the dishonest option.

The formula, in public

The categories combine into a weighted base, then blend fifty-fifty with the rater's overall gut score: final = (weighted base + overall) / 2. Why keep the gut score? Because a place can tick every box and still be nothing, or fail a category and still be somewhere you would defend with your life. The blend keeps the arithmetic honest and the humanity intact. Every listing shows the category bars, so you can see why a number is what it is — an 8.7 built on value and vibe is a different animal from an 8.7 built on marble and turndown service.

Why category spreads matter more than the headline

Take two real patterns from our data. A hostel in Lisbon might run: quality 7.2, location 9.0, service 8.5, value 9.5, vibe 9.1. A caldera resort in Santorini: quality 9.6, location 9.9, service 9.2, value 6.4, vibe 9.0. Similar headline scores. Utterly different purchases. The spread is the review; the single number is just its shadow. This is also why we let you re-read any score through a persona lens — an 8.9 "for foodies" and an 8.9 "for families" are answering different questions, and pretending one number serves everyone is how the big platforms end up meaning nothing.

What we refuse to do

  1. No paid placement in scores or charts. Rankings are computed. There is no sales team with a thumb available.
  2. No burying the count. A 9.1 from 24 ratings and a 9.1 from 2,400 are different claims. We show the count next to every score and weight our charts to respect it — it is exactly how our Hidden Gems list (high score, low count) and Overrated Index (huge count, mid score) get computed.
  3. No scrubbing context. Reviews here carry persona, trip type, and month visited. A couple in May and a family in August stayed at different hotels that happen to share an address.

A rating should be an argument you can check, not a verdict you have to trust.

Disagree with a score? Good — rate it yourself, and say what the number cannot. That is the system working.

More from the Journal

All posts