Ireland, in hindsight › How we rank tips

How we rank tips

Every tip and answer on this site shows the evidence behind it. Here is how the numbers work.

Compiled by DP Mooney

What a report is

The tips are counted from 336 trip reports: posts by visitors on TripAdvisor, Fodor's, Rick Steves forums, Reddit and travel blogs, plus trips sent in through this site. A tip's number is how many different reports gave it. It is not likes, upvotes or replies. One person is one report: if the same author on the same site wrote two reports, they count once, and saying the same thing twice in one report counts once.

Tips: strength labels

Beside the number you may see a small label. For tips it comes from the count alone:

Tips with fewer than 3 reports are hidden from the ranked lists, and none gets a label. Where visitors flatly contradict each other (we mark these “Visitors disagree”), we also show what share of the visitors who took a side said this one. A tip with 10 or more reports needs 75% or more of that share to stay Strong; 60 to 74% makes it Moderate; below 60% it gets no label.

Questions and answers

The answers on /questions/ come from replies in public trip-planning groups. The number shown is how many threads asked the question, and each thread counts as one independent question. We store no names of the people who posted or replied. Replies that are businesses advertising themselves (a driver, guide, tour firm or hotel promoting its own service or posting its own contact details) are left out of every count.

Questions come in two kinds, and they are measured differently.

Yes/no and either/or questions (for example “Is November to February a good time to visit?”) have one consensus position, so we measure agreement. Each question has an explicit frame: Option A, Option B, and which one the consensus picks. Each top-level reply is judged in two steps. First, does it directly answer the question? If not (a fact, off-topic, side advice, a question, thanks, or a very short reply that only makes sense in context) it counts as neither. Only then is it judged as picking the consensus option (supporting) or the other option (contradicting). A thread counts as supporting or contradicting by the majority of its replies that took a side; ties and threads with no side are left out. Agreement is supporting threads divided by supporting plus contradicting threads, and it is shown only once 5 threads have taken a side. With fewer than 10 threads taking a side we show the plain count (“4 of 5 threads agree”) instead of a percentage; from 10 threads it is a percentage. Under 70% agreement the page adds a “Where visitors disagree” line.

For these questions the label follows the number of threads that took a side, not the number that asked: Strong is 10 or more threads taking a side and 75% or more agreement; Moderate is 7 to 9 threads taking a side, or 60 to 74% agreement; Early signal is 5 or 6 threads taking a side. Under 60% agreement there is no label, and neither is there with fewer than 5 threads taking a side.

How we checked it. The judgements are made by reading the replies, not by counting, so we test them. An independent reviewer labelled a blind sample of 20 replies using the same frames and rules, and we required at least 17 to match before turning agreement on. The latest check matched 17 of 20. Treat any percentage as a careful estimate.

“Which” and “where” questions (for example “Which castle hotel is worth a night?” or “Best day trips from Dublin”) have many valid answers, so a single agreement figure would mislead. Anything with three or more named options, or that asks which or where, is treated this way. These show no percentage and no label. Instead the page lists Most named: the options mentioned in at least 3 separate threads, ranked by how many threads. An option counts once per thread, however many times it comes up there. A reply that only names a business, with no experience described (“Emerald Transfers” as opposed to “we used Emerald Transfers and our driver was great”), counts as promotion and is left out, because we cannot tell a customer from the owner. Places are never treated this way: a bare “Kinsale” is a fair answer.

How often it updates

The counts are rebuilt every Friday morning. Trip reports are added in batches (none added in the last 30 days), and the question answers are re-checked when new threads arrive. “Last reviewed” on an answer is the month its figures were last checked against the replies.

Numbers on this page and across the site are computed from the data on each rebuild, never typed in by hand.