TL;DR: We run the same profession task in every tool, score five things vendors tend to hide, and publish the failures next to the wins. As of July 10, 2026, our hands-on bench has not run yet. Until it does, every page reports the verifiable public record and dated practitioner reports, each number labeled for what it is.
This is the standard we hold ourselves to. It is also a commitment, because in this first cycle we have not yet run the hands-on suite. Budget for tool subscriptions is still pending. So this page describes the protocol we are building, states plainly where it stands, and gives you a way to check our work. Every review, comparison, and roundup on the site links back here.
What we test: profession tasks, not feature lists
We do not score marketing pages. We run one real job from the profession through every tool in a category, using the same file each time.
For bookkeeping, that is one deliberately messy month of books. Task one: sort a month of mixed personal and business transactions. Task two: reconcile a bank account with a planted discrepancy. Task three: close a month with two missing statements. Then we count errors by type.
For contract review, the same NDA and services agreement go into every tool, each seeded with known defects. The NDA carries a missing carve-out for court-ordered disclosure, a survival clause that contradicts the term clause, and uncapped indemnification. We count catches, misses, and false flags.
For legal research, we ask the same five questions in every tool, then pull each returned citation and read it in a primary-law database. A citation that does not exist counts as an error.
What we score: the five things vendors hide
Feature checklists are easy to game. These five columns are not, and top-10 vendor pages rarely publish them.
| Criterion | The question we answer |
|---|---|
| Setup time | How long from sign-up to first useful output, in minutes. |
| Behavior when a number is wrong | Does the tool flag its own doubt, or state a wrong figure with full confidence? |
| PII and upload policy | What the vendor does with the client data you feed it, quoted from its own terms. |
| The real price | The actual monthly cost, not "from $X". Verified against the live pricing page. |
| Refund policy | Whether you can get your money back, and the window, in the vendor's words. |
The honest-negative rule
We publish where every tool fails. Each review carries a "where it failed" section, and every comparison table can end with a "neither, for now" row. A page that only praises is a page we do not trust, so we do not write one. Reporting a weakness protects readers. A stated flaw is also checkable, which protects us from a defamation claim.
When hands-on is impossible: the Harvey case
Some tools have no trial. Harvey , for example, is demo-gated and sells only through sales calls. We do not pretend to test what we cannot open. Instead we use the pricing-gap method: verify everything public, gather dated practitioner reports, reconcile the numbers, and label the page clearly as no hands-on access. That page still beats a vendor brochure, because it splits verified fact from claim. What it never does is imply a test we did not run.
Screenshots, dates, and corrections
Our screenshots are our own, and every one is dated. We never post a vendor image as our result. The 2024 FTC rule on consumer reviews treats a staged "we tested" as deceptive, and we agree with the reason behind it.
Prices and features move, so every price, limit, and feature carries a "Last verified" date against the live source. When something we published turns out wrong, we fix it in place and log the change in a dated changelog entry, rather than quietly editing the record. Corrections are part of the method, not a break in it.
Where this stands today
To be exact about cycle one: zero hands-on hours so far, stated on every page. The protocol above is written and waiting on tool budget. When the bench runs, dated screenshots and per-task error counts land in the single-tool reviews first, and the linking pages get changelog entries.
We are the neutral layer over vendors who cannot compare themselves honestly. That only works if we stay independent. We hold no affiliate or business ties to any vendor named on the site as of this writing. When affiliate relationships begin, a disclosure will run before the first partner link on each page, spelled out in our editorial policy . More on who stands behind this work is on the about page.
Originally published July 10, 2026. Last updated July 10, 2026. Methodology status last verified July 10, 2026: hands-on suite not yet run. Pages reflect the verified public record plus dated practitioner threads. Hands-on results and dated screenshots will be added to individual reviews with changelog entries as the bench runs. Independence and disclosure terms are in our editorial policy .