What it measures
Search, triage, and extraction as separate tasks
The public benchmark tracks target-driven retrieval, retained review rows, and structure-extraction outputs separately so a single score does not hide weak spots.
Benchmark
Patent analysis and structure extraction benchmark.
Public proof should show what is measurable now, keep private data redacted, and separate retrieval metrics from chemistry-extraction metrics.
13 blinded programs
21,133 retrieved publications
456 structure rows
433 rows with structure-linked fields
206
target-centered evidence cases retained for review
110
rows with normalized name fields in the anonymous chemistry benchmark
57.2
minutes total wall time for the structure-heavy benchmark snapshot
Retrieve
Retain
Confirm
Dataset design
Blinded survey plus anonymous chemistry-heavy patent set
Target identities and source identifiers stay redacted. The structure benchmark uses anonymous documents to prove schema-level extraction without exposing private source files.
Public proof assets
Benchmark page, sample outputs, and case-study layouts
Each asset answers a different question: how the benchmark is framed, what the output looks like, and how a pilot engagement is explained to a buyer.
Tasks
Target-driven patent analysis
- Retrieve broadly from the target, disease area, or program theme.
- Screen quickly into keep, target-pending, and exclude candidates.
- Return a shortlist with evidence-bearing rationale for human review.
Tasks
Patent PDF structure extraction
- Take representative PDFs or a narrowed patent batch.
- Extract structure-heavy rows, normalized name fields, and text-linked signals.
- Return review-ready rows rather than screenshots or OCR text alone.
Tasks
Target-search scoping
- Start from the biology question and chosen search window.
- Show how broad retrieval narrows before deeper extraction starts.
- Keep the public story aggregate while private runs can expand later.
Results snapshot
Public metrics currently shown on PatWinnow
These numbers are the ones already exposed on the homepage and are suitable for a public benchmark page because they stay aggregate or anonymous.
| Block | Metric | Value | Why it matters |
|---|---|---|---|
| Blinded survey | Retrieved publications | 21,133 | Shows broad search coverage before aggressive narrowing. |
| Blinded survey | Retained evidence cases | 206 | Shows the size of the review-ready set after screening. |
| Structure benchmark | Rows with structure-linked fields | 433 / 456 | Shows that structure-heavy extraction is already possible in a public proof asset. |
| Structure benchmark | Rows with normalized name fields | 110 | Shows text-forward chemistry capture beyond image-only extraction. |
| Structure benchmark | Rows with assay-linked evidence | 13 | Signals bioactivity-linked extraction when evidence is present. |
Sample CSV proof
Public redacted rows that make the benchmark more concrete
Numbers alone are not enough. These preview rows show the shape of the deliverables behind the benchmark without exposing private source patents or client work.
| CSV | Preview row | Why it matters |
|---|---|---|
| Patent analysis shortlist | Redacted Family A, keep, deep_review |
Shows how broad retrieval becomes a small review-ready shortlist. |
| Structure extraction rows | Record A, structure field redacted, assay note retained |
Shows row-level chemistry outputs rather than only a row-count claim. |
Failure cases
What this benchmark does not hide
- OCR quality and scheme density still change extraction difficulty.
- Ambiguous chemistry names may require manual review to confirm identity.
- Target-relevance hits should stay separated from context-confirmed evidence.
Methodology
Rules for public-facing benchmark pages
- Keep target names, source identifiers, and client identities redacted.
- Publish benchmark framing, not private workflow code.
- Use case studies and sample outputs to explain the product surface.
Next proof assets
What to add as public content grows
- More redacted sample outputs with stable output schema.
- More target-analysis examples with clearer buyer questions.
- A short methodology page if the benchmark expands materially.
Proof assets
Use the benchmark as the hub, then link out to outputs and case studies.
That gives search engines and buyers a clean content structure without forcing any code disclosure.