Dimension 2 of 10 · methodology v1.0

Independently measured performance

What does the implementation actually cost to run, measured by someone with no stake in the answer?

Dimension
2 of 10
Requirements
6
Conformance gate
No
Applies to
All categories
References
0
Methodology
v1.0
In full

Requirements, evidence source, and references

This page is the citable form of this dimension — a finding elsewhere can link straight to the bar it was judged against.

02

Independently measured performance

6 requirements

What does the implementation actually cost to run, measured by someone with no stake in the answer?

Requirements · 6
  • Keygen / encapsulate / decapsulate and keygen / sign / verify timings, measured rather than quoted.

    • Reported with a distribution, not a single mean — median, p95 and p99 at a stated iteration count.

    • Measured on named hardware, at a stated liboqs or library version, traceable to a reproducible public run.

  • Measured on more than one architecture (x86-64 and ARM) so the numbers travel to the reader's estate.

  • Handshake-level cost measured in situ, not inferred by adding primitive timings together.

    TLS / networkPKI / CA
  • Key, ciphertext and signature sizes checked against the standard rather than taken from the datasheet.

  • Operations per second under sustained load, and the point at which the device saturates.

    HSM
  • Where an implementation is not independently testable, that is recorded as not assessed — never inferred from the vendor's own published figures, and never scored as zero.

Evidence sourceQ-Shield, Q-Advantage's daily benchmark harness — measurements taken on named hardware and published openly.
Scoping applies on this dimensionSome requirements here apply only to certain kinds of vendor. A requirement that is out of scope for a vendor is not assessed for that vendor — it is never scored as a failure, and never counted in the denominator.
Take the whole bar
Download .md
The ratings are not published yet

The criteria are public so you can judge the bar before anyone is measured against it.

If you are choosing between post-quantum suppliers, or you are one and want to know how you will be assessed, get in touch. We will tell you when there are results to read.

A vendor in the pool, or think you should be? Cooperation gets you advance notice of the criteria and a pre-publication window to correct factual errors. It does not buy a better result.