Post-Quantum Cryptography: Multi-layer Bottleneck Analysis

International Journal of Information Security (2026) 25:89
https://doi.org/10.1007/s10207-026-01242-0
REGULAR CONTRIBUTION
Post-Quantum Cryptography: A Multi-layer Bottleneck Analysis
Ruben Rios1·José A. Montenegro1
Received: 28 July 2025 / Accepted: 2 March 2026
© The Author(s) 2026
Abstract
Advances in quantum computing have led the research community to develop new post-quantum cryptographic algorithms.
While most efforts have focused on designing and analyzing the security of these algorithms, their integration into real-
world systems demands a multi-layer performance analysis to identify and mitigate potential bottlenecks. Aligned with
NIST’s post-quantum transition strategy, this work addresses that need by introducing a performance evaluation framework
that enables to assess and analyze the impact of post-quantum key encapsulation mechanisms (KEMs) and signatures at
multiple layers of the protocol stack—from standalone cryptographic primitives, to their integration into the TLS handshake,
and finally into full application scenarios such as VPNs—on two hardware platforms. Our results show that signature and
verification operations dominate the overall cryptographic cost, while KEMs such as MLKEM introduce minimal overhead. In
contrast, hybrid and HQC-based solutions significantly increase handshake duration, particularly on ARM platforms. Notably,
the relative performance between platforms shifts when comparing these cryptographic primitives in isolation versus their
integration into higher protocol layers. This highlights the critical need to evaluate post-quantum algorithms in realistic,
system-level scenarios.
Keywords Post-Quantum Cryptography (PQC) ·Key Encapsulation Mechanisms (KEM) ·Transport Layer Security (TLS) ·
Virtual Private Networks (VPN) ·Performance Benchmarking
1 Introduction
The rapid progress in quantum computing poses a signifi-
cant threat to traditional cryptographic algorithms [1], such
as elliptic curve cryptography (ECC) and RSA, sparking
a global effort to develop and standardize post-quantum
cryptographic (PQC) algorithms capable to resist quantum
attacks. A leading role in this effort was taken by U.S.
National Institute of Standards and Technology (NIST) who
initiated a competition to standardize PQC algorithms [2],
culminating in several FIPS standards for key encapsulation
(KEM) and digital signatures.
While the primary focus of the research community has
been on the security guarantees of these new algorithms, eval-
uating the impact of their deployment in real-world systems
is critical to a post-quantum transition strategy [3]. Modern
BJosé A. Montenegro
[email protected]
Ruben Rios
[email protected]
1Network, Information and Computer Security (NICS) lab,
Universidad de Malaga, Malaga 29071, Spain
systems and applications will heavily rely on secure commu-
nication protocols, and the integration of PQC algorithms are
not sufficiently explored and may significantly affect latency,
scalability, bandwidth, resource consumption and even pro-
tocol operation [4].
Effective integration of PQC into existing protocols, such
as Transport Layer Security (TLS), and applications, such
as Virtual Private Networks (VPNs), requires a thorough
understanding of performance trade-offs across different lay-
ers of the communication stack and across various hardware
platforms. Consequently, in this work, we present a com-
prehensive performance evaluation framework tailored for
the post-quantum transition. We systematically assess the
performance of post-quantum KEMs and digital signatures—
both in isolation and within different layers of the protocol
stack—on two representative hardware platforms. Our evalu-
ation allows exploration of architectural differences and their
influence on cryptographic performance, providing insights
into potential bottlenecks.
Our analysis spans from low-level cryptographic prim-
itive benchmarks to evaluations in TLS handshakes and
TLS-based VPN deployments, revealing that signature and
0123456789().: V,-vol 123

89 Page 2 of 28 R. Rios, J.A. Montenegro
verification operations dominate cryptographic overhead,
while certain post-quantum KEMs (e.g., ML-KEM)intro-
duce minimal latency. In contrast, hybrid and HQC-based
solutions significantly increase handshake duration, par-
ticularly on ARM platforms. This multi-layer evaluation
demonstrate that performance rankings can shift dramati-
cally when primitives are embedded in real-world contexts,
with some signature schemes introducing notable overheads.
These findings underscore the critical need for system-level
evaluations to guide the practical adoption of post-quantum
cryptography.
The structure of this paper is organized as follows. Sec. 2
analyzes related work and positions our main contributions.
Sec. 3presents our evaluation setup, including the selec-
tion of cryptographic primitives, protocols, and applications
selected for evaluation, as well as the details of evaluation
framework for performance assessment at different layers of
the communication stack. Sec. 4,5and Sec. 6present our
performance analysis at the primitive, TLS handshake, and
VPN application levels, respectively. Sec. 7discusses the key
cases in our work. Finally, Sec. 8concludes with key findings
and outlines future research directions.
2 Related Work
The development of PQC algorithms has been an active area
of research, particularly after the announcement of the NIST
competition in 2016 [2]. Several works have focused on the
analysis of these algorithms from a security perspective (e.g.,
[5,6]) while others have concentrated on their applicability
and cost.
Some authors have focused on benchmarking different
aspects of PQC algorithms. For example, an extensive com-
pilation of performance metrics across various families of
algorithms is presented in [7]. Other works have analyzed
the energy consumption of these algorithms [8]. In particular,
some studies have examined their suitability for power-
constrained scenarios, such as the Internet of Things [9].
Similarly, other authors have worked on the optimization and
benchmarking of PQC algorithms. For instance, Dang et al.
[10] implemented three lattice-based KEMs from the second
round of the NIST PQC standardization process, showing that
hardware acceleration can substantially improve their execu-
tion times. In [11], two competition finalists—one KEM and
one signature scheme—are benchmarked on a cryptoproces-
sor optimized for their execution.
The integration of post-quantum KEMs and digital sig-
natures into security protocols has also been extensively
studied. Notably, [12] evaluates NIST signature algorithm
candidates at that time and investigates the latency they
impose on TLS 1.3 connection establishment. [13] goes
one step further and analyzes the impact of both several
post-quantum key encapsulation and signature algorithms in
TLS. In [14], the authors devise a modular framework to
evaluate the impact of integrating post-quantum and hybrid
cryptographic primitives in the TLS protocol under various
network conditions. Similarly, [15] presents an evaluation
across three security levels, based on the traditional and
post-quantum candidate algorithms available at the time
of the study. Their work includes a breakdown of library-
level computational workload, highlighting an increased
CPU usage at the protocol level, but without a detailed
analysis of the underlying cryptographic primitives. Inter-
estingly, Farooq et al. [16] observed that some KEMs in
TLS implementations— particularly BIKE and Classic
McEliece—performed better on Linux platforms than
on Windows on x86 systems. Additionally, the authors in
[17] focus on analyzing vectorized instruction sets on x86
platforms, demonstrating performance improvements over
non-vectorized implementations. However, these improve-
ments are not reflected when integrated into the TLS protocol.
That work conducts a two-level analysis, whereas our study
performs a three-level analysis across two different plat-
forms.
Some works have also investigated the impact of integrat-
ing PQC algorithms in virtual private network applications.
[18] evaluated the number of CPU instructions of sev-
eral post-quantum cryptography from the NIST competition
in OpenVPN and HTTPS. Similarly, [19] compared exe-
cution time of OpenVPN with traditional RSA and with
Dilithium. Paquin et al. [20]havealsoworkedonthe
integration of PQC algorithms into the OpenVPN protocol.
Some authors have focused on different types of VPN. For
example, Bae et al. [21] evaluate the execution speed of
PQC algorithms from NIST Round 3 finalists into IPSec.
Wireguard VPNs have also received attention lately [22,
23].
The analysis of the literature reveals that the performance
of PQC algorithms is influenced by hardware platform;
however, most works tend to focus on a single platform.
In general, research also tends to concentrate on a sin-
gle layer of the protocol stack and does not trace the
impact of these algorithms across multiple layers. In con-
trast, our analysis is not only multi-layered but also considers
two widely used hardware architectures. Additionally, we
focus on standardized PQC algorithms or those recently
selected for standardization. Compared to our previous
work [14], this study extends the analysis by incorporat-
ing performance comparisons across CPU architectures and
evaluating the latest PQC standards. Most notably, it intro-
duces an additional layer of complexity to the assessment
of PQC impact on network protocols, namely the VPN
layer.
123

Post-Quantum Cryptography: A Multi-layer Bottleneck Analysis Page 3 of 28 89
3 Evaluation setup
This section first presents the selection of cryptographic
primitives—traditional, hybrid and post-quantum— proto-
cols and tools to be evaluated. Next, it describes the software
platform devised for the evaluation of the selected primitives,
protocols and tools.
3.1 Primitives, protocols and applications
Cryptographic primitives are typically categorized based
on the level of security they provide, which reflects the
computational effort required to break them. In traditional
cryptography, this is expressed in terms of bits of security,
whereas post-quantum cryptography uses security levels—
from Level I to V. Although there is no exact correspondence
between these two metrics, they are conceptually aligned:
for instance, a Level I post-quantum cryptosystem offers
resistance to quantum attacks comparable to the resistance
of AES-128 against classical attacks. Similarly, Levels III
and V correspond approximately to the security offered by
AES-192 and AES-256, respectively.
Table 1presents the cryptographic primitives selected for
evaluation. A baseline set of primitives for both key exchange
and authentication has been chosen. Schemes based on ellip-
tic curves are prioritized over finite-field alternatives due
to their superior efficiency [24]. For post-quantum cryp-
tography, we selected efficient algorithms that have been
standardized or selected for standardization in the NIST com-
petition [2]. Hybrid algorithms combining traditional and
post-quantum primitives were considered for key encapsula-
tion, but not for digital signatures. This reason for this is that
hybrid schemes are primarily designed to mitigate “harvest
now, decrypt later” attacks, which do not pose a significant
threat to signature schemes.
These cryptographic primitives are commonly used in
security protocols to provide authentication and to ensure
confidentiality and integrity following key exchange mech-
anisms. Among the various protocols that incorporate key
exchange and authentication, we select the Transport Layer
Security (TLS) protocol due to its widespread adoption and
versatility across diverse application scenarios. TLS func-
tionality is implemented by various cryptographic libraries,
each offering different levels of performance, configurabil-
ity, and support for algorithms. For our evaluation, we select
OpenSSL, an actively maintained open-source library that
supports the integration of external cryptographic providers,
enabling the use of both traditional and post-quantum cryp-
tographic primitives.
Several applications leverage the TLS protocol to pro-
vide security at higher layers of the protocol stack. Many
application-layer protocols–such as HTTP, SMTP, and FTP–
rely on TLS to secure their communication channels and
Fig. 1 Evaluation Framework Components.
ensure confidentiality, integrity, and authentication. Among
the various use cases, we select Virtual Private Networks
(VPNs) as the target application for testing. In particular, we
choose OpenVPN, a widely used open-source VPN solution
that employs TLS for secure key exchange and authentica-
tion, making it suitable for our evaluation.
3.2 Evaluation Framework
A systematic evaluation of post-quantum cryptography bot-
tlenecks demands for a benchmarking platform that facili-
tates the configuration, deployment, execution and analysis
of cryptographic primitives, protocols and applications.
Fig. 1presents a layered structure of the tools and libraries
that compose the framework, where each layer fulfills a spe-
cific function. These components are encapsulated within a
containerized environment to ensure consistent deployment
of both client and server systems, regardless of the platform
architecture. By leveraging containerization, the framework
maintains a standardized environment across all experimen-
tal setups. For this purpose, Docker has been adopted as the
core container technology.
Cryptography layer: this layer provides the primitives (sig-
natures and KEMs) necessary for the protocols to perform
the cryptographic handshake. While both OpenSSL and
MsQuic include traditional signature and KEM primitives,
they do not include post-quantum primitives by default. This
123

89 Page 4 of 28 R. Rios, J.A. Montenegro
Table 1 Cryptographic
primitives selected for
evaluation
Security KEM Signature
Level Trad Hybrid Post Trad Post
IX25519 x25519_mlkem512 mlkem512 Ed25519 mldsa44
x25519_hqc128 hqc128
III X448 x448_mlkem768 mlkem768 secp384r1 mldsa65
x448_hqc192 hqc192
VP-521 p521_mlkem1024 mlkem1024 secp521r1 mldsa87
p521_hqc256 hqc256
layer relies on the liboqs library, from the Open Quantum
Safe (OQS) Project, which provides numerous post-quantum
cryptographic implementations, including algorithms stan-
dardized by NIST as well as proposals from previous
standardization rounds.
The integration of these post-quantum primitives into both
OpenSSL and MsQuic is facilitated by their modular archi-
tecture, which supports the use of cryptographic providers.
The oqsprovider acts as bridging component, allow-
ing both protocol implementations to access the primitives
offered by liboqs.
Protocols Layer: this layer comprises the protocols under
evaluation, in this case, TLS. Specifically, OpenSSL has
been selected due to its widespread adoption and robust
support for cryptographic standards. It provides implementa-
tions for both client and server components, enabling precise
measurement of handshake performance with minimal devel-
opment effort.
Application Layer: this layer hosts the applications that rely
on the TLS protocol implemented in the Protocols layer.
In this work, we selected OpenVPN as the representative
application for evaluation. However, the specific choice of
application is not critical, as long as it employs TLS-based
authentication to establish a secure communication channel.
The primary objective at this layer is to enable a compari-
son of the TLS handshake duration when executed directly at
the protocol level versus during the application runtime, with
the aim of analyzing how the handshake time is distributed
across both contexts.
Network Layer: this layer provides the capability to emulate
various network conditions between the client and server.
This makes it possible to evaluate handshake performance in
both ideal and realistic scenarios, including packet loss and
latency.
Emulating ideal conditions allows for isolating the per-
formance of cryptographic primitives during the handshake,
eliminating external factors that could obscure the analysis.
Nonetheless, it is equally important to assess performance
under realistic network conditions. To this end, our frame-
work integrates Pumba, a flexible and powerful tool for
manipulating network behavior at the container level.
In addition, the network layer incorporates EdgeShark,
which enables Wireshark to capture packets directly
within Docker containers. This facilitates the verification of
protocol-compliant message exchanges, the calculation of
actual network traffic generated by each configuration, and
the validation of handshake duration measurements based on
packet transmission timestamps.
Orchestration and Analysis Layer: this layer is responsi-
ble for managing scenario execution and coordinating all
components of the framework via a set of bash scripts.
Execution logs and captured network traffic are processed
using Python scripts, which generate CSV files containing
the relevant data. These files serve as the basis for generating
plots and performing statistical analyses for each evaluated
scenario. In addition to basic statistical metrics, our analysis
includes the computation of the coefficient of variation, and
outliers.
The evaluation framework is available at https://github.
com/montenegro-montes/TLS-VPN. The repository con-
tains all the necessary files to run the platform, including
Docker installation instructions, scenario execution scripts,
and analysis tools. Additionally, it provides access to the logs
and plots generated and used in this work.
3.3 Hardware Platforms and Execution Environment
The evaluation was conducted on two hardware platforms:
an ARM-based system and an x86-based system. The ARM
platform is a Macbook Pro 13 fitted with an Apple M2 Sys-
tem on Chip that offers 8 cores: 4 performance cores (up
to 3.5 GHz), and 4 power-efficiency cores (up to 2,4 GHz).
The x86 system is a server with two Intel Xeon Silver 4214
CPUs, each with 12 physical cores running at 2.20 GHz, but
it can run at Turbo speed (3.20 GHz) on demand. The exper-
iments were run on a virtualized Ubuntu Desktop machine
with 24 logical cores running on top of the VMware ESXi
version 8.0 hypervisor. The x86 processor supports advanced
vector instruction sets, including AVX, AVX2, and AVX-
512 (F, DQ, BW, and VL extensions). No CPU frequency
scaling or turbo boost mechanisms were active during the
123

Post-Quantum Cryptography: A Multi-layer Bottleneck Analysis Page 5 of 28 89
experiments, ensuring stable and deterministic performance
measurements.
Both platforms running Ubuntu 24.10, aligned with
cloud-native and container-oriented deployment practices.
To ensure reproducibility and prevent ABI incompatibilities
among cryptographic libraries, the complete evaluation envi-
ronment is encapsulated within a Docker container. The
cryptographic stack relies on OpenSSL 3.4 and liboqs
v0.12.0, with oqsprovider v0.8.0 acting as the
integration layer between both components. All software is
compiled using GCC 14.2.0 with using -O3 optimization.
A core design principle of the framework was to mini-
mize modifications to standard software distributions. This
approach ensures compatibility with upstream updates and
avoids introducing dependencies on custom or patched
software components, thereby enhancing maintainability
and ease of integration. Although only standard software
distributions are used, all components are locally com-
piled, enabling platform-specific optimizations based on
the underlying hardware and available CPU instruction
sets.
For the evaluation of cryptographic primitives, each
benchmark consisted of fifty consecutive executions of the
openssl speed command, automated through the frame-
work. All signature and KEM operations were executed
within a containerized environment configured with strict
CPU and I/O isolation to minimize system-induced variabil-
ity. For each primitive, fifty latency samples were collected
to compute the mean execution time.
For the measurement of TLS handshake duration, we
developed a minimal custom program based on s_client
from the OpenSSL toolkit. This tool establishes multiple
handshake connections within a configurable time window
and reports the mean duration of successful handshakes over
that period. Although this approach allows for automated
and repeatable measurements, it introduces a relatively high
degree of variability due to concurrent execution and internal
timing mechanisms.
For the application-level measurements, we adhered to
the principle of avoiding any modifications to the original
OpenVPN distribution. Consequently, the evaluation was
conducted by capturing network traffic and analyzing packet
timestamps to determine connection establishment durations.
This strategy enhances the framework’s adaptability to future
versions of the application, as it avoids introducing soft-
ware dependencies or requiring changes to the source code.
However, it also incurs additional delays due to the over-
head introduced by packet capture and logging mechanisms.
This effect becomes particularly evident when comparing the
TLS handshake durations measured during VPN connection
setup with those obtained using the custom measurement
tool.
3.4 Metrics and Statistical Analysis
In all experiments, in addition to computing the mean and
standard deviation (SD) of the connection duration, we also
calculate the median and Interquartile Range (IQR) as well as
the coefficient of variation (CV), and the percentage of out-
liers. We use Tukey’s method for the identification of outliers,
which establish as outliers those values outside 1.5 ×IQR
below the first quartile or above the third quartile. All our
results are obtained from continuous executions (‘hot’ runs),
as suggested by [25], and warm-up runs are excluded from
the reported data.
The mean and median are measures of central tendency
and comparing them indicate the skewness (asymmetry) of
the distribution. The SD and IQR quantify spread in the orig-
inal data units, while the CV normalizes this spread. In case
the SD is disproportionately large compared to the IQR, it
confirms that the outliers are heavily influencing the overall
variability. The CV, calculated as the ratio of the SD to the
mean, is a unitless measure often express as a percentage. A
low CV (below 10%) indicates stable and predictable hand-
shake timing, whereas higher values reveal greater variability
and jitter, which could impact the reliability of real-world
handshake performance.
4 Cryptographic Primitives Evaluation
This section evaluates the execution time of key encapsula-
tion/exchange mechanisms (KEMs) and signature schemes
across two processor architectures—ARM and x86— quan-
tifying the computational overhead of each primitive.
The evaluation consists of fifty consecutive runs of
OpenSSL’s speed command for each of the algorithms
introduced in Sec. 3.1. These executions were launched using
the evaluation framework inside a container with strict CPU
and I/O isolation. Precise evaluation details are provided in
Sec. 3.3.
4.1 Benchmarking KEM primitives
Key encapsulation mechanisms consist of three main opera-
tions: (i) KeyGen, a probabilistic algorithm that generates a
public/private key pair (pk,sk); (ii) Encaps, a probabilis-
tic algorithm that takes pk as input and outputs a ciphertext
ct along with a shared secret ss; and (iii) Decaps, a deter-
ministic algorithm that recovers ss from sk and ct.The
results of executing these operations on both ARM and x86
architectures are detailed in Table 2. Additional statistical
metrics, including median, IQR and outlier percentage, are
provided in Table A.1 in Appendix A.
The results show that ARM outperforms x86 across all
cryptographic algorithms and security levels, except for
123