2026-10-20 02:26:50
Nvidia’s Vera server chip aims for high per-core performance. Its Olympus cores have massive out-of-order structures and use high per-clock throughput to meet that goal within a high core count server chip. Core design however is only one part of the equation. Off-core memory access can heavily impact core performance, and higher core counts complicate interconnect design. The task of feeding Vera’s Olympus cores falls to Nvidia’s second generation Scalable Coherency Fabric, or SCF. SCF plays a similar role as Arm or Intel’s mesh interconnect, acting as a large chip-wide network that connects cores, cache, and off-chip IO. Like the first generation SCF used in Nvidia’s Grace chip, Vera’s SCF probably uses a mesh topology.
SCF is built from several node types. Coherency Switch Nodes (CSNs) host cores and cache, acting a bit like Intel’s mesh stops. Unlike Intel’s mesh stops, Nvidia has each CSN host a pair of cores. Doing so helps keep node count under control, and is a tactic also used by Ampere’s Altra Max. Ampere uses Arm’s CMN-600, and has each fully coherent request node (RN-F) host an Arm DynamIQ Shared Unit (DSU) containing a pair of Neoverse N1 cores.
Memory Switch Nodes (MSNs) serve as a bridge from SCF to Vera’s LPDDR5X memory controllers. Vera has a 1024-bit LPDDR5X-9600 setup, so the MSNs can move up to 1.228 TB/s of bandwidth. Bridge Switch Nodes (BSNs) host PCIe controllers and the cross-socket link. Vera has 88 PCIe lanes, which together can provide 665.5 GB/s of bandwidth to peripherals. NVLink C2C serves as the cross-socket link, and provides 1.8 TB/s of theoretical bandwidth.
Grace broke ground in the server scene by using LPDDR5X memory, but had mediocre DRAM performance compared to its peers. On Grace, memory latency was high at over 200 ns. Grace’s memory bandwidth was fine in an absolute sense, but fell a bit short compared to contemporary DDR5-based server chips. Vera turns everything around. The massive 1024-bit LPDDR5X interface now gives Nvidia’s server chip more bandwidth than its peers. Latency has been fixed and is now comparable to that of DDR5-based setups. Vera also adopts SOCAMM2 modules, making its memory setup field serviceable and potentially upgradeable. Grace by contrast used soldered memory, so memory chip failures could be problematic.
Vera maintains excellent control over latency under increasing bandwidth load. A pointer chasing thread doesn’t observe more than 200 ns of latency, even with other threads pushing the chip’s memory bandwidth limits. Other server platforms show a more distinct “hockey stick” pattern, where latency rises sharply as bandwidth load approaches the memory controller’s limits.
Nvidia’s combination of high bandwidth and low latency under load should let it control noisy neighbor effects better than other servers. Another way of looking at it is that Vera can out-scale other chips in DRAM-bound workloads, even if Vera has fewer cores.
Even a massive LPDDR5X subsystem won’t come close to being able to handle every program memory access, so Vera tackles the memory wall with large caches like other high performance CPUs. Vera’s caching strategy resembles the ones used on other large mesh-based designs. A 164 MB L3 cache is shared across all cores, and maintains a non-inclusive relationship with core-private L2 caches. This arrangement gives Vera up to 340 MB of effective last level cache capacity if there’s no data duplication across the L2 caches. For addresses that are frequently accessed by different cores, the L3 can economize cache capacity by only keeping one copy of the data at a shared address. Vera’s L3 read bandwidth comes out to about 3.35 TB/s, or 33.1 GB/s per core. A single core can achieve 88.5 GB/s without contention from its peers, so 33.1 GB/s represents a significant drop. However, that’s still a reasonable amount of bandwidth in an absolute sense, and is likely to be adequate because the 2 MB L2 caches should reduce L3 bandwidth requirements.
Nvidia’s caching strategy contrasts with AMD’s, which emphasizes cache capacity across the chip while delivering very high L3 bandwidth. AMD’s Genoa-X for example can give a single core 90 GB/s of L3 bandwidth, and only drops that to 85 GB/s when loading all cores in the die. Dividing Genoa-X (and other AMD server chips) into octa-core clusters means that each L3 instance only has to deliver high bandwidth to eight cores. Genoa-X’s 1248 MB of L2+L3 capacity could also be advantageous for multithreaded applications that work primarily on thread-private data, or when running many different jobs in parallel. However, a workload that shares a lot of data between threads could end up using that cache capacity inefficiently because shared data may be duplicated across multiple L3 instances. Finally, a single thread can only allocate into its 96 MB L3 instance on Genoa-X, while Vera can theoretically let one thread use its entire 164 MB L3.
NVLink C2C acts as a point-to-point link between sockets, and lets Vera scale to dual socket configurations. A dual socket Vera system has two NUMA nodes, each of which represents a socket and its directly attached memory. Cross-node memory accesses have 309.27 ns of load-to-use latency, which is 177.41 ns more than a local memory access. Achieving that 309.27 ns figure curiously requires generating a few dozen GB/s of bandwidth on the link. A pointer chasing test run in isolation measures over 370 ns of latency, suggesting that NVLink C2C only transitions to a higher performance state if there’s enough bandwidth demand and requests start backing up in front of it.
NVLink C2C delivers very high bandwidth for a cross-socket link. Reads achieve just over 587 GB/s, while a read-modify-write pattern that moves data in both directions can reach 780 GB/s. Vera not only has more cross-socket bandwidth than any other dual-socket configuration I’m aware of, but also has more cross-socket bandwidth than most servers have bandwidth to directly attached memory. However, NVLink C2C falls behind in latency terms. AMD’s Genoa-X for comparison has 222.7 ns load-to-use latency for cross-socket memory accesses. Vera also doesn’t maintain the same tight control over NVLink C2C latency that it does for local DRAM accesses. Latency spikes at high bandwidth demand, and funnily enough, reach 780 ns when using a read-modify-write pattern to achieve 780 GB/s of cross-socket bandwidth.
Interconnects have to present a coherent view of memory by making a core’s writes globally visible, even if written data may only be present in that core’s private cache. A “core to core” latency test bounces a value between two threads using atomic operations, and measures an average of 148.5 ns on core pairs in the same socket. Latencies range from 87.5 to 199.3 depending on the tested core pair, and on which agent owns the tested cache line. SMT siblings have less than 10 ns of latency, indicating that a core can maintain coherency across its SMT threads without involving the slower cross-core interconnect.
Latency between cores on different sockets is approximately 660 ns, which is unfortunately high. I remember Anandtech measured similar figures in a dual-socket Ampere Altra system, but latency above 600 ns isn’t typical.
AMD’s Genoa-X (EPYC 9V33X) achieves much better cross-socket core to core latency, at roughly 209 ns. Amazon’s Graviton 4 is even better, with cross-socket latency on par with older, lower core count designs.
Vera’s intra-socket latency is consistent, matching an advantage mentioned in Nvidia’s whitepaper. However, that latency is also consistently high, and worse than on Nvidia’s older Grace chip. It’s also worse on average than Amazon’s Graviton 4 and AMD’s Genoa-X.
SPEC’s CPU rate suite can be used to measure throughput by running multiple copies of each workload. When running one copy per hardware thread, Vera has no trouble beating Graviton 5 node-for-node. Unsurprisingly, Vera also beats a lower core count Zen 5 part that was curiously configured to have one NUMA node spanning both sockets.
However, Vera falls behind Graviton 5 at the socket level. It’s not a completely fair comparison because Graviton 5 can’t expose the entire socket as a single NUMA node, but it does show a tradeoff between core count and having more powerful cores. A density optimized design with less powerful cores can often provide higher throughput, assuming there’s work available to bring all those cores into play. SMT helps close the gap, but it’s still hard for Vera to take on a design with more than twice as many cores.
Vera still has an advantage even against Graviton 5 in a socket-for-socket comparison in workloads with high memory bandwidth demands. 749.fotonik3d tends to be very heavy on DRAM bandwidth even when running a single copy on the Ryzen 7 9800X3D.
Performance monitoring data wasn’t collected from the socket-wide Vera and Graviton 5 runs, but it’s safe to assume 749.fotonik3d is DRAM bandwidth bound for both of those chips. Running multiple copies will mean less cache capacity on average for each copy, and bandwidth demands from each copy will stack. Despite having a much lower core count than Graviton 5, Vera achieves nearly a 20% performance lead over Graviton 5 in 749.fotonik3d. That advantage extends to several other SPEC CPu2026 floating point tests. On the flip side, Graviton 5 has an overwhelming advantage in more cache friendly workloads.
Vera is a very strong contender in today’s server landscape, with exceptionally strong per-core performance backed by tremendous memory bandwidth. It’s a far more convincing offering than Nvidia’s older Grace chip, which suffered from having small 1 MB L2 caches and very high L2 miss latency. Compared to other server chips from Arm, and even AMD and Intel, Vera trades core count for per-core performance. In use cases where individual tasks need to be finished quickly and workloads may not fill all cores, Vera will be hard to beat.
For the Arm server landscape as a whole, Vera’s entrance brings a welcome increase in diversity. When Arm first gained a foothold in the server market, Arm chips competed purely on density. Cores like Neoverse N1 couldn’t match their AMD or Intel counterparts head-to-head, so chips like Ampere Altra competed by offering higher core counts. Arm server chips today still compete on that density focused front with chips like AmpereOne, but chips based on Arm’s Neoverse V series cores have expanded Arm’s boundaries. Vera pushes that boundary even further in the per-core and per-thread performance direction. More diversity and more competition in the server scene is always a good thing, and Vera’s strength is a sign of more to come.
2026-10-06 06:49:26
Server chips traditionally offered less single threaded performance than contemporary client parts. Higher core counts translate to less power per core, and the complex interconnect required to connect many cores often suffer from higher latency as well. However, that trend has been narrowing. AMD’s recent server chips have been pushing higher frequencies over the last couple generations. NVIDIA’s Olympus is another aggressive move to push server single threaded performance forward, but approaches the problem from a different angle by focusing more on performance per clock.
To deliver on that front, Olympus takes a similar approach to Arm’s Cortex X925. Growing core structures tends to run into diminishing returns, so Olympus diverges from Arm’s approach with a simultaneous multi-threading (SMT) implementation. The result is a monster core that is a force to be reckoned with in the server scene, and one that gets very close to desktop performance.
Olympus is a 10-wide out-of-order core with almost comically large out-of-order structures depending on where you look. It runs at a modest 3.3 GHz clock speed, which isn’t high these days even by server standards, so Olympus needs very strong per-cycle performance to meet NVIDIA’s performance goals.
At a high level, Olympus’s general layout has a lot in common with Arm’s Cortex X925. Both cores use a semi-distributed scheduler with a similar execution unit layout. Both aim to deliver high performance without having to reach the clock speeds that AMD and Intel’s cores do. However, Olympus takes that goal even further by increasing out-of-order structure sizes compared to X925. NVIDIA’s core also differs in a number of areas, like having a non-scheduling queue in front of the floating point schedulers.
A wide core with deep reordering capacity has a lot to lose from branch mispredicts, so Olympus needs a strong branch predictor. In a test with various numbers of branches that are taken/not-taken with different pattern lengths, Olympus stops just short of Arm’s X925. X925’s direction predictor can track approximately 16-24K global history patterns with a few branches in play, or over 64K patterns with 512 branches. Olympus can get to roughly 5-6K patterns with a couple branches, or 48K with 512 branches.
In SPEC CPU2026, prediction accuracy is right up there with the best of Intel and AMD’s cores. Olympus overall is slightly behind AMD’s Zen 5, and slightly ahead of Lion Cove. Each core has individual wins and losses on different workloads, highlighting the difficulty of optimizing a branch predictor to cater for a wide range of program behaviors.
SPEC CPU2026’s floating point suite is generally easier on branch predictors, but still includes a couple of workloads with slight branch prediction challenges. Olympus comes out ahead of Intel and AMD in the floating point suite, mostly because of a large win in 731.astcenc.
Branch prediction speed is also crucial to feeding a wide, high throughput design because taken branches are quite common in code. Like AMD’s Zen 5, Olympus can do two taken branches per cycle across a large branch footprint. Unlike Zen 5 and Cortex X925, Olympus’s ability to do two taken branches per cycle is tied to instruction footprint rather than branch count, which could mean it’s tied to the instruction cache. Instruction footprints beyond 48 KB break the two branches per cycle capability. For those larger instruction or branch footprints, Olympus has a 16K entry BTB with 4 cycle latency.
NVIDIA partitions the branch target caching structures to give each thread about half of available capacity when the core is running two threads. The two threads cannot share targets, even if they belong to the same process and therefore share an address space. Sustained taken branch throughput remains at two branches per cycle from the fastest BTB level. When servicing two threads, the 16K entry L2 BTB can deliver a taken branch target every five cycles from each thread’s point of view, or roughly every 2-3 cycles from the core’s point of view.
NVIDIA and AMD both use ahead indexing to improve branch predictor throughput, meaning that one lookup provides predictions for the next two branches rather than a single branch. NVIDIA implies they can do this for their direction predictor and not just the BTB, letting them carry out more predictions per cycle for branches with dynamic behavior.
Olympus implements advanced predictors with robust ahead pipelining mechanisms delivering up to 2.3x higher predictions per cycle vs the competition
-Vera Whitepaper
Predicting with BTB pairs allows two fetches to be predicted in one prediction cycle
-AMD’s Zen 5 Optimization Guide
I tried setting up a test with conditional branches that were not-taken 1/8 of the time in a trivially predictable manner. I was hoping that would force the core to use the conditional predictor to at least validate what comes out of the BTB via ahead indexing. But if that causes an impact, I can’t see it in either Zen 5 or Olympus. Both cores can manage two taken branches per cycle if they have to use the branch predictor. I tried making the branches taken/not-taken in more complex patterns, but stopped putting more time into after the first couple attempts caused mispredicts on both cores.
NVIDIA further noted that Olympus could sustain more predictions per cycle compared to the competition, and cited specific SPEC CPU2026 integer workloads.
Validating this is difficult because neither architecture provides hardware counters to track how many predictions they made. I also don’t think predictions per cycle is a useful metric by itself, because predictions made after a mispredict are useless and more predictions per cycle may not impact performance if the frontend is already adequately fed. Another factor is the core’s design goals around performance per cycle rather than clock speed. Olympus may be averaging more predictions per cycle, but that could be because it’s moving along more of the instruction stream in a given number of cycles compared to Zen 5. Zen 5 in contrast is designed to clock high while still being able to sustain two taken branches per cycle, and the rate at which the two designs complete branches isn’t too far apart when clock speed is taken into account.
Fetch addresses from the branch predictor go through address translation with a 64 entry fully associative instruction TLB. Olympus then fetches instructions from a 64 KB, 4-way set associative instruction cache that can deliver 128 bytes per cycle into a 48 instruction decode queue. A 128 byte fetch can contain up to 32 instructions, which far exceeds the core’s decode throughput and possibly serves to let the frontend catch up if it meets a straight run of instructions after a long latency branch or mispredict.
Larger instruction footprints are handled well, with the core able to fetch 32B/cycle of instructions from the L2 cache. Per-cycle L2 code throughput for a single thread is much higher on Olympus than on Zen 5 or Cortex X925. Like many other cores, code read throughput drops sharply if instructions spill into L3.
Running two threads in the core causes a sharp drop in instruction fetch bandwidth before 1 MB, indicating that Olympus is statically partitioning the L2 cache. This applies even if the sibling thread is running a trivial loop that consists of several instructions, and doesn’t make any memory accesses that could cause data-side cache capacity contention. Throughput for a high IPC thread (NOPs only) is also cut in half even if the other thread is latency bound and stuck at low IPC.
Micro-ops from decoded instructions need to go through register renaming and have appropriate tracking resources allocated in the backend. This process provides opportunities to carry out other optimizations like move elimination. Olympus can do move elimination to a limited extent. A chain of dependent MOVs executes at just over 1 per cycle, indicating that some MOVs executed with zero latency. AMD’s Zen 5 can impressively do this across 5 MOVs per cycle.
If the core runs two threads, Olympus’s per-thread move elimination capability strangely gets better. Per-thread dependent MOV throughput increases to 1.7 MOVs/cycle per thread, or 3.5 across both threads. Zen 5 improves as well, with per-core throughput increasing to 7 MOVs per cycle. However, that represents a drop in per-thread throughput from 5 to 3.5 MOVs/cycle.
Olympus’s backend has massive register files and queues just about everywhere. It’s a bit like Cortex X925, but with many things taken up a notch. The only structure with a somewhat lower entry count is the FP/vector register file, which has fewer entries than Zen 5’s. However, Olympus compensates by often letting a 128-bit vector register hold a pair of scalar FP values written by separate instructions. It’s a useful optimization because a lot of code cannot be vectorized.
Testing with NOPs puts a very high ceiling on the number of instructions that Olympus can have in flight. The core can overlap two long latency loads with more than 1000 NOPs spaced between them, but the practical number of in-flight instructions is likely to be significantly lower because Olympus can only have a bit over 606 registers allocated. The ~606 allocated register limit applies on the architectural register side, because using the scalar FP optimization mentioned above still results in the number of allocated architectural registers capping out at 606.
Olympus’s schedulers are laid out similarly to those on Arm’s Cortex X925. Eight integer ALUs are arranged into pairs, with each pair fed by a scheduling queue with approximately 25 entries. One of the ALU pairs is special and can handle less common operations. Olympus and Cortex X925 both use a six-pipe FPU, again with a scheduling queue for each pair of pipes. However, NVIDIA uses a non-scheduling queue in front of smaller scheduling queues, while Arm uses three very large scheduling queues.
All of the backend structures I checked are statically partitioned, meaning that each thread gets half (or less) of a structure’s capacity when the sibling thread is not idle. That applies even to scheduling queues, which are often watermarked or competitively shared on other architectures. Static partitioning even extends to the execution units, with execution throughput cut in half regardless of whether the sibling thread is running instructions that hit the same ports. AMD’s SMT implementation for comparison flexibly shares most structures, though it sets limits (watermarking) to ensure one thread cannot unfairly monopolize a resource.
Olympus executes memory accesses down a set of four pipelines, all of which can handle loads, and two of which can handle stores. The core’s L1 data cache has a nominal latency of four cycles, but a simple memory latency test can measure 2-3 cycles of L1D latency possibly because of value prediction.
Memory disambiguation behavior on Olympus has more in common with older cores from Arm or Intel’s Atom line. Olympus can do fast forwarding from a 64-bit store to a dependent 32-bit load as long as the load is aligned to either half of the store. Forwarding latency is 3-4 cycles for exact address matches, or 5-6 cycles when forwarding the upper half. All other overlap cases lead to a 12-13 cycle penalty, likely because the core blocks the load and waits for the store to commit.
Olympus encounters a strange hiccup when doing store forwarding across the 32B boundary in the middle of a 64B cacheline. It strangely doesn’t hiccup in the same way when accesses cross a 64B cacheline boundary, even though every 64B aligned boundary is of course also 32B aligned. Also, independent loads and stores that touch the same 32-bit block at the end of the first 32B half of a cacheline sometimes aren’t allowed to proceed in parallel.
Wider vector accesses show the same strange behavior at the middle 32B boundary, and show an additional hiccup at the first 16B aligned boundary into the cacheline. Latency for vector store forwarding is higher, which is typical for many cores.
For address translation, Olympus has a large 112 entry fully associative data TLB, backed by the unified 3K entry L2 TLB. Zen 5 for comparison has a 96 entry fully associative data TLB backed by a 4K entry L2 data TLB. AMD doesn’t use a unified L2 TLB, and stores instruction-side translations in a separate 2048 entry L2 instruction TLB. NVIDIA ran a kernel with 64 KB pages on the system we had access to, which extends TLB reach compared to using typical 4 KB pages. 64 KB pages basically meant that address translation latency was a non-factor for a lot of testing, but made direct comparisons difficult.
Large out-of-order structures, prefetchers, branch prediction, and value prediction help a core tolerate memory access latency. Caches attack the problem from the other side by cutting memory latency for frequently accessed addresses. Vera’s caching strategy is greatly improved compared to NVIDIA’s prior Grace CPU. Olympus’s data memory hierarchy starts with a large 96 KB 6-way set associative data cache, which offers more capacity than the 32, 48, or 64 KB data caches often found on AMD, Arm, and Intel’s server cores.
L1 misses can be caught by a 2 MB 8-way set associative L2, which delivers 10-cycle load-to-use latency. The low cycle count latency means that Olympus’s L2 can deliver data nearly as fast as the Cortex X925’s 12-cycle L2, despite the X925’s higher 4 GHz clock speed. Having 2 MB of L2 capacity is a welcome improvement compared to Grace’s Neoverse V2 cores, which had to make do with 1 MB L2 caches. Vera’s L3 latency is comparable to Grace’s and isn’t great in absolute terms at over 120 cycles. However, that’s typical of designs that implement a L3 cache over a massive monolithic interconnect. Vera at least increases L3 capacity from 114 to 164 MB.
NVIDIA covers for high L3 latency with aggressive prefetching. A linear read pattern from a single thread achieves 88.53 GB/s from L3, and applying Little’s Law indicates that Olympus was able to sustain 50-51 outstanding requests to L3 (88.53 GB/s /(64B lines * 3.3 GHz) = 0.42 cachelines per cycle, multiplied by 121.5 cycle latency = 50.9 requests in flight). Doing the same calculation for Grace/GH200 gives 35-36 requests in flight. As with many other cores, Olympus likely has a deeper queue for L2 misses than L1 misses. Testing with Lemire’s memory level parallelism microbenchmark shows that Olympus can have 18-19 demand L1D misses in flight. Lemire’s test uses a randomized pointer chasing pattern that won’t work with L2 prefetchers, and therefore won’t take advantage of a larger L2 miss queue.
Olympus’s SMT implementation partitions the L1D and L2 caches for data access too, meaning that each thread gets half the capacity of all core-private caches if the sibling thread is active. As with the instruction side, this partitioning applies even if the sibling thread makes no data-side accesses.
SPEC CPU2026 scores show Olympus delivering very strong single threaded performance for a server core. It isn’t far off Intel and AMD’s latest cores in desktop settings, represented here with the Core Ultra 9 285K and Ryzen 7 9800X3D. Against Arm server competitors like Neoverse V2 and Neoverse N2, Olympus has no trouble pulling off a significant lead. A significant problem here is that the kernel on the test system we had access to uses 64 KB pages, while all other results are obtained with 4 KB pages. Estimating page size impact will be difficult, but I expect Olympus will still compare well against its Neoverse counterparts even with a smaller page size.
Examining individual scores show NVIDIA doing well across many integer workloads including the ones highlighted in their whitepaper (723.llvm_r, 727.cppcheck_r, 721.gcc_r, 714.cpython_r). Olympus’s large loss to Zen 5 in 706.stockfish_r has more to do with instruction set differences than core architecture, because Olympus had to execute roughly twice as many instructions as Zen 5 did in that workload. ISA differences also affect 772.marian_r on the floating point side. Instruction count can go the other way too, with aarch64 requiring fewer instructions for 750.sealcrypto_r, but Zen 5 is able to maintain a strong performance in that workload thanks to its overall higher throughput design when taking clock speed into account.
Olympus’s goal is to achieve high single threaded performance by emphasizing performance per clock, and performance counters unsurprisingly indicate very high IPC. NVIDIA’s IPC is often higher than Zen 5’s, even in workloads where Zen 5 manages to deliver higher performance.
Wider pipelines are difficult to feed, and checking top-down performance events reveals that nearly all workloads leave over 50% of dispatch slots unused on average. SPEC CPU2026’s integer workloads lose a lot of throughput because the frontend couldn’t supply micro-ops fast enough (frontend bound), showing that even a fast two-ahead branch predictor and large instruction cache can’t always keep up. The same is true of Zen 5 as well.
SPEC’s floating point workloads tend to be more backend bound, meaning that micro-ops couldn’t dispatch because a backend resource was full. In other words, the core reached a limiting factor for how many instructions it could keep in flight.
SMT is a way to claw back unused core width and more efficiently utilize backend resources when a second thread is available. NVIDIA’s “spatial multithreading” takes a rather unconventional approach to SMT, and definitely shows promise when looking at SPEC CPU2026’s integer workloads. There, Olympus’s SMT gains are on par with Zen 5’s. Zen 5 is ahead in actual performance, but I’m focusing on gains here because again a client CPU is expected to maximize per-core performance.
Olympus’s gains are less convincing on the floating point side, though SPEC CPU2026’s floating point suite still enjoys a significant overall uplift.
Individual workloads show varied gains, with Zen 5 or Olympus getting more out of SMT on different subtests compared to the other. However, the general pattern is the same for both cores. Low IPC, frontend bound workloads like 723.llvm tend to see massive gains from SMT. Higher IPC, throughput bound workloads like 750.sealcrypto and 714.cpython don’t gain as much.
SPEC CPU2026’s floating point suite tends to be more backend bound, and Olympus’s SMT implementation surprisingly suffers throughput losses on 709.cactus and 782.lbm. SMT implementations ideally don’t lose overall throughput when using both threads, so Olympus’s spatial multithreading doesn’t seem ideal in those two workloads. Zen 5 also gains less from SMT in SPEC’s floating point workloads, but manages to avoid situations where using both SMT threads would be worse than running two tasks in serial.
SPEC CPU2017 remains an interesting suite because it features difficult workloads without analogues in SPEC CPU2026. I also have more SPEC CPU2017 data on hand, including from systems I no longer have access to. That data shows Olympus beating several more server cores, including a Zen 5 server implementation. It also shows Olympus drawing even with AMD’s top end Zen 4 desktop chip, which is an impressive feat for a server core.
SMT gains in SPEC CPU2017 are even higher than in SPEC CPU2026, thanks to workloads that wreak havoc on branch predictors and suffer from backend memory latency. 505.mcf comes to mind.
Olympus again takes a few concerning SMT losses in the floating point suite. AMD’s Zen 4 isn’t immune, but its losses are comparatively light. A 3% or 1% performance decrease is far from ideal, but is likely less noticeable than 11.98% or 9.78% losses.
Explaining Olympus’s SMT behavior is difficult, because analyzing SMT test cases is harder than looking at single threaded ones. 709.cactus’s situation can be explained by L2 data MPKI increasing from 3.58 to 6.52 when the core is running two copies of the workload in its SMT threads. Then, the partitioned resources in the backend may be insufficient to deal with high L3 latency.
However, I can’t apply the same explanation for SPEC CPU2017’s 503.bwaves and 549.fotonik3d. 503.bwaves thrashes the L2 in both cases, with 32.12 MPKI for a single thread and 33.03 MPKI when running two threads. Top-down events indicate performance losses for non-memory bound reasons. 549.fotonik3d also suffers a lot of L2 misses (52.45 MPKI for a single thread, 57.13 for two threads), but again the relative difference isn’t too big. In general, Olympus’s spatial multithreading seems to deliver strong gains on par with a conventional SMT implementation for frontend bound workloads, but doesn’t handle as well with backend bound ones.
NVIDIA’s spatial multithreading is less like a conventional SMT implementation, and more like dynamically splitting Olympus into two smaller cores when it’s running two threads. Just about every imaginable core resource down to the L2 cache is statically partitioned, so each thread can only access half of what’s available. This approach certainly eases performance analysis, because viewing the two SMT threads as two 5-wide cores is simpler than thinking about how two threads might dynamically use various core resources in, say, AMD’s Zen 5. I imagine it’s easier on the core design front too. NVIDIA doesn’t have to tune complicated mechanisms that ensure thread fairness because splitting everything in half already gives a fairness guarantee.
I think the jury is still out on whether spatial multithreading is a good idea or a simplification meant to ease NVIDIA’s first generation SMT implementation. Spatial multithreading shows promise by delivering gains on par with conventional SMT implementations in SPEC CPU’s integer workloads. In the best cases, Olympus demonstrates that how efficiently you can let two threads share resources might not matter if your core resources are sufficiently massive in the first place. However, gains can be inconsistent and even negative in backend-bound scenarios in SPEC’s floating point tests. It’s hard to tell whether those rough spots come from teething issues in a first generation SMT implementation, or from fundamental inefficiencies that arise from locking each thread into half of a core. I imagine a certain portion of SMT gains come from dynamically letting each thread dip into resources left under-utilized by its sibling, and limiting each thread’s ability to do that could negatively impact SMT gains.
As with any architectural feature, it’s probably best to wait and see if NVIDIA continues the same strategy. Successful architectural techniques tend to stay around for multiple generations and gain adoption across the industry. Whether that’ll happen with spatial multithreading is anyone’s guess.
Arm’s early server chips emphasized core count over per-core performance, but the Arm server scene has diversified as newer Arm cores took aim at higher performance targets. Now, NVIDIA’s Vera delivers top-notch single threaded performance and turns the situation around. NVIDIA is now the one with a lower core count (though 88 cores is still a lot in an absolute sense), and AMD’s server chips have more cores with less single threaded performance. There is some wiggle room in the SPEC results because of different page sizes, but I suspect that would only narrow the gap between Olympus and the fastest AMD server cores rather than changing their relative rankings.
Against its Arm Neoverse competition, Olympus dominates on per-core throughput terms thanks to both its high single threaded performance and its ability to increase throughput by running a second thread. Debating the merits of a conventional SMT approach versus Olympus’s spatial multithreading one doesn’t matter when Arm cores can’t simultaneously run a second thread at all. Olympus gains throughput from running a second thread in the vast majority of cases, and any throughput it gains widens the per-core performance gap between it and its Neoverse contemporaries. NVIDIA’s prior Grace chip is included in that mix. Vera is a huge step up from Grace, with both faster cores and more cores.
Olympus overall is a very exciting server core. Its performance shows the merits of a low frequency design that aims to maximize work done per clock cycle, and definitely pushes boundaries for single-threaded performance in a server. Other server core designers should take note, because performance in low-threaded workloads continues to be important even as server core counts continue to increase. I look forward to seeing NVIDIA further evolve this impressive core.
2026-09-30 00:07:30
Laptop chips have gotten very powerful in recent years, with increasing CPU core counts, more powerful iGPUs, and higher DRAM bandwidth to feed that compute. All of that creates pressure on the chip’s system level architecture, which must deliver high bandwidth, service an increasing number of blocks, and keep latency under control. Qualcomm’s Snapdragon X2 Elite Extreme is a good example of how powerful recent laptop chips have become, with 18 CPU cores and an iGPU packing nearly as much compute as a Nvidia GeForce GTX 1070 Ti. It’s an aggressive entry into the laptop market, and needs a strong interconnect design to support everything packed into it.
Qualcomm simplifies the chip-level interconnect by splitting CPU cores into clusters, much like AMD. All cores within a cluster can be treated as a single client of the system level interconnect, reducing the number of clients that interconnect has to deal with. A shared cache within a cluster only has to concern itself with that cluster’s cores, simplifying cache design and making it easier to ensure fast cache access. Qualcomm uses 6-core clusters on the Snapdragon X2 Elite Extreme, compared to quad core clusters on the older Snapdragon X Elite. The first cluster contains E-Cores, or “Performance” cores in Qualcomm parlance, and has 12 MB of shared L2 cache. The two other clusters contain P-Cores, or “Prime” cores, and have 16 MB of shared L2 cache each.
Qualcomm employs a caching strategy with parallels to the one used in Apple’s M1. Each cluster’s L2 is designed to provide very low latency and high bandwidth, while each CPU core comes with relatively large L1 caches. This combination lets Qualcomm dispense with the mid-level caches often found on AMD, Arm, and Intel designs. To deliver the required performance, Qualcomm runs the L2 caches at core clocks and tightly couples them to the cores. L2 latency comes out to 20-21 cycles on both the P-Core and E-Core clusters.
Curiously, a couple cores within each P-Core cluster have slightly higher latency at 22-23 cycles, while all E-Cores have the same L2 latency. I suspect Qualcomm uses a centralized crossbar design for their L2. The high 5 GHz might mean two P-Cores need a couple extra pipeline stages to get to L2 while meeting cycle time requirements. Perhaps the E-Cores didn’t need to make that concession because of their lower 3.6 GHz clock speed.
L2 bandwidth is 32B/cycle in both the read and write direction, so each core can achieve 64B/cycle of off-core bandwidth with a read-modify-write pattern. This is true even in the E-Core cluster, even though the load/store unit on Qualcomm’s E-Cores can only handle two 16 byte accesses per cycle. With a test where an E-Core only accesses half of each 64B cacheline’s data, it’s possible to infer that the interface to L2 can sustain 64B/cycle of fills and writebacks.
Bandwidth scales well with increasing active core counts. Each P-Core is able to get more than 100 GB/s of read bandwidth on average when hitting all cores in a cluster with a bandwidth test. AMD does come out ahead with a read-modify-write pattern, but Qualcomm should have more than enough shared cache bandwidth on hand for the vast majority of use cases.
Going outside the cluster reveals an impressive level of external bandwidth. Each cluster appears to have a 32B/cycle path to the system, but cycle here is a cycle at the cluster’s clock. AMD’s Strix Halo for comparison has 32B/cycle read and write paths to the system, but that’s running at the chip’s 2 GHz Infinity Fabric clock (FCLK). The result is that Qualcomm’s CPU core clusters enjoy higher DRAM read bandwidth than Strix Halo clusters.
AMD again does well with a read-modify-write pattern that exercises both of the cluster’s external 32B/cycle paths. However, most applications will make far more reads than writes, so Qualcomm’s ability to get higher bandwidth with reads alone is a notable advantage.
DRAM latency on the Snapdragon X2 Elite Extreme is rather good for a LPDDR5X setup at 115 ns. It can’t compare with the sub-100 ns latencies typical on desktop DDR5 setups, but it’s up there with the best LPDDR5(X) implementations. As CPU bandwidth demands increase, Qualcomm’s system fabric maintains excellent control over latency and doesn’t let bandwidth-hungry threads excessively penalize a latency sensitive one. CPU read bandwidth in absolute terms is impressive too, surpassing even chips with 256-bit memory buses. AMD’s Strix Halo and Nvidia’s GB10 have 256 GB/s and 273 GB/s of theoretical DRAM bandwidth, respectively. The Snapdragon X2 Elite Extreme has 228 GB/s of DRAM bandwidth with a 192-bit bus. However, AMD and Nvidia’s CPU clusters don’t have enough external bandwidth to fully utilize their beefier DRAM setups.
Switching to a read-modify-write pattern lets Strix Halo’s CPU cores utilize more DRAM bandwidth, but hurts Qualcomm. Mixing reads and writes can challenge the memory controller, because the DDR memory bus can only be in read or write mode at any given time. Switching the bus between read and write mode (a turnaround) wastes bus cycles and therefore reduces achievable bandwidth. Perhaps Qualcomm suffers more from that, but getting to 68.5% of theoretical DRAM bandwidth is still a good result.
Cluster designs can experience contention at cluster boundaries, but that’s very well controlled on Qualcomm’s design. Latency stays under 200 ns from a CPU core regardless of how much DRAM bandwidth its peers are pulling through the off-cluster interface. It’s a better showing than some of AMD’s designs, which could let bandwidth-hungry threads squeeze out a latency sensitive one.
Adding in bandwidth demands from other clusters pushes latency up, but it never reaches unacceptable levels.
Qualcomm’s performance in this testing also compares well to Nvidia’s GB10, which could let latency creep over 200 ns when X925 cores contend with each other for off-cluster bandwidth.
Interconnects have to maintain cache coherency by ensuring that a write from one core can be made visible to another. A “core-to-core” latency test evaluates how long it takes for one core to observe another’s write using compare and exchange operations. Qualcomm probably handles transfers within a cluster through the L2 cache, which maintains an inclusive relationship with core-private caches within the cluster. Because the L2 includes L1 contents, it can easily tell when it might need to probe another core. Intra-cluster transfers enjoy very low latency, which is up there with the best of what Intel or AMD’s designs can achieve.
Cross-cluster accesses go through Qualcomm’s system level fabric and incur higher latency. However, that latency still remains well controlled and is similar to cross–cluster latency on AMD’s client designs. Latency to the E-Core cluster is somewhat higher than between the two P-Core clusters.
In all cases, Qualcomm’s newest design is a huge improvement over the older Snapdragon X Elite. Core clustering still means there’s a cross-cluster latency penalty, but it’s now much better controlled than on Qualcomm’s previous design.
Intel, and more recently Nvidia, have criticized cross-cluster penalties in their marketing publications. However, everything is a tradeoff, and focusing on uneven latencies misses the benefits of clustering. Clusters let Qualcomm deliver excellent shared cache latency and low core-to-core latency within a cluster, and that’s true of AMD’s designs too. I doubt Qualcomm could deliver L2 latencies in the low 20 cycle range if L2 accesses have to traverse an interconnect servicing 18 cores. Similarly, sub-20 ns core-to-core latencies would be hard to achieve with larger, logically monolithic interconnects.
I think Qualcomm’s clustering approach strikes a good balance. Worst case latencies aren’t much higher than average latency on a high core count, mesh-based server chip. Trying to build a big monolithic interconnect risks getting consistent core-to-core latency at the cost of making that core-to-core latency consistently bad. A monolithic interconnect would also impact cache performance for typical accesses that don’t need cross-cache transfers.
GPUs can be huge bandwidth consumers on modern laptop chips. When mixing CPU and GPU bandwidth demands, the Snapdragon X2 Elite Extreme can let a CPU thread’s DRAM latency reach nearly 350 ns. It’s not too bad though. Worst case latency is on the same level as Meteor Lake, and achieved bandwidth is much higher than what a 128-bit LPDDR5(X) setup can hope to deliver.
CPU+GPU bandwidth doesn’t approach theoretical DRAM bandwidth as closely as it does on other chips. I achieved the highest bandwidth in this test run with the GPU using 74.86 GB/s and the CPU pulling 94.78 GB/s, for a total of 169.335 GB/s. That’s 74.2% of theoretical, and just short of the 176.98 GB/s (77.6% of theoretical) achievable with a CPU-only bandwidth test. For comparison, AMD's Strix Halo and Nvidia's GB10 can achieve 226.08 GB/s and 243.05 GB/s, or 88% and 81% of theoretical respectively.
Unlike Nvidia’s GB10 and AMD’s Strix Halo, the Snapdragon X2 Elite Extreme doesn’t let the GPU squeeze out the CPU. If I let the CPU and GPU grab as much bandwidth as they can get their hands on (very little compute relative to memory accesses), Qualcomm gives the CPU and GPU each about half of available bandwidth. Decreasing GPU bandwidth demands lets the CPU pick up whatever’s left. GB10 and Strix Halo can both give the CPU more bandwidth if the GPU isn’t asking for too much. However, they leave DRAM bandwidth on the table because their interconnects can’t deliver the same level of bandwidth to the CPU side.
Qualcomm’s bandwidth patterns under this test are closest to Meteor Lake’s, though the Core Ultra 7 155H has much lower bandwidth in general. It’s an older chip, and its 128-bit LPDDR5X-7467 configuration can’t hope to compete with a faster 192-bit one.
High CPU core counts and large iGPUs can challenge interconnect design, but Qualcomm’s interconnect is more than up to the task. The Snapdragon X2 Elite ensures its CPU cores can enjoy low latency and high bandwidth access to shared caches thanks to its clustered setup. Then, Qualcomm’s system level fabric offers a compelling combination of high bandwidth and well-controlled latency. It compares well against other mainstream laptop chips, though admittedly I only have Intel’s slightly dated Meteor Lake to compare against. More impressively, it beats out AMD’s Strix Halo and Nvidia’s GB10 in terms of CPU-side bandwidth. AMD’s Strix Halo and Nvidia’s GB10 both have larger 256-bit DRAM buses, but they can feel like large GPUs that happen to have a pile of CPU cores integrated. Qualcomm’s Snapdragon X2 Elite doesn’t give off the same vibes, and treats its CPU cores as first class citizens.
Qualcomm’s excellent interconnect performance and high bandwidth DRAM setup speak to the company’s ambitions. I remember when years ago, Gerard Williams III (who no longer works at Qualcomm today) mentioned at Hot Chips that architectural features can be built with a vision to enable advances in future designs, even if those features don’t deliver a huge benefit for a current product. Back then, he was talking about CPU core design, but the same principle can apply at the chip level too. The Snapdragon X2 Elite’s 192-bit LPDDR5X setup and system level fabric borders on being over-engineered considering its CPU and GPU setup. CPU cores typically hit cache for the vast majority of memory accesses provided the cache design is competent, so I expect typical CPU-side bandwidth demands to be well within what a 128-bit DRAM bus can deliver. GPU workloads often demand high sustained bandwidth, but Adreno X2 isn’t sized to deliver the level of performance that Strix Halo or GB10’s iGPUs can, and can’t consume the same level of bandwidth.
The Snapdragon X2 Elite Extreme’s system level design is something of a warning to Intel and AMD. For now, AMD’s Strix Halo and Intel’s Panther Lake stand out as halo-segment mobile chips. But no one designs an interconnect and memory subsystem as strong as the one in the Snapdragon X2 Elite if they haven’t at least thought about building a monster to take on AMD and Intel’s halo-segment products. With that in mind, I’m excited to see what Qualcomm comes up with next, on both the CPU and GPU side.
2026-09-20 02:03:20
Integrated GPUs have become a crucial component in recent laptop chips, thanks to a push for better graphics performance in ultraportable devices. AMD and Intel have both launched impressive iGPU implementations in recent years. Qualcomm’s laptop push needs a powerful iGPU in this landscape, and that’s where the Snapdragon X2 Elite’s Adreno X2-90 enters the picture.
Adreno X2 expands on the foundation set by the Snapdragon X Elite’s Adreno X1, but brings an enhanced architecture that aims for higher throughput. Shader Processors (SPs) act as the basic building block for Qualcomm’s Adreno GPUs. Each SP contains a pair of Micro Shader Processor Texture Processor (uSPTPs), which have execution units, register files, and private texture caches. At a higher level, Qualcomm groups pairs of SPs into slices. A fully enabled X2-90 part has eight SPs grouped into four slices, compared to six SPs in three slices on Adreno X1. Qualcomm also scaled up clock speeds from 1.5 to 1.85 GHz. A larger, higher clocked GPU will put more pressure on the memory subsystem, so Qualcomm scaled up cache sizes and increased bandwidth throughout the memory hierarchy.
With all these changes, Adreno X2 has nearly twice the theoretical compute throughput of its predecessor. It’s dwarfed by AMD’s highest end iGPU offering, which brings in more of everything. AMD’s more standard “Strix Point” chip may be a better comparison, but I don’t have that on hand, so I’ll be doing performance comparisons across a smattering of interesting devices. A special thanks goes out to ASUS for supplying the Zenbook A16 for testing.
Qualcomm’s Adreno line traditionally emphasizes high throughput for basic FP32 and FP16 operations. Adreno X2’s uSPTPs continue that emphasis, with 128 FP32 lanes that appear to be arranged in two execution unit partitions. FP16 benefits from double rate execution like on many modern GPUs.
Other instruction categories don’t necessarily enjoy the same high throughput, but Adreno X2 improves over its predecessor in several areas. INT32 adds appear to execute at full rate, though it’s hard to approach theoretical throughput on them compared to FP32 operations. 32-bit integer multiplies execute at half rate, which compares well to Intel and AMD’s GPU architectures, and aligns with Nvidia. Integer performance overall is a huge improvement over the rather poor situation on Adreno X1, where even basic integer operations executed at less than half rate. Better INT32 units help 64-bit integer performance too, since GPUs carry those out using multiple INT32 operations. Like previous Adreno GPUs, Adreno X2 lacks FP64 support.
Testing with Nemes’s Vulkan-based suite shows substantial compute improvements compared to Adreno X1. Adreno X2 also compares well to older iGPUs. AMD’s Radeon 780M can achieve higher FP32 throughput if AMD can use VOPD dual-operation instructions or wave64 mode, but Adreno X2 isn’t far off even in that case. However, Adreno X2 seems to need a lot of work in flight to approach theoretical compute throughput, even compared to larger GPUs like AMD’s Strix Halo.
OpenCL experimentation with varying dispatch sizes shows that Adreno X2 can reach beyond TFLOPS with FP32 adds, or 6 TFLOPS with multiply-adds. Adreno X2 just needs roughly 8x more parallelism to reach its maximum throughput. Further testing with OpenCL with large dispatches shows Adreno X2 getting close to theoretical throughput in floating point operations. However, INT32 throughput barely improves over Vulkan.
Adreno X2 continues to lack hardware fused multiply-add (FMA) capability like its predecessor. FMA avoids intermediate rounding between the multiply and add, reducing numerical error. Asking for that using OpenCL’s fma() built-in function results in very poor throughput. AMD, Intel, and Nvidia’s GPUs all have native FMA hardware and can provide FMA behavior without throughput loss.
Each Adreno X2 uSPTP has 128 KB of register file capacity, down from 192 KB on Adreno X1. Register file design tends to be difficult because they have to provide immense bandwidth to feed a GPU’s wide vector units. Smaller register files might have let Qualcomm increase clock speeds while keeping power and area under control. Adreno X2 also ditches Adreno X1’s wave128 mode, and only supports wave64. Smaller wave sizes can let register file capacity go further because each thread can allocate more registers before impacting theoretical occupancy.
With the register file capacity change, Adreno X2’s occupancy should behave like Adreno 730’s.
Qualcomm didn’t disclose Adreno X2’s theoretical occupancy, but I tried to guess by running a DRAM latency test with different dispatch sizes, and the same access pattern used across all workitems. If the GPU can keep all workitems in flight, latency shouldn’t be much higher than with a single workitem. However, running out of wave slots would force some workitems to wait until another finishes and its wave slot frees up, increasing observed runtime.
I suspect each Adreno X2 uSPTP partition has somewhere between 6 to 8 wave slots, with 8 being a more likely figure. Latency starts increasing with a 12288 workitem dispatch (6x wave64 per uSPTP partition), and more than doubles when going from 16384 to 18432 workitems.
Qualcomm also did not disclose instruction cache capacity, but basic testing with increasingly large loops shows that throughput from a single thread stays high with up to 8192 FP32 add statements in the loop. Absolute throughput figures suggest Adreno X2 can’t dual-issue FP32 adds from a single thread, possibly because each wave64 thread is locked to a single uSPTP partition. Throughput remains reasonable at larger instruction footprints.
Adreno X2 gets a beefed up memory hierarchy compared to its predecessor, but continues to inherit Adreno-style memory subsystem features that make it unusual next to other modern GPUs. uSPTPs continue to have texture caches that don’t serve compute global memory accesses. These texture caches are now 4 KB in size, but that’s still very small. The smallest texture cache I’ve seen outside of Qualcomm’s Adreno GPUs is the 8 KB one on AMD’s Terascale.
Global memory accesses go straight to Adreno X2’s 128 KB cluster caches. Cluster cache accesses have slightly higher latency than texture cache hits, but remains reasonable in an absolute sense. However, that means Qualcomm has higher minimum latency for global memory accesses than AMD. The gap is especially wide if AMD can carry out scalar optimizations. Qualcomm does not have a separate scalar cache hierarchy. Even if I use the constant memory type, that appears to be backed by the same cluster cache. The cluster cache does fine in bandwidth terms, and can deliver 64B/cycle to each uSPTP.
A 2 MB L2 cache sits after the cluster caches, and matches AMD’s 2 MB L2 on Strix Halo and Strix Point. L2 latency is reasonably good, and better than AMD’s. L2 capacity is small next to what Intel has been doing, but Qualcomm does have a system level cache to catch L2 misses, making them less reliant on L2 capacity. AMD’s Strix Halo uses a similar strategy.
L2 bandwidth appears to be a bit above 1 TB/s, or just over 32B/cycle per uSPTP. Like upper level caches on many GPUs, Adreno X2’s cluster cache is either write-through or read-only. Writes go to L2, which services them at about half the read rate.
Atomic operations are often handled at L2 because it’s the first cache level shared across the GPU. Adreno X2 provides decent throughput for INT32 atomic adds. Counting each add as two 4B accesses (read-modify-write) gives 632.64 GB/s of effective bandwidth.
Testing cross-thread latency using atomic_cmpxchg on global memory shows excellent performance from Adreno X2, and a big improvement over Adreno X1. “Core to core” latency on Adreno X2 is on par with AMD’s latest GPU architectures, and better than on Intel Meteor Lake’s iGPU.
The Snapdragon X2 Elite’s 8 MB system level cache (SLC) has rather high latency at just over 200 ns, or roughly 150 ns above L2 hit latency. SLC bandwidth appears to be around 380 GB/s, which is comfortably higher than DRAM bandwidth. AMD’s high end Radeon 8060S has both higher bandwidth and lower latency to its 32 MB SLC.
For DRAM, Qualcomm has a rather aggressive setup with a 192-bit LPDDR5X-9523 configuration. I could get just over 150 GB/s of DRAM bandwidth from the GPU. That’s better than what a typical 128-bit LPDDR5X setup can achieve, but sits under 70% of theoretical. A potential positive there is that GPU bandwidth demands can’t pressure CPU-side accesses to the same degree as other iGPU implementations, where the iGPU can gobble up a larger fraction of theoretical DRAM bandwidth.
Adreno X2’s DRAM latency situation is a bit strange. Latency plateaus at around 274 ns with moderate test sizes, but heads for the hills when test coverage exceeds 64 MB. Testing with a 128B stride doubles apparent cluster cache capacity, suggesting Adreno X2’s cluster cache has 64B lines. Apparent L2 capacity remains unchanged until stride length exceeds 4 KB, after which larger strides push out both the L2 and 64 MB inflection points. My interpretation is that Adreno X2 has a virtually addressed L2, and uses 4 KB pages for address translation with a 16K entry TLB placed after the L2. High latencies at large test sizes imply very high L2 TLB miss penalties. Raw DRAM access latency, excluding address translation penalties, is likely around that reasonable 274 ns figure. However, applications making accesses with poor locality over a large memory footprint may experience much higher latency.
Qualcomm has a long history of embracing tiled rendering, and Adreno X2 doubles down on that with a massive 21 MB block of on-chip storage called “Adreno High Performance Memory”. This HPM fills the same function as GMEM (Graphics Memory) in prior Adreno GPUs, and holds render targets to contain intermediate tile state while a tile is being rasterized. For scale, Adreno X1 had 3 MB of GMEM, while the Snapdragon 8+ Gen 1’s Adreno 730 had 2 MB of GMEM. Qualcomm says they sized HPM to handle QHD+ frames, and the math checks out. A 1440P frame with 32 bits per pixel (RGBA, 8 bits per channel) would occupy 14.7 MB. ASUS equipped the Zenbook A16 with a 2880x1800 OLED screen. Doing the same math would give 20.7 MB per frame, which should barely fit within HPM.
Rather than doing tiled rendering in the traditional sense with perhaps a dozen or so tiles per frame, Qualcomm seems to be throwing a curveball by trying to make the whole screen one tile. Tiled rendering comes with potential inefficiencies. For example, a triangle that overlaps two tiles will have to go through the rasterizer twice, creating extra rasterizer work. As one tile finishes, wave slots might start freeing up across the shader array. The GPU might have extra pixel work available, but has to hold it back because the whole point of tiled rendering is to serialize the rasterization process at the tile level. Those potential inefficiencies suddenly go away if the entire screen is one tile. At the same time, Qualcomm would still get the advantage of being able to contain intermediate tile state in on-chip memory. A large cache could do this naturally of course, but a cache requires extra power and area for tag and state arrays.
HPM is an excellent move if applications stick to the conventional rasterization API, but I’m worried that HPM’s applicability might be limited for general purpose compute. While rasterization remains the bedrock of PC gaming and likely will for the foreseeable future, the modern rasterization process sometimes includes a compute component. Newer rendering engines often have features that leverage compute, like Unreal Engine 5’s Nanite. HPM isn’t useless in these scenarios, because Qualcomm can allocate some HPM for use as a software managed scratchpad. Then, HPM can take on the same role as AMD’s Local Data Share or Nvidia’s Shared Memory.
I tried to see how many workgroups the GPU could run simultaneously with different amounts of OpenCL local memory allocated. From those experiments, Adreno X2 only appears capable of allocating 1 MB of HPM as local memory. Local memory allocations that aren’t a power of two bring that down further. 1 MB of local memory isn’t bad for a GPU of Adreno X2’s size. Adreno X1 could only allocate 384 KB of local memory across the GPU. AMD’s Radeon 8060S has 2.5 MB of Local Data Share (LDS) across the GPU, which makes sense considering its larger size. Still, having 21 MB of storage and only being able to use a small fraction of it for compute doesn’t feel great.
Local memory latency improves over Adreno X1, which is impressive because HPM is seven times larger than Adreno X1’s GMEM. However, AMD, Intel, and Nvidia’s latest GPUs still provide better latency to local memory. It’s easier to do that when accesses don’t have to traverse a cross-GPU interconnect.
Thread-to-thread latency through local memory also improves on Adreno X2. But again, AMD’s core-private LDS on their latest GPUs is faster.
Atomic adds on local memory usually enjoy higher throughput than ones through global memory, because most GPUs have dedicated atomic ALUs implemented at per-core local memory instances. That’s not the case for Adreno X2, which is able to sustain 32 INT32 atomic adds per cycle across the entire GPU, or 4 per cycle on a SP basis. For comparison, AMD’s RDNA3.5 can do 32 INT32 atomic adds per cycle at each WGP’s Local Data Share. Intel’s Xe-LPG in Meteor Lake also provides high throughput, with 16 INT32 atomic adds per Xe Core cycle.
Local memory bandwidth testing shows that HPM can deliver as much bandwidth as the GPU’s texture or cluster caches.
Software conventionally uses copy APIs to get data to and from the GPU. With OpenCL’s clEnqueueReadBuffer and clEnqueueWriteBuffer, copying data to the GPU is much faster than going the other way around. Prior Adreno GPUs acted like this too. Curiously, pinning the workload to different cores changes measured bandwidth, but only affects movement from CPU to GPU memory. Qualcomm might be using a CPU core to do the transfer, rather than offloading data movement to DMA engines like many other GPUs.
Newer APIs can enable zero-copy behavior. OpenCL’s Shared Virtual Memory (SVM) API is one example, and also maintains the same virtual addresses to let CPU and GPU code use the same pointers. Adreno X2 advertises atomics support with SVM, meaning that the CPU and GPU can theoretically exchange data while a GPU kernel is running. However, I wasn’t able to get the GPU and CPU to see each other’s writes with atomic operations.
I was able to test with fine grained sharing, which lets the CPU see the GPU’s writes after the GPU kernel finishes. Making writes visible between the CPU and GPU is fast enough to show that Qualcomm isn’t copying the entire 256 MB test buffer under the hood. However, kernel launch and synchronization overhead is higher than on other platforms especially if the test is run from an E-Core.
FluidX3D simulates fluid behavior using the lattice Boltzmann method (LBM), and has historically been a challenging workload for Adreno GPUs. FluidX3D uses FMA operations in its default configuration, resulting in poor performance on Adreno compared to just about anything else. FluidX3D can be modified to use a “legacy_fma” setting, which uses multiply-add operations instead of fused multiply-add. But even with that modification, Adreno X2 still struggles. It regresses compared to Adreno X1, and is worlds away from last generation iGPUs.
Memory capacity and memory bandwidth tend to be persistent constraints for FluidX3D. Therefore, FluidX3D can be built to use FP16S and FP16C modes, which use 16-bit floating point formats for storage. These 16-bit storage formats are converted back to FP32 for computation to minimize precision loss, effectively increasing compute pressure to reduce pressure on the memory subsystem. FP16S uses the standard FP16 format, letting format conversions take advantage of fast-path hardware on most GPUs. FP16C is a custom format and needs software format conversion, but provides better precision for the typical values that FluidX3D sees.
Neither of these modes help Adreno X2, indicating that compute is the limitation rather than bandwidth. FP16C puts more pressure on compute because of the extra instructions required for format conversion, and actually regresses compared to the full FP32 mode. I’ve included the GTX 1050 3 GB as a comparison because it’s the kind of older, lower-midrange discrete GPU that newer iGPUs should easily beat. Adreno X2 has more than twice as much compute throughput, memory bandwidth, and cache capacity than the GTX 1050 3 GB, so losing to it is disappointing.
Folding at Home runs protein folding simulations. The project has a rather dated benchmark called FAHBench, which I’m running under binary translation. I briefly tried to build an arm64 version, but it seems like the time investment to do so isn’t worth it. Adreno X2 turns in a passable result on the default dhfr (dihydrofolate reductase) workitem, though losing to the GTX 1050 3 GB is again not a good look.
FAHBench also includes a larger nav (voltage gated sodium channel) workload that tends to place more pressure on the memory subsystem, compared to the more compute bound dhfr workload. I had several GPUs fail the accuracy check in this workload. I’m showing their performance anyway because their calculation results only differ in the 5th or 6th significant digits (for the GTX 1050 and Adreno X2 respectively). They weren’t off by orders of magnitude, indicating that they were making a good faith effort at the work involved.
Nvidia’s GTX 1050 3 GB is 36% faster than the Adreno X2 with the nav workitem, while it was just 13.7% faster with dhfr. Adreno X2 seems to struggle with memory bound workloads. I wonder if its relatively low TLB coverage (for a GPU) combined with high TLB miss costs hold it back in less cache-friendly compute workloads.
3DMark’s Wild Life Extreme should be a familiar workload for Qualcomm, because it’s a Vulkan-based test that targets mobile devices. Adreno X2 performs very well in Wild Life Extreme, pulling comfortably ahead of Meteor Lake’s iGPU and taking up a reasonable position relative to AMD’s largest iGPU. Fire Strike Extreme is a heavy DirectX11 workload, which I had to run under binary translation. I’m only presenting the graphics score for that test. Fire Strike Extreme is a tougher challenge for Adreno X2, but it still manages a comfortable lead over Intel’s older iGPU.
Adreno X2 also does well in Cyberpunk 2077’s built-in benchmark, where it provides twice the performance of Intel’s Meteor Lake iGPU and Nvidia’s old GTX 1050 3 GB.
Cyberpunk 2077 performance is vastly better on Adreno X2 compared to the prior Adreno X1, which achieved 24.3 FPS at 1080P with the low preset. Adreno X2 easily beats this at the medium preset, which I’m using because Cyberpunk 2077’s CPU-side load isn’t trivial like in 3DMark’s graphics test. High framerates could bring binary translation overhead into the picture, and I wanted to avoid that. Overall Qualcomm seems to be doing well in rasterization, in contrast to the poor performance in GPGPU workloads.
3DMark’s Solar Bay test showcases raytraced reflections in a simple scene. It’s a lighter workload meant to fit within the capabilities of typical cell phones while still using raytraced effects. Adreno X2 turns in an exceptional performance here. It outperforms Meteor Lake’s iGPU and AMD’s Radeon 780M by more than a factor of two, and isn’t too far off AMD’s Radeon 8060S. It’s also a huge improvement over Adreno X1.
Adreno X2 brings DirectX Raytracing (DXR) 1.1 to Qualcomm’s GPU line, letting a wide range of PC games tap into Adreno’s hardware raytracing accelerators. 3DMark has a more demanding Port Royal test that uses DXR, which targeted high end gaming PCs from a few generations ago. Compared to Solar Bay, Port Royal makes heavier use of compute shaders and places less emphasis on the conventional rasterization pipeline.
Curiously, Adreno X2 achieves the same score as Meteor Lake’s iGPU in Port Royal. Qualcomm also loses ground relative to Strix Halo’s massive iGPU, and only achieves 50.8% of the Radeon 8060S’s score compared to 74.5% in Solar Bay. It’s also fun to note that a score of 1678 corresponds to 7.77 average FPS, which isn’t ideal for most games.
Cyberpunk 2077 is something of a raytracing showcase title. I’m testing that by taking the medium preset above, enabling raytraced reflections, and setting the raytraced lighting slider to medium. Bringing those raytraced effects into the picture destroys performance on all three iGPUs. None of them deliver a comfortably playable framerate, but there is a concerning shift in placement. Adreno X2 no longer ties or beats Meteor Lake’s iGPU, and now loses by a large margin.
There’s certainly potential in Qualcomm’s raytracing implementation, as 3DMark’s tests show. Qualcomm’s slides indicate that each uSPTP has a raytracing unit (RTU) with vaguely similar capabilities to Intel’s RTAs. Both implementations accelerate BVH traversal and intersection testing, and both embed a L0 cache into the raytracing unit.
I wonder what holds Qualcomm back in Cyberpunk 2077 with raytracing enabled. I doubt it’s a CPU-side bottleneck, because the GPU stays at or near 100% throughout the benchmark run. Maybe it’s because Cyberpunk 2077 has a huge BVH that covers the entire city. When profiling with AMD’s tools on RDNA2, Cyberpunk 2077’s BVH had over 59K nodes, compared with 11.6K nodes in 3DMark Port Royal. Or, maybe Cyberpunk 2077’s raytraced effects put a higher burden on the compute pipelines. I haven’t done the profiling necessary to figure out why Adreno X2 behaves the way it does.
Adreno X2 is a step forward for Qualcomm’s GPU line in many respects. General performance benefits from higher clocks, more SPs, and a beefier memory subsystem. Specific classes of workloads can stand to benefit more thanks to Adreno X2’s huge HPM block and improved raytracing units. Adreno X2 subjectively feels good enough to not hold Qualcomm’s laptop push back. I tried a few games that I’ve been playing recently, and performance felt appropriate for a mainstream iGPU.
However, Qualcomm’s latest iGPU still contains a lot of glass jaw performance cases for a modern GPU. Adreno X2 inherits Adreno X1’s tendency to fall over in GPGPU compute. Architectural improvements are certainly evident in compute microbenchmarks, but those observations don’t translate into good performance in the compute workloads I tried. It feels like Qualcomm took a specific set of graphics workloads and optimized for them, while AMD, Intel, and Nvidia set out to build more general purpose designs with robust performance characteristics across a broader range of use cases.
For now, I think Qualcomm has bigger fish to fry. Binary translation overhead was the biggest issue I faced when trying to game on the device, because it’s easy to get bound by CPU performance when running x86-64 binaries. When playing back an Age of Empires II game through CaptureAge for example, the Snapdragon X2 Elite Extreme couldn’t provide enough CPU performance to maintain realtime playback, while Strix Halo had enough spare CPU power to fast forward. But in the years to come, Qualcomm’s GPU will have a tough matchup on their hands. Today’s iGPU landscape is characterized by brutal competition. AMD’s Radeon 8060S and Intel’s Arc B390 can approach the performance of lower-midrange mobile discrete GPUs in both graphics and compute. The thin and light laptop trend will likely continue, and drive AMD and Intel to create ever more ambitious designs. I look forward to seeing what Qualcomm has in store for their future designs, and hope they’ll challenge the best that AMD and Intel have to offer.
2026-09-10 17:46:18
The PC market is one of the toughest areas for a CPU designer to compete in. Consumers in the PC segment expect high performance across a wide range of applications, and famously expect their devices to reach beyond a constrained, curated set of use cases. Microsoft’s own Surface RT prominently failed more than a decade ago because it could not run the programs that PC users expect to just work on Windows. Recent 64-bit Arm cores from Qualcomm and others are fast enough to satisfy the performance component of the PC equation, but software compatibility presents a tougher roadblock. PC software is traditionally built for x86-64. Getting developers to offer 64-bit Arm (aarch64) versions of their programs is a slow and gradual process. Some programs may never get aarch64 ports because they’re no longer under active development and were only distributed in binary form. Arm’s PC market chances therefore ride on Microsoft’s efforts to ensure x86-64 binaries can run seamlessly on aarch64 hosts.
Windows 11’s newest binary translator, dubbed Prism, enables this by translating x86-64 instructions to aarch64 ones. Binary translation is challenging because x86-64 instructions sometimes don’t map to aarch64 ones in a straightforward manner. Translation also has to be fast to minimize program launch delays and avoid consuming excessive CPU time for doing the translation. All of this means binary translation comes with a performance penalty compared to running equivalent native code.
Here, I’m checking out what that penalty looks like using Geekbench 7. Geekbench 7 comes in both aarch64 and x86-64 versions, providing an opportunity to compare performance with binary translation against native aarch64 performance. John Poole (Founder of Primate Labs, Creator of Geekbench) has kindly provided a pro key, which lets me profile individual workloads. Of course, one benchmark suite can’t reflect behavior across a wide range of applications, but I consider this a good starting point. As for hardware, I’m testing with the Snapdragon X2 Elite Extreme X2E-96-100 in an Asus Zenbook A16 laptop that was sampled by Asus as well as Arm instances available on Microsoft Azure.
Like Geekbench 6, Geekbench 7 has a number of workloads that leverage vector extensions when available. The suite is slanted towards high IPC, compute-bound workloads. Running workloads through Intel’s Software Development Emulator (SDE) set to expose Haswell’s feature set shows more than half the workloads using AVX and 256-bit vector width. I’m running each workload with 200 iterations to ensure the vast majority of counted instructions come from the workload rather than Geekbench’s test harness.
With 200 iterations, each workload executes roughly a few trillion instructions. Performance counter data from various aarch64 cores show roughly similar instruction counts when executing Geekbench 7’s native aarch64 binary, with occasional exceptions. Qualcomm’s cores curiously report higher retired instruction counts than Neoverse N1 and N2 in many tests, which suggests there may be some inaccuracy in hardware performance monitoring.
Executing Geekbench 7’s x86-64 version under binary translation results in dramatically inflated instruction counts across the board, except in PDF Viewer. A Geekbench 7 workload will execute roughly twice as many aarch64 instructions as x86-64 ones when going through binary translation, as a rule of thumb.
Microsoft hints that binary translation doesn’t work the same way across all aarch64 CPUs. Different CPUs support different instruction set extensions, some of which may let Prism more closely map x86-64 instructions to aarch64 ones. An aarch64 CPU can also hypothetically provide guarantees beyond what the ISA requires, like maintaining store ordering even though aarch64 lets one core observe stores from another out of program order.
Prism is optimized and tuned specifically for Qualcomm Snapdragon processors. Some performance features within Prism require hardware features only available in the Snapdragon X series, but Prism is available for all supported Windows 11 on Arm devices with Windows 11 24H2.
If Prism is optimized specifically for Qualcomm’s cores, it doesn’t make a difference from the instruction count side. Ampere Altra’s Neoverse N1 is the oldest core I’ve tested on, and performance counters show very similar executed instruction counts.
Similar instruction counts can of course hide large differences. Performance is ultimately what matters in the end, and that’s affected both by the core architecture and the nature of the generated instructions.
Geekbench 7’s score shows harsh penalties from binary translation. Every core, including Qualcomm’s, suffers several generations worth of performance loss. However, the cores in the Snapdragon X2 Elite have much higher baseline performance than Neoverse N1 and N2. The Snapdragon X2 Elite has a 6-core cluster of “performance” cores, and two 6-core cluster of “prime” cores. “Performance” cores are 6-wide and run at 3.6 GHz. Since they fill the same role as efficiency cores in other chips, I’ll call them E-Cores for simplicity. “Prime” cores are 9-wide and run at 5 GHz. They aim for maximum performance, so I’ll call them P-Cores.
Qualcomm’s E-Cores manage to outperform Neoverse N1 even when taking a binary translation penalty. The same applies to the P-Cores against Neoverse N2. There’s really no arguing with much wider cores running at very high clock speeds, even with binary translation penalties in play.
Qualcomm’s cores also take a slightly lower penalty from binary translation. It’s hard to tell whether this is down to Qualcomm-specific optimizations in Prism, or whether it’s down to Qualcomm’s cores having much higher throughput. Instruction count increases from binary translation remind me of playing with Claude’s C Compiler (CCC). CCC’s case involved creating a lot of extra instructions off the critical path, which an out-of-order CPU can often absorb. Recent high performance cores usually leave most of their core width unused, so extra core width can mitigate higher instruction counts to some extent. But I’m not sure that’s the case here, because Qualcomm’s 4-wide E-Core comes off with a lighter penalty than 5-wide Neoverse N2.
Individual workloads suffer to varying degrees when run under binary translation. Navigation is a low IPC workload that’s heavily bound by branch prediction and backend memory latency, and takes a comparatively low penalty when run under binary translation. At the other end of the spectrum, Video Player is a well vectorized workload that takes advantage of AVX instructions. Score differences in that test are absolutely massive. Photo Editor and Photo Library are in a similar situation.
Workload characteristics under binary translation stay mostly the same on Arm’s Neoverse N1 and N2.
Performance counters show Qualcomm’s cores putting a dent in binary translation overhead by dipping into unused core width. However, the IPC increase is minor compared to the instruction count overhead, which explains the large binary translation penalty. Snapdragon X2 Elite Extreme’s P-Core manages a huge IPC increase in the Video Player workload, but gains elsewhere tend to be muted.
Qualcomm’s E-Core is in a similar situation, though as a narrower core it’s able to make much better use of its core width. Text Processing, HDR, and Video Player basically have the core running up against its 4-wide width limitation when running under binary translation. However, the meager IPC increases seen on Qualcomm’s P-Core in Text Processing and HDR suggest that widening the E-Core might do little, as the workload might soon run into other limitations.
Neoverse N2 starts off at lower IPC and curiously fails to gain much IPC when running x86-64 under binary translation. The core performs fine in an absolute sense when looking at IPC for native workloads, but it doesn’t look great next to Qualcomm’s latest efficiency optimized core. No workload on Neoverse N2 can average above 3 IPC, with or without binary translation in play.
Neoverse N1 is the oldest core in this set, and unsurprisingly has the hardest time. N1 in Ampere Altra spearheaded Arm’s push to get a foothold in the server market, and succeeded because the core could deliver adequate performance while being more density optimized than its x86-64 contemporaries. But that was years ago. Qualcomm’s E-Core is also 4-wide, and leaves it in the dust.
Comparatively good performance from Qualcomm’s E-Core serves as a reminder that core width means relatively little compared to other architecture characteristics. Qualcomm gives their modern E-Core a much larger out-of-order engine than Neoverse N1. Doing so lets the core keep more instructions in flight, reducing the impact of short duration delays on individual instructions.
Accounting for dispatch-stage pipeline throughput shows a largely similar picture for workloads running natively and under binary translation, which isn’t a surprise because they should be doing the same high level work. On Qualcomm’s P-Core, the biggest difference is more utilized pipeline slots, along with some movement towards being less frontend bound.
Qualcomm’s narrower E-Core uses a huge portion of its core width in many tests, especially the x86-64 versions running under binary translation. A narrower core is much easier to feed, and pipeline slots lost to frontend reasons are far fewer compared to on the wider P-Core.
Detailed performance monitoring is harder on Azure because the platform doesn’t pass through top-down counters that account at the pipeline slot granularity. Roughly sketching things out with cycle-level accounting (STALL_FRONTEND and STALL_BACKEND) however shows that Neoverse N2 loses more pipeline slots to both backend and frontend reasons than Qualcomm’s E-Core. On one hand, feeding a 5-wide pipeline is harder than a 4-wide one. Also, Qualcomm has larger core-private caches and a sizeable 12 MB L2 for its E-Core cluster with 21 cycle latency. Missing the 1 MB L2 on Neoverse N2 exposes the core to ~100 cycle L3 latency, which will be difficult for any core to cope with. Binary translation doesn’t change much on that front.
Backend stalls are even more of a problem on Neoverse N1, which also has to contend with >100 cycle L2 miss latency. N1 is more severely affected because of its smaller out-of-order engine.
A comparison with Qualcomm’s E-Core shows just how rough the situation is for Neoverse N1. Qualcomm’s core is designed to keep more than twice as many instructions in flight, even though it has the same core width and similar execution resources. Extra instructions from binary translation increase pressure on out-of-order resources, and Qualcomm’s beefier core has a better chance of absorbing that overhead.
Prism stores translated code in C:\Windows\XtaCache. Writing generated code to disk serves two functions. First, it lets Prism avoid the overhead of re-translating code every time a binary gets launched. Second, it lets Windows users examine translated code to understand how x86-64 binaries are able to run on aarch64 hosts. This is important because it lets them empathize with the hardware. XtaCache’s efficacy on the latter point has been limited because Microsoft forgot to document their .jc cache format, but thankfully others have stepped in to do so. Unfortunately, digging through translated code is still a very time consuming exercise. So here, I’ll stick to giving a couple of examples to give an idea of what Prism’s translation looks like.
Geekbench 7’s Video Player workload stands out because it runs at much higher IPC when under binary translation, and also suffers from a huge executed instruction count increase. Profiling Video Player on Skylake with VTune shows that the hottest basic block is a 17 instruction loop. This loop traverses an array and performs fused multiply-add operations using AVX FMA3 instructions.
Prism translates this to a 69 instruction sequence on both Neoverse N1 and Snapdragon X2 Elite. AVX instructions get mapped onto NEON ones, which only have 128-bit vector width. Qualcomm’s latest cores support SVE, but using it likely wouldn’t make much difference because Qualcomm’s cores implement SVE with 128-bit vector length. Also, fixed width x86-64 instructions wouldn’t map well to SVE’s variable vector length anyway.
Instruction overhead comes from performing 256-bit vector operations using multiple NEON instructions, replicating x86-64’s more flexible addressing modes, and having to perform memory accesses with separate instructions rather than specifying both a load and a math operation with a single instruction. For example, vbroadcastss ymm8, [r8+r10*4] loads a single precision scalar (32-bit) value and broadcasts it across all vector lanes. Prism uses three aarch64 instructions. One loads the value, another replicates it across a 128-bit NEON register’s lanes, and a final one copies the value to another NEON register that represents the upper 128 bits of ymm8.
Fused multiply-add instructions get even more expansion. vfmadd231ps ymm7, ymm8, [rax + r9*4 - 0x13c] uses x86-64’s most complex addressing mode, with a base, scaled index, and immediate offset. Prism does the addressing calculation with two instructions, and is able to mash both the index scaling and addition to the array base address into a single add-with-scale. Then, a load-pair (LDP) instruction specifies a 256-bit load into a pair of 128-bit NEON registers. Prism carries out the FMA operation with a pair of FMLA instructions, but curiously spills NEON register holding the high half of ymm7 to the stack. It does this again for all eight of the loop’s FMA instructions, which doubles cache bandwidth demands. None of the spilled registers are modified before the next loop iteration, so spilling and reloading them is completely unnecessary, and potentially harmful in a loop that already has high bandwidth demands. Toward the end of the loop body, Prism faithfully translates software prefetch instructions into aarch64 equivalents, including extra ALU instructions to generate addresses. Then, loop counter increment and control flow instructions translate directly into aarch64 equivalents, though of course with a different jump target address.
A human translator would avoid Prism’s unnecessary register spills. They could also omit prefetch instructions, because linearly traversing an array creates a predictable access pattern that a hardware prefetcher would likely pick up on. They could also eliminate a few instructions here and there, like by using a single register to hold both halves of ymm8 because the broadcast load’s destination is never modified. Prism does deserve credit though for simplifying address generation for six out of the eight FMA operations by emitting just a subtraction, recognizing that rax + r9*4 has already been generated.
I also took a peek at Geekbench 7’s HDR workload because its binary translated version barely gains any IPC. The hottest block from profiling on Skylake is a 36 instruction loop with scalar AVX operations. That should make it easier to translate because Prism won’t have to map AVX’s wider vector width onto NEON.
Prism translates this into somewhere over 60 instructions. The complete translation didn’t seem to be available in the translation cache when I grabbed it from my Neoverse N1 VM.
Prism introduces significant instruction overhead by trying to maintain a register mapping where x86 SSE registers map to the equivalently numbered aarch64 NEON register. For example, within the instruction sequence Geekbench 7 does a FP add (vaddss xmm5, xxm4, [rdi+rbx*4]) and stores the result to memory (vmovss [rdi+rbx*4], xmm5). Prism’s aarch64 translation puts the FP add result in s18, and then moves it to v5 before storing it in order to use v5 to represent xmm5. Additional overhead comes from inexplicable stores to the stack, as well as extra instructions that seemingly generates the carry flag for a right shift instruction. Many x86-64 instructions set flags, and not all have flag-setting equivalents in aarch64. Prism seems to prefer preserving the flag generating behavior rather than expend compute power seeing if the flags are used.
Prism generates mostly the same sequence for Qualcomm’s Snapdragon X2 Elite, but with differences around loads and stores. Qualcomm’s newer cores support FEAT_LPCPC2, which lets it use ldapurb (load-acquire with an immediate offset) instead of ldaprb (load-acquire). Being able to take an immediate offset simplifies address generation, cutting out some extra instructions. In both cases, Prism translates loads to load-acquire instructions rather than aarch64’s basic LDR to ensure memory ordering.
Binary translation is key to making Arm on Windows viable in the PC market. Prism’s translations are mostly straightforward. x86-64 instructions get their behavior faithfully replicated by equivalent aarch64 sequences, so much that it’s possible to look through translated code and easily see which sequences map to x86-64 instructions. I suspect Prism does this to avoid consuming excessive compute power, because what it’s doing looks achievable with a linear pass through each block with minimal state kept in memory. However, doing the translation in such a straightforward manner misses potential optimizations that a human could carry out. Replicating x86-64 addressing in aarch64 often requires extra instructions. x86-64 compilers also take advantage of instructions that combine a load and math operation. Those have to be broken into two instructions in aarch64, and may require using an extra temporary register to hold the loaded value.
On the hardware side, high performance cores can often absorb dead code with unused core width. That sometimes happens when Prism generates dead code, but Prism doesn’t generate excessive amounts of dead code the way Claude’s C Compiler does. Instruction overhead from replicating address generation or copying values between registers to maintain Prism’s register mapping can easily lengthen critical dependency chains. There’s not much an out-of-order engine can do about that.
The ultimate answer to binary translation’s consequences is to have a high performance architecture with few weaknesses in the first place. Qualcomm seems to be going for that with their latest Snapdragon X2 Elite. A 9-wide core running at 5 GHz is nothing to sneeze at. Large instruction caches (128 KB on the E-Core, 192 KB on the P-Core) and ample out-of-order resources are good for performance in general, and crucially help mitigate the instruction overhead from binary translation. I’m still trying different use cases on the Asus Zenbook A16 UX3607OA laptop, but it feels surprisingly normal to use outside of a few corner case scenarios like gaming. I assume the Windows on Arm situation will only get better over time as more applications get aarch64 ports, and higher performance cores mean performance gets more acceptable even when applications have to run through binary translation.
2026-09-08 10:00:00
Editor’s Note (9/8/2026): Arm has reached out to clarify that the maximum IPC increase of the C2-Ultra core over the C1-Ultra core using the parameters shown in the endnotes is 7% along with the memory bandwidth improvement not factoring into the IPC improvement. The article has been edited accordingly.
Hello you fine Internet folks,
While Arm’s last announcement was about their new datacenter focused CPU, the Arm AGI CPU. Their latest set of announcements bring a much wider focus. The incoming C2-Ultra CPU and G2-Ultra NX GPU IP will see broad use across flagship phones, with latter G2 Ultra already shipping in Xiaomi’s XRING O3 chip that launched on August 24th. While the new Neoverse CSS N4 IP will see usage in the datacenter space.
Hope y’all enjoy!
Starting off with a comparison to the Cortex X925, the last ARM core we have good data on, with the limited information provided to us by Arm on C2 Ultra. We see limited changes when comparing to the two generation old Cortex X925.
Across both cores you have the same overall layout featuring 10 wide decode, 8 simple ALUs, 6 lanes of FP, 3 branch ports, and same 4 load/2 store config. So for C2 Ultra to get it’s performance boost, we’re looking at more iterative changes targeting a few critical structures like the branch predictor and OoO buffers.
Arm was unfortunately very vague with how it accomplished these branch predictor improvements. We don’t know if they modified BTB sizes, return stacks, or the branch algorithm as a whole.
Likewise they mention C2-Ultra has a larger execution window compared to C1-Ultra along with improved speculation, but go into no explicit details here.
All this means that C2-Ultra spends less time waiting for data compared to C1-Ultra.
Now, looking at the claimed uplift from C1-Ultra to C2-Ultra we see that Arm is claiming a 15% peak performance uplift with an average 12% uplift in traditional benchmarks and workloads.
However, the endnotes provide some important context for these claims.
Firstly, these numbers are from FPGA simulations so actual hardware may see different uplifts. Secondly, the C2-Ultra platform’s results are estimated with an 8.5% higher clock compared to the C1-Ultra along with a larger 3 MB L2 cache on the C2-Ultra platform that C1-Ultra does support and does ship with in the 3 MB L2 configuration in Xiaomi’s XRING O3 and Samsung’s Exynos 2600. According to Arm, while the memory system is providing nearly twice the bandwidth for the CPU benchmarks, the improvement to the memory subsystem is not factoring into the performance uplift.
When factoring the increased clock speed, the average uplift of C2-Ultra over C1-Ultra is 3.2% and this increase does include the extra 1 MB of L2 that the C2-Ultra has which may be adding a boost in some workloads.
Moving to the peak increase of 7% IPC over C1-Ultra that Arm reached out with, I assume that this number also does factor in the L2 cache increase as well. Geekbench 6 does appear to benefit from a larger L2 cache and a 3MB L2 seems to catch most L1 misses where a smaller L2 cache may not.
From my perspective, it does appear as if in the tested workloads, C2-Ultra hasn’t improved the performance per clock much compared to C1-Ultra on average with some workloads benefiting from a larger L2 cache that some designers may opt for.
Arm says that C2-Ultra uses 38% less power compared to C1-Ultra. However, this number does factor in node and implementation improvements so how much of this 38% decrease comes from the microarchitectural improvements is up in the air.
Arm also announced C2-Nano and C2-Pro as well, however these use the same underlying microarchitecture as the C1-Nano and C1-Pro cores.
Moving to the GPU IP, Arm says that Mali G2-Ultra NX is “The largest re-architecting of the GPU IP in 7 generations."
The amount of math that a G2-Ultra NX shader core can do hasn’t changed. It still has 128 FMA units which equates to 256 FP32 FLOPs per clock or 512 FP16 FLOPs per clock. The maximum number of shader cores allowed in a G2-Ultra NX GPU, 24 G2-Ultra NX shader cores, also hasn’t changed.
G2-Ultra NX has increased the number of registers that a warp can access from 64 to 128 along with improving the granularity of register access so that G2-Ultra can now allocate registers at a granularity of 16 registers. And along with these improvements, the register file has also increased by 25%.
Arm has also improved the RT units in G2-Ultra NX by making the triangle structure that the RT unit works on more compact which according to Arm reduces the DRAM traffic by 13% due to an elimination in redundant data which allows more data to fit into cache.
Arm has also added Opacity Micromaps to G2-Ultra NX which brings a desktop GPU feature down into the mobile sector.
Another desktop GPU feature that Arm is integrating into G2-Ultra NX is a matrix accelerator into the Shader Core. This matrix accelerator can do up to 1,024 INT8 MACs per clock or 512 INT16 MACs per clock and it can clock up to twice the clock that the Execution Engines can clock to. However, a glaring omission is that the matrix accelerator only supports INT8 and INT16 but not formats such as lower precision like FP8 or the larger yet very popular format BF16 which also isn’t supported on the standard ALUs either.
Arm also doesn’t require every Shader Core in a G2-Ultra NX GPU to have a matrix accelerator but does require a minimum of 6 “NX cores” for the Ultra branding.
Xiaomi decided for their G2-Ultra NX implementation on their brand new XRING O3 that only half of the Shader Cores would implement the matrix accelerator.
Comparing the structure sizes of the cores with and without the matrix accelerator, a G2-Ultra NX shader core with a NX unit ends up being about 1.88 mm^2 and without the matrix unit a G2-Ultra shader core is about 1.55 mm^2 which means that the matrix unit adds about 21% to the area of a shader core.
The added matrix units have allowed Arm to introduce their own version of ML-powered upscaling called Neural Super Sampling (NSS).
Arm is also introducing their Frame Rate upscaling technology with the G2 generation.
With all of changes to the GPU IP, Arm is claiming up to 14% improvement in games and up to 24% in Ray Tracing benchmarks compared to last generation.
However, the G2-Ultra NX GPU is clocking about 11% higher compared to the G1-Ultra GPU in these comparisons which imply that these changes may not improve non-RT games much.
On the server side, Arm has also announced that Neoverse N4 CSS will become available for their partners.
Sadly, this is the one slide that Arm had about Neoverse CSS N4. When asked, Arm did give out more information:
CPU: 8–128 Neoverse N4 cores per die, the widest core-count range offered in a Neoverse CSS, with frequencies up to 3.8 GHz.
Scaling: Supports multi-chiplet and multi-socket designs for scaling beyond a single die.
L1 cache: 64KB instruction and 64KB data cache per core.
L2 cache: Up to 2MB private L2 cache per core.
System-level cache: Up to 256MB shared cache per die.
Memory: Supports DDR5 or LPDDR6, providing flexibility across capacity, bandwidth, power and system design.
I/O: Up to 128 lanes of PCIe Gen 6/7 and CXL 4.0.
Chiplet connectivity: Arm chip-to-chip interconnect with support for UCIe or partner-specific PHYs.
There are still a number of questions about what core is N4 using, what is the width of the memory bus on a single die, what is the number of PCIe lanes on a single die, among others.
Something that I should mention is that Arm’s technical disclosure this time around is disappointing. To Arm’s credit, the company did answer questions when asked and that responsiveness is appreciated, however it is frustrating that those questions were necessary to fill gaps that should have been addressed in the presentation. If we had been given the quality of information comparable to prior Arm announcements, we would have spent longer on analyzing the microarchitecture in our more typical fashion.
C2-Ultra is a good example of this problem, with claims about improved branch prediction and speculation giving little information of what actually changed, while the performance figures combine those core changes with higher clocks, a larger L2 cache, and more memory bandwidth. Moving to the power figures which include process and implementation improvements, we are left with the question about how much is the new microarchitecture contributing to the power decrease. These numbers are particularly frustrating for an announcement about CPU IP where comparisons at equivalent clocks and memory configurations would have been useful.
G2-Ultra NX gives more details, particularly around register allocation, the addition of matrix accelerators, and the improvements to the ray tracing units, however describing it as “The largest GPU rearchitecting in seven generations” creates an expectation of technical depth that wasn’t delivered. The matrix accelerator’s limited precision support also raises questions about its usefulness beyond the workloads Arm has chosen to target and I would have liked Arm to spend time explaining in the presentation why. And CSS N4 takes the lack of detail to an extreme with one slide with basic information about the product having to be asked.
Arm has IP and products worth talking about, but the presentation does a poor job of communicating them. The technical substance should be in the presentation from the beginning rather than something we have to assemble through follow-up questions and emails.