GPU memory has a geography problem. High-bandwidth memory has to sit within millimeters of the GPU die, which caps how much any one accelerator can have. Agentic AI and long-context inference keep asking for more, and once the KV cache outgrows HBM, expensive GPUs sit around waiting on memory.
Penguin Solutions thinks the answer is light. At SC26 in Chicago the company unveiled the MemoryAI Photonic Appliance, which it bills as the first photonics-based, HBM-class memory appliance. The same day, Marvell and Penguin announced a broader collaboration on AI memory, and the two pieces fit together.
The appliance
The box sits meters away from the GPUs and connects over silicon photonics, using CXL 3.0 and Marvell photonic NICs. Penguin says optics draw less power than copper over those distances. The targeted specs:
- 57.6 Tbps of total appliance bandwidth
- Up to 16 TB of DDR5 RDIMM and 576 GB of HBM3E per appliance
- Three access modes: HBM Direct, HBM-DDR5 Cached and DDR5 Direct
- Memory pooled between CPUs and GPUs, with static or dynamic allocation handled by Penguin software
Sharing is transparent, so applications don’t need special access routines or data copies. The appliance adds to the HBM already on the GPU rather than replacing it. Penguin is also showing a MemoryAI Photonic Memory Module built on Marvell’s Photonic Fabric. Sampling to select customers is planned for the second half of 2027.
“Photonics is the step-function change that makes this possible,” said Andy Mills, VP of advanced product development at Penguin.
The Marvell side
The wider collaboration runs on two tracks. The near-term one is unglamorous and probably useful sooner. Marvell Structera X memory-expansion controllers go into Penguin’s SMART CXL add-in cards, which reuse high-density DDR4 modules pulled from decommissioned servers. They connect over PCIe 5.0 and CXL 2.0, and the same card can build CXL memory appliances that pool DDR4 across several hosts. Penguin expects these cards ready for qualification and enterprise deployment by the end of 2026.
The longer track brings Marvell Photonic Fabric into Penguin’s MemoryAI appliances and ComputeAI systems, for memory shared across multiple racks in pools reaching petabytes. Those servers are due to sample in the second half of 2027. The pitch is a single upgrade path, from direct-attached expansion to multi-rack sharing, without re-architecting anything.
Old DDR4 getting a second life next to HBM3E is neat economics, given how tight memory supply is right now. Kash Shaikh, Penguin’s president and CEO, said memory is becoming increasingly critical to the performance and economics of AI factories. Preet Virk, who runs Marvell’s Photonic Fabric business, put it in terms of tokens: inference has to deliver them more cheaply before it can scale.
The photonic products are 2027 hardware, so treat the numbers as targets. The direction is clear enough. Memory is turning into its own tier of the data center, with its own boxes and its own network.