$MU $SKHY $CXMT Marvell's simulation baseline was a server with 2 TB of CPU DR… · @carrioresearch · TwKit

@carrioresearch

$MU $SKHY $CXMT Marvell's simulation baseline was a server with 2 TB of CPU DRAM, using compute traces from an eight-H200 system running Llama-405B, per Converge Digest's report of Mahatme's Aug. 20 Hot Interconnects talk. The appliance setup had 32 TB. That's 16x the memory for roughly 6x the conversations, 300 versus 50. So the gain comes from adding DRAM, not saving it. One appliance holds 128 standard DDR5 DIMMs at up to 256 GB each, plus about 1 TB of HBM3e. If shared KV-cache tiers like this ship, they're new demand for both kinds of memory Micron ($MU) and SK hynix ($SKHY) sell, sitting outside the GPU package and outside the server.

曝光 436 · 评论 1 · 点赞 3 · 书签 0