Understanding the Intel Optane Shutdown

84
5
5 days
(techtarget.com)
by lichtenberger

Comments

Veliladon
2 days

Optane was an interesting solution that was still looking for a question. One of the things that drove mass SSD adoption was the experience was plainly better. Even the most illiterate user could see how fucking fast an SSD machine booted and how much more responsive they were.

SSD to Optane? There's no practical difference. The high end desktop users and serious gamers that have serious cash for their rigs turned up their noses at Optane because it turns out SSD sequential read was good enough for most use cases. At least good enough to not pay 3x more per GB. Nobody really cared about insane sustained write speed when the SLC caches did almost as well in most use cases. Plus as the interfaces scaled up, plain old NAND drives were ready to flood the bus with as much sequential read bandwidth that the bus would take.

What could have saved it? I dunno. If Intel wanted to be anti-competitive they could have slapped on 8 PCIe lanes direct to the CPU exclusively for Optane on their desktop products. As we've found out, 4 lanes of NVMe to the CPU and the rest having to go through the chipset is some sort of unfunny, market segmentation joke. It could have forced high end users to grudgingly accept Optane as the path to the absolute best I/O and an actual, tangible difference in performance.

paulmd
2 days

> One of the things that drove mass SSD adoption was the experience was plainly better. Even the most illiterate user could see how fucking fast an SSD machine booted and how much more responsive they were.

> SSD to Optane? There's no practical difference.

No, that's false. That's true of NVMe in general but Optane specifically is really good at the low-queue-depth random-4k workloads that characterize consumer use-cases.

Optane actually is extremely fast at things like OS patches that normally take a few minutes even on NVMe flash let alone sata. They boot faster, applications launch faster, etc. Normally those things don't have lots of parallel threads running so queue depth is low and flash doesn't perform very well (as in, SATA and NVMe are nearly indistinguishable), but Optane really helps.

Is that worth paying 10x as much for? No, probably not, but that's the real problem, not the general lack of performance. Optane is actually noticeable specifically for being the first thing that actually provides a noticeable improvement above and beyond SATA SSDs for consumer use-cases. It's just also too expensive to be justifiable.

PaulHoule
2 days

Although "client computing" is still the largest source of Intel revenue, and its scale was what made the x86 architecture able to crush legacy server architectures, I think Intel takes it for granted relative to the data center.

It would have been very interesting to see an "all Optane" tablet device but it would have been hard to make work: a modest amount of storage would be affordable but it would take a clean sheet OS to maximize performance and economics.

I was skeptical about Optane DIMMs.

Optane SSDs were the fastest SSD you'd ever seen, but the DIMMS were the slowest you'd seen in a while. Although they were faster than Optane SSDs, they were slower than RAM if you replaced RAM 1-1. Optane was denser and cheaper than ordinary RAM so you could pack your machine with a huge quantity of slow RAM.

If access patterns and cache behavior permit, you could build systems of a certain problem size that perform well with Optane. Advanced programming techniques can help.

For every big problem there are many little problems (don't need Optane) and some problems are too big for Optane. Many problems in the size range for Optane can be implemented to conserve RAM and have access patterns that stream well to mass storage. Thus Optane faced fierce competition.

paulmd
2 days

In practical terms the only real use for Optane DIMMs was very high capacity RAM systems. Optane let you get 512GB in a stick when LRDIMM was 128GB. And of course you can still go multisocket etc too. So you run the same software, but in large-VRAM scenarios that couldn't be handled optimally by LRDIMMs without paging. Things like SAP HANA and so on.

But a system that only makes sense in super high end configurations is setting an uphill battle for itself. You can't even put together a demo machine without using one of a handful of processors at the top of Intel's stack, how do you even demo that to your boss as a concept? And in the end the commodity solutions just ended up outpacing it at lower cost, flash still sucks at random but it's generally fine and people don't want to pay the extra $500 for optane on their 1tb drive.

Software getting entirely rewritten for NVRAM is a pipe dream lol. Maybe you could have defined a "high range" ala HIMEM that is persistent across boots and will never be allocated, just used it as a block device so your SAP database would never unload, but, yeah, there are non-trivial problems.

monocasa
2 days

> and its scale was what made the x86 architecture able to crush legacy server architectures, I think Intel takes it for granted relative to the data center

This is true; the reason is that the margins on the data center chips are what keeps the lights on.

Comment was deleted :(
luma
2 days

I think one of the major missing factors is lack of OS support for the sorts of features that NVDIMM could potentially bring to the table. Super-fast wake from deep hibernation is just a starting point, the real work is re-writing every line of code written since the dawn of computing which starts from the two assumptions "one kind of storage is persistent and slow" and "the other kind of storage is volatile and fast".

HPE made noise about challenging those assumptions with The Machine but predictably lost interest as their own memristor technology lost steam.

orlp
2 days

> What could have saved it?

Persistent memory was/is a big topic in the database research world. The idea was that you would not need to worry about ACID since all your data is still there in the event of a power loss/crash. It would massively simplify database code.

However as it turns out... persistent memory doesn't buy you all that much. You still effectively need a replay log to restore the state you were in before a crash event, and you still need all the restoration code. In the end it in most systems it just behaved as a very fast (and expensive) SSD. The benefits vs. price just wasn't there as a modern DB setup on a fast SSD + more RAM would get you much more performance per dollar.

What you'd really need is transactional memory, not just persistent memory. And you'd need a database engine written completely from the ground up to use this memory. Probably several hundred KLoC of tricky code (and all the joys that come with parallel programming), targeted at hardware that doesn't even exist yet. Sounds very expensive and risky to me.

I think eventually we'll see something in this line of technology become commercially available. But I'm not surprised that Intel pulled out.

gavinray
2 days

  > " And you'd need a database engine written completely from the ground up to use this memory."
You mean like the one from one of CMU's research projects, that Pavlo et al have written about?

https://db.cs.cmu.edu/projects/nvm/

  "How to Build a Non-Volatile Memory Database Management System," in Proceedings of the 2017 ACM International Conference on Management of Data, 2017

  "Spitfire: A Three-Tier Buffer Manager for Volatile and Non-Volatile Memory," in Proceedings of the 2021 International Conference on Management of Data, 2021, pp. 2195-2207.
xyzzy123
2 days

Optane was pretty great for fs and db logs / journals.

That was pretty much its killer (practical) use case.

mort96
2 days

What's more, Optane was sold as... a cache to have in front of your spinning rust. In an age where SSDs were cheap enough that you could just get a few TB of SSD and have all your data access sped up, not just what happens to be in cache.

Melatonic
2 days

It could have made a big difference but it was adoption that was the problem. Swapping an SSD from a HDD is very easy and straight forward for most people - SATA cables are also well designed and easy to understand. Its almost like a USB cable in an L shape.

Completely changing the format? That is a huge obstacle.

NVME SSD's also came along soon enough we got something that was much, much faster than SATA SSD's and eventually I think it just became clear it was diminishing returns. That being said I am still hoping for an Optane replacement.

christkv
2 days

They should have sold it as level 4 cache

skyde
2 days

you mean swap drive ?

cleansingfire
2 days

That's what I used the <$10 16gb Optane sticks for, when I had a batch of terrible small zero cache SSDs. Just to extend the performant life of the hardware I was stuck with. Worked ok.

neverartful
5 days

Surprised that there was not even a single mention of Micron! My understanding is that it was a joint venture of some variety between Intel and Micron.

I'm sad to see Optane being discontinued. I had very good performance results with it being incorporated into SDS cluster architectures.

fluoridation
2 days

It had good performance, but part of the problem is that Intel way oversold it at the beginning. They said they would be "1000x faster, 1000x more enduring, and 10x more dense than NAND". In the end none of these promises materialized. They released these puny little drives that had a bit better latency than NVMe SSDs, that were locked to Intel systems, that OSs didn't know what to do with them, and that didn't quite fit anywhere in the memory hierarchy.

Yeah, it found a few niche uses, but it was never going to be sustainable.

nolanhergert89
2 days

Don't confuse the Optane SSD products (worse binned media over a slower NVMe link) with the higher-performance (and more expensive) Optane Persistent Memory product. For the latter, the latency looks to be sub-microsecond (~100X? SSDs), bandwidth is ~3-5X SSDs, ~10X more endurance, and up to 10X more dense than DRAM (512GB sticks anyone?). Don't have a specific good source though, just from Googling.

I wasn't a marketing person, but I was on the engineering side.

Melatonic
2 days

That really sounds like they need better marketing people lol

Why would they name those two things the SAME? Call one Optane and one of them Budge-Tane or something

paulmd
2 days

Not OP but marketing was obviously a huge problem for Optane. Intel Marketing did an absolutely terrible job.

* You've got Optane Memory, the PDIMM, which only works with a limited set of Intel products

* You've got Optane the NVMe drive, which works with anything, and are available in sizes usable as a boot disk or for databases/ZIL/etc

* You've got Optane Memory, the 16gb/32gb cache drive used in shitty laptops to try and boost performance, but actually those are just really small NVMe SSDs and can be used as such if you want.

* You've got Optane the caching software, which only works with Intel motherboards/laptops, and basically does the same thing as PrimoCache/StoreMI.

etc. And Intel marketing just made zero effort to distinguish or clarify these products in any way... to this day you get people thinking you had to own an intel system to use optane drives.

wtallis
2 days

Probably the worst bit of Optane marketing was laptops with 8GB of RAM and 16GB of Optane SSD cache being advertised as having 24GB memory.

paulmd
1 day

Yep that's super deceptive. I guess if you pin it to swap it's not terrible with a latency that low (optane is ideal for a swap drive, super low latency) but it's not really the same as 24gb.

also, current-day: this "intel processor" being the new name for the low-end processor is fucking lmao, did raja's marketing buddies burn the whole place down already? oh my god imagine using your primary brand name as a sub-brand for low-quality products

like man intel marketing has never been all that on the ball, but wat

fluoridation
2 days

Like I said in the other comment, all I cared about was having many gigs of memory on desktop. I never got a whiff of those 512 GB sticks, although I sure would have loved to get my hands on them. If Intel segmented them for the datacenter then whoop-de-fucking-doo. All I ever saw were shitty little useless cache drives.

wtallis
2 days

> "1000x faster, 1000x more enduring, and 10x more dense than NAND"

Their claims were actually a mix of comparisons against NAND and DRAM; I don't think they ever claimed it would be denser than NAND, just denser than DRAM.

Also, the Optane SSDs were never locked to Intel systems, just their caching software for Windows and the Optane DIMMs.

fluoridation
2 days

>Their claims were actually a mix of comparisons against NAND and DRAM; I don't think they ever claimed it would be denser than NAND, just denser than DRAM.

You're right. I looked up the slide before commenting to look at the numbers and misread it. Actually, that reminds me that the first time I saw the slide years ago I hoped we were going to have Optane memory sticks and desktops with hundreds of gigabytes of main memory, even if with worse latency. I remember being so disappointed when Optane finally came out and reading the actual numbers. Yeah, the latency was okayish, but the throughput was barely better than NAND.

>Also, the Optane SSDs were never locked to Intel systems, just their caching software for Windows and the Optane DIMMs.

If anything, that just makes it worse. I was quite interested in Optane, and I can't remember which things were what and what they worked on. The product line was so confusing.

anonymousDan
2 days

I wonder if standardisation efforts around things like UCIe and CXL will solve the locked to Intel problem, potentially enabling a sustainable market for such devices in future?

Comment was deleted :(
PAPPPmAc
2 days

IIRC Intel bought Micron out of the 3DXpoint/Optane project in 2018, and (based a some hallway conversations at conferences) folks at Micron were not exactly enthusiastic about the early results from the technology or Intel's overhyping marketing years before then.

anonymousDan
2 days

Interesting - do you mean Optane PMem (i.e. where it is connected as a DIMM)? Or the Optane SSDs?

tpetry
2 days

Optane SSDs had been great! They have the fastest write latency which made them e.g. perfect for databases or the ZFS ZIL which is a write of any new block to a special space (on that optane ssd) that will later be rewritten to the slower ssd.

fiat_fandango
2 days

Low latency optane ssd's are incredible for so many reasons - really hope those don't become discontinued as well :(

Many here seem to miss how deeply complex SSD's are and how truly different optane is from most consumer and enterprise SSD offerings. Most consumer ssd's only optimize for sequential reads and writes and at that max throughput, not latency for random access. Glossing over a lot of detail, but this is where optane truly shines.

paulmd
1 day

> Many here seem to miss how deeply complex SSD's are and how truly different optane is from most consumer and enterprise SSD offerings. Most consumer ssd's only optimize for sequential reads and writes and at that max throughput, not latency for random access. Glossing over a lot of detail, but this is where optane truly shines.

Yep. Optane was much much closer to an "ideal" block device. There is no block write/page erase mechanism, no garbage collection, no interpolation layer between you and the hardware at all actually. Reads and writes onto a platonic block device. Latency is very low for xpoint so that's fine, on paper.

It also means, that for ZFS, it's basically fine even without the battery backup. The xpoint isn't lying to you about a committed write so there's no reason to have a caching layer to manage the physical flash pages, in the way that flash SSDs need backup power for their controllers to flush the cache.

HarHarVeryFunny
2 days

It was never just about Optane. The joint Micron/Intel 3DXPoint technology was meant to be a new type of non-volatile memory who's performance was closer to memory than disk, leading to all sorts of revolutionary changes in the way memory was used.

The trouble is the performance goals were never realized, so the revolutionary non-volatile main memory use case was not realized. Optane/SSD-usage seems to have been an attempt to at least salvage something from the investment given that it was fast enough for that, but I suppose it the end it can't have been cost competitive with NAND-based SSDs. Perhaps the economy of scale would have been different if the technology had met it's performance goals and been more widely used ?

gavinray
2 days

At least the PMDK (Persistent Memory Development Kit) code works interoperably with both Optane and CXL:

https://github.com/pmem/pmdk