Intel Optane Memory is SRT Cache...Again (Updated)

Update, 4/24/17, 09:25am PT: Intel announced that the Optane Memory M.2 2280 modules are now available. The company hasn't shared pricing information or from what retailers you'll be able to purchase the modules.

Original article: 3/27/17, 09:00am PT:

Intel's Smart Response Technology (SRT) was so successful that we haven't talked about it since it was first introduced in 2011. We say that with a heavy dose of sarcasm, because Intel plans to bring the technology back with a faster media. Cached I/O isn't anything new. For the last decade, Microsoft and Intel have given users several options, ranging from USB thumb drives to accelerate I/O to speedy purpose-built single-level cell (SLC) SSDs. With Optane, the technology might just catch on, but not the way Intel sees it.

Intel just announced two new products that bring Optane technology to the consumer desktop. Optane is loosely defined as the company's products built with 3D XPoint technology, a next generation non-volatile memory structure built from the ground up to reduce latency. The new Optane Memory products will ship in two capacities (16GB and 32GB) and give users access to a whole new performance tier--as long as you have the supporting technology in place, mainly a 200-series chipset.

Pricing for Optane Memory M.2 2280 modules start at just $44 (16GB) and peak at $75 (32GB). The operating system recognizes the new products as addressable storage, just like a regular hard disk drive or solid-state drive. Intel told us that support for the drives as cache starts with the latest 200-series chipset products that feature an additional four PCI Express lanes over the older 100-series chipset.

The magic happens when you enable a "modified" version of Smart Response Technology and build a cache array with the Optane Memory standing invisibly in front of an HDD or SSD. The Optane Memory becomes a cache device that accelerates I/O for data retained in its memory structure from previous I/O requests.

Despite spending time with Intel, we have a lot of unanswered questions about all of the Optane family of products. There are a few theories about the memory technology, but almost everyone agrees that 3D XPoint is some type of phase change memory. We would like to learn more about the controller and how it communicates with the memory. The controller is quite small; to date, the only SSD controllers we've seen of this size are two-channel DRAM-less models used in non-performance environments, like point of sale (POS) machines. We have a few theories about this but will hold our tongues until we've had a chance to dig deeper.

A Pyramid Scheme

Image 1 of 2

Intel re-purposed the memory latency/price graphic to show where Optane Memory fits in different system configurations. The graphic shows an Intel Optane SSD (more on that in a bit) for high-end systems like what an "eSports Gamer" would utilize. The product announced today, Optane Memory, fits in mid-tier systems as a cache device for traditional SSDs using NAND flash technology and in lower priced systems as a cache device paired with a traditional hard disk drive.

Although the media is new, the technology and the practice were introduced in the Intel ecosystem in 2011. Smart Response Technology is a feature built into the Rapid Storage Technology driver. Intel's first attempt at "SSD Cache" used a 20GB SSD (model 311 "Larson Creek") featuring 34nm single-level cell (SLC) technology. This was later refreshed in 2012 with a 25nm SLC SSD using model number 313 branding (Hawley Creek).

Both products worked remarkably well and spawned additional products sold as "cache SSDs" from third-party vendors. Intel took steps to retain an advantage over competing solutions. In 2014, TRIM was removed from the SRT feature list, which left the SSD to handle all collection activities without the guidance from the operating system. This left lower-priced, MLC-based products that had less advanced controllers with a severe disadvantage. Intel also limited the cache size to just 64GB. If your SSD was larger than 64GB, the additional capacity could be used as a separate addressable volume with a drive letter. The limits imposed kept users from building a large cache to place in front of the hard disk drive cold storage tier.

Third Time's The Charm

Smart Response Technology (SRT) was introduced during a transitional time for consumer SSDs. Crucial had just released the first SATA 6Gbps SSD (the C300), and SandForce proved that in-house firmware led to increased sales. Users had a taste of flash and started to consume a lot of it. That drove the cost of the technology down. Older SSDs appeared on the secondary market for pennies on the dollar, and that started a boom that is now a thriving industry. For a few dollars, users could build a hybrid array with a little flash in front of a spinning disk; or, for a few dollars more, you could boot to a full-speed SSD. The market spoke, cache technology lost its I and was just on the way Out.

In contrast, Optane Memory reshuffles the board and starts off with an advantage over traditional NAND-based SSDs. Intel and Micron own 3D XPoint technology, and we don't expect to see the two competing against each other. Intel has stated that 3D XPoint will not be for sale to third-party customers, so you won't see, for example, an Adata Optane product on store shelves to drive costs down. Optane-based products are the new SLC, but this time no one plans to share the secret sauce to outside companies. If you want a 5x to 8x increase in low-queue depth performance, only two companies are in the kitchen.

Optane Memory comes during another transitional period for flash. The market has shifted to lower-cost, high-capacity products that use internal cache technology to mimic performance found in previous generations. Two-bit per cell NAND is nearly nonexistent in consumer SSDs, barring a few standout SSD manufacturers that entertain Toshiba executives. MLC will disappear within a year for all but very high-cost SSDs designed for prosumers.

Toshiba will finish the factories for 3D BiCS technology. Its partner, SanDisk, has already publicly stated that TLC is the new normal for consumer SSDs. If you thought TLC was bad, just wait for quad-level cell (QLC) flash that will certainly require an appetizer of Optane Memory to be palatable.

Image 1 of 3

Intel has made several bold claims about Optane Memory technology. The new storage media reaches high speeds much faster than current NAND technology, which requires parallel reads and writes to achieve high performance. Optane Memory can surpass consumer SSD performance at high queue depths while operating at very low queue depths. This benefits users because most I/O comes at just queue depth 1 or 2. It's rare to reach queue depth 4 even while multitasking with several applications. 

Swing And A Miss: The Cost Of Failure

Cache technology brings a set of inherit risks. Stacking cache on top of cache amplifies the penalty of a cache miss. As an analogy: In baseball, you may go through several batters to get a hit. That takes time, and you could end up on the losing side of performance when an I/O request first strikes out the Optane cache, then the SLC layer cache, and finally has to reach out to the third layer. In this case, it's the TLC. The third batter gets a hit, but there are already two strikeouts on the scoreboard for the inning. Holding more data in the high-speed tier one cache helps to increase your chances of a hit. The more applications you use, the wider the strike zone becomes. Your core, daily use software is ready to swing for the fences, but the application you haven't run in six months has to be flown in from the Triple-A league. If your primary media is a hard disk drive, then you see a massive 20ms delay after swinging at fast balls for most of the day.

Cache Isn't King

Prosumers and working professionals that rely on performance consistency and use large data sets are not the core audience for Optane Memory. I'll be one of the first to take advantage of the new 3D XPoint technology in a RAID 0 array with three 32GB M.2 NVMe drives configured in a 96GB array to test what the technology is capable of.

Intel spoke softly about an Optane SSD, which will serve as a normal boot volume, and gave us permission to acknowledge its existence. So, we can confirm that it's going to be a Real Thing. Unfortunately, Intel didn't give us much more than that. We can speculate that the Optane SSD will have a relationship to the just-announced DC P4800X that's similar to the one between the DC P3700 (enterprise SSD) and the 750 Series (consumer NVMe SSD). Intel has reconfigured enterprise-class products in the past for those willing to spend the money for extreme performance. We expect an add-in card form factor but were also told that, "Optane becomes very interesting with U.2."

The message is just as clear as it is cryptic. The DC P4800X will see two larger capacity models around the same time Intel releases the technology in a U.2 form factor.