David Fair - SNIA on Data, Networking & Storage

New Webcast: Cloud File Services: SMB/CIFS and NFS…in the Cloud

September 18, 2014September 18, 2014 David Fair Leave a comment

Imagine evaporating your existing file server into the cloud with the same experience and level of control that you currently have on-premises. On October 1^st, ESF will host a live Webcast that introduces the concept of Cloud File Services and discusses the pros and cons you need to consider.

There are numerous companies and startups offering “sync & share” solutions across multiple devices and locations, but for the enterprise, caveats are everywhere. Register now for this Webcast to learn:

Key drivers for file storage
Split administration with sync & share solutions and on-premises file services
Applications over File Services on-premises (SMB 3, NFS 4.1)
Moving to the cloud: your storage OS in a hyperscalar or service provider
Accommodating existing File Services workloads with Cloud File Services
Accommodating cloud-hosted applications over Cloud File Services

This Webcast will be a vendor-neutral and informative discussion on this hot topic. And since it’s live, we encourage your to bring your questions for our experts. I hope you’ll register today and we’ll look forward to having you attend on October 1^st.

Relentless Advance Of Ethernet – And Ethernet Storage Networking

March 31, 2014March 31, 2014 David Fair Leave a comment

As one Cisco colleague once said to me, “After the nuclear holocaust, there will be two things left: cockroaches and Ethernet.” Not sure I like Ethernet’s unappealing company in that statement, but the truth it captures is that Ethernet, now entering its fifth decade (wow!), is ubiquitous and still continuing to advance at a breathtaking pace. And as it advances, it advances the capabilities of storage networking based on the Ethernet backbone, be it file storage like NFS or SMB or block storage like iSCSI or FCoE.

Most recent evidence of Ethernet’s continuing and relentless evolution is illustrated in the 28 March 2014 announcement from the Ethernet Alliance congratulating the IEEE on formation of their IEEE P802.3bsâ„¢ Task Force:

The new group is chartered with the development of the IEEE P802.3bs 400 Gigabit Ethernet (GbE) project, which will define Ethernet Media Access Control (MAC) parameters, physical layer specifications, and management parameters for the transfer of Ethernet format frames at 400 Gb/s. As the leading voice of the Ethernet ecosystem, the Ethernet Alliance is ideally positioned to support this latest move towards standardizing and advancing 400Gb/s technologies through efforts such as the launch of the Ethernet Alliance’s own 400 GbE Subcommittee.

Ethernet is in production today from multiple vendors at 40GbE and supports all storage protocols, including FCoE, at those speeds. Market forecasters expect the first 100GbE adapters to appear in 2015. Obviously, it is too early to forecast when 400GbE will arrive, but the train is assuredly in motion. And support for all the key storage protocols we see today on 10GbE and 40GbE will naturally extend to 100GbE and 400GbE. Jim O’Reilly makes similar points in his recent Information Week article, “Ethernet: The New Storage Area Network“ where he argues, “Ethernet wins on schedule, cost, and performance.”

Beyond raw transport speed, the rich Ethernet infrastructure offers techniques to catapult your performance even beyond the fastest single-pipe speed. The Ethernet world has established techniques for what is alternately referred to as link aggregation, channel bonding, or teaming. The levels available are determined by the capabilities provided in system software and what switch vendors will support. And those capabilities, in turn, are determined by what they respectively see as market demand. VMware, for example, today will let you bond eight 10GbE channels into a single 80GbE pipe. And that’s today with mainstream 10GbE technology.

Ethernet will continue to evolve in many different ways to support the needs of the industry. Serving as a backbone for all storage networking traffic is just one of many such roles for Ethernet. In fact, precisely because of the increasing breadth of usage models Ethernet supports, it will also continue to offer cost advantages. The argument here is a very simple volume argument:

Total Server-class Adapter and LOM Market Ports

crehan-relentless-ethernet-420

Enough said, except to also note that volume is what funds speed roadmaps.

Software Defined Networks for SANs?

September 26, 2013March 1, 2014 David Fair Leave a comment

Previously, I’ve blogged about the VN2VN (virtual node to virtual node) technology coming with the new T11-FC-BB6 specification. In a nutshell, VN2VN enables an “all Ethernet” FCoE network, eliminating the requirement for an expensive Fibre Channel Forwarding (FCF) enabled switch. VN2VN dramatically lowers the barrier of entry for deploying FCoE. Host software is available to support VN2VN, but so far only one major SAN vendor supports VN2VN today. The ecosystem is coming, but are there more immediate alternatives for deploying FCoE without an FCF-enabled switch or VN2VN-enabled target SANs? The answer is that full FC-BB5 FCF services could be provided today using Software Defined Networking (SDN) in conjunction with standard DCB-enabled switches by essentially implementing those services in host-based software running in a virtual machine on the network. This would be an alternative “all Ethernet” storage network supporting Fibre Channel protocols. Just such an approach was presented at SNIA’s Storage Developers Conference 2013 in a presentation entitled, “Software-Defined Network Technology and the Future of Storage,” Stuart Berman, Chief Executive Officer, Jeda Networks. (Note, of course neither approach is relevant to SAN networks using Fibre Channel HBAs, cables, and switches.)

Interest in SDN is spreading like wildfire. Several pioneering companies have released solutions for at least parts of the SDN puzzle, but kerosene hit the wildfire with the $1B acquisition of Nicira by VMware. Now a flood of companies are pursuing an SDN approach to everything from wide area networks to firewalls to wireless networks. Applying SDN technology to storage, or more specifically to Storage Area Networks, is an interesting next step. See Jason Blosil’s blog below, “Ethernet is the right fit for the Software Defined Data Center.”

To review, an SDN abstracts the network switch control plane from the physical hardware. This abstraction is implemented by a software controller, which can be a virtual appliance or virtual machine hosted in a virtualized environment, e.g., a VMware ESXi host. The benefits are many; the abstraction is often behaviorally consistent with the network being virtualized but simpler to manipulate and manage for a user. The SDN controller can automate the numerous configuration steps needed to set up a network, lowering the amount of touch points required by a network engineer. The SDN controller is also network speed agnostic, i.e., it can operate over a 10Gbps Ethernet fabric and seamlessly transition to operate over a 100Gbps Ethernet fabric. And finally, the SDN controller can be given an unlimited amount of CPU and memory resources in the host virtual server, scaling to a much greater magnitude compared to the control planes in switches that are powered by relatively low powered processors.

So why would you apply SDN to a SAN? One reason is SSD technology; storage arrays based on SSDs move the bandwidth bottleneck for the first time in recent memory into the network. An SSD array can load several 10Gbps links, overwhelming many 10G Ethernet fabrics. Applying a Storage SDN to an Ethernet fabric and removing the tight coupling of speed of the switch with the storage control plane will accelerate adoption of higher speed Ethernet fabrics. This will in turn move the network bandwidth bottleneck back into the storage array, where it belongs.

Another reason to apply SDN to Storage Networks is to help move certain application workloads into the Cloud. As compute resources increase in speed and consolidate, workloads require deterministic bandwidth, IOPS and/or resiliency metrics which have not been well served by Cloud infrastructures. Storage SDNs would apply enterprise level SAN best practices to the Cloud, enabling the migration of some applications which would increase the revenue opportunities of the Cloud providers. The ability to provide a highly resilient, high performance, SLA-capable Cloud service is a large market opportunity that is not cost available/realizable with today’s technologies.

So how can SDN technology be applied to the SAN? The most viable candidate would be to leverage a Fibre Channel over Ethernet (FCoE) network. An FCoE network already converges a high performance SAN with the Ethernet LAN. FCoE is a lightweight and efficient protocol that implements flow control in the switch hardware, as long as the switch supports Data Center Bridging (DCB). There are plenty of standard “physical” DCB-enabled Ethernet switches to choose from, so a Storage SDN would give the network engineer freedom of choice. An FCoE based SDN would create a single unified, converged and abstracted SAN fabric. To create this Storage SDN you would need to extract and abstract the FCoE control plane from the switch removing any dependency of a physical FCF. This would include the critical global SAN services such as the Name Server table, the Zoning table and State Change Notification. Containing the global SAN services, the Storage SDN would also have to communicate with initiators and targets, something an SDN controller does not do. Since FCoE is a network-centric technology, i.e., configuration is performed from the network, a Storage SDN can automate large SAN’s from a single appliance. The Storage SDN should be able to create deterministic, end-to-end Ethernet fabric paths due to the global view of the network that an SDN controller typically has.

A Storage SDN would also be network speed agnostic, since Ethernet switches already support 10Gbps, 40Gbps, and 100Gbps this would enable extremely fast SANs not currently attainable. Imagine the workloads, applications and consolidation of physical infrastructure possible with a 100Gbps Storage SDN SAN all controlled by a software FCoE virtual server connecting thousands of servers with terabytes of SSD storage? SDN technology is bursting with solutions around LAN traffic; now we need to tie in the SAN and keep it as non-proprietary to the hardware as possible.

Q&A Summary from the SNIA-ESF Webcast – “How VN2VN Will Help Accelerate Adoption of FCoE”

August 29, 2013March 1, 2014 David Fair Leave a comment

Our VN2VN Webcast last week was extremely well received. The audience was big and highly engaged. Here is a summary of the questions attendees asked and answers from my colleague, Joe White, and me. If you missed the Webcast, it’s now available on demand.

Question #1:

We are an extremely large FC shop with well over 50K native FC ports. We are looking to bridge this to the FCoE environment for the future. What does VN2VN buy the larger company? Seems like SMB is a much better target for this.

Answer #1: It’s true that for large port count SAN deployments VN2VN is not the best choice but the split is not strictly along the SMB/large enterprise lines. Many enterprises have multiple smaller special purpose SANs or satellite sites with small SANs and VN2VN can be a good choice for those parts of a large enterprise. Also, VN2VN can be used in conjunction with VN2VF to provide high-performance local storage, as we described in the webcast.

Question #2: Are there products available today that incorporate VN2VN in switches and storage targets?

Answer #2: Yes. A major storage vendor announced support for VN2VN at Interop Las Vegas 2013. As for switches, any switch supporting Data Center Bridging (DCB) will work. Most, if not all, new datacenter switches support DCB today. Recommended also is support in the switch for FIP Snooping, which is also available today.

Question #3: If we have an iSNS kind of service for VN2VN, do you think VN2VN can scale beyond the current anticipated limit?

Answer #3: That is certainly possible. This sort of central service does not exist today for VN2VN and is not part of the T11 specifications so we are talking in principle here. If you follow SDN (Software Defined Networking) ideas and thinking then having each endpoint configured through interaction with a central service would allow for very large potential scaling. Now the size and bandwidth of the L2 (local Ethernet) domain may restrict you, but fabric and distributed switch implementations with large flat L2 can remove that limitation as well.

Question #4: Since VN2VN uses different FIP messages to do login, a unique FSB implementation must be provided to install ACLs. Have any switch vendors announced support for a VN2VN FSB?

Answer #4: Yes, VN2VN FIP Snooping bridges will exist. It only requires a small addition to the filet/ACL rules on the FSB Ethernet switch to cover VN2VN. Small software changes are needed to cover the slightly different information, but the same logic and interfaces within the switch can be used, and the way the ACLs are programmed are the same.

Question #5: Broadcasts are a classic limiter in Layer 2 Ethernet scalability. VN2VN control is very broadcast intensive, on the default or control plane VLAN. What is the scale of a data center (or at least data center fault containment domain) in which VN2VN would be reliably usable, even assuming an arbitrarily large number of data plane VLANs? Is there a way to isolate the control plane broadcast traffic on a hierarchy of VLANs as well?

Answer #5: VLANs are an integral part of VN2VN within the T11 FC-BB-6 specification. You can configure the endpoints (servers and storage) to do all discovery on a particular VLAN or set of VLANs. You can use VLAN discovery for some endpoints (mostly envisioned as servers) to learn the VLANs on which to do discovery from other endpoints (mostly envisioned as storage). The use of VLANs in this manner will contain the FIP broadcasts to the FCoE dedicated VLANs. VN2VN is envisioned initially as enabling small to medium SANs of about a couple hundred ports although in principle the addressing combined with login controls allows for much larger scaling.

Question #6: Please explain difference between VN2VN and VN2VF

Answer #6: The currently deployed version of FCoE, T11 FC-BB-5, requires that every endpoint, or Enode in FC-speak, connect with the “fabric,” a Fibre Channel Forwarder (FCF) more specifically. That’s VN2VF. What FC-BB-6 adds is the capability for an endpoint to connect directly to other endpoints without an FCF between them. That’s VN2VN.

Question #7: In the context of VN2VN, do you think it places a stronger demand for QCN to be implemented by storage devices now that they are directly (logically) connected end-to-end?

Answer #7: The QCN story is the same for VN2VN, VN2VF, I/O consolidation using an NPIV FCoE-FC gateway, and even high-rate iSCSI. Once the discovery completes and sessions (FLOGI + PLOGI/PRLI) are setup, we are dealing with the inherent traffic pattern of the applications and storage.

Question #8: Your analogy that VN2VN is like private loop is interesting. But it does make VN2VN sound like a backward step – people stopped deploying AL tech years ago (for good reasons of scalability etc.). So isn’t this just a way for vendors to save development effort on writing a full FCF for FCoE switches?

Answer #8: This is a logical private loop with a lossless packet switched network for connectivity. The biggest issue in the past with private or public loop was sharing a single fiber across many devices. The bandwidth demands and latency demands were just too intense for loop to keep up. The idea of many devices addressed in a local manner was actually fairly attractive to some deployments.

Question #9: What is the sweet spot for VN2VN deployment, considering iSCSI allows direct initiator and target connections, and most networks are IP-enabled?

Answer #9: The sweet spot if VN2VN FCoE is SMB or dedicated SAN deployments where FC-like flow control and data flow are needed for up to a couple hundred ports. You could implement using iSCSI with PFC flow control but if TCP/IP is not needed due to PFC lossless priorities — why pay the TCP/IP processing overhead? In addition the FC encapsulation/serializaition and FC exchange protocols and models are preserved if this is important or useful to the applications. The configuration and operations of a local SAN using the two models is comparable.

Question #10: Has iSCSI become irrelevant?

Answer #10: Not at all. iSCSI serves a slightly different purpose from FCoE (including VN2VN). iSCSI allows connection across any IP network, and due to TCP/IP you have an end-to-end lossless in-order delivery of data. The drawback is that for high loss rates, burst drops, heavy congestion the TCP/IP performance will suffer due to congestion avoidance and retransmission timeouts (â€˜slow starts’). So the choice really depends on the data flow characteristics you are looking for and there is not a one size fits all answer.

Question #11: Where can I watch this Webcast?

Answer #11: The Webcast is available on demand on the SNIA website here.

Question #12: Can I get a copy of these slides?

Answer #12: Yes, the slides are available on the SNIA website here.

VN2VN: “Ethernet Only” Fibre Channel over Ethernet (FCoE) Is Coming

February 26, 2013March 1, 2014 David Fair Leave a comment

The completion of a specification for FCoE (T11 FC-BB-5, 2009) held great promise for unifying storage and LAN over a unified Ethernet network, and now we are seeing the benefits. With FCoE, Fibre Channel protocol frames are encapsulated in Ethernet packets. To achieve the high reliability and “lossless” characteristics of Fibre Channel, Ethernet itself has been enhanced by a series of IEEE 802.1 specifications collectively known as Data Center Bridging (DCB). DCB is now widely supported in enterprise-class Ethernet switches. Several major switch vendors also support the capability known as Fibre Channel Forwarding (FCF) which can de-encapsulate /encapsulate the Fibre Channel protocol frames to allow, among other things, the support of legacy Fibre Channel SANs from a FCoE host.

The benefits of unifying your network with FCoE can be significant, in the range of 20-50% total cost of ownership depending on the details of the deployment. This is significant enough to start the ramp of FCoE, as SAN administrators have seen the benefits and successful Proof of Concepts have shown reliability and delivered performance. However, the economic benefits of FCoE can be even greater than that. And that’s where VN2VN — as defined in the final draft T11 FC-BB-6 specification — comes in. This spec completed final balloting in January 2013 and is expected to be published this year. The code has been incorporated in the Open FCoE code (www.open-fcoe.org). VN2VN was demonstrated at the Fall 2012 Intel Developer Forum in two demos by Intel and Juniper Networks, respectively.

“VN2VN” refers to Virtual N_Port to Virtual N_Port in T11-speak. But the concept is simply “Ethernet Only” FCoE. It allows discovery and communication between peer FCoE nodes without the existence or dependency of a legacy FCoE SAN fabric (FCF). The Fibre Channel protocol frames remain encapsulated in Ethernet packets from host to storage target and storage target to host. The only switch requirement for functionality is support for DCB. FCF-capable switches and their associated licensing fees are expensive. A VN2VN deployment of FCoE could save 50-70% relative to the cost of an equivalent Fibre Channel storage network. It’s these compelling potential cost savings that make VN2VN interesting. VN2VN could significantly accelerate the ramp of FCoE. SAN admins are famously conservative, but cost savings this large are hard to ignore.

An optional feature of FCoE is security support through Fibre Channel over Ethernet (FCoE) Initialization Protocol (FIP) snooping. FIP snooping, a switch function, can establish firewall filters that prevent unauthorized network access by unknown or unexpected virtual N_Ports transmitting FCoE traffic. In BB-5 FCoE, this requires FCF capabilities in the switch. Another benefit of VN2VN is that it can provide the security of FIP snooping, again without the requirement of an FCF.

Technically what VN2VN brings to the party is new T-11 FIP discovery process that enables two peer FCoE nodes, say host and storage target, to discover each other and establish a virtual link. As part of this new process of discovery they work cooperatively to determine unique FC_IDs for each other. This is in contrast to the BB-5 method where nodes need to discover and login to an FCF to be assigned FC_IDs. A VN2VN node can login to a peer node and establish a logical point-to-point link with standard fabric login (FLOGI) and port login (PLOGI) exchanges.

VN2VN also has the potential to bring the power of Fibre Channel protocols to new deployment models, most exciting, disaggregated storage. With VN2VN, a rack of diskless servers could access a shared storage target with very high efficiency and reliability. Think of this as “L2 DAS,” the immediacy of Direct Attached Storage over an L2 Ethernet network. But storage is disaggregated from the servers and can be managed and serviced on a much more scalable model. The future of VN2VN is bright.

Flash Webcast Q&A

October 3, 2012March 1, 2014 David Fair Leave a comment

Our recent Webcast: Flash – Plan for the Disruption was very well received and well attended. We thank everyone who was able to make the live event. For those of you who couldn’t make it, it’s now available on demand. Check it out here.

There wasn’t enough time to respond to all of the questions during the Webcast, so we have consolidated answers to all of them in this blog post from the presentation team. Feel free to comment and provide your input.

Q. Are you going to cache both read and writes in NetApp FlashCache?
A. Flash Cache is a level 2 Read cache and it is used to accelerate random read operations. NetApp offers an additional capability called Flash Pool which caches both random reads and random overwrites. Both technologies are part of the NetApp Virtual Storage Tier family within the Data ONTAP operating environment.

Q. Is eMLC flash available today?
A. Yes, a number of Flash vendors are shipping eMLC today.

Q. Also can you review the write cycle performance of SLC vs. MLC?
A. Write cycles for SLC are typically around 100,000. With eMLC, write cycles of 30,000 per bit can be achieved.

Q. Has specific analysis been conducted on what applications and relative data can be cached at the server versus at the storage controller (tolerance for latency, user patience, etc.)?
A. This varies but server caching will typically be used for applications with the most hot spots such as databases. If there is a particular requirement for ultra low latency such as in OLTP environments, server caching may be appropriate. Server caching can also yield significant benefit to increase VM density. Generally, server caching will be deployed to accelerate a specific application while storage controller caching will be used to accelerate storage which is shared across multiple applications.
Q. Does the data running over the network storage PDUs or Ethernet Layer2/IP traffic?
A. Ethernet Layer 2 in this demo, thought it could have been scaled to for L3 IP routed traffic.
Q. What is the difference between flash tier and flash cache?
A. A flash tier is persistent storage whereby datasets are pinned to flash technology for some period of time (or permanently). In Automated Storage Tiering, data may be migrated to and from the flash tier based on the temperature of the data. A flash cache, on the other hand is a caching technology in which the most frequently accessed data is copied to flash for data access but then evicted as the data cools down. Data is copied to the flash cache either on the basis of calculated data temperature or on a first-in first-out basis.
Q. Given the large advantages of flash on power (direct), cooling, and DC footprint, why do enterprise data centers not just completely switch out their HDDs? It seems like there is a good ROI even without considering performance. Is it the operational complexities that make this challenging?
A. For many applications, this is not cost justified given the significant price difference of the SSD and HDD devices. Since hot data typically amounts to less than 20% of total data, a small amount of flash can be deployed successfully. In the caching case, this can be around 1%.

Flash Webcast – Are You Ready for the Disruption?

September 12, 2012March 1, 2014 David Fair Leave a comment

There’s no doubt that flash is a game changer. Even a relatively small percentage of flash can drive a significant improvement in peak storage performance. How are you planning for the disruption? Join me and my SNIA colleague, Paul Feresten, for a live Webcast next week, Thursday, September 20th (11:00 a.m. ET, 8:00 am. PT) as we discuss the impact of flash. We’ll take a look at how flash is being deployed in storage systems, key considerations and tradeoffs, performance benefits, trends in non-volatile memory and more. And because it’s live we’ll take your questions on the spot. We hope to see you there. Register now.

Will Ethernet storage move to 10GBASE-T?

August 2, 2012March 1, 2014 David Fair Leave a comment

10GBASE-T is a technology that runs 10Gb Ethernet over familiar Category 6/6a cables for distances up to 100m and is terminated by the ubiquitous RJ-45 jack. Till now, most datacenter copper cabling has been special Direct Attach cables for distances up to 7m terminated by an SFP+ connector. To work, data center switches need matching SFP+ connectors, meaning new switches are required for any data center making the move from 1GbE to 10GbE. 10GBASE-T is generating a lot of interest in 2012 as the first single-chip implementations at lower power (fanless) and lower cost (competitive with Direct Attach NICs) come to market. A data center manager now has an evolutionary way to incorporate 10GbE that exploits the cabling and switches already in place. The cost savings from preserving existing cabling alone can be tremendous.

But is 10GBASE-T up to the task of carrying storage traffic? The bit-error rate technical tests of 10GBASE-T look promising. 10GBASE-T is meeting the 10-12 BER requirements of all the relevant Ethernet and storage specifications. We expect NAS and iSCSI to move rapidly to take advantage of the deployment cost savings offered by 10GBASE-T. Admins responsible for NAS and iSCSI storage over Ethernet should find 10GBASE-T meets their reliability expectations.

But what about Fibre Channel over Ethernet (FCoE)? Note that storage admins responsible for FC and/or FCoE are among the most risk-adverse people on the planet. They especially need to be confident that any new technology, no matter how compelling its benefits, doesn’t appreciably increase the risk of data loss. For this reason, they are adopting FCoE very slowly, though the economics make FCoE very compelling. So a broad market transition to FCoE over 10GBASE-T is likely to take some time regardless.

Cisco announced in June 2012 a new 5000-series Nexus switch supporting up to 68 ports of “FCoE-ready” 10GBASE-T. Cisco has made the investment to support storage protocols, including FCoE, over 10GBASE-T in this switch and is committed to working with the industry to do the testing to prove its robustness. In fact, some eager end-users are getting ahead of this testing, and, based on results from their own stress tests, moving now to storage over 10GBASE-T deployments, including FCoE.

Every major speed and capabilities transition for Ethernet has engendered skeptics. The transition to running storage protocols over 10GBASE-T is no different. General consensus is that the “jury is out” for FCoE over 10GBASE-T. The interoperability and stress testing to prove reliability isn’t complete. And storage admins will generally want to see reports from multiple deployments before they move. But the long-term prognosis for storage – NAS, iSCSI, and FCoE — over 10GBASE-T is looking very encouraging.

Author: David Fair