Network storage can offer several key advantages for organizations, including cost reduction and increased efficiency, but it also presents certain challenges. Storage area networks (SANs) can have their own inherent complexity, and organizations often require higher levels of performance to connect network storage to servers as the business grows.
The arrival of 10 Gigabit Ethernet (10 GbE), along with the Data Center Bridging (DCB) and Fibre Channel over Ethernet (FCoE) specifications, keeps alive the promise of a truly converged network fabric. These technologies offer system administrators a clear path to unify Internet SCSI (iSCSI) and Fibre Channel SANs while delivering improved storage efficiency, increased throughput, and cost-effective deployment of network storage in data centers.
Growth of SANs in network storage environments
SANs are essential elements in the shift to data center virtualization. In virtualized environments, images and data are stored on a shared SAN to facilitate live migration of virtual machines. SANs are also growing in popularity because they offer value in key areas including storage consolidation, improved disk utilization, disaster recovery, and centralized data protection.
Implementing SANs presents a number of challenges for system administrators. As the virtualized environment grows, for example, SANs require multiple networks, and each network necessitates the addition of ports and cabling for each server, which can increase costs and energy consumption. Servers and storage require advanced integration and management to realize the full benefits of virtualization, further increasing costs. In addition, a consolidated and virtualized infrastructure also generates more I/O requirements: running multiple virtual machines means supporting multiple I/O flows, and the pool of flows increases the I/O bandwidth and performance requirements of storage arrays and physical servers.
However, another source of complexity is the fact that many organizations deploy two types of networks: Fibre Channel for storage and Ethernet for data. Organizations typically maintain both types because each protocol has its own advantages and disadvantages. The latest Fibre Channel storage devices offer relatively high throughput (hardware is currently available for 8 Gb/s Fibre Channel and is expected to be available for 16 Gb/s Fibre Channel as well); however, Fibre Channel can also represent high acquisition and management costs. Ethernet is generally more cost-effective than Fibre Channel and connects with IP networks to help overcome long distances; however, the Gigabit Ethernet (GbE) networks that predominate in today's data centers offer lower throughput and higher latency compared to Fibre Channel.
Implementing SANs presents a number of challenges for system administrators. As the virtualized environment grows, for example, SANs require multiple networks, and each network necessitates the addition of ports and cabling for each server, which can increase costs and energy consumption. Servers and storage require advanced integration and management to realize the full benefits of virtualization, further increasing costs. In addition, a consolidated and virtualized infrastructure also generates more I/O requirements: running multiple virtual machines means supporting multiple I/O flows, and the pool of flows increases the I/O bandwidth and performance requirements of storage arrays and physical servers.
However, another source of complexity is the fact that many organizations deploy two types of networks: Fibre Channel for storage and Ethernet for data. Organizations typically maintain both types because each protocol has its own advantages and disadvantages. The latest Fibre Channel storage devices offer relatively high throughput (hardware is currently available for 8 Gb/s Fibre Channel and is expected to be available for 16 Gb/s Fibre Channel as well); however, Fibre Channel can also represent high acquisition and management costs. Ethernet is generally more cost-effective than Fibre Channel and connects with IP networks to help overcome long distances; however, the Gigabit Ethernet (GbE) networks that predominate in today's data centers offer lower throughput and higher latency compared to Fibre Channel.
With two networks, managing growth and optimizing utilization can become much more difficult, costly, and complex. Both types of networks require separate computing resources, including different hardware and technical expertise, which increases infrastructure and management costs. The emergence of iSCSI has enabled the use of cost-effective Ethernet infrastructure as a SAN fabric and has driven greater adoption of iSCSI SANs, such as the Dell™ EqualLogic™ PS Series arrays.
Emergence of 10 GbE
10GbE is expected to emerge as the future of data center networks, preserving the advantages of Ethernet while offering new possibilities. For example, new 10GbE components are expected to maintain the cost advantage of existing Ethernet over Fibre Channel, and cost-effective 10GbE interfaces are expected to help reduce management complexity.
10GbE offers an effective way to expand bandwidth for virtualized environments by providing highly scalable and simplified connectivity that allows multiple virtual networks to be distributed over the same physical connection. Typically, using 10GbE connectivity also means greater energy efficiency and cost-effectiveness compared to using multiple GbE network interface cards.
Furthermore, 10GbE offers a clear path for unifying iSCSI and Fibre Channel storage into a single network fabric. It enables the increased performance needed to unify communications and ensure network consistency, while building on the familiar and cost-effective IP and Ethernet technologies that businesses typically already use. Original equipment manufacturers (OEMs) are preparing products to support the rise of 10GbE. Dell introduced a 10GbE iSCSI I/O module for Dell/EMC CX4 series storage and plans to add 10GbE capability to its broad range of storage arrays.
10GbE offers an effective way to expand bandwidth for virtualized environments by providing highly scalable and simplified connectivity that allows multiple virtual networks to be distributed over the same physical connection. Typically, using 10GbE connectivity also means greater energy efficiency and cost-effectiveness compared to using multiple GbE network interface cards.
Furthermore, 10GbE offers a clear path for unifying iSCSI and Fibre Channel storage into a single network fabric. It enables the increased performance needed to unify communications and ensure network consistency, while building on the familiar and cost-effective IP and Ethernet technologies that businesses typically already use. Original equipment manufacturers (OEMs) are preparing products to support the rise of 10GbE. Dell introduced a 10GbE iSCSI I/O module for Dell/EMC CX4 series storage and plans to add 10GbE capability to its broad range of storage arrays.
Unified network structure for a convergence paradigm
Organizations are seeking ways to combine their data and storage networks into a single converged fabric that helps reduce the total cost of ownership of data center infrastructure, connect multiple storage islands, and improve scalability. 10 GbE offers the performance needed to achieve this goal, and the DCB specification is the latest piece of the convergence paradigm to be incorporated (see the separate article “Enhancing Ethernet Bridging” on this page). The DCB specification provides a set of standards-based extensions to traditional Ethernet, delivering a lossless data center transport layer that enables the convergence of LANs and SANs into a single, unified network fabric.
DCB offers advantages for iSCSI and Fibre Channel storage. Organizations can use the FCoE specification (which relies on DCB capabilities and is supported by a large number of storage and networking vendors) to connect legacy infrastructures to a 10 GbE and DCB Ethernet network. FCoE maps Fibre Channel frames across the Ethernet network while preserving the Fibre Channel protocol. The DCB specification is designed to maintain the assured delivery characteristics of the Fibre Channel link and physical layers. The DCB specification also ensures iSCSI storage by providing enhanced congestion management and comprehensive high bandwidth for Ethernet traffic, including iSCSI traffic.
DCB offers advantages for iSCSI and Fibre Channel storage. Organizations can use the FCoE specification (which relies on DCB capabilities and is supported by a large number of storage and networking vendors) to connect legacy infrastructures to a 10 GbE and DCB Ethernet network. FCoE maps Fibre Channel frames across the Ethernet network while preserving the Fibre Channel protocol. The DCB specification is designed to maintain the assured delivery characteristics of the Fibre Channel link and physical layers. The DCB specification also ensures iSCSI storage by providing enhanced congestion management and comprehensive high bandwidth for Ethernet traffic, including iSCSI traffic.
Network structure options with 10 GbE and DCB
Together, 10 GbE and DCB can provide a single, efficient network fabric that offers organizations strategic alternatives in data center network planning.1 Organizations can use iSCSI, FCoE, or both: using FCoE would require new infrastructure investments, while iSCSI simply waits for the future availability of cost-effective 10 GbE interfaces.
Organizations that choose to use FCoE as a bridge to legacy Fibre Channel SANs are expected to benefit from the ongoing evolution of FCoE (see Figure 1). The first generation of FCoE-enabled devices is expected to focus on server I/O convergence via an Ethernet switch. In the second phase of this evolution, large FCoE networks supported by DCB-enabled switches are expected to offer Assured Delivery over Ethernet (ADE) capabilities, equivalent to Fibre Channel switches. Finally, the third phase is expected to offer native FCoE storage availability for connectivity to the FCoE network, which requires FCoE services to run on the DCB network.
Convergence to consolidate network storageNetwork convergence allows organizations to consolidate storage by providing improved efficiency and cost reductions. Multiple networks can share a single 10GbE host connection, minimizing server adapters, cabling, and power consumption (see Figure 2). Combining SAN and LAN traffic on the same network significantly reduces the number of adapters, cables, and switches. SAN traffic in the converged network can use iSCSI or FCoE.
The additional bandwidth provided by the unified 10GbE fabric helps address the I/O challenges of server and storage virtualization. Networks deploying 10GbE and DCB are expected to support up to 20 Gb/s of bandwidth with two 10GbE adapters for redundancy. Additional benefits include:
- Reduced support costs: Convergence can reduce management complexity, and resources no longer need to be split between Ethernet and Fibre Channel.
- Expanded bandwidth for high-performance computing (HPC): 10 GbE is designed to expand bandwidth to connect HPC clusters to the network.
- Energy savings: By using fewer adapters, cables, and switches in a unified network fabric compared to a legacy data center network, the physical infrastructure is reduced, improving energy efficiency and cooling.
- Security: Ethernet features such as access control lists can be used for Ethernet bridging and virtual LANs to provide traffic isolation and security for various traffic flows. Security remains robust because storage traffic (iSCSI or FCoE) is simply carried within the Ethernet frames.
The additional bandwidth provided by the unified 10GbE fabric helps address the I/O challenges of server and storage virtualization. Networks deploying 10GbE and DCB are expected to support up to 20 Gb/s of bandwidth with two 10GbE adapters for redundancy. Additional benefits include:
- Reduced support costs: Convergence can reduce management complexity, and resources no longer need to be split between Ethernet and Fibre Channel.
- Expanded bandwidth for high-performance computing (HPC): 10 GbE is designed to expand bandwidth to connect HPC clusters to the network.
- Energy savings: By using fewer adapters, cables, and switches in a unified network fabric compared to a legacy data center network, the physical infrastructure is reduced, improving energy efficiency and cooling.
- Security: Ethernet features such as access control lists can be used for Ethernet bridging and virtual LANs to provide traffic isolation and security for various traffic flows. Security remains robust because storage traffic (iSCSI or FCoE) is simply carried within the Ethernet frames.
Flexibility for migration to unified network storage
Organizations have considerable flexibility as they prepare to unify network storage. FCoE can be used to connect FCoE servers to legacy Fibre Channel SANs over Ethernet, without impacting the Fibre Channel user experience while the organization migrates to 10GbE. Meanwhile, iSCSI offers the ability to perform storage in native Ethernet environments and route traffic across LANs and wide area networks (WANs). As the organization migrates from GbE to 10GbE, iSCSI can operate in a combined GbE and 10GbE network environment, and iSCSI traffic can take advantage of enhanced networking features when the infrastructure is upgraded to the DCB specification.
Many businesses can ultimately benefit from migrating to either iSCSI or FCoE connectivity. Now is a good time for organizations to conduct a thorough review of their storage strategy. Companies can continue to consolidate operations without worrying about difficult investments, as the DCB specification for Ethernet can equally benefit both iSCSI network storage and FCoE.
Many businesses can ultimately benefit from migrating to either iSCSI or FCoE connectivity. Now is a good time for organizations to conduct a thorough review of their storage strategy. Companies can continue to consolidate operations without worrying about difficult investments, as the DCB specification for Ethernet can equally benefit both iSCSI network storage and FCoE.
term "
Data Center Ethernet" to refer to a set of enhancements to Ethernet bridging standards designed to improve
Layer 2 Ethernet congestion management and enable the convergence of different traffic types—including not only storage area network (SAN) traffic but also LAN, management, and inter-process communication (IPC) traffic—on the same network. These enhancements led to the creation of the IEEE 802.1 Data Center Bridging (DCB) working group, and Cisco now directly references the DCB specification, which is expected to be formally adopted soon and will conform to the following standards.
Priority-based flow control (IEEE 802.1Qbb) at the link layer helps ensure that no packets are lost during congestion on DCB networks.
Enhanced broadcast selection (IEEE 802.1Qaz) allows administrators to reserve a specific amount of bandwidth for each traffic type to ensure good quality of service.
Congestion notification (IEEE 802.1Qau) helps improve overall congestion management and prevent recurring congestion and frame loss.
The DCB Exchange (DCBX) protocol helps ensure consistent configuration across the network.
Data Center Ethernet" to refer to a set of enhancements to Ethernet bridging standards designed to improve
Layer 2 Ethernet congestion management and enable the convergence of different traffic types—including not only storage area network (SAN) traffic but also LAN, management, and inter-process communication (IPC) traffic—on the same network. These enhancements led to the creation of the IEEE 802.1 Data Center Bridging (DCB) working group, and Cisco now directly references the DCB specification, which is expected to be formally adopted soon and will conform to the following standards.
Priority-based flow control (IEEE 802.1Qbb) at the link layer helps ensure that no packets are lost during congestion on DCB networks.
Enhanced broadcast selection (IEEE 802.1Qaz) allows administrators to reserve a specific amount of bandwidth for each traffic type to ensure good quality of service.
Congestion notification (IEEE 802.1Qau) helps improve overall congestion management and prevent recurring congestion and frame loss.
The DCB Exchange (DCBX) protocol helps ensure consistent configuration across the network.
Along with other IEEE 802.1 standards, the DCB specification is expected to help
systems organizations take advantage of improved communication quality for converged networks.
Authors:
- Achmad Chadran is a marketing manager for storage solutions in Dell's large enterprise business unit.
- Gaurav Chawla is a technology strategy specialist in the Enterprise Storage Architecture and Technology Group
at Dell's chief technology officer offices.
at Dell's chief technology officer offices.
- Ujjwal Rajbhandari is a product marketing advisor for Dell storage solutions.
"Reprinted with permission from Dell Power Solutions Magazine, September 2009."
