10Gbit FCoE drives data center convergence

Following a broad group of vendors' release of their first products for Fibre Channel over 10Gbit, startup SolarFlare is expected to announce a transceiver that can power 10Gbit Ethernet up to 100m over copper on a single 65nm CMOS chip that dissipates just 5.5W.

The moves fuel a broad industry drive to run networking, storage and clustering traffic over a single, mainstream pipe in tomorrow's data centers. The aim is to create one converged fabric, reducing the cost and power requirements of supporting today's multiple switches, adapter cards and cables.

"Somewhere between 50 and 70 percent of the Fortune 1000 companies are going to be building data centers in the next three years," said Renato Recio, a chief engineer for server networking at IBM Corp. "They are looking for technologies to make them more green, and this network convergence group has that value—this rings for customers," he said.

Cisco Systems, Emulex, Intel, Mellanox and QLogic were among the many vendors that announced the first crop of Fibre Channel over Ethernet (FCoE) products at Storage Networking World in last week. But with the exception of a few, including Cisco, many of the vendors are waiting for ASICs and standards to be completed before they attempt to field high-volume FCoE products.

The work on the 10GBase-T standard for Ethernet over copper lines only indirectly fuels the network convergence. Its primary aim is to lower the cost of and expand the market for 10Gbit Ethernet, which has been limited to expensive optical and short-reach copper cables to date.

Lower prices
Running 10Gbit signals over copper typically has required multiple chips using as much as 12W. The SolarFlare SFT 9001 transceiver could slash the power budget in half, but the company is still characterizing first silicon on the part, which it hopes to sample in May at less than $100.

"This is first silicon coming back, and it is very complex, but we are bringing it up now and we may productize on this stepping," said Bruce Tolley, VP of marketing for SolarFlare. "We could be a quarter to a third the price of the cheapest optics."

The 21mm² part is the third effort by the company, which was formed in 2001 and has raised $110 million in venture capital. The chip is intended for sockets in a broad range of servers, switches and network appliances.

Other startups, including Aquantia and Teranetics, are pursing a similar target, while established vendors such as Broadcom and Marvell may also be on the trail. One startup, KeyEye Communications, went bust in March pursuing the goal.

Without a long-reach copper option, the rise of 10Gbit Ethernet has been sluggish to date. Less than 400,000 10GbE switch ports were shipped last year—perhaps half of them not populated—and only about 30,000 server cards have been sold.

"Our goal is to put together a 10GbE server card that could sell for a $500 street price" before June 2009, said Tolley. "With this new silicon, we expect the market will start accelerating."



10GBase-T technology
"Getting 10GBase-T chips down to a 5W level will enable some realistic deployment," said Bob Wheeler, senior analyst with The Linley Group, "but it's still a horse race to see who can get that to production and what the exact power consumption levels will be."

Vendors are divided over the technology; IBM gave it high marks, but Cisco's new switch group gave it a pass.

"In my opinion, 10GBase-T is a very important piece because it significantly reduces my price point to use copper," said Recio of IBM. "I'd rather not use fiber in a rack, and it's an even better deal if my end-of-row switch can use copper."

The 10GBase-T technology "is great in terms of compatibility and simplicity," said Dante Malagrino, director of product marketing for data center solutions at Nuova Systems, acquired last week by Cisco. "But the additional power, cost and latency mean it is not really feasible for us, and I don't think we will use it."

Nuova, a 200-person startup, announced its first product—a 10GbE switch supporting FCoE—and plans to use a hybrid solution, based on a new copper cable terminated by SFP+ optical transceivers, that it claims has lower power requirements and lower latency than the 10GBase-T options. Malagrino pegged 10GBase-T at 2 to 3 microseconds in latency and 4- to 8W per link in power consumption. The new hybrid option will initially be limited to 1-, 3- and 5m lengths but could shave 30 percent off the overall costs of an optical fiber link, he said.



FCoE venture
On the FCoE front, most vendors have not yet completed work on new ASICs and software, with the exception of a few companies, such as Cisco. Because of the still-evolving nature of the underlying standards, even Cisco's new Nexus 5000 switch may need a software upgrade to comply with work on FCoE in separate T11 and IEEE groups.

The Wheeler noted that the new Emulex LightPulse LP21000 server card for FCoE uses four chips: separate Ethernet and Fibre Channel controllers, along with a PCIe switch and an address translation chip. The Emulex card runs existing Fibre Channel software drivers, tested on most current storage systems.

Emulex would not state the power consumption of the card, except to say it comes in under the 25W limit for an Express slot. The company plans a more integrated offering in 2009.

"It's not the most cost- or power-efficient product and looks more like a proof of technology than a high-volume product," Wheeler said of the Emulex card.

"The first half of 2009 is our target as an industry to get there with a competitive solution," said Recio of IBM. "If I get there with two or three times the parts, it's not worth it," he added, underlining the need for ASICs.

A top technologist at Fibre Channel switch maker Brocade Communications agreed. "Participating vendors are releasing prestandard products in 2008 for evaluation and test purposes, which will drive development of the integrated silicon in 2009," said John Hufferd, senior executive director of technology at Brocade. "Work done in 2009 will validate the use cases and contribute to actionable, deployable solutions for the data center in 2010 and going forward."

Cisco appears to be one of the few companies delivering an ASIC-based product at this point. The Nexus 5000 uses as many as five ASICs, including multiple instances of two key chips, said Nuova's Malagrino. "You need hardware to handle these transactions at line speeds."

One main ASIC in the Cisco system is a port controller that acts as an Ethernet controller, handling packet buffering and virtual queuing for lossless traffic. The other is a non-blocking crossbar switch with an aggregate bandwidth of more than a terabit/ second and an integrated traffic scheduler.

The system also uses a supervisory unit based on an Intel CPU running a new operating system co-developed by Cisco and Nuova. That OS is a hybrid based on the storage operating system used in Cisco's first Fibre Channel switch and Cisco's classic IOS networking software.

"All of the PHY definitions to make hardware compliant have been done in the standards groups," said Malagrino. "Most likely we will be able to claim [standards] compliance out of the gate [when the product ships in May], or, in the worst case, we will need a software upgrade."

The T11 spec for FCoE will probably be completed by August, according to Recio. However, the IEEE work, which is defining a lossless version of Ethernet to be a more robust foundation for FCoE, still has a ways to go.

"We saw that not enough progress was being made; we were stuck in a rut," said Recio. "So I called all these people in December, and we formed a new group, CEE Authors."

That group—adopting the name Converged Enhanced Ethernet, which many have used for the lossless standard—hopes to complete proposals before the end of May in three specific areas. The proposals will then be submitted to the IEEE group.

Some vendors want to implement products based on the proposals as a "Version 0" that they would upgrade later to the final standard set by the IEEE group.

"We think it's a good thing, and we are telling our customers a Version 0 product is fine," said Recio.

The CEE Authors group is chaired by Recio and includes representatives from Broadcom, Brocade, Cisco, Emulex, Fujitsu, IBM, Intel, Juniper, QLogic and Sun.

- Rick Merritt
EE Times