FCoE TechBook update for EMC

An updated version of the FCoE TechBook (v11.0) has been posted to powerlink.


New for this version is a case study that describes how to configure FCoE on the Nexus 7000 and MDS.  The topology that is configured within the case study is shown below. 




Currently it contains Cisco UCS and IBM BladeCenter related information and we are expecting to add HP Blade server related information in a month or two.


FCoE TechBook (v11.0)

VMWARE VSA

VMware VSA – all you need to know (technical paper)

In the release of VMware vSphere 5.0, VMware also released a new software storage appliance called the VMware vSphere Storage Appliance (VSA). VMware VSA provides an alternative shared storage solution to our Small to Medium Business (SMB) customers who might not be in a position to purchase a Storage Area Network (SAN) or Network-Attached Storage (NAS) array for their virtual infrastructure. Without shared storage configured in a vSphere environment, customers have not been able to exploit the unique features available in vSphere 5.0, such as vSphere High Availability (HA), vSphere, vMotion and vSphere Distributed Resource Scheduler (DRS). The VSA is designed to provide “Shared Storage for Everyone”

New EMC TechBooks are available!

Backup and Recovery in a SAN TechBook : Provides information on traditional backup and recovery architecture, SAN-based backup and recovery technologies, and disk and tape backup and recovery. Some case studies are included.


Building Secure SANs TechBook :Identifies and exemplifies some common SAN security attacks, presents some built-in and bolt-on mechanisms to enhance SAN security, and provides some insight on how to implement various product-specific security mechanisms.


Extended Distance Technologies TechBook: Provides a basic understanding of distance extension technologies and information to consider when working with extended distance. IP-based distance extension solutions are also included.


Fibre Channel over Ethernet (FCoE): Data Center Bridging (DCB) Concepts and Protocols TechBook:Provides a high-level overview of Fibre Channel over Ethernet (FCoE), the supported configurations and features, and examples for incorporating FCoE into an existing data center. Discussion of Ethernet basics is included. Setup examples for Nexus Series, Virtual PortChannel (vPC), and MP-8000B are provided. EMC RecoverPoint and Celerra MPFS solutions are discussed. Basic troubleshooting techniques are also presented.


Fibre Channel SAN Topologies TechBook :Provides a high-level overview of Fibre Channel SAN topologies, discusses simple and complex Fibre Channel SAN topologies, and provides case studies for Brocade Virtual Fabrics and EMC RecoverPoint. FICON connectivity is also discussed.


iSCSI SAN Topologies TechBook:Provides a high-level overview of iSCSI SAN topologies and includes basic information about TCP/IP technologies and iSCSI solutions.


Networked Storage Concepts and Protocols TechBook : Provides fundamental information about Fibre Channel. It presents fabric design considerations, explains how SAN technology works, and describes IP SAN concepts. Extended distance technologies and solutions are also discussed.


Storage Virtualization and Replication Technologies TechBook:Provides a high-level overview of storage virtualization and replication, including EMC’s Invista, RecoverPoint, and VPLEX. It also provides information for SAN connectivity for Invista and RecoverPoint. Basic information on interswitch link hops, along with examples showing hop counts in four different fabric configurations is included.


WAN Optimization Controller Technologies TechBook :Provides a high-level overview of the WAN Optimization Controller (WOC), including network and deployment topologies, storage and replication application, FCIP configurations, and WAN Optimization Controller appliances.

Powerful of Linux CD Command for Admins,Devlopers,Testers,Supporters

Powerful of Linux CD Command for Admins,Devlopers,Testers,Supporters

CD is one of the most frequently used command during a UNIX and LINUX session. I am going to add some useful practical examples of CD command. This will increase your productivity and make it easier to navigate the directory from CLI.

CDPATH: If you are performing CD to sub directories  of a any specific parent directory and you can set the CDPATH variable to the parent directory and perform CD to to the sub directories without Giving parent directory path.

Example:

To make this change permanent, add export CDPATH=/etc to your

~/.bash_profile

export CDPATH=.:~:/var:/usr:/etc

Directory stack

dirs: Display the directory stack

pushd: Push directory into the stack

popd: Pop directory from the stack and cd to that directory

Automatically Correct Mistyped Directory Names using shopt -s

Use shopt -s cdspell to correct the typos in the cd command

automatically as shown below

FC SAN components and Zoning

Components of FC-SAN

While SAN configurations can become very complex, a SAN can be simplified to three basic entities;

 The host system or systems, the network and the storage device

1. Host System(s)

• Application Software (SAN Management Software, CLI Interface and others)

• Middleware (e.g., Volume Manager or Host RAID)

• Operating System/File System

• Host Bus Adapter (HBA) Driver

• Host Bus Adapter (HBA)

• Host Bus Adapter Firmware

2. Storage Network/Communications Infrastructure

• Physical Links (FC, ISCSI, Ethernet)

• Transceivers (GBIC & SFP or any other Transreceiver)

• Switches and Switch Firmware (Switches & Directors)

• Routers and Router Firmware

• Bridges or Extenders and their Firmware

3. Storage Device(s)

• Interface Adapter

• Interface Adapter Driver/Firmware

• Storage Controller Firmware

• Storage Device (e.g., disk, JBOD, Storage Arrays, Tape or Tape Library)

• Storage Media

Switch or Fabric Zoning:

SAN implementations make data highly accessible; as a result, there is a need for data-transfer optimization and finely tuned network security. Fabric zoning sets up the way devices in the SAN interact, establishing a certain level of management and

security.

What is zoning?

Zoning is a fabric-centric enforced way of creating barriers on the SAN fabric to prevent set groups of devices from interacting with other devices. SAN architectures provide port-to-port connections between servers and storage subsystems through bridges, switches, and hubs. Zoning sets up efficient methods of managing, partitioning, and controlling pathways to and from storage subsystems on the SAN fabric, which improves storage subsystem utilization, data access, and security on the SAN. In addition, zoning enables heterogeneous devices to be grouped by operating system, and further demarcation based on application, function, or department.

Types of zoning

There are two types of zoning: soft zoning and hard zoning.

• Soft zoning uses software to enforce zoning. The zoning process uses the name server database located in the FC switch. The name server database stores port numbers and World Wide Names (WWNs) used to identify devices during the zoning process.

When a zone change takes place, the devices in the database receive Registered State Change Notification (RSCN). Each device must

correctly address the RSCN to change related communication paths. Any device that does not correctly address the RSCN, yet continues to transfer data to a specific device after a zoning change, that device will be blocked from communicating with its targeted device.

• Hard zoning uses only WWNs to specify each device for a specific zone. Hard zoning requires each device to pass through the switch’s route table so that the switch can regulate data transfers by verified zone.

For example, if two ports are not authorized to communicate with each other, the route table for those ports is disabled, and the communication between those ports is blocked.

Zoning components

Zone configurations are based on either the physical port that devices plug into, or the WWN of the device. There are three zoning components:

• Zones

• Zone members

• Zone sets

What is a zone?

A zone is composed of servers and storage subsystems on a SAN that access each other through managed port-to-port connections. Devices in the same zone recognize and communicate with each other, but not necessarily with devices in other zones unless a device, in that zone, is configured for multiple zones

Zone types

•          Port zoning (all zone members are ports)

•          WWN zoning (all zone members are WWNs)

•          Session-based zoning (zone members are a mixture of WWNs and ports)

Zone database

•          Zone database consists of zone objects. 

•          A zone object can be an alias, a zone, or a configuration

•          Configurations contain zones which contain aliases

•          For any object, the commands available allow you to create, delete, add, remove, or show

–         cfgcreate/delete/add/remove/show

–         zonecreate/delete/add/remove/show

–         alicreate/delete/add/remove/show

•          Every switch in the fabric has the same copy of the entire database.

•          To clear the zone database from a switch, use cfgclear

Alias

•          An alias is a name for a device in the fabric

•          The alias contains the name of the devices, and either the WWN of the device, or the domain and port the device is attached to

•          WWN alias: alicreate “alias1”,”10:00:00:00:01:01:02:02”

•          Port alias: alicreate “alias2”,”100,15”

What is a zone member?

Zone members are the devices within the same assigned zone. See Figure 2. Zone member devices are restricted to intra-zone communications, meaning that these devices can only interact with members within their assigned zone. A zone member

cannot interact with devices outside its assigned zone unless it is configured in other zones

How is a zone member identified?

Each zone member is identified by a WWN or port number. each device has a unique WWN. A WWN is a 64-bit number that uniquely identifies each zone member.

What is a zone set?

A zone set is a group of zones that function together on the SAN. Each zone set can accommodate up to 256 zones. All devices in a zone see only devices assigned to their zone, but any device in that zone can be a member of other zones. In Figure 3,

all 4 zones see Member A.

Configurations

•          A configuration is a set of zones.

•          You can have multiple defined configurations, but only one active configuration in a fabric at any time.

•          cfgcreate “cfg1”,”zone1”

•          To enable a configuration, use ‘cfgenable “config1”.  This is now called the effective configuration

To disable the effective configuration, use cfgdisable command.  Note when you disable zoning that all devices can now see each other

Zone Commit

•          A zone commit is the process of updating all switches in the fabric when making a zone change

•          Zone commit is executed for cfgdisable, cfgenable, or cfgsave commands

•          Zone commit uses RCS protocol.  The switch making the commit communicates with each switch individually to ensure commit took place

•          When zone commit takes place, entire zoning database is sent to all switches even if only a little change has taken place.

RCS [Reliable Commit Service]

•          RCS is used for zoning, security, and some other things.

•          For zoning, RCS ensures a zone commits happens for every switch in the fabric, or none at all

•          4 phases to RCS:  ACA, SFC, UFC, RCA

Zoning limitation

Currently, fabric zoning cannot mask individual tape or disk storage LUNs that sit behind a storage-subsystem port. LUN masking and persistent binding are used to isolate devices behind storage-subsystem ports.

Fibre Channel Switch setup

====================

 I. FC Switch configuration  (Ex: Brocade )                                                                                                                               

1. Open up a HyperTerminal
2. Login as admin
3. Enter password for password
4. Type in configure
1. Configure entire switch
5. Type in Help
1. List commands possible
6. Type ipAddrSet
7. Enter the Ethernet IP address
1. Get from Tom York
8. Enter the common SubNet Mask
1. 255.255.252.0
9. Hit enter twice after SubNet Mask
1. Use default values
10. Enter the gateway number, which is the same throughout the lab
1. 147.145.175.254
11. When ask to set respond by entering <y>
12. Now type ipAddrShow
1. Make sure the ip address held
13. Now reboot
1. This will take several minutes

 

II. Enable the Switches

1. Connect to the internet
2. Enter the address http:// ip address of the switch
3. Now click on zone admin
4. Enter admin for user name
5. And enter password for password
6. For zone selection select switch/port level zoning
7. And click <ok>
8. Click port zone tab
9. Click create zone
10. Name zone
11. Go to switch port, domain and select ports 0 through 7
12. Click add mem =>
13. Create another zone
14. Select ports 8 through 15
15. Now select port config tab
16. Highlight both new zone add them
1. Under file zones
17. Add mem =>
18. Click enable config and then apply and okay
1. All located on bottom of screen

                                                                                                                   

Switch Initialization:

·         At the Power ON, boot PROM diagnostics :

Ø       Verify CPU DRAM Memory.

Ø       Initialize base Fabric Operating System (FOS).

·         The Initialized FOS does the following:

Ø       Execute Power-On Self Test (POST) on switch.

Ø       Initialize ASICs & Front panel.

Ø       Initialize link for all ports (put online).

Ø       Explore the Fabric and determine the Principal Switch.

Ø       Assign Addresses to Ports.

Ø       Build Unicast routing table.

Ø       Enable N-Port operations.

Fabric Port Initialization Process: (From Switch Prospective)

Transition 1: At the beginning, verify if anything is plugged to Switch Port.

Transition 2: At FL-Port, Is there any Loop connections present in the Switch.

Transition 3: At G-Port, Verifying if any other (switch or Hubs) devices connected.

Transition 4: After G-Port, Verifying if Switch or Point-to-Point devices connected.

Communication Protocols:

• Fabric Devices typically

Ø       FLOGI à PLOGI to Name Server à SCR to Fabric Controller à Register & Query [using FC Common Transport (FC_CT) Protocol] à LOGON.

• Loop Devices typically

Ø       PRIVATE NL: LIP (PLOGI & PRLI will enable private storage devices that accept PRLI & thus “appear” Fabric capable)

Ø       PUBLIC NL: LIP à FLOGI à PLOGI à SCR à Register & Query à LOGO & then PLOGI à & Communicate with other end nodes in the fabric.

Ø       LIP Process include: LIP, LISM, LIFA, LIPA, LIHA, LISA and LIRP & LILP.

Create SWAP file system

What is swap partition?

Swap partition is used for paging in memory management method by which computer can store and retrieve data from secondary storage (Hard disk) and then for use in primary storage (RAM).

What is paging?

Operating system retrieves data from secondary storage in same size blocks called  pages

Why its required?

The main purpose of this file system is when an Application or process data not fit into physical memory (RAM) then disk storage (Hard disk) is used as temporary memory.

Example:Your system having 1GB RAM and running Apache web server and because of huge traffic on server 1 GB RAM exceded. Then memory management schemes moved the low proiprty process (tasks) to Swap file sysem and started execute remaining high priority process first.

If system needs run moved process then it copies pages from swap partion to System RAM.

what is the size of swap partition?

Common recommendation is 2 times of your physical RAM.

But According to Open BSD FAQ:

Nonsense rule: Twice the size of your main system RAM for Servers

Many people follow an old rule of thumb that your swap partition should be twice the size of your main system RAM. This rule is nonsense. On a modern system, that's a LOT of swap, most people prefer that their systems never swap. You don't want your system to ever run out of RAM+swap, but you usually would rather have enough RAM in the system so it doesn't need to swap

My recommendation: If you are a kernel hacker (debugging and fixing kernel issues) and generating core dumps, you need twice the RAM swap space.

example: Your server having 2GB RAM then Swap partion should be 4GB

How to create swap partition?

There are 2 methods

1.While installing new OS

2.After installing OS

While installing new OS

While installing new OS you have to select new partion of file system type SWAP with Size.

After installing OS

Check previous swap partitions and their size as below.

cat /proc/swaps

swapon -s

free -m

top

Create a file for swap usage as shown below

Use the mkswap command to make our file swap-consumable for the Linux kernel. Again as root, carefully 

To turn on our swap file, we should run swapon

Add the following line to /etc/fstab and reboot the system for the swap

to take into effect.

/root/swap-fs swap swap defaults 0 0

100 Storage Interview questions

[ Hard Disk ]

1. Explain the inner structure or layout of a Hard disk ?

2. Explain what is meant by CHS ?

3. What is Cylinder,Head,Sector,Track,clusters ?

4. What is LBA ?

5. Name some of the hard disk vendors ? What are the specifications of a hard disk you consider when you are working as storage admin or storage tester ?

6. Explain CPACS( Capacity,Performance,Availability,Cost,Scalability) importance in Storage ?

7. Name some of the Storage components ?

8. Explain each of below Storage components

            Hosts [ Servers ]

            Internetworking Components

            Network Devices [ Switches , Hubs ]

            Cable Connectivity

            Storage Subsystems

            Hard drives

            Tape drives

            JBOD [ Just a Bunch Of Disks ]

            Enclosures

            Arrays

            NAS [ Network attached Storage ]

            SAN [ Storage Area Network ]

9. Explain the difference between DAS-NAS-SAN ?

10.Explain below standards or Protocol interfaces ?

            IDE   : Integrated Device Electronics

            ATA   : Advanced Technology Attachment

            SATA : Serial ATA

            SCSI  : Small Computer System Interface

            SAS   : Serial attached SCSI

            FC     : Fibre Channel

            ISCSI : SCSI over IP

11. Sort above mentioned protocols interms of Performance  and Cost ?

[ MAGNETIC TAPE ]

12. What is a magnetic tape ?

13. Mention different types of magnetic tapes ?

[ STORAGE OPERATING SYSTEMS ]

14. Mention some of the major operating systems which you have worked on with respect to Storage experience ?

15. Mention different processor types & latest processors what you have used in your work ?

16. Why we need to be aware of processor & OS combinations ?

17. What difference a 64 bit Windows 2003 Server from normal 32 bit ? Which processors are 32 bit & 64 bit from Intel ?

[  SCSI  ]

18. What was the name for SCSI before it was standardised by ANSI ?

19. Why SCSI had very good market share compared to ATA/IDE/PATA ?

20. Whats Ultra1,Ultra2,Ultra3 in SCSI ?

21. Mention the difference between Narrow & Wide SCSI ?

22. What is the maximum no. of devices supported in narrow,wide SCSI?

23.I have a SCSI peripheral but I do not know if it has the single-ended, HVD or LVD interface. Is there an easy way to tell?

24. Explain SE ?

25. What is LVD & HVD ?

26. Differentiate between SE, LVD, HVD ?

27. Mention the advantages of SCSI ?

28. WHat is meant by Hot plug or Hot swap ?

29. Among IDE/ATA, SCSI, SAS, SATA , FC which ones support Hot plug/Hot swap ?

30. What is NCQ ?

31. What is TCQ ?

32. What is SCSI ID & why is it important ?

33. HOw SCSI id is configured for a SCSI hard disk drive (HDD) ?

34. Mention device types supported in SCSI 3 ?

35. Mention device types supported in SCSI 2 ?

36. Mention device types supported in SCSI 1 ?

37. Give examples for each of the below :

            Block devices (SBC)

            Stream devices (SSC)

            Graphic devices (SGC)

            Medium changer devices (SMC)

            RAID Controllers (SCC)

            CD-ROMS (MMC)

38. Explain connector types & their no. of pins in SCSI ?

39. What is the maximum length of cable permitted in SCSI ?

40. What is daisy chaining ?

41. What are the different types of SCSI cards ?

42. Mention different SCSI phases ?

43. Explain what happens in each of the below phases :

            arbitration

            selection

            message out

            message in

            command data out

            data in           

            status

            bus free

44. What is a terminator & why is it needed in SCSI ?

45. Mention some SCSI commands ?

46. Explain each of below

            -Test unit ready:

            -Inquiry:

            -Request sense:

            -Start/Stop unit:

            -Read capacity:

            -Format unit

            -Read (four variants)

            -Write (four variants)

            -Log sense:

            -Mode sense:

            -Mode select:

47. What is SCSI arbitration and selection ? How is this resolved ?

48. Name some of the vendors of SCSI HBA,SCSI hard drives ?

49. Does SCSI work in both directions (is it a bidirectional bus ) ?

50. What is "multimode LVD" or LVD/MSE SCSI?

51. What are the benefits of LVD SCSI?

52. Is LVD SCSI backward compatible?

53. What is Ultra160 or U160 SCSI?

54. I have heard of U160/m SCSI. What is it?

55. Is Ultra160 SCSI backward compatible?

56. Is Ultra 160 SCSI better than fibre channel?

57. Is Ultra 160 SCSI better than EIDE?

58. What is Fast-20 [or Fast-40 or Fast-80] SCSI?

59. Can I connect an Ultra 2 Wide (LVD) disk to an Ultra Wide adapter?

60. What is Double Transition clocking?

61. What is "Domain Validation"?

62. What is CRC?

63. I recently purchased a new SCSI drive with an SCA (or SCA-2) connector. How do I connect this to my cabled SCSI system?

64. What is QAS?

65. What is SCSI "packetization" (also called information units)?

66. What are the five optional features of Ultra 160 (Ultra 3) SCSI?

67. All the new hard disk drives have the LVD/MSE interface. Can I use them on a Narrow single-ended SCSI bus?

68. I know I can place a LVD/MSE peripheral on a single-ended bus, but the LVD interface is Wide (34 pair cables) and the single-ended bus is Narrow (25 pair cables). Any problems here?

69. Can I attach Narrow devices to a Wide SCSI bus?

70. I have heard the term "idc" applied to internal SCSI ribbon cables. What does it mean?

71. Can I attach Wide peripherals to a Narrow SCSI bus?

72. I note that in several of your FAQs you recommend the use of 68-pin to 50-pin adapters. Aren't they "impedance lumps" in the SCSI transmission line that can cause reflections and associated problems?

73. I had a problem with my SCSI bus. It was much slower than I thought it should be. When I used a new SCSI cable that was six feet longer than the one it replaced, the system performance returned to what I think it should be. Why?

74. How can I connect a HVD device to my single-ended bus? [or vice versa]

75. How can I connect a HVD device to my LVD bus?

76. I have a LVD/MSE bus with several multimode peripherals, and I want to add a single-ended peripheral to it. I know that LVD/MSE is backward compatible through the single-ended interface, but when I add the single-ended peripheral, the LVD peripherals really slow down. Can something be done to resolve this.

77. How can I connect a LVD peripheral to my single-ended SCSI bus?

78. Why does my LVD/MSE bus slow down when I connect even one single-ended peripheral?

79. I connected what I believed to be a single-ended peripheral to my multimode (LVD/MSE) bus, thinking the bus would switch to the single-ended mode and operate, although at a slower speed. The bus simply shut down. What happened?

80. How can I connect a LVD peripheral to an HVD SCSI bus?

81. What are LUNs? why are they important ?

82. Can I have more than one computer on my SCSI bus?

83. What is the difference between SCSI and IDE (or EIDE or ATAPI)?

84. Among SCSI or IDE which is better for enterprise storage  and for desktop ?

85. Will my single-ended to differential converter also do differential to single-ended conversion?

86. terminators. Where should they be placed?

87. What is the difference between passive and active terminators?

88. Should I use passive or active terminators?

89. I need a terminator for my SCSI bus consisting of a number of HVD disk drives. I do not need a pass-through HVD terminator, but that is all I can find. Can I use it for the terminator at the end of this HVD bus?

90. How can I tell if an unmarked terminator is SE or differential?

[ SAN Related Interview Questions ]

91. What is LUN masking?

92. What is SAN zoning?

93. What are different types of zoning & how do they differ ?

94. What are hard and soft zoning? What is port zoning ? What is WWN zoning ?

95. What is a World Wide Name (WWN) & why is it important ?

96. In a FC HBA how can i find WWN ?

97. How are iSCSI, iFCP and FCIP secured over IP networks?

98. What are the differences between FC & iSCSI ?

99. Mention some of the issues you faced while working on SAN & how did you resolve them ?

100.What was the most challenging troubleshooting you did while working on SAN. Please give the steps & commands used ?