XtremSW Cache User Guide 2.0.1
XtremSW Cache User Guide 2.0.1
XtremSW Cache User Guide 2.0.1
XtremSW Cache™
v2.0.1
User Guide
302-000-001
REV 02
Copyright © 2013 EMC Corporation. All rights reserved. Published in the USA.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to
change without notice.
The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any
kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability
or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication
requires an applicable software license.
EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and
other countries. All other trademarks used herein are the property of their respective owners.
For the most up-to-date regulatory document for your product line, go to EMC Online Support
(https://support.emc.com).
Preface
Appendix D Troubleshooting
General issues............................................................................. 261
Adding a cache device ........................................................... 262
Adding GPT source devices in Microsoft Cluster Service
environment .......................................................................... 262
Partitioning source devices .................................................... 265
VMAX support........................................................................ 265
Card issues............................................................................ 265
Log files................................................................................. 273
“Please re-run start cache <cache_device> manually” error
message ............................................................................... 273
Slow response times in the CLI .............................................. 274
Cache counters (read hits and read IOs) may be incorrect ...... 274
Cache devices enter passthru mode unexpectedly ................. 274
Error messages ...................................................................... 274
VSI plug-in and VMware............................................................... 278
Installation issues ................................................................. 278
Enabling features................................................................... 278
Activating XtremSW Cache administration ............................. 279
Title Page
Title Page
PREFACE
As part of an effort to improve its product lines, EMC periodically releases revisions of
its software and hardware. Therefore, some functions described in this document
might not be supported by all versions of the software or hardware currently in use.
The product release notes provide the most up-to-date information on product
features.
Contact your EMC technical support professional if a product does not function
properly or does not function as described in this document.
Note: This document was accurate at publication time. Ensure that you are using the
latest version of this document, and all related documents, by downloading the
current version from EMC Online Support (https://support.emc.com).
Purpose
This document describes how to use EMC XtremSW Cache.
Sections include:
◆ Part 1, “XtremSW Cache Basics,”describes basic concepts of operation and
XtremSW Cache management. Chapters include:
• Chapter 1, “Introduction to Managing XtremSW Cache”
• Chapter 2, “Before You Begin”
◆ Part 2, “Using the Command Line Interface (CLI),” describes how to manage
XtremSW Cache with the command line interface. Chapters include:
• Chapter 3, “Getting Started—CLI”
• Chapter 4, “Managing XtremSW Cache—CLI”
• Chapter 5, “Advanced Management Topics—CLI”
• Appendix A, “Command Reference—CLI”
◆ Part 3, “Using the VSI Plug-in for VMware,” describes how to manage XtremSW
Cache with the VSI Plug-in for VMware. Chapters include:
• Chapter 6, “Getting Started—VSI Plug-In”
• Chapter 7, “Managing XtremSW Cache—VSI Plug-In”
• Chapter 8, “Advanced Configuration—VSI Plug-In”
◆ Part 4, “Using the XtremSW Management Center,” describes how to manage
XtremSW Cache with the XtremSW Cache Management Center. Chapters include:
• Chapter 9, “Getting Started—XtremSW Management Center”
• Chapter 10, “Monitoring and Managing Systems”
• Chapter 11, “Managing Cache and Source Devices”
• Chapter 12, “Configuring XtremSW Management Center Settings”
• Appendix B, “Servicing the XtremSW Management Center”
• Appendix C, “XtremSW Management Center REST API.”
◆ Part 5, “Using the XtremSW Lite Client,” describes how to manage XtremSW Cache
with the XtremSW Lite Client. Chapters include:
• Chapter 13, “Getting Started—XtremSW Lite Client”
• Chapter 14, “Configuring the XtremSW Lite Client”
• Chapter 15, “Viewing Statistics and System Information”
• Appendix D, “Troubleshooting”
Related documentation
The following EMC publications provide additional information:
◆ EMC XtremSW Cache Installation Guide
◆ EMC XtremSW Cache Release Notes
◆ EMC XtremSW Cache Quick Start
NOTICE is used to address practices not related to personal injury.
IMPORTANT
An important notice contains information essential to software or hardware operation.
Typographical conventions
EMC uses the following type style conventions in this document:
Bold Use for names of interface elements, such as names of windows,
dialog boxes, buttons, fields, tab names, key names, and menu paths
(what the user specifically selects or clicks)
Italic Use for full titles of publications referenced in text
Monospace Use for:
• System output, such as an error message or script
• System code
• Pathnames, filenames, prompts, and syntax
• Commands and options
Monospace italic Use for variables.
Monospace bold Use for user input.
[] Square brackets enclose optional values
| Vertical bar indicates alternate selections — the bar means “or”
... Ellipses indicate nonessential information omitted from the example
Your comments
Your suggestions will help us continue to improve the accuracy, organization, and
overall quality of the user publications. Send your opinions of this document to:
techpubcomments@emc.com
The chapters in this part of the guide describe issues to consider before managing
XtremSW Cache. Chapters include:
◆ Chapter 1, “Introduction to Managing XtremSW Cache,” describes how XtremSW
Cache works, installation options, and system requirements.
◆ Chapter 2, “Before You Begin,”describes issues to consider before beginning to
manage XtremSW Cache.
Introduction to Managing XtremSW Cache
CHAPTER 1
Introduction to Managing XtremSW Cache
EMC XtremSW Cache™ is a server flash caching solution that reduces latency and
increases throughput to dramatically improve application performance by leveraging
intelligent caching software and PCIe flash technology.
XtremSW Cache accelerates reads and protects data by using a write-through cache to
the networked storage to deliver persistent high availability, integrity, and disaster
recovery.
XtremSW Cache coupled with array-based EMC FAST software creates the most
efficient and intelligent I/O path from the application to the data store. The result is a
networked infrastructure that is dynamically optimized for performance, intelligence,
and protection for both physical and virtual environments.
This chapter describes how XtremSW Cache works, and the management utilities you
can take advantage of to maximize XtremSW Cache benefits. Topics include:
◆ Understanding XtremSW Cache ........................................................................ 19
◆ XtremSW Cache management utilities .............................................................. 20
XtremSW Cache is most effective for workloads with 70% (or more) read-write ratio
with random I/O. By default, I/O up to 64KB (in Windows, 128KB) will be cached.
XtremSW Cache is optimized to work with XtremSF cache cards, but it can work with
other cards as well. For a description of supported cards, see the “System
Requirements” section of the XtremSW Cache Installation Guide.
When using XtremSF cards, you benefit from the following enhancements:
◆ Card health monitoring
◆ Firmware management
◆ Log extract feature
◆ Software compatibility verification
The Management Center, a vApp that runs on a dedicated virtual machine, can
manage Windows, Linux, and ESX servers. You connect to the vApp from any web
browser.
The Management Center is best for environments with multiple XtremSW Cache
installations, in both VMware and physical environments, where historical
information is important.
The Management Center can also integrate with EMC Unisphere Remote to
simplify monitoring and managing of caching and flash cards, directly from
Unisphere Remote.
◆ Lite Client
The Lite Client, a “Lite” version of the Management Center, manages a single
instance of XtremSW Cache. The Lite Client runs on a Windows machine, and
supports remote systems running on Windows and Linux physical servers.
The Lite Client is recommended for environments with one, or a few, physical
machines, and it can be used post-configuration for virtual machines and ESX
hosts, too. The Lite Client does not store history from session to session.
You can use multiple management utilities in your environment. For example, you
could manage your physical machines with the CLI and your virtual machines with the
VSI plug-in and/or the Management Center.
Before using the management utilities, you must install the caching software and the
relevant management utilities. These tasks are described in the XtremSW Cache
Installation Guide.
This document is divided into separate parts, each of which describes the use of a
different management utility:
◆ Part 2, “Using the Command Line Interface (CLI),” describes how to manage
XtremSW Cache with the command line interface.
◆ Part 3, “Using the VSI Plug-in for VMware,” describes how to manage XtremSW
Cache with the VSI plug-in.
◆ Part 4, “Using the XtremSW Management Center,” describes how to manage
XtremSW Cache with the XtremSW Cache Management Center.
◆ Part 5, “Using the XtremSW Lite Client,” describes how to manage XtremSW Cache
with the XtremSW Lite Client.
Proceed to the part that describes the management utility you want to use.
CHAPTER 2
Before You Begin
This chapter describes basics to consider before you begin to install and manage
XtremSW Cache. Topics include:
◆ Supported devices and multi-pathing............................................................... 23
◆ VMware considerations .................................................................................... 27
◆ Configuring XtremSW Cache support for clusters .............................................. 30
Note: This chapter is identical to the chapter with the same name in the XtremSW
Cache Installation Guide.
The previous filter rule is the default filter rule for Linux.
When using PowerPath and LVM together, you should use the PowerPath filter. This
will avoid duplicate PVIDs. For more information, refer to the PowerPath Installation
and Administration guide.
multipath -F
5. To verify the change, run the following command: multipath -v2
The flash device should not appear in the output.
Device name
should start
Device type with Device full path Examples
Device name
should start
Device type with Device full path Examples
Note: You can accelerate LVM logical devices, or the physical devices that these LVM
are based on. You cannot accelerate both physical and logical devices.
VMware considerations
This section describes issues to consider in your XtremSW Cache implementation in
the VMware environment.
RDM
To enable live migration, the flash device must appear to the source and target ESX
hosts as a shared resource. This is accomplished using a multipath plug-in (MPP) over
RDM.
On each virtual machine, a virtual SCSI device is created with a fixed ID. This ID will be
identical over all ESX hosts on the cluster. The virtual machine accesses the flash
device using an RDM disk over that SCSI device.
Now that the ESX hosts share the same SCSI ID, the migration can succeed.
Writes to cache flow through the RDM disk to the MPP on the ESX and from there
straight to the flash device.
VMware considerations 27
Before You Begin
XtremSF2200 PCIEHHM-2200V
XtremSF1400 PCIEHHM-1400M
XtremSF350S PCIEHHS-3XXM2
XtremSF700S PCIEHHS-7XXM
Therefore, to use any card for DAS, no other cards from that group can be added as
caching, or the DAS-intended card will be used for caching, too.
The following examples illustrate this principle:
◆ An ESX host on which XtremSF550 and XtremSF2200 are installed. Both cards are
from the same group, thus both cards must be used for caching or both cards
must be used for DAS.
◆ An ESX host on which XtremSF550 and XtremSF700 are installed. The cards are
from different groups, thus the cards can be used in any combination of caching
and DAS with no limitations.
VMware considerations 29
Before You Begin
When XtremSW Cache is used in a cluster, quorum disks should not be defined as
source devices.
When an application within a cluster is being accelerated using XtremSW Cache, the
data is written to the XtremSW Cache device as well as to a shared LUN. In the case of
a failover, the application could be moved to a different node and will continue to
write to the shared LUN, but will not write to the cache device of the previously active
node.
When the application fails back to the original node, the application will retrieve data
from the cache device, but now, this device could contain stale data.
Some environments (RHEL Cluster Suite, Veritas Cluster Server, and AIX PowerHA)
require configuration of the supplied XtremSW Cache Clustering script, to ensure that
stale data is never retrieved. The scripts use Cluster Management events that relate to
an application service start/stop transition to trigger a mechanism that purges the
cache. Other environments, such as Microsoft Cluster Service and Oracle Real
Application Clusters, do not require script configuration.
Cluster support is currently provided for clusters configured to operate in
active/standby mode, where XtremSW Cache is installed and operating on the single,
active node and on any combination of the standby nodes. In addition, Oracle Real
Application Clusters (RAC) is supported.
In Microsoft Cluster Server environments, XtremSW Cache supports multi-application
cluster configurations. In other environments, only one application in a cluster can
use XtremSW Cache. To use XtremSW Cache for several applications, you must
configure these applications as separate resources in a single application, so that
they fail over between hosts as a single unit.
Note: Before adding cache or source devices in a cluster environment, ensure that you
have configured cluster support, as described in the following sections.
Increase
config_version by 1
Note: If your version of ccs_tool does not support the update option, manually copy
the file to all cluster nodes and then restart the cluster.
After successful completion of the cluster configuration, perform the following steps,
on each passive node, before configuring the active nodes:
1. Configure cache devices and source devices on the passive nodes, using the
vfcmt add command.
2. On each passive node, run vfcmt set -clustermode passive.
This command releases the source devices so that an active node can acquire
them.
3. Repeat the previous steps for all passive nodes in the cluster.
4. After completing the previous steps, if there is an active node in the cluster,
configure its cache devices and source devices, using the vfcmt add command.
Note: Any time you add cache devices to a passive node, you must run the vfcmt set
-clustermode passive command on that node before adding any source devices to
that cache device.
To verify proper XtremSW Cache configuration within the cluster, you can run the vfcmt
display -all command. On each passive node, the state of each source device should
be PASSIVE. On the active node, the state of each source device should remain as
originally configured (for example, STARTED, PT-RD or PT-WR).
Logging
Script logs are written to /var/log/messages.
To verify proper XtremSW Cache configuration within the cluster, you can run the vfcmt
display -all command. On each passive node, the state of the relevant source device
should be PASSIVE. On the active node, the state of all cache devices should be
STARTED.
Application mode
By default, from XtremSW Cache v2.0, multiple applications in a cluster can use
XtremSW Cache, whereby when there is a failover, only the source devices related to
the failed application went into passthru mode. This is known as application-group
mode.
From v2.0.1, you can revert source devices to single-application mode, in which when
there is a failover, all source devices go into passive mode.
Note: Ensure that the file type is DEF, not TXT or something else.
c. If, during the installation of XtremSW Cache v1.5.x, you performed the steps
described in “Configuring support for Microsoft Cluster Service,” including:
extracting the cluster script, creating the resource, and defining
dependencies, then skip to the next step. Otherwise, perform those steps
now.
d. Create source devices.
2. Repeat the previous steps on every node in which XtremSW Cache is installed in
this cluster.
Note: You do not need to create a resource on every node, as the first resource
was created as a shared resource.
To add GPT source devices, see “Troubleshooting” in the XtremSW Cache User Guide.
For Linux
1. Extract the vfcache_State_Control.sh script file from
VFCache_Cluster_Support.zip to the /etc/emcsfc/ directory.
2. Make the script executable by issuing the following command:
chmod 555 /etc/emcsfc/vfcache_State_Control.sh
3. Repeat the previous steps on all nodes where XtremSW Cache is installed.
For Windows
1. Extract vfcache_State_Control.vbs script file from
VFCache_Cluster_Support.zip to the XtremSW Cache installation folder
(by default, C:\Program Files\EMC\VFC).
2. Repeat the previous step on all nodes where XtremSW Cache is installed.
3. On all cluster nodes, regardless of whether XtremSW Cache is installed, perform
the following:
a. Extract the contents of the EMC_vfcache folder from
VFCache_Cluster_Support.zip into <VCS_HOME>\bin. This should
create the <VCS_HOME>\bin\EMC_vfcache folder.
b. Copy <VCS_HOME>\bin\Default50Agent.dll to
<VCS_HOME>\bin\EMC_vfcache\EMC_vfcache.dll. If the file exists,
you should replace it.
4. Ensure that all the files you extracted in the previous steps are accessible and
executable by the user running the VCS application.
Defining dependencies
This section describes how to add the following dependencies to the XtremSW Cache
resource created in the previous step:
◆ The XtremSW Cache resource should depend on the shared LUN or LUNs.
◆ The application should depend on the XtremSW Cache resource.
To create dependencies, perform the following steps:
1. Create the XtremSW Cache resource dependency by executing the following
command:
hares -link emc_vfcache_resource storage_resource
where storage_resource is the name of your storage resource
2. Create the application dependency by executing the following command:
Note: Any time you add cache devices to a passive node in the future, you must run
the vfcmt set -clustermode passive command on that node before adding any source
devices to that cache device.
To verify proper XtremSW Cache configuration within the cluster, you can run the vfcmt
display -all command. On each passive node, the state of all cache devices should be
PT-RW. On the active node, the state of all cache devices should be STARTED.
cause of the problem, caching may restart once the problem is solved, or you may
need to restart it manually. The first read is always read directly from storage. Data is
not passed between XtremSW Cache on the various nodes via interconnect.
For example, if there is no access to one of the nodes during a time-out period, or if
there are no resources available for some other reason, XtremSW Cache will switch to
PT-RW mode, and will remain in that mode for at least 60 seconds. This state can
affect a single node or all nodes. After 60 seconds, and once access to other nodes is
attained, XtremSW Cache operation will return to PT-Normal (started) mode.
In cases when XtremSW Cache is not installed on a node, that node only (the
corresponding sources (LUNs) being accelerated by XtremSW Cache) is prevented
from being an active member of the RAC cluster. SCSI-3 persistent reservation blocks
IO to nodes that are not part of the Dcache IO update relay interconnect network. In
this way, data integrity is maintained across the cluster.
Note: The storage array must support SCSI-3 persistent reservation, and the option
must be set to active. If you do not set the option to active, blocking of additional
sources may occur. Refer to your storage array documentation for instructions about
configuring SCSI-3 persistent reservation.
EMC recommends using XtremSW Cache with Oracle RAC to cache LUNs that hold data
files. It is not recommended to cache redo logs, archives, temporary and Clusterware
files.
XtremSW Cache supports the following environments:
◆ Oracle 11g on Windows, RHEL, or OEL (running the operating system versions
supported by XtremSW Cache, as described in the XtremSW Cache Installation
Guide.)
◆ Oracle Clusterware 11g
◆ Up to eight nodes per cluster
◆ VMware with source devices using RDM (Raw Device Mapping)
The following requirements are necessary for Oracle RAC support:
◆ Oracle RAC must be installed on all nodes before XtremSW Cache is installed (if
XtremSW Cache was installed prior to Oracle, uninstall XtremSW Cache and
reinstall it).
◆ A supported flash device must be installed in every node.
IMPORTANT
Do not add source devices with the following assignments:
- Quorum devices
- Redo logs
- TempDB
- Archives
The following figure depicts how XtremSW Cache is deployed in an Oracle RAC
environment.
Limitations:
• In cases when EMC PowerPath is deployed, acceleration of LUNs managed by
PowerPath, and masked by UDEV, may not work in all scenarios. A workaround
for this limitation is to use PowerPath persistent names.
• Multiple Oracle RAC clusters on the same interconnect network or sub-network
are not supported. Do not install XtremSW Cache in such environments.
• In cases where SCSI-3 support is required in Linux guest virtual machine
environments, each virtual machine must be installed on a separate ESX host,
and must be configured as Pass-Through RDM / Physical Compatibility Mode
(as shown in Figure 2).
• When the command vfcmt add source is used cluster-wide, the source is not
added using SCSI ID, but it is recognized using SCSI ID.
• In cases where multiple virtual machines reside on the same ESX host, they
must be configured as Pass-Through RDM / Virtual Compatibility Mode (as
shown in Figure 2).
Troubleshooting:
When a vfcmt stop command is executed, the operation might only succeed on some
of the cluster nodes. On those nodes, caching will be stopped. On nodes where the
operation failed, the source will be in pass-through mode (PT-RW). Try to perform the
stop operation again. If it still is not successful, a node restart will set the source to
Stopped (in other words, this situation does not persist system restarts).
You must perform the steps in this section after installing XtremSW Cache, and before
adding cache devices. Oracle Clusterware should already be running.
During the installation of XtremSW Cache, a script is copied to the default installation
directory:
• On Linux systems: /etc/emcsfc/
• On Windows systems: C:\Program Files\EMC\VFC\
After installing XtremSW Cache, you must run this script to create the resource and its
dependencies.
To run the script, perform the following:
1. Create the resource:
• On Linux, type the command:
/etc/emcsfc/oraxswc.sh set
• On Windows, navigate to the location: C:\Program Files\EMC\VFC\ and
then type the command:
oraxswc.cmd set
Note: I If the file extension is cnd instead of cmd, rename the file by typing:
ren "C:\Program Files\EMC\VFC\oraxswc.cnd" oraxswc.cmd
NAME=emc.xswc
TYPE=local_resource
TARGET=OFFLINE, OFFLINE
STATE=OFFLINE, OFFLINE
ora.LISTENER.lsnr:START_DEPENDENCIES=hard(type:ora.cluster_vip_
net1.type,emc.xswc)
pullup(type:ora.cluster_vip_net1.type,emc.xswc)
OK.
Note: Before removing XtremSW Cache, be sure to remove the resource, as described
in “Removing XtremSW Cache components” on page 52.
Error states in the cluster cause caching to be suspended, and XtremSW Cache enters
PT_RW mode. When the problem is solved, and communication between all nodes is
restored, XtremSW Cache returns to normal caching operation (Started state), after a
60 second time-out period. The following events will suspend caching:
◆ No communication with one of the other nodes in the cluster
◆ Memory allocation problems
The following table lists possible node states that can be displayed by the XtremSW
Cache:
State Comment
Waiting for the Oracle RAC grid Oracle RAC grid infrastructure is not operating,
services and XtremSW Cache cannot operate until the
grid services are running
Querying the cluster configuration Oracle RAC grid infrastructure is operating, and
XtremSW Cache is checking the status of the
nodes in the cluster
Connecting to other nodes Information about the cluster and its nodes has
been collected successfully, and XtremSW
Cache is attempting to connect to the nodes
3. Add cache devices to the nodes using the vfcmt add -cache_dev
<device> command.
4. Run the vfcmt display -all command to verify that:
• each cache node state is Operational
• each cache device on each node is Started
Example:
[root@oracls3n1 ~]# vfcmt display -all
Node 'oracls3n1 (local node)':
Node State: Operational
Node 'oracls3n2':
Node State: Operational
Node 'oracls3n3':
Node State: Operational
...
5. Configure the source device on one node only, using the vfcmt add
-source_dev <device> command.
• Windows
C:\Program Files\EMC\VFC\oraxswc.sh reset
2. Select the following menu options in the order that they appear, by pressing Enter
to progress through the sets of menu options: Communications Applications and
Services > PowerHA SystemMirror > Cluster Applications and Resources >
Resources > Configure User Applications (Scripts and Monitors) > Application
Controller Scripts > Add Application Controller Scripts. A list of fields appears.
3. Complete the fields as follows:
• Application Controller Name—type a text name of your choice
• Start Script—type:
/etc/opt/emcsfc/VFCache_State_Control.sh start
• Stop Script—type:
/etc/opt/emcsfc/VFCache_State_Control.sh stop
• Application Monitor Name(s)—leave this field blank
• Application Startup Mode—accept the default value [background]
Note: If the source device has the same name across all cluster nodes, you can pass
the source device name to the cluster script. This is important in cases when one node
has a local device accelerated (which is not part of the cluster and will not move
between nodes), and another accelerated device that is shared (part of the cluster). In
such cases, specify the name of the device after the first argument.
For example, if the shared device is hdisk6, the start/stop scripts would look like this:
If there is more than one shared device, create a separate Application Controller script
for each device. If the disk name is different, you may need separate scripts.
This solution only works if the devices have the same name; otherwise, you must
rename them so that they have identical names.
5. Return to the Cluster Applications and Resources menu, and then select the
following menu options: Resource Groups > Change/Show Resources and
Attributes for a Resource Group.
6. From the list displayed in the box at the bottom of the screen, select the resource
group to which the accelerated device belongs, and press Enter. A list of
configuration parameters appears.
7. In the Application Controllers field, add the Application Controller name that was
defined in Step 3.
8. Press Enter to save the configuration.
To synchronize the cluster nodes:
1. Return to the Resources menu (Communications Applications and Services >
PowerHA SystemMirror > Cluster Applications and Resources > Resources).
2. Select Verify and Synchronize Cluster Configuration, and press Enter. Accept any
further confirmation messages that are displayed. The configuration is
propagated throughout the cluster. This process may take several minutes. When
the process is completed successfully, the Command field at the top of the screen
displays the message OK.
Note: Any time you add cache devices to a passive node in the future, you must run
the vfcmt set -clustermode passive command on that node before adding any source
devices to that cache device.
To verify proper XtremSW Cache configuration within the cluster, you can run the vfcmt
display -all command. On each passive node, the state of each source device should
be PASSIVE. On the active node, the state of each source device should remain as
originally configured (for example, STARTED, PT-RD or PT-WR).
Logging
Script logs are written to /var/hacmp/adm/cluster.log
The chapters in this part of the guide describe how to use the vfcmt command line
interface (CLI) to manage XtremSW Cache. Chapters include:
◆ Chapter 3, “Getting Started—CLI,” describes first-time configuration XtremSW
Cache.
◆ Chapter 4, “Managing XtremSW Cache—CLI,” describes how to manage cache and
source devices.
◆ Chapter 5, “Advanced Management Topics—CLI,”describes advanced
management topics.
◆ Appendix A, “Command Reference—CLI,” describes all of the vfcmt command line
commands.
Getting Started—CLI
CHAPTER 3
Getting Started—CLI
This chapter describes how to use the command line interface to configure XtremSW
Cache. Topics include:
◆ “Identifying cache and source devices” on page 59
◆ “Executing command line interface commands” on page 61
◆ “Basic steps to begin caching with XtremSW Cache” on page 63
◆ “Example of first-time configuration” on page 65
You configure XtremSW Cache by executing vfcmt command line interface commands.
These commands are added to your system during the XtremSW Cache installation.
Some of these commands are relevant to the first-time configuration and others are
used for managing and maintenance.
This chapter describes first-time configuration. For information about managing and
maintenance, refer to Chapter 4, “Managing XtremSW Cache—CLI.”
For information about advanced configuration, refer to Chapter 5, “Advanced
Management Topics—CLI.”
Note: Resizing is not supported on logical volumes used as cache devices. If devices
are resized, they should be deleted from caching and then added again.
After installing your cache card and device drivers, and verifying that it is properly
installed, you should see the disk in the Device Manager, as shown in the following
example
Windows devices can be referred to either by their XtremSW Cache handle, described
below, or by the disk number, for instance harddiskX, where X is the number of the
disk in the Disk Management console list (starting from 0).
The following figure illustrates how an XtremSF cache device is shown using the lsdev
command.
Note: To run vfcmt commands on ESXi hosts, use the full path to the command:
/opt/emcsfc/bin/vfcmt <command>.
The XtremSW Cache caching device is referred to in this document as the cache
device. The device containing the data to be accelerated is referred to in this
document as the source device, or LUN.
When executing command line interface commands, you can specify the cache device
and source device names in either of the following ways:
In Windows:
◆ As an operating system-specific device name, for example, harddisk3.
◆ As an XtremSW Cache instance ID (handle), such as 9, where the 9 represents the
handle.
For example, either of the following commands could be used to delete the cache
device harddisk3 with a XtremSW Cache handle of 9:
• vfcmt delete -cache_dev harddisk3
• vfcmt delete -handle 9
In Linux:
◆ As an operating system-specific device name, for example, /dev/rssda
◆ As an XtremSW Cache instance ID (handle), such as 9, where the 9 represents the
handle.
For example, either of the following commands could be used to delete the cache
device /dev/rssda with a XtremSW Cache handle of 9:
• vfcmt delete -cache_dev /dev/rssda
• vfcmt delete -handle 9
You can retrieve the handle by executing the vfcmt display -all command. Each
XtremSW Cache device and source device will be displayed with a unique handle.
In AIX:
◆ As an operating system-specific device name, for example, /dev/hdisk1.
◆ As an XtremSW Cache instance ID (handle), such as 9, where the 9 represents the
handle.
For example, either of the following commands could be used to delete the cache
device /dev/hdisk1 with an XtremSW Cache handle of 9:
• vfcmt delete -cache_dev /dev/hdisk1
• vfcmt delete -handle 9
You can retrieve the handle by executing the vfcmt display -all command. Each
XtremSW Cache device and source device will be displayed with a unique handle.
Your configuration settings are stored and saved so that when you reboot, you do not
need to perform them again.
The following sections describe each of these steps. At the end of this section, the
commands are put together in a series so you can see the complete process.
XtremSW Cache will display a message upon successful command execution, or an
error message if the command is not successful.
If you are running XtremSW Cache in a cluster environment, you must configure cluster
support before defining any devices. If you wish to add cache devices on a passive
node after the initial setup, you must run the vfcmt set -clustermode passive
command. For more information, refer to “Support for Hyper-V” on page 85
IMPORTANT
If your cache device is partitioned, or if you expect to partition it, refer to “Partitioning
a PCIe flash device” on page 76 before adding the cache device.
You can enable multiple XtremSW Cache devices, one per PCIe card. Each cache
device can accelerate multiple source devices, but each source device can be
accelerated by only one XtremSW Cache device.
To enable multiple XtremSW Cache devices, repeat the following procedure for each
XtremSW Cache device to be added.
To enable an XtremSW Cache cache device, run the following command:
vfcmt add -cache_dev <device>
where device is specified as an operating-system specific device
Before executing this command, ensure that:
◆ the cache device is supported by XtremSW Cache
◆ the cache device is not reported to be in use by the operating system
In addition to adding the cache device, this command also starts the caching process
on the added device, thus there is no need to run the start command.
When you add an XtremSW Cache device, you can also set the following parameters:
◆ Data deduplication
◆ Maximum I/O size
◆ Page size
For more information on setting these other parameters, as well as how to change
them after adding the cache device, see “Adding a cache device” on page 77.
Note: When you add a cache device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command. For more information on vfcmt commands in this environment,
see “Configuring support for Oracle Real Application Cluster” on page 40.
By adding a device by name, you are declaring that the device name is permanent. If
the device name is going to change, remove the old device name before the name
changes, with the vfcmt remove -source_dev command.
When adding mpath source devices, such as mpathc, in the Linux environment, add
the source device by the full name of the mpath device.
When multipath is installed (such as EMC PowerPath or Veritas DMP), only use the
multipath devices as source devices, not the physical paths. For example, use
PowerPath device /dev/emcpowerk and not /dev/sdb.
You can accelerate a source device with one cache device only.
If there is only one cache device, you do not need to specify to which device to attach
the source. If there are multiple cache devices, you must specify which cache device
will be performing the acceleration, otherwise the source device will be attached to
the cache device with the lowest handle.
Note: Oracle RAC source devices are always added to the default cache device, usually
the one that was added first. The -cache_dev argument is not supported.
To use a different cache device for acceleration, use the vfcmt migrate command after
adding the source device.
You must perform this command for each source device to be cached.
In addition to adding the source device, this command also starts acceleration for the
added device, thus there is no need to run the start command.
Note: When you add a source device in the Oracle RAC environment, you are adding
the device to all server nodes in the cluster. Before adding source devices, all RAC
nodes must be reported as Operational. You can determine the node status by using
the vfcmt display -all command. For more information on vfcmt commands in this
environment, see “Configuring support for Oracle Real Application Cluster” on
page 40.
1. Enable and start the cache device using the add -cache_dev command.
2. Enable and start caching of the source device using the add -source_dev
command.
Windows example
For the purposes of this example, we are assuming the following devices:
◆ The cache device is harddisk5.
◆ The source devices are harddisk6, harddisk3 and harddisk1.
1. Enable and start caching on the cache device by executing the following
command:
vfcmt add -cache_dev harddisk5
2. Enable and start caching of the source devices by executing the following
commands:
• vfcmt add -source_dev harddisk6
• vfcmt add -source_dev harddisk3
• vfcmt add -source_dev harddisk1
3. You can review the configuration by executing the following command:
vfcmt display -all
The following figure shows the output of the display -all command.
Note that the XtremSW Cache handles do not match the Windows disk names.
To manage XtremSW Cache, refer to Chapter 4, “Managing XtremSW Cache—CLI.”
Linux example
For the purposes of this example, we are assuming the following devices:
AIX example
For the purposes of this example, we are assuming the following devices:
◆ The cache device is /dev/hdisk4.
◆ The source devices are /dev/hdiskpower1 and /dev/hdiskpower2.
1. Enable and start caching on the cache device by executing the following
command:
vfcmt add -cache_dev /dev/hdisk4
2. Enable and start caching of the source devices by executing the following
commands:
• vfcmt add -source_dev /dev/hdiskpower1
CHAPTER 4
Managing XtremSW Cache—CLI
This chapter describes how to manage the cache and maintain XtremSW Cache flash
devices and source devices.
The following topics are described:
◆ Managing the cache ......................................................................................... 69
◆ Displaying system statistics ............................................................................. 73
◆ Managing flash devices.................................................................................... 75
◆ Maintaining flash devices................................................................................. 79
Note: When you add a source device in the Oracle RAC environment, you are adding
the device to all server nodes in the cluster. Before adding source devices, all RAC
nodes must be reported as Operational. You can determine the node status by using
the vfcmt display -all command.
To add GPT source devices in the Microsoft Cluster Service environment, see “Adding
GPT source devices in Microsoft Cluster Service environment” on page 262.
To add a source device to acceleration, run the following command:
vfcmt add -source_dev <device> [-cache_dev <device>
-dev_id_type by-name]
where source device is specified as an operating-system specific device.
where cache device can be specified as an operating-system specific device or as an
XtremSW Cache handle.
You can identify the device by name, by using the -dev_id_type by-name argument.
This can be used when the device cannot otherwise be found, for example if it’s a
native SCSI device that Linux cannot identify in /dev/disk/by-path or
/dev/disk/by-id.
By adding a device by name, you are declaring that the device name is permanent. If
the device name is going to change, remove the old device name before the name
changes, with the vfcmt remove -source_dev command.
When adding mpath source devices, such as mpathc, in the Linux environment, add
the source device by the full name of the mpath device.
When multipath is installed (such as EMC PowerPath or Veritas DMP), only use the
multipath devices as source devices, not the physical paths. For example, use
PowerPath device /dev/emcpowerk and not /dev/sdb.
You can accelerate a source device with one cache device only.
If there is only one cache device, you do not need to specify to which device to attach
the source. If there are multiple cache devices, you must specify which cache device
will be performing the acceleration, otherwise the source device will be attached to
the cache device with the lowest handle.
Note: Oracle RAC source devices are always added to the default cache device, usually
the one that was added first. The -cache_dev argument is not supported.
To use a different cache device for acceleration, use the vfcmt migrate command after
adding the source device.
To accelerate a partitioned source device, you must specify the whole device name,
not the partition, as described below:
• To accelerate a Windows device on which the E: partition resides, you would
add it as follows:
vfcmt add -source_dev harddisk2
• To accelerate a Linux device on which /dev/emcpowera1 resides, you would
add it as follows:
vfcmt add -source_dev /dev/emcpowera
• To accelerate a logical volume in AIX, specify it as the source device. For
example, to accelerate the logical volume /dev/hdiskpower0, you would
add it as follows:
vfcmt add -source_dev /dev/hdiskpower0
After the successful completion of the add -source_dev command, the added source
devices are automatically started upon system start-up.
To start the devices manually, type the following command:
vfcmt start -source_dev <device>| -handle <handle>
where source device can be specified as an operating-system specific device or as an
XtremSW Cache handle.
Cache harddisk3 1
Source harddisk1 2
Source harddisk2 3
Note: In the Oracle RAC environment, the vfcmt display -all command returns output
from all nodes.
The following table describes the fields listed with the display -all command:
Data Description
Data Description
Pages/Used Number of pages occupied in the cache, out of all cache pages
Dedup Gain The percentage of the amount of data in your cache that is duplicated.
Deduplication uses RAM, in proportion to this amount.
Exceeds The percentage of I/O on this device whose size exceeds the defined
MaxIO maximum I/O
Skips The sum percentage of write requests that were not found in cache (write
misses) plus the IOs that were directly read from, or written to, the source
device because they were larger than the specified maximum IO
◆ If you add a logical volume mapped to the device, for example /dev/hd3,
XtremSW Cache will use only that part of the disk for acceleration. You must
allocate a minimum of 20 GB for acceleration.
Note: When you add a cache device, all parameters that you do not specify are set to
their default values.
You can also use edit these parameters after the XtremSW Cache device has been
added using the vfcmt edit_props command. For more information, refer to “Editing
cache device properties” on page 79.
Note: When you add a cache device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
To add a cache device with default values, run the following command:
vfcmt add -cache_dev <device>
where device is specified as an operating-system specific device
To add a cache device and set its parameters, run the following
command:
vfcmt add -cache_dev <device> [-enable_ddup [-ddup_gain
<percentage>]] [-set_max_io_size <size>] [-set_page_size
<KB>]
where
• device is specified as an operating-system specific device.
• percentage represents the expected amount of data in your cache that is
duplicated. By default, deduplication is disabled.
• size represents the desired maximum I/O size, from 32KB to 128KB. By
default, set to 64KB (in Windows, 128KB).
• KB represents the cache page size. You can set this to 4, 16, 32, 64, or 128. By
default, set to 8KB (in Windows, 64KB).
Before executing this command, ensure that the device is supported by XtremSW
Cache and that the device is not in use.
After the successful completion of the add -cache_dev command, the
start -cache_dev command is automatically performed upon system start-up, thereby
starting the caching on the specified device.
For a list of supported cache devices and multi-pathing information, refer to
“Supported devices and multi-pathing” on page 23.
For more information about these commands, see Appendix A, “Command
Reference—CLI.”
Note: Any property that is not set explicitly with this command will be returned to its
default value.
After issuing this command, restart the cache device, using the start -cache_dev, and
restart the source devices, using the start -source_dev, commands.
The migrate command has many prerequisites. Be sure to read through this entire
section before performing the migrate command.
Note: Executing the migrate command will cause all cached data in the source and
cache devices to be erased.
Migration steps
To migrate source devices, perform the following steps:
1. Record the handle of the existing cache drive.
2. Shut down the host.
3. Insert the new flash device.
4. Restart the host.
5. If you had previously split the existing device to use a part as DAS, and you need
to copy data to the new device, use standard, host-based utilities to do that now.
6. Perform the migrate command (after fulfilling all the prerequisites listed above),
by executing the following command:
vfcmt migrate -existing_dev <device> | -existing_hd
<handle> -new_dev <device> | -new_hd <handle>
where -existing_dev and -existing_hd represent the name or handle of the existing
cache device
where -new_dev and -new_hd represent the name or the handle of the new cache
device
7. To remove the existing card, shut down the machine, disconnect the power cord
from the power supply, and remove the card.
8. Restart the machine, and restart the cache and source devices.
9. Delete the old XtremSW Cache device, using the delete -cache_dev command,
and specifying the drive by its handle.
For more information about these commands, see Appendix A, “Command
Reference—CLI.”
To erase the data from a cache device, type the following command:
vfcmt secure_erase -<device>| -handle <handle>
where device can be specified as an operating-system specific device or as an
XtremSW Cache handle.
Confirm the operation and the device’s data will be erased.
When the operation is complete, a confirmation message appears.
CHAPTER 5
Advanced Management Topics—CLI
When installing XtremSW Cache in the Hyper-V environment, pay attention to the
following details:
◆ The XtremSF card and driver, as well as the XtremSW Cache software are installed
on the Hyper-V host machine. This results in the following:
• Virtual disks can be defined either before or after configuring the LUN as a
source device.
• All virtual disks allocated on a LUN source device will be accelerated.
◆ The installation procedure is identical to the procedure for Windows. For more
information, refer to the XtremSW Cache Installation Guide.
The following figure illustrates the output of the LUN selector recommendation list:
The LUNS with the higher score will benefit more from being accelerated with
XtremSW Cache. If resources are limited, these LUNs should be accelerated first.
Note: You can also perform this test on an existing cache device by using the
vfcmt edit_props command to change its deduplication properties. For more
information, refer to “Editing cache device properties” on page 79.
Note: This command displays the deduplication statistics, not the XtremSW Cache
device statistics that are displayed with the vfcmt display -all command.
4. Calculate the observed deduplication hit ratio, and compare it the configured
ratio.
• Calculate the observed deduplication hit ratio by dividing the Write Hits by the
Writes Received. This is the amount of duplicate data in the cache.
– If the observed ratio is less than 10%, it is recommended to turn off
deduplication.
Note: Both commands can result in changes in other cache device parameters, and
will result in starting with a cold cache. Be sure to review the relevant sections in the
documentation before using these commands.
The following table depicts the host memory used by enabling deduplication.
Configured deduplication
gain 350 GB card 700 GB card
50% 500 MB 1 GB
Note: If you back up to and from the flash device, IOs are likely to be larger than
the default (64 or 128KB). You do not want to cache the backup IOs, so in this
case, it is recommended to allow more than 25% Exceeds MaxIO and not to
increase the max I/O.
Note: The process of changing the maximum I/O on an XtremSW Cache device
requires that you first stop all source devices. This will result in starting with a cold
cache.
2. Stop the cache device, using the vfcmt stop -cache_dev command.
3. Change the current maximum I/O ratio using one of the following methods:
• Use the vfcmt edit_props command, and set the maxio parameter:
vfcmt edit_props -cache <device> | -handle <handle>
-set_max_io_size <size>
Note: After using the edit_props command, you need to restart the cache and
source devices.
• Remove the XtremSW Cache device, using the vfcmt delete -cache_dev
command, and recreate a new one using the vfcmt add -cache_dev command,
with the new maxio parameter:
vfcmt add -cache_dev <device> -set_max_io_size
<max_io_size>
where device is specified as an operating-system specific device.
where max_io_size is the maximum size of I/O to be cached.
Note: The add -cache_dev command is also used to set deduplication, as described in
the previous section. You can set maximum I/O size and deduplication in the same
command. If you do not set deduplication when issuing this command, deduplication
will be set to the default (disabled).
To set the page size on a current device, stop the cache device, and use the
-set_page_size flag in the vfcmt edit_props command and set KB to the desired size.
For more information, refer to “Editing cache device properties” on page 79.
2. If the previous rule does not appear, add it by typing the following command:
esxcli storage core claimrule add -u -t vendor
--vendor=EMC --model=VFC-PSAN --plugin=PSAN_MPP --force
3. Enter claiming rules for the flash devices installed on the ESX hosts in your cluster
by typing the following commands:
• XtremSF350s, XtremSF700, XtremSF700s, any XtremSF1400
esxcli storage core claimrule add -u
-t vendor --vendor=ATA --model=mtip32xx
--plugin=PSAN_MPP --force
• XtremSF550 and XtremSF2200
esxcli storage core claimrule add -u
-t vendor --vendor=EMC --model=VFStore
--plugin=PSAN_MPP -force
• XtremSF300S
esxcli storage core claimrule add -u
-t vendor --vendor=EMC --model=SLP-300
--plugin=PSAN_MPP -force
Note: You only need to claim the devices once per device group. For example,
if you have XtremSF350s and XtremSF700 installed, you only need to run the
command once.
By default, pseudo devices and flash cards are claimed by NMP. Before the PSAN_MPP
rules can claim them, you must first unclaim them.
4. Unclaim the devices by typing the following command: esxcli storage core
claiming unclaim -t vendor --vendor=<vendor_name>
--model=<model_name>
where <vendor_name> and <model_name> are according to the following
examples, per XtremSF device:
• XtremSF350s, XtremSF700, XtremSF700s, and XtremSF1400
esxcli storage core claiming unclaim -t vendor
--vendor=ATA --model=mtip32xx
• XtremSF550 and XtremSF2200
esxcli storage core claiming unclaim -t vendor
--vendor=EMC --model=VFStore
• XtremSF300S
esxcli storage core claiming unclaim -t vendor
--vendor=EMC --model=SLP-300
Note: You only need to unclaim the devices once per device group. For
example, if you have XtremSF350s and XtremSF700 installed, you only need to
run the command once.
d. Load the claim rules by typing the following command: esxcli storage
core claimrule load
e. Run the claim rules by typing the following command: esxcli storage
core claimrule run
This appendix describes the command line interface commands. For more information
about the symbols used in the command syntax, refer to “Typographical conventions”
on page 15.
Note: To run vfcmt commands on ESXi hosts, use the full path to the command:
/opt/emcsfc/bin/vfcmt <command>.
Executable commands
Table 6 Executable commands
Refer to
Command Function page
vfcmt add -cache_dev <device> instructs XtremSW Cache to use the device page 107
[-enable_ddup [-ddup_gain device for caching data blocks
<percentage>]] [-set_max_io_size
<size>] [-set_page_size <KB>]
vfcmt add -source_dev <device> instructs XtremSW Cache to provide cache page 108
[-cache_dev <device> -dev_id_type services for the block device device.
by-name]
vfcmt analyze -all | -src <device> runs the LUN selector utility page 109
[-collect <minutes>]
vfcmt delete -cache_dev <device> instructs XtremSW Cache to stop using the page 110
| -handle <handle> cache device device
vfcmt delete -source_dev <device> instructs XtremSW Cache to stop caching page 110
| -handle <handle> services for the block device device
vfcmt disable -maxio -cache_dev instructs XtremSW Cache to process all IOs, page 111
<device> | -handle <handle> regardless of size
vfcmt display -cache_dev <device> displays statistics, source devices, and page 112
| -handle <handle> caching information for the specified cache
device
vfcmt display -firmware -device displays the firmware version of the flash page 112
<device> device
Refer to
Command Function page
vfcmt display -license displays information about the license file page 112
on the current physical host
vfcmt display -serial -device displays the serial number of the flash page 113
<device> device
vfcmt display -source_dev <device> displays the state of the source device as page 113
| -handle <handle> well as specific performance statistics
associated with the source device
vfcmt display -tunable rft [-src displays the state of the read full track page 114
<device>] features
vfcmt edit_props [-cache <device> | edits the properties of a previously defined page 114
-handle <handle>] [-enable_ddup XtremSW Cache device
[-ddup_gain <percentage>] ]
[-set_max_io_size <size>]
[-set_page_size|-ps <KB>]
vfcmt extractlog -device <device> creates a log file with debugging data page 115
[-path <logpath>]
vfcmt migrate -existing_dev instructs XtremSW Cache to move all source page 115
<device>|-existing_hd <handle> devices from one cache device to another
cache device.
-new_dev <device> |-new_hd <handle>
vfcmt purge -cache_dev <device> instructs XtremSW Cache to discard all page 116
| -handle <handle> [-stats_only] current cache entries
vfcmt purge -source_dev <device> instructs XtremSW Cache to discard current page 116
| -handle <handle> [-stats_only] cache entries related to a specific source
device
Refer to
Command Function page
vfcmt release -source_dev <device> releases the lock from an XtremSW page 116
Cache-enabled VMAX device
vfcmt reservation -clear_reservation removes all registration keys and the page 117
-device <device> -reservation_key reservation group
<key>
vfcmt reservation removes the issuer node's reservation key page 117
-unregister_reservation -device from the reservation group associated with
<device>|-all a specified source device
vfcmt secure_erase -<device>| enables you to erase the data from a cache page 117
-handle <handle> device in a secure way
vfcmt set -clustermode passive enables the configuring of XtremSW Cache page 117
in clusters
vfcmt set_license -license_file copies the XtremSW Cache license file to page 118
<license_file_path> the physical host
vfcmt set -opmode <passthru_mode> enables manually changing the state of a page 118
-cache_dev <dev_name> | -source_dev cache or source device
<device> | -handle <dev_handle>
vfcmt set -tunable rft -mode <mode> enables the configuring of the read full page 118
[-src <device>] track mode
vfcmt start -cache_dev <device> instructs XtremSW Cache to prepare the page 119
| -handle <handle> cache device for cache operations
vfcmt start -source_dev <device> instructs XtremSW Cache to start caching page 120
| -handle <handle> services for the block device source device
vfcmt stop -cache_dev <device> instructs XtremSW Cache to stop caching page 120
| -handle <handle> operations for all source block devices
currently using the cache device
vfcmt stop -source_dev <device> instructs XtremSW Cache to stop caching page 121
| -handle <handle> services for the source device
vfcmt updatefirmware -device updates the PCIe card firmware page 121
<device> -path <path> -force
vfcmt version displays the current version of the XtremSW page 121
Cache software
If you are running XtremSW Cache in a cluster environment, you must configure cluster
support before defining any devices. If you wish to add cache devices on a passive
node after the initial setup, you must run the vfcmt set -clustermode passive
command. For more information, refer to “Configuring XtremSW Cache support for
clusters” on page 30.
You can enable data deduplication with the -enable_ddup option. For more
information, refer to “Configuring data deduplication” on page 90.
You can change the maximum size of I/O to be cached with the -set_max_io_size
option. For information, refer to “Setting the maximum I/O size” on page 95.
You can change the page size with the -set_page_size option. For more information,
refer to “Changing the page size” on page 96.
You can set all of the above parameters in the same command. Any parameter that is
not set, will be set at its default value.
Before executing this command, ensure that the device is supported by XtremSW
Cache and that the device is not in use.
Note: When you add a cache device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
By adding a device by name, you are declaring that the device name is permanent. If
the device name is going to change, remove the old device name before the name
changes, with the vfcmt remove -source_dev command.
This command is used as workaround for extreme cases where the device has no SCSI
ID. After reboot, you must reconfigure the device.
When adding mpath source devices, such as mpathc, in the Linux environment, add
the source device by the full name of the mpath device.
When multipath is installed (such as EMC PowerPath or Veritas DMP), only use the
multipath devices as source devices, not the physical paths. For example, use
PowerPath device /dev/emcpowerk and not /dev/sdb.
add -source_dev instructs XtremSW Cache to provide cache services for the block
device source_dev.
By default, the block device will be attached to only one cache device, the first one
that is defined. To attach the block device to a different cache device, use the optional
-cache_dev device parameter.
Note: Oracle RAC source devices are always added to the default cache device, usually
the one that was added first. The -cache_dev argument is not supported.
To use a different cache device for acceleration, use the vfcmt migrate command after
adding the source device.
Use this command only on a source device that has not been previously configured.
Note: When you add a source device in the Oracle RAC environment, you are adding
the device to all server nodes in the cluster. Before adding source devices, all RAC
nodes must be reported as Operational. You can determine the node status by using
the vfcmt display -all command.
Example:
vfcmt analyze -all -collect 10
This command analyzes all VMAX LUNs, and returns a recommendation report based
on ten minutes of activity.
Note: When you delete a cache device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
vfcmt delete_license
This command removes the license file from the physical host. On ESX hosts, you
must use this command before you install a new license file with the set_license
command.
On ESX hosts, precede vfcmt commands with /opt/emcsfc/bin/vfcmt
<command>
For more information on licensing, see Appendix A, “Licensing.”
Note: When you delete a source device in the Oracle RAC environment, you are
removing the device from all nodes. Before removing source devices, all RAC nodes
must be reported as Operational. You can determine the node status by using the
vfcmt display -all command.
vfcmt display
On an ESX host, displays information about the following:
◆ Active LUNs—LUNs that are being used by the guest machines as communication
ports
◆ Active devices—physical devices that were added to the ESX local cache pool
◆ Global counters—counters used by the XtremSW Cache ESX driver
Note: Any property that is not set explicitly with this command will be returned to its
default value.
For information about the meaning and use of these flags, see Chapter 5, “Advanced
Management Topics—CLI.”
When you use the add -cache_dev command to make these cache device
configuration changes, you must first remove all attached source devices, and remove
the cache device. The benefit of the edit_props command, is the ability to reconfigure
cache device properties without losing the configuring of the cache and source
devices.
Before using this command, ensure that the source devices being accelerated by this
cache device, and the cache device itself, are in a stopped state. The easiest way to
do this is to stop the cache device using the stop -cache_dev command.
After issuing this command, restart the cache device, using the start -cache_dev, and
restart the source devices, using the start -source_dev, commands.
vfcmt export
This command generates a list of all the vfcmt commands necessary to recreate your
cache devices and the source devices that are attached to them.
vfcmt help
help displays a brief summary of each of the vfcmt commands.
Note: Executing the migrate command will cause all cached data in the cache devices
to be erased.
After executing this command, restart the source devices, using the start -source_dev
command.
Disable RFT for all source devices vfcmt set -tunable rft -mode off
Disable RFT for a specific source vfcmt set -tunable rft -mode off
-src <device>
Enable RFT for all source devices vfcmt set -tunable rft -mode on
Enable RFT for a specific source vfcmt set -tunable rft -mode on
-src <device>
Note: When you start a cache device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
Note: When you start a source device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
Note: When you stop a cache device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
Note: When you stop a source device in the Oracle RAC environment, you are adding
the cache to a specific node. Before adding cache devices, all RAC nodes must be
reported as Operational. You can determine the node status by using the vfcmt
display -all command.
vfcmt version
version displays the current version of the XtremSW Cache software.
The chapters in this part of the guide describe how to use the VSI plug-in to manage
XtremSW Cache in the VMware environment. Chapters include:
◆ Chapter 6, “Getting Started—VSI Plug-In,”describes the tasks required before
beginning to use the plug-in to manage and monitor XtremSW Cache as well as
first-time configuration.
◆ Chapter 7, “Managing XtremSW Cache—VSI Plug-In,” describes the tasks used to
manage XtremSW Cache.
◆ Chapter 8, “Advanced Configuration—VSI Plug-In,” describes advanced
configuration options.
This chapter describes the steps required to perform caching. Topics include:
◆ About the VSI plug-in ..................................................................................... 125
◆ Before you begin ............................................................................................ 125
◆ First-time caching on a virtual machine........................................................... 136
Note: If all of the relevant ESX hosts are using the same user and password (such as
root), you can skip this step. Use this user and password in the XtremSW ESX Agent
Authentication in the next step.
Note: When using the plug-in to manage all virtual machines from one
management station, you must use the same user name and password for all
ESX hosts.
d. To ensure enhanced security, clear the Grant shell access to this user option.
e. In the Group membership section, make this user a member of the root group.
3. Set the user permissions:
a. From the Permissions tab, double-click the new user.
b. Select a permission.
– Read-only
Enables read-only access
– Administrator
Enables read and write access, for actions such as add and remove devices
for local cache pool
The ESX Agent credential configuration is complete.
c. From the Features area, on left side of the client window, click XtremSW.
Note: If XtremSW is not displayed in the Features area, but is listed in the
Installed Features list, you may simply need to enable it. Right-click and select
Enable. If it is still not displayed, see “Installation issues” on page 166.
• User (case-sensitive)
VFCacheAdmin (default user)
• Password
The password defined for the XtremSW agent user during the installation of
the XtremSW Cache remote agent.
Note: To manage all virtual machines from one management station, you must
use the same password across all virtual machines running XtremSW Cache.
For more information, refer to the XtremSW Cache Installation Guide.
Note: To manage all virtual machines from one management station, you must
use the same user name and password across all virtual machines running
XtremSW Cache.
Note: The first time that a vCenter administrator logs in to a vSphere upon which
XtremSW Cache is installed, the XtremSW administration custom attribute is
automatically added to the schema. This step is necessary in order for XtremSW
Cache to function.
This datastore may have been previously created, during the licensing of XtremSW
Cache. If that is so, you will be directed in the procedure below what parts to skip.
The datastore must be created on a LUN that is visible to all ESX hosts in the
datacenter that may host a virtual machine with XtremSW Cache on it. The LUN does
not need to be larger than 1 GB.
Note: Before you continue to create the datastore, ensure that such a LUN exists.
Note: If this link is not displayed, the shared datastore has already been
created, and you should skip to “Task 3: Populating the ESX cache pools with
XtremSF devices” on page 132.
The Welcome dialog offers links to create the shared datastore and to populate the
local cache pools.
To create the shared datastore, perform the following:
Hint: You can do this 1. Click Create the XtremSW shared datastore (required).
task without the
Welcome dialog, too: The Create Shared Datastore dialog appears.
Select a cluster, and
click EMC VSI >
XtremSW > Cluster
details.From the
Commands panel,
click Create Shared
Datastore, then
follow the steps
below.
The first time you log in to a virtual machine or ESX, or if a certificate has changed,
you will encounter a certificate trust request. Review the certificate and click Trust
if it’s trusted.
This dialog lists the free LUNs in the datacenter. If there are LUNs detected that
are visible to all of the ESX hosts in the datacenter, then only those LUNs are
listed. If there are no such LUNs detected, then all of the free LUNs are listed.
Note: You must create the datastore on a LUN that is visible to all of the ESX hosts
in the datacenter that are running XtremSW Cache. If there are many ESX hosts,
but only one cluster with XtremSW Cache, it is adequate that the datastore LUN be
visible to the ESX hosts in that cluster.
Do not create the shared datastore on a flash device that will be used for caching.
A yellow warning icon to the left of the LUN name indicates that this LUN is not
visible to all ESX hosts, and therefore may not be a candidate for the shared
datastore.
2. Select the LUN where to create the datastore, and click Create Datastore.
3. When a message displays that indicates that the shared datastore has been
created, click Close.
The shared datastore is named XtremSW_Cache_DS. Do not change this name.
After creating the shared datastore, use the vfcmt set_license command to install your
license file. For more information, see the “Licensing” appendix in the XtremSW Cache
Installation Guide.
Continue with populating the cache pools.
After creating the shared datastore, you need to populate the ESX cache pools with
XtremSF devices.
Every XtremSW Cache-enabled ESX host comes with an empty XtremSW Cache local
pool. The cache pool is a resource from where we allocate cache capacity to the virtual
machines on an ESX. The pool is populated from the flash devices that are installed
on the ESX hosts in the cluster whose cache capacity is added to the cache pool.
The free XtremSF devices are displayed. Devices that will be populated into their
ESX hosts are checked.
When an ESX host contains multiple devices of the same vendor and model, the
amount of devices, indicated in the # of devices column, will be greater than 1.
Selecting to add those devices to the ESX pool will add all of these devices to the
ESX pool.
Adding devices to the pool erases their contents. Ensure that none of your
devices contain any data that needs to be saved.
The number of devices of a given model appears in the # of Devices column, and
the aggregate size appears in the Total Size column.
2. Click Apply to populate all the checked devices into their ESX hosts.
3. When the process finishes, click OK, then Close.
Adding devices to the pool erases their contents. Ensure that none of your
devices contain any data that needs to be saved.
4. Add devices.
If this screen does not appear, XtremSW Cache monitoring has already been
enabled.
3. The first time you log in to a virtual machine or ESX, or if a certificate has changed,
you will encounter a certificate trust request. Review the certificate and click Trust
if it’s trusted.
If you click the link and an error message appears, it is due to one of the following
reasons:
• Your user is missing the Global > set custom attribute privilege over this
specific virtual machine. This could happen if the user role is read-only on this
virtual machine.
• The custom attribute is not defined properly. For more information, refer to
Appendix C, “Troubleshooting the VSI Plug-in.”
You can monitor all virtual machines with XtremSW remote management enabled in
the ESX XtremSW > Virtual Machines screen.
The last step in the process is to enable UUID mapping. This enables the virtual
machine to locate the datastores that will be configured.
You can enable UUID data mapping with the XtremSW Cache management window or
using the vSphere client settings.
To enable UUID mapping with the XtremSW Cache management window, perform the
following steps:
1. From the tree view of the vSphere client window, select a virtual machine.
2. Select the EMC VSI tab.
3. From the Features section, click XtremSW Cache. The XtremSW Cache
management window, Figure 8 on page 137, appears.
4. In the Commands section, click Enable Disk File Mapping to Guest. If this
command does not appear in the Commands section, then it is already enabled.
5. Click OK, then reset the virtual machine. This is not necessary if the UUID mapping
was previously enabled.
To enable UUID mapping with the vSphere settings, perform the following steps:
1. Stop the virtual machine.
2. Add a configuration parameter of disk.EnableUUID=TRUE. You can set the
configuration parameters by choosing Edit Setting > Options > Advanced (General)
for your virtual machine.
3. Shut down and restart the virtual machine.
You manage and XtremSW activities from the XtremSW management window.
◆ The Status section shows the status of XtremSW Cache on the current machine.
◆ The Commands section shows context-relevant command links.
◆ The Device Configuration section shows the XtremSW Cache devices and source
devices that are added to the current machine.
Note: On RHEL 5.x machines, you may be prompted to reset the virtual machine.
Do so, and then start this process again.
2. Click Next.
6. Click OK.
Note: When you add an XtremSW Cache device, you can also enable data
deduplication.
Before enabling deduplication, it is recommended to review “Configuring data
deduplication” on page 159.
Note: In Oracle RAC environments, you must use the CLI to add source devices. For
more information, see “Adding or removing source devices” on page 70.
This chapter describes how to manage the XtremSW Cache feature. Topics include:
◆ Managing XtremSW Cache pools .................................................................... 143
◆ Managing XtremSW Cache devices ................................................................. 148
◆ Viewing virtual machines................................................................................ 156
3. Click OK.
The added device is now shown in this ESX’s local cache pool.
Note: You remove devices from the XtremSW Cache pool from a specific vendor and
model unit; when you remove one device from that vendor, all of the same devices
from that vendor are removed.
To remove a flash device from the local cache pool, perform the following:
1. Remove all XtremSW Cache devices from virtual machines on the ESX host:
a. From the navigation pane, select the ESX host.
All guest virtual machines appear in the Virtual Machines screen, as illustrated
in the following figure:
b. On every virtual machine that has 1 or more Cache Devices, remove these
cache devices (from the virtual machines EMC VSI tab).
2. From the navigation pane, select the ESX host.
3. Select Cache Pool.
4. In the Cache Pool section, select the device to remove, then click Remove Device.
5. Confirm the removal by clicking OK.
6. When the process is complete, click Close.
The removed flash device is no longer displayed in the pool.
To display aggregated cache pool information for a whole cluster, perform the
following:
The Local Cache Pool Summary section displays the total capacity, available
capacity, and the number of cache devices defined in all of the ESX hosts in the
cluster combined.
Hint: To quickly open The ESX Local Cache Pools section displays the total capacity, available capacity,
the ESX in the
vSphere client, click and number of cache devices defined per ESX.
its name. If there is
an error state, click You can use the Commands panel to perform the following actions:
anywhere else to see
the error message ◆ Refresh the displayed data.
displayed in the
Details section. ◆ Populate the XtremSW local cache pools with XtremSF devices. For more
information, see “Task 3: Populating the ESX cache pools with XtremSF devices”
on page 132.
◆ Create the shared datastore. This link is only visible if the datastore hasn’t
already been created. For more information, see “Task 2: Creating the
XtremSW shared datastore” on page 129.
Note: When RAC-controlled source devices are attached to the cache device, you
must stop them first using the CLI. For more information, see “Starting or stopping
the caching of a source device” on page 73.
Note: To change deduplication parameters, delete the cache device and create it
again with the new deduplication parameters.
Removing devices
You can remove a cache or source device.
Note: In Oracle RAC environments, you must use the CLI to remove source devices. For
more information, see “Adding or removing source devices” on page 70.
To remove a source device, select the device, click Remove, and then confirm.
To remove a cache device, perform the following:
1. Remove all attached source devices.
2. Select the cache device.
3. Click Remove, and then confirm.
4. You are prompted to remove the created RDM device.If you are not expecting to
use it for future caching, you can remove it.
Displaying statistics
You can display cache device and source device statistics.
To display statistics, select the device and click the Statistics link for that device.
Cache Size and Utilization (only for cache • Cache Size (pages)
devices) total cache size, in pages of 8 kb
• Pages used
number of available cache pages
that are being used, expressed
as a percentage
• Write Hits
amount of write requests that were found in the cache
• Write Misses
amount of write requests that were not found in cache
• Writes Pending
amount of write requests waiting for completion of outstanding IOs
• Reads Received
total amount of read requests received
• Writes Received
total amount of write requests received
• I/Os Received
total amount of IOs received
• Unaligned I/Os (source device only)
number of IOs that are unaligned to the VFC cache page
• Partial I/Os
total amount of IOs that were smaller than the cache page size
• I/Os Outstanding
total amount of IOs in progress
• Skipped I/Os (source device only)
number of IOs that are not cached, either because they are larger than
the max I/O size or because the cache device is in passthru mode
• Evictions
number of pages that have been replaced with other data
• Cache Pages
total pages in cache
• Cache Pages Used
total used pages in cache
• Down Reads (source device only)
number of read IOs that are sent to the source device
• Down Writes (source device only)
number of write IOs that are sent to the source device
• Read Hits
amount of read requests that were found in the cache
• Read Misses
amount of read requests that were not found in cache
• Reads Received
total amount of read requests received
• I/Os Received
total amount of IOs received
• Down Reads
number of actual reads to the flash device
• Down Writes
number of actual writes to the flash device
• Validate Hits
number of XtremSW Cache hits that exist in the deduplication cache
• Validate Misses
number of VFC cache hits not found in the deduplication cache
because they have been evicted
• Evictions
number of pages that have been replaced with other data
• Evicted During Read
number of deduplication entries evicted while there are reads in
progress for these entries
• Evicted During Write
number of deduplication entries evicted while there are writes in
progress for these entries
• Entries Number
number of deduplication cache entries
• Occupied Entries Number
number of occupied deduplication cache entries
Note: In many cases, the statistics in the cache device and the source device will not
match.
The XtremSF Devices screen displays the following information for each attached
XtremSF device:
Parameter Description
Parameter Description
The Virtual Machines screen displays the following information for each virtual
machine:
Parameter Description
This chapter explains advanced XtremSW Cache configuration topics. Topics include:
◆ Configuring data deduplication ...................................................................... 159
◆ Changing the maximum I/O size..................................................................... 161
◆ Changing the page size .................................................................................. 162
You can view the Deduplication Hit Ratio in the Key Statistics window. This ratio,
calculated by the dividing the amount of deduplication write hits by the deduplication
writes received, shows the amount of duplicate data in the cache.
You can view other deduplication statistics that appear on the right side of the Expert
View window.
To determine the best use of deduplication in your environment, perform the following
steps:
1. Add an XtremSW Cache device, as described in “Creating an XtremSW Cache
device” on page 137.
a. In the Add XtremSW Cache Device dialog, select Use Data Deduplication.
Leave the slider at the default value, 20%.
b. Set maximum I/O and page size.
c. Click OK.
2. Add a source device to this XtremSW Cache device, described in “Starting source
device acceleration” on page 141.
3. After the cache is warm (after letting it run through a typical business cycle), from
the Device Configuration area of the XtremSW Cache management window, select
the cache device and click Statistics.
4. In the Key Statistics window, view the Deduplication Hit Ratio.
Hint: To benefit from • If the Deduplication Hits Ratio is less than 10%, it is recommended to turn off
extended device life,
keep deduplication deduplication.
enabled, and set the
Deduplication Gain • If the Deduplication Hits Ratio is over 35%, it is recommended to raise the
to 0%. deduplication gain to match the observed deduplication.
• If the Deduplication Hits Ratio is between 10% - 35%, it is recommended to
leave the deduplication gain as is.
5. To change the configured gain, remove the XtremSW Cache device, and recreate a
new one using the Add XtremSW Cache device command, as described in
“Creating an XtremSW Cache device” on page 137.
Note: Changing the deduplication gain on an XtremSW Cache device requires that you
first stop all attached source devices, remove the XtremSW Cache device, and then
add a new XtremSW Cache device. This will result in starting with a cold cache.
Note: If you backup to and from the source device, IOs are likely to be larger than
the default. You do not want to cache the backup IOs, so in this case, it is
recommended to allow more than 25% Exceeds MaxIO and not to increase the
max I/O.
5. To change the maximum I/O, select the cache device, and modify this parameter,
as described in “Modifying an XtremSW Cache device” on page 149.
The chapters in this part of the guide describe how to use the XtremSW Management
Center to manage XtremSW Cache. Chapters include:
◆ Chapter 9, “Getting Started—XtremSW Management Center,” describes how to log
in and begin using the XtremSW Management Center.
◆ Chapter 10, “Monitoring and Managing Systems,” describes how to use the
Management Center to view and manage systems.
◆ Chapter 11, “Managing Cache and Source Devices,” describes how to set up
caching operations, and how to manage cache and source devices.
◆ Chapter 12, “Configuring XtremSW Management Center Settings,”describes how
to configure Management Center settings.
◆ Appendix B, “Servicing the XtremSW Management Center,”describes how to
perform service activities.
◆ Appendix C, “XtremSW Management Center REST API,”describes how to use the
REST API.
This section describes how to use the XtremSW Management Center to manage
multiple systems. If you only have one or two systems, you may want to use the
XtremSW Lite Client. For more information about the Lite Client, see Part 5, “Using the
XtremSW Lite Client.”
The XtremSW Management Center provides all the functionality of the Lite Client. In
addition, it retains machine history and enables you to manage multiple machines,
physical, virtual, and ESX hosts from a single view.
VNX users can benefit from the integration of the management center with Unisphere
remote for VNX. For VNX LUNs being accelerated by XtremSW Cache, this integration
simplifies the monitoring of the cache performance by displaying the information
directly on the Unisphere Remote management screens. In addition, you can see the
health information of XtremSF flash cards that are managed by the Management
Center from Unisphere Remote.
To enable this integration, register the XtremSW Management Center in the Unisphere
Remote, by providing the IP and credentials of the Management Center. The
registration is from the Settings > Management Settings > Network Menu tab.
CHAPTER 9
Getting Started—XtremSW Management
Center
This chapter introduces the XtremSW Management Center and the Dashboard, the
main management console. Topics include:
◆ Logging in ...................................................................................................... 165
◆ The Dashboard............................................................................................... 166
Logging in
This section describes how to log in to the XtremSW Management Center.
Communication between the Management Center and managed systems uses the
CIM/XML protocol over HTTPS on the port 5989.
1. To log in, enter the IP address of the XtremSW Management Center in any browser
window, as follows:
https://<IP_address>
The login screen appears.
Note: The first time you attempt to log in, you may encounter a security certificate
warning. You can proceed past this warning.
Note: On first login, you are prompted to change the default password.
• Domain
The LDAP domain. Required if the user you are logging in with is a member of
an LDAP domain.
3. Click Login.
Upon successfully logging in, the XtremSW Management Center Dashboard appears.
In this document, we’ll refer to it as the Dashboard.
The Dashboard, described in the following section, is the center of your management
activities.
The menu bar is displayed at the top of every XtremSW Management Center window.
You can use the menu bar to access the main functions of the Management Center:
◆ “Viewing and managing managed systems” on page 169
◆ “Viewing XtremSF cards” on page 179
◆ “Viewing alerts” on page 180
◆ “Configuring XtremSW Management Center Settings” on page 199
These tasks are described in the following chapters.
The Dashboard
This section describes how to use the Dashboard.
The Dashboard displays a system view of activities that are relevant to your managed
systems. Managed systems are physical machines, virtual machines, or ESX hosts
upon which XtremSW Cache is running. You add managed systems from the Managed
Systems window, described in “Viewing and managing managed systems” on
page 169. Until you add managed systems, the Dashboard is empty.
This chapter describes how to use the XtremSW Management Center to monitor and
manage your systems. Topics include:
◆ Viewing and managing managed systems ...................................................... 169
◆ Viewing XtremSF cards ................................................................................... 179
◆ Viewing alerts ................................................................................................ 180
◆ Viewing audit log messages ........................................................................... 181
The XtremSW Management Center gives you a single pane-of-glass to all systems that
you want to manage. To manage systems, you must first add them to the Management
Center.
You can add physical servers, ESX hosts, and virtual machines that are running
XtremSW Cache or in which there are XtremSF cards installed.
Note: To add a system, it must have the XtremSW Cache remote agent installed, and
its authentication details must have been set. These steps are typically performed
during product installation.
To enter the Managed Systems window, click Managed Systems in the menu. The
Managed Systems window appears.
From the Managed Systems window, you can view and manage systems that are
running XtremSW Cache.
When you first open the Managed Systems window, it will be blank. The next section
describes how to add machines.
Note: If you have applied filters, a new system that does not meet the filter criteria
will not be displayed.
The following figure illustrates the Managed Systems window populated with various
systems:
The following table describes the parameters shown for each managed system. To
display the advanced settings, select Show Advanced Parameters.
XtremSW Cache Usage and Total cache size and how much source device Standard
Allocation space is being cached
XtremSF Card Detected Yes or blank (card presence is only detected Standard
if the XtremSW Cache remote agent is
installed on the machine)
XtremSW Cache Installed Yes or blank (only shows if the XtremSW Standard
Cache remote agent is installed on the
machine)
Remote Agent Version The version of the XtremSW Cache remote Advanced
agent
Last Sample Time The last time the machine was sampled Advanced
successfully
Certificate Expiry Date The date on which the current certificate Advanced
expires
Filter Description
Filter Description
These filters are common throughout the other XtremSW Management Center
windows.
The status bar at the bottom left of the managed systems window gives you a quick
summary of the status of the systems that are displayed, according to the symbols in
the previous table.
You can also filter which systems are displayed by choosing one of the filters in the All
Systems pane. Choose from:
◆ Physical systems
◆ VMware systems, of:
• VMware Clusters
• Non-Clustered ESX Servers
• Other Virtual Machines
Note: To filter VMware systems by clusters or ESX servers, you must first register a
vCenter. For more information, see “Registering a vCenter and adding a shared
datastore” on page 208.
For example, ESX hosts and physical servers have physical devices installed, whereas
virtual machines do not. In the former, you can access card-related activities, which
do not appear for virtual machines.
On the other hand, on virtual machines and physical servers, you can perform
activities related to XtremSW Cache devices, such as adding devices, which would not
be relevant to ESX hosts.
To view machine details, from the Managed Systems window, click the machine
hostname or IP address, as illustrated in the following figure:
The following figure illustrates the machine details view for the physical server
selected in the previous figure:
The following table lists the sections and tabs that appear for each kind of managed
system. The actions that can be performed from the various tabs are described as
parts of specific tasks later in this guide.
After drilling down to a machine’s details, you can return to the Managed Systems
window by clicking Managed Systems in the menu bar, or by clicking Managed
Systems in the bread crumb menu:
Note: When selecting a system, select it from anywhere other than its host name/IP.
This selects the entire line.
Note: When selecting a system, select it from anywhere other than its host
name/IP. This selects the entire line.
Note: When selecting a system, select it from anywhere other than its host
name/IP. This selects the entire line.
Once you confirm the certificate update, your certificate will be invalid, and you will
not be able to communicate with the managed system until you confirm a new
certificate.
In the top part of the screen, the following aggregate information is displayed:
◆ Total number of cards
◆ Total capacity (GB) of the cards
◆ The number and ratio of cards per temperature rating
◆ The number and ratio of cards per life expectancy rating
In the bottom part of the screen, the following information is displayed, per card:
Parameter Description
Viewing alerts
The Alerts window displays information on all alerts in the managed systems.
To open the Alerts window, click Alerts from the main menu.
Parameter Description
Parameter Description
Managed System The managed system where the action was performed
This section describes how to start caching services, and to manage XtremSW Cache
devices and source devices. Topics include:
◆ Understanding the caching process................................................................ 183
◆ First-time caching........................................................................................... 185
◆ Managing XtremSW Cache pools .................................................................... 191
◆ Managing cache devices ................................................................................ 196
◆ Managing source devices ............................................................................... 196
Note: Before you continue to create the datastore, ensure that such a LUN exists.
To add a vCenter and a shared datastore, refer to “Registering a vCenter and adding a
shared datastore” on page 208.
After creating the shared datastore, use the vfcmt set_license command to install your
license file. For more information, see the “Licensing” appendix in the XtremSW Cache
Installation Guide.
First-time caching
To configure caching, you need to perform the following steps:
◆ “Task 1: Adding a managed system” on page 186
◆ “Task 2: Adding a cache device” on page 186
◆ “Task 3: Adding a source device” on page 189
You add XtremSW Cache devices that will provide acceleration to source devices
(LUNs). The process of adding cache devices is very similar in the physical and virtual
environment, with the following exception:
In the VMware environment, when you create a cache device, the following activities
go on in the background:
◆ A search is performed to find an available RDM device that is connected to the
cache pool. If one is not detected, it will be created.
◆ A scan is performed to identify the RDM device.
◆ A search is performed to find a free “cache-pool” LUN to which to connect the
RDM device. If no LUN is found, a new one will be added.
◆ A new SCSI controller is created for the RDM device. If all the controllers (up to 4)
already exist, the device will be added to an existing controller and a warning will
be displayed that you might not get the best performance as the cache device is
not on a dedicated controller.
These additional activities result in an additional step when adding a cache device, as
described in the following section.
This dialog box In the VMware environment, the Create Cache Device dialog appears.
is relevant only
for virtual
machines
XtremSW Cache searches for an available RDM device through which to create the
link for caching services. When a device is found, the Click Next to configure the
cache message appears.
Note: On RHEL 5.x machines, you may be prompted to reset the virtual machine.
Do so, and then start this process again.
Note: When you add an XtremSW Cache device, you can also enable data
deduplication. Before enabling deduplication, it is recommended to review “Data
deduplication” on page 246.
Note: In Oracle RAC environments, you must use the CLI to add source devices. For
more information, see “Adding or removing source devices” on page 70.
1. From the Managed Systems tab, drill down to the machine on which the XtremSW
Cache device is located, by selecting the IP or host name.
The machine details view appears.
2. Select the LUNs (potential sources) tab.
3. Select source devices to add, then click Add (attach) LUNs to Cache. You can
select multiple LUNs.
4. Select the cache device to use for acceleration of the selected LUNs, then click OK.
If you accepted the default setting, the selected source devices are now being
accelerated, and appear as started in the LUNS (potential sources) tab.
To view all source devices that are being accelerated by a cache device, perform the
following:
1. Drill down to the machine details view, and click the XtremSW Cache Devices tab.
2. Select the cache device, and click Show All Cached LUNs.
The cache pool details appear in the top section of the screen.
2. Select the XtremSW Cache Pool Cards tab and click Add Device.
In the Add Flash Disk to XtremSW Cache Pool dialog, the free XtremSF devices are
displayed.
To view non-XtremSF devices, clear the Show only XtremSF devices option.
When an ESX host contains multiple devices of the same vendor and model, the
amount of devices, indicated in the Number of Devices column, will be greater
than 1. Selecting to add those devices to the ESX pool will add all of these devices
to the ESX pool. For more information, see “Enabling use of disks for DAS” on
page 28.
Adding devices to the pool erases their contents. Ensure that none of your
devices contain any data that needs to be saved.
3. Select the devices to add to the local cache pool, and click OK.
4. When the devices have been added to the pool, click Close.
Note: You remove devices from the XtremSW Cache pool from a specific vendor and
model unit; when you remove one device from that vendor, all of the same devices
from that vendor are removed.
To remove a flash device from the local cache pool, perform the following:
1. Remove all XtremSW Cache devices from virtual machines on the ESX host:
a. From the Managed Systems tab, locate all registered virtual machines hosted
by the ESX by filtering by the ESX host.
The following figure illustrates the ESX host in the Managed Systems window:
b. In the left pane, filter by the ESX machine. The hosted virtual machines appear.
The virtual machines that are indicated as accelerating are those with
XtremSW Cache devices,
c. Drill down to the machine details for each virtual machine, then from the
XtremSW Cache tab select the cache device and click Remove Cache.
d. Confirm the removal.
When you remove a cache device, all attached source devices are also removed.
Note: To remove a flash device to which there are RAC source devices attached,
you must first remove the source devices using the CLI, as described in “Adding or
removing source devices” on page 70.
Note: In Oracle RAC environments, you must use the CLI to manage source devices.
For more information, see “Managing the cache” on page 69.
The actions in this section are identical to those performed in the XtremSW Lite Client
and are described there. Where there are differences, they are noted.
You can perform the following source device management actions:
◆ “Adding a LUN to a cache device” on page 252
◆ “Removing a LUN from a cache device” on page 252
◆ “Starting and stopping LUN acceleration” on page 253
The actions in this section are identical to those performed in the XtremSW Lite Client
and are described there. Where there are differences, they are noted.
You can view the following statistics and performance details:
◆ “Viewing current accelerated LUNs performance” on page 256
◆ “Viewing performance statistics” on page 257
◆ “Viewing source device raw counters” on page 258
This chapter describes how to configure the XtremSW Management Center settings.
Topics include:
◆ Management settings..................................................................................... 199
◆ Managing users ............................................................................................. 201
◆ vCenter settings ............................................................................................. 207
◆ Viewing system log messages ........................................................................ 210
Management settings
To configure management settings, choose Settings > Management Settings from the
main menu.
◆ Gateway
The default gateway for the server
DNS servers
◆ Domain Name
The name of the domain the DNS server
◆ IP address
The IP address of the DNS server serving the domain
◆ Add
Add a new DNS (by IP address) to the DNS servers list
◆ Remove
Remove a DNS from the DNS servers list
◆ Move up
Move the DNS server down in the list. When performing DNS queries, the DNS
servers will be asked sequentially, in the order of the list, until one of them
answers. If a DNS server answers that a record cannot be found, the other DNS
servers will not be questioned.
◆ Move down
Move the DNS server up in the list.
Managing users
To manage users, choose Settings > User Management from the main menu. Only
users with the administrator role can access this window.
Parameter Description
Type The user type: Local user, LDAP user, or LDAP group
Adding users
You can define additional users, with various roles. You might want to define users
that do not have administrative rights.
You can use the User Management window to add users.
To add users, perform the following:
1. From the User Management window, click Add User. The User wizard appears.
2. Choose from the available types of users, then click Next.
• Local
User that is defined internally
• LDAP User
User that is defined in the LDAP server
• LDAP Group
An LDAP group that is defined in the LDAP server
3. Enter the User name, password, and password verification, according to the
following guidelines, then click Next.
• Passwords must be 9 to 40 characters in length and cannot contain spaces or
a single quote.
• Passwords must include mixed case, a number, and a special character from
this list: ! , @ # $ % ^ * ? _ ~
• When changing a password, you cannot reuse your last password.
4. Specify the role for the user, then click Next:
• Monitor
View configuration only
• Operator
View and change caching configuration of the managed systems
• Administrator
View and change configuration, and manage the Management Center: register
managed systems, manage users, perform maintenance tasks
5. Click Finish to create the user.
Removing a user
You use the User Management window to remove a defined user or group.
Note: The default user, admin, cannot be removed, though you can change its role.
• User Settings
– User ID Attribute
LDAP user object attribute that specifies the user ID (for example, in Active
Directory it is samaccountname).
– User Object Class
LDAP object describing a user
– User Search Path
The base LDAP path from which the LDAP user is searched. If you have a
very large LDAP tree, and all of the XtremSW Management Center users are
in a specific part of the tree, you may want to change this value.
• Group Settings
– Group Member Attribute
LDAP group object attribute that specifies the list of group members
– Group Name Attribute
LDAP group object attribute that specifies the group name
– Group Object Class
LDAP object that describes a group
– Group Search Path
The base LDAP path from which the LDAP group is searched. If you have a
very large LDAP tree, and all of the XtremSW Management Center groups
are in a specific part of the tree, you may want to change this value.
Note: If you remove an LDAP server, and then add another server for the same LDAP
domain, the user and group permissions for this LDAP domain will return.
vCenter settings
To manage vCenters, choose Settings > vCenter Settings from the main menu.
The vCenter Settings window appears.
You can use the vCenter Settings window to perform the following tasks:
◆ “Registering a vCenter and adding a shared datastore” on page 208
◆ “Editing vCenter settings” on page 209
Note: Before you continue to create the datastore, ensure that such a LUN exists.
9. Expand a cluster and select an ESX upon where the datastore should be created
and click Next.
Note: Do not create the shared datastore on a flash device that will be used for
caching.
The datastore must be created on a LUN that is visible to all ESX hosts in the
vCenter that may host a virtual machine with XtremSW Cache on it.
10. Select a LUN and click Next.
11. Confirm the shared datastore details and click Finish.
Upon successful completion, the message The datastore was created successfully
appears.
Removing a vCenter
To remove a vCenter, perform the following:
1. Select the vCenter, then click Remove vCenter.
2. Confirm the deletion.
Once you confirm the certificate update, your certificate will be invalid, and you will
not be able to communicate with the managed system until you confirm a new
certificate.
Parameter Description
This appendix describes how to service the XtremSW Management Center. Topics
include:
◆ Accessing the service menu ........................................................................... 211
◆ Performing service activities........................................................................... 212
This appendix describes the REST API exposed by the XtremSW Management Center.
You can use this API to monitor XtremSW Cache-enabled systems as well as their
installed XtremSF flash devices.
Topics include:
◆ Before you begin ............................................................................................ 215
◆ Monitoring XtremSF devices ........................................................................... 216
◆ Monitoring XtremSW cache devices ................................................................ 218
◆ Data types and enumerations......................................................................... 232
◆ Supported URI patterns .................................................................................. 232
The XtremSW management Center Rest API uses JSON encoding and adheres to EMC's
EDAA style conventions for all read operations.
Description String
Name String
Capacity Long
wwn String
linkSpeed Int
Relationships
partOfStoragePool StoragePool
FlashDiskMeasurement
The FlashDiskMeasurement has a reference to its FlashDevice object, as well as to any
alerts that the FlashDeviceMeasurement triggered. Use this class to monitor the
amount of wear on a device, its temperature, etc.
Example alerts include device approaching end-of-life” and device temperature above
threshold. The FlashDeviceMeasurement refers to its FlashDevice object, as well as to
any alerts that the FlashDeviceMeasurement triggered.
Properties
Relationships
Properties
alertParams Map
Relationships
Cache
The cache class contains information about an XtremSW cache device.
Properties
revision Int
Relationships
Relationships
CacheMeasurement
The CacheMeasurement class represents a performance measurement for a cache. A
measurement contains performance counters for a cache accumulated over a period
of time. If the cache serves multiple LUNs data exposed by the CacheMeasurement
will reflect all of them. Use CachingServiceMeasurement.
Properties
dedupData CachingMeasurementDedupData
evictions long
partialIos long
CachingMeasurementDedupData
The CachingMeasurementDedupData class contains performance data for the
deduplication mechanism of a XtremSW Cache or cachingServiceForSource. The
measurement period for this data can be retrieved from the corresponding
CacheMeasurement or CachingServiceforSourceMeasurement.
Properties
deDupReadHits Long
deDupReadMisses Long
deDupWriteHits Long
deDupWriteMisses Long
deDupIOsReceived Long
deDupReadsReceived Long
deDupDownReads Long
deDupDownWrites Long
deDupIOErrors Long
deDupWritesReceived Long
deDupEntries Long
deDupOccupiedEntries Long
deDupEvictions Long
deDupEvictedDuringRead Long
deDupEvictedDuringWrite Long
deDupEvictionFailed Long
deDupSignatureErrors Long
deDupCacheIsPurgedErrors Long
deDupValidateMisses Long
deDupValidateHits Long
CachingServiceForSource
The CachingServiceForSource class represents the XtremSW cache service for a single
cached LUN
Properties.
revision Int
description String
Relationship
hostedOnComputerSystem ComputerSystem
cachingServiceForSource CachingServiceForSource
Measurements Measurement
latestMeasurement CachingServiceForSource
Measurement
usesCache Cache
cachingServiceForSourceAlerts CachingServiceForSourceAlert
cachesLogicalDisk: LogicalDisk
CachingServiceForSourceMeasurement
The CachingServiceForSourceMeasurement class contains performance data about
the XtremSW cache service provided to a single cached LUN.
Properties
readsReceived Long
writeHits Long
writeMisses Long
writeReceived Long
iosReceived Long
dedupData CachingMeasurement
DedupData
unalignedIos Long
skippedIos Long
downReads Long
downWrites Long
totalReadLatency Long
totalWriteLatency Long
totalReadSize Long
totalWriteSize Long
alertType CachingServiceForSourceAlertType
severity Severity
Relationships
computerSystem ComputerSystem
PhysicalDisk
Properties
description String
name String
capacity Long
wwn String
Relationship
hostedOnComputerSystem ComputerSystem
partOfStoragePool StoragePool
StoragePool
Properties
logicalId String
description: String
name: String
Relationships
logicalDisks LogicalDisk
LogicalDisk
The LogicalDisk class contains data about a local logical disk on a computer system
Properties.
Relationships
flashDiskAlerts FlashDiskAlert
flashDisks FlashDisk
storagePools StoragePool
cachingServicesForSources CachingServiceForSource
logicalDisks LogicalDisk
physicalDisks PhysicalDisk
cacheAlerts CacheAlert
cachingServicesForSourceAlerts CachingServiceForSourceAlert
caches Cache
Properties
Relationships
storagePools StoragePool
cachingServicesForSources CachingServiceForSource
logicalDisks LogicalDisk
cacheAlerts CacheAlert
cachingServicesForSourceAlerts CachingServiceForSourceAlert
guestOnHostComputerSystem HostComputerSystem
Relationships
flashDiskAlerts FlashDiskAlert
flashDisks FlashDisk
storagePools StoragePool
physicalDisks PhysicalDisk
cacheAlerts CacheAlert
hostedComputerSystems GuestComputerSystem
FlashDiskAlertType { "DeviceTemperatureAboveThermalLimitTemperature",
"DeviceTemperatureAboveShutdownTemperature",
"EndOfLifeNear", "EndOfLifeReached", "WriteProtectIsHigh",
"AllDeviceIsWriteProtect", "FirmwareNotSupported" }
CacheAlertType { "CacheFailed" }
CachingServiceForSource { "AllReadsAreSkipped",
AlertType "WriteReadProportionAboveThreshold" }
HTTP
meth
URI Pattern Example ods Comments
/api/instances/{id} • /api/instances/{id}/api/ GET Get one instance. Note that the MSA
instances/Cache::12341234 {id} includes the
• /api/instances/ type_name::instance_id for our
Cache::12341234?expand=* objects.
By default only the id will be provided
for related objects. You may use
expand=* in order to get the related
objects in-place. This may reduces the
number of requests needed, but will
cause larger responses. Note that only
direct relationships of the requested
object will be expanded.
The chapters in this part describe how to use the XtremSW Lite Client to manage
XtremSW Cache instances. Chapters include:
◆ Chapter 13, “Getting Started—XtremSW Lite Client,” describes how to log in and
the basic functions of the XtremSW Lite Client.
◆ Chapter 14, “Configuring the XtremSW Lite Client,” describes how to configure
and manage XtremSW Cache instances.
◆ Chapter 15, “Viewing Statistics and System Information,” describes how to view
performance statistics and system data for XtremSW Cache instances.
◆ Appendix D, “Troubleshooting,” describes how to solve issues that may arise.
CHAPTER 13
Getting Started—XtremSW Lite Client
Overview
You can use the XtremSW Lite Client to manage a single system at one time. To
manage multiple systems, either open multiple instances of the Lite Client, or use the
XtremSW Management Center. For more information about the Management Center,
see Part 4, “Using the XtremSW Management Center.”
Hint: To manage and You can use the XtremSW Lite Client to view and manage a physical machine or to
view VMware
environments, it is view a virtual machine or ESX host that is running XtremSW Cache. If that machine has
recommended to use a cache card installed, you can also monitor that card.
the VSI plug-in.
Communication between the Lite Client and managed systems uses the CIM/XML
protocol over HTTPS on the port 5989.
Logging in
This section describes how to log in to the XtremSW Lite Client.
Note: To manage machines, the XtremSW Cache remote agent must be installed on
the machine, and authentication credentials must be defined. If these steps were not
completed during installation, refer to the “Troubleshooting” section of the XtremSW
Cache Installation Guide.
1. Start the XtremSW Lite Client program. The login screen appears.
Note: If the Lite Client desktop icon does not appear, the default installation
folder is C:\Program Files (x86)\EMC XtremSW Lite Client.
2. Enter the login credentials and click Login. All values are required.
• User (case-sensitive)
Initial user name is VFCacheAdmin
• Password
The password for user VFCacheAdmin, set during the installation process or by
using the setcredentials script.
• IP
The IP address of a machine upon which XtremSW Cache is running.
Note: You cannot use localhost as the IP, even if the Lite Client is installed on
the managed system.
3. A security certificate warning appears. Review the certificate and, if it’s OK, click
Approve.
Upon successful authentication, the XtremSW Lite Client Managed System window
opens.
The following figure illustrates a physical machine with one XtremSF card installed.
You use the Managed System window to view and to manage systems.
Logging in 239
Getting Started—XtremSW Lite Client
This chapter describes how to use the XtremSW Lite Client to view and manage your
system. Topics include:
◆ First-time configuration .................................................................................. 241
◆ Managing cache devices ................................................................................ 246
◆ Managing source devices ............................................................................... 252
First-time configuration
This section describes how to begin using the XtremSW Lite Client to manage caching.
You configure caching from the Managed System window.
When you first open the Managed Systems window, it shows you information about
the machine into which you are logged in as well as about installed XtremSF cards.
However, if no XtremSW Cache devices have been added, and no source devices
(LUNs) are being accelerated, there will be no performance activity.
First, you need to add a cache device and then you can start acceleration.
3. Choose a flash device to use for the caching. To show non-XtremSF devices, select
Show all devices.
4. Set the Maximum I/O request size to cache by sliding the bar to the desired
position, between 32-128KB. The default value is 64KB (in Windows 128KB).
For more information on this option, see “Maximum I/O size” on page 248.
5. Set the Requested page size by sliding the bar to the desired position, between
8-64KB. You can choose 8, 16, 32, and 64KB. The default value is 8KB (in
Windows, 64KB).
For more information on this option, see “Page size” on page 249.
6. To start the cache device automatically, select Start cache automatically. This
option is selected by default.
7. To enable deduplication, select Use deduplication. For more information on using
deduplication, and how to determine the optimal gain percentage, see “Data
deduplication” on page 246.
8. Click OK.
The new device is added and is displayed in the XtremSW Cache Devices tab.
Note: In Oracle RAC environments, you must use the CLI to add source devices. For
more information, see “Adding or removing source devices” on page 70.
You manage source devices from the LUNs (potential sources) tab.
On Linux machines, when MultiPath devices are present, only MultiPath devices are
displayed. You can select the Show all LUNs (potential sources) option to display
other source devices, as well as MultiPath “dummy” devices. You should not
accelerate the dummy devices.
Parameter Description
Cached On The cache device to which this source has been attached
VMAX Read Full Track Shows whether a VMAX source device has the read full track
option enabled
2. Select one or more LUNs, and click Add (attach) LUNs to Cache.
The Add Selected LUNs to Cache dialog appears.
3. Select a cache device to which to attach the source device. You can scroll over to
see the properties of available cache devices.
4. Select Start acceleration after adding LUNs. This option is selected by default.
5. Click OK.
The source device is now attached to the selected cache device, and acceleration of
the source device begins.
Initial configuration is now complete!
Now that you have started acceleration for a source device, you can view the
performance statistics.
Data deduplication
Data deduplication eliminates redundant data by storing only a single copy of
identical chunks of data, while at the same time, providing access to the data from the
cache.
XtremSW Cache enables you to take advantage of the benefits of data deduplication,
such as reduced storage and bandwidth requirements and extended cache device life
expectancy.
In general, deduplication is most likely to benefit environments where unstructured
data, such as files, is being cached. Deduplication will not be helpful when the data
being cached contains internal signatures, such as database blocks (for example,
Oracle and SWQL Server).
Deduplication is enabled per cache device, so the requests from all applications using
the enabled XtremSW Cache device will be deduplicated.
By default, XtremSW Cache deduplication is disabled.
When enabling deduplication, you set a deduplication ratio (default 20%) that
represents a percentage of the amount of data in your cache that you expect is
duplicated. Deduplication uses RAM, in proportion to this deduplication ratio.
Before you set deduplication, it is recommended to determine the optimum use for
your environment.
To determine the optimum use of deduplication in your environment, perform the
following steps:
1. Add an XtremSW Cache device, as described in the section below:
• For Lite Client
“Adding a cache device” on page 242, and select Use deduplication.
• For Management Center
“Adding a cache device” on page 186, and select Use deduplication.
Note: You can also perform this test on an existing cache device by editing the
existing cache device. For more information, refer to “Editing cache device
properties” on page 251.
3. After the cache is warm (after letting it run through a typical business cycle), use
the data in the Counters tab to calculate the observed deduplication hit ratio, and
compare it to the configured ratio.
• Calculate the observed deduplication hit ratio by dividing the Write Hits by the
Writes Received. This is the amount of duplicate data in the cache.
Hint: To benefit from – If the observed ratio is less than 10%, it is recommended to turn off
extended cache
device life, instead of deduplication.
turning
deduplication off, – If the observed ratio is over 35%, it is recommended to raise the
enable deduplication deduplication gain to match the observed deduplication.
with a gain of 0%.
– If the observed ratio is between 10%
35%, it is recommended to leave the deduplication gain as is.
To change the configured ratio, edit the cache device properties, as described in
“Editing cache device properties” on page 251, or add the cache device from the
start.
1. Add an XtremSW Cache device and allow it to run throughout a typical business
cycle. This enables the cache to warm up.
2. Open the Performance tab for the accelerated source device. For more
information, see “Viewing performance statistics” on page 257.
3. If the percentage of Exceeds MaxIO is higher than 25%, you could benefit from
increasing the maximum I/O size.
Note: If you back up to and from the flash device, IOs are likely to be larger than
the default. You do not want to cache the backup IOs, so in this case, it is
recommended to allow more than 25% Exceeds MaxIO and not to increase the
max I/O.
To edit the maximum I/O size, edit the cache device properties, as described in
“Editing cache device properties” on page 251.
Page size
This section describes how to configure the cache page size.
By default, the page size is 8KB (in Windows, 64KB). This size is correct for most
applications. For some applications, however, caching will be more effective by
increasing the page size, to match the application block size.
You can set the page size when adding a new cache device, and you can change the
page size after the device has already been added.
To add a cache device, see the steps described in:
◆ Lite Client
“Adding a cache device” on page 242.
◆ Management Center
“Adding a cache device” on page 186.
To set the page size for an existing cache device, see “Editing cache device
properties” on page 251.
Note: In Oracle RAC environments, you must use the CLI to remove source devices
before you remove a cache device to which the source device is attached. For more
information, see “Adding or removing source devices” on page 70.
When you stop caching services, attached source devices are also stopped. When
caching is restarted, it will start with a cold cache.
Note: In Oracle RAC environments, you must use the CLI to remove source devices
before you stop a cache device to which the source device is attached. For more
information, see “Adding or removing source devices” on page 70.
To start or stop caching service on a cache device, perform the following steps:
1. From the XtremSW Cache Devices tab, select the cache device and then click Start
Cache or Stop Cache.
Note: The Start Cache button is not available unless the cache device is stopped.
Stopping caching services will result in a cold cache when it is restarted.
Note: If the Edit Cache option is not available, the cache has not been stopped.
Select the cache, click Stop Cache, confirm, then click Edit Cache.
Note: In Oracle RAC environments, you must use the CLI to remove source devices
before you disable caching. For more information, see “Adding or removing source
devices” on page 70.
Note: In Oracle RAC environments, you must use the CLI to manage source devices.
For more information, see “Managing the cache” on page 69.
This chapter describes how to view performance statistics, system alerts and logs.
Topics include:
◆ Viewing a managed system ............................................................................ 255
◆ Viewing alerts ................................................................................................ 259
◆ Viewing log messages .................................................................................... 260
For a better view of a specific source device, select that device from the list on the
right of the graph.
2. Select the cache or source device whose performance statistics you want to see.
3. You can choose which statistics to see by selecting or clearing the desired boxes.
The following table describes the performance statistics:
Parameter Description
Read Hits/sec Amount of read requests that were found in the cache, per
second
Parameter Description
Write Hits/sec Amount of write requests that were found in the cache, per
second
Skipped IOs/sec Total number of IOs that are not cached, per second, either
because they are larger than the max I/O size or because the
cache device is in passthru mode
Average Write Response The average time that it took each write I/O (both hits and
Time misses) to be served, in milliseconds
Average Read Response The average time that it took each read I/O (both hits and
Time misses) to be served, in milliseconds
3. You can filter the counter display by adding text to the Filter for box.
Viewing alerts
The Alerts window displays information on all alerts in the managed systems.
To open the Alerts window, click the Alerts tab.
Parameter Description
Parameter Description
General issues
◆ Adding a cache device.................................................................................... 262
◆ Adding GPT source devices in Microsoft Cluster Service environment.............. 262
◆ Partitioning source devices ............................................................................ 265
◆ VMAX support ................................................................................................ 265
◆ Card issues .................................................................................................... 265
◆ Error messages .............................................................................................. 274
General issues
When you have any concern about caching performance, check these issues first:
◆ Confirm that I/O is flowing through the cached source device.
◆ Verify that the cache source device(s) are those being used by the application.
◆ Verify the application access pattern to make sure it can be accelerated with
XtremSW Cache:
Troubleshooting 261
Troubleshooting
• Use OS tools to confirm that request sizes are smaller than 64k, and that
read-write ratio is around 70% and higher.
• If the application does mostly reads, but the statistics show more writes, it
may be that most of the reads are satisfied by the application caching. Use the
application tools to verify this.
• Verify that the card is operating correctly. For more information, refer to “Card
issues” on page 265.
Workaround:
◆ On a virtual machine, delete the VMDK, recreate it, and try again.
◆ On a physical machine, use the vfcmt secure erase command, then try again. If
this doesn’t work, see http://support.microsoft.com/kb/2539626.
The physical disk appears as Disk: DiskX, as illustrated in the following figure:
Before adding the serial number, the value would appear blank.
4. From the Failover Cluster Manager, bring the new resource online by right-clicking
it and choosing Bring this resource online.
The resource should now appear as being online.
5. Create a new file, vfcManualSingleApp.def, in
C:\ProgramData\EMC\VFC\system.
6. Add the source device by running this command: vfcmt add -source_dev
harddiskX
where X is the number of the drive discovered in step 2 on page 262
VMAX support
You may encounter a device is busy error in the error log, when attempting to add a
source device or if caching for a source device is not started after reinstalling.
It is likely that a VMAX LUN has been locked by XtremSW Cache. The lock is created
when using XtremSW Cache with VMAX LUNs in order to prevent data corruption.
This error is likely to appear if the server was reinstalled.
To verify if the LUN is locked, try to start it manually. If a busy error is returned, it is
locked.
To unlock the LUN, use the vfcmt release -source_dev command.
Card issues
This section describes the following card issues:
◆ “Operational states” on page 266
◆ “Flash device health” on page 266
◆ “Unplanned shutdown” on page 269
◆ “Finding the card’s serial number” on page 269
◆ “Replacing cards” on page 269
Operational states
The XtremSF Cache device can be one in the following operational states:
State Description
Started Reads and writes are being processed by the cache device. This
is the normal state.
PT-RD New source reads bypass the cache. This state would be
entered if the cache encounters more than 64 read I/O errors on
cache device.
PT-WR New source writes bypass the cache. This state would be
entered if the cache encounters more than 64 write I/O errors.
PT-RW New reads and writes bypass the cache. This state would be
entered due to health of the device. For more information, refer
to “Flash device health” on page 266. Check the log for
additional information.
You can manually change the operational state of a cache device or source device
using the vfcmt set -opmode command.
The following table summarizes the preventative measures that XtremSW Cache will
take to keep your flash device in good shape:
All flash program erase cycle (“life”) have Change cache status from started to PT-RW.
been consumed.
Drive enters write protect mode because the Change cache status from started to PT-RW
internal temperature is getting too hot
Life consumed reaches 90% Write the following message to the log: - %s:
Attention! - XtremSW Cache cache device
Approaching End of Specified Lifetime.
Continued usage may lead to loss of data
and/or degraded performance”
%s is replaced with the actual OS device
name
Life consumed reaches 100% Write the following message to the log:
%s: Attention!
XtremSW Cache cache device
End of Specified Lifetime has been Reached.
Any Additional Usage will Invalidate Warranty
and Data Retention Specification.
%s is replaced with the actual OS device
name
The flash device reaches 100 degrees C Write the following message to the log:
%s: Attention!
XtremSW Cache cache is Now in Decreased
Performance Mode due to Thermal Limiting.
%s is replaced with the actual OS device
name
The flash device exceeds 120 degrees C The flash device is taken offline, and
XtremSW Cache writes the following message
to the log:
%s: Attention!
XtremSW Cache cache is Now in Thermal
Shutdown mode to Prevent Over Heating.
%s is replaced with the actual OS device
name
The following table describes error messages relating to flash device health.
Attention! - XtremSW Cache cache device - If you are using the card for caching, nothing
Approaching Write Protect (Read Only) Mode. needs to be done. If you are using the card for
Please Ensure Any Pending Data is Saved DAS, you should copy your data to another
Prior to Repurposing the Drive as Read Only card as the card will soon be in read-only
or Retiring the Drive. mode.
Attention! - XtremSW Cache cache device is If you are using the card for caching, nothing
Now in Write Protect (Read Only) Mode. Any needs to be done. If you are using the card for
Pending Data Should be Saved to an DAS, you should copy your data to another
Alternative Storage Device. card as the card is now in read-only mode.
Attention! - XtremSW Cache cache is Now in The card overheated, and has stopped
Decreased Performance Mode due to Thermal working. To return the card to normal
Limiting. functionality, run the following command:
vfcmt set -opmode pt-normal
<cache device>.
If this message recurs, check the cooling in
the area, or reduce the amount of source
devices being accelerated to reduce the load.
Attention! - XtremSW Cache cache is Now in The card overheated, and has stopped
Thermal Shutdown mode to Prevent Over working. To return the card to normal
Heating. functionality, run the following command:
vfcmt set -opmode pt-normal
<cache device>.
If this message recurs, check the cooling in
the area, or reduce the amount of source
devices being accelerated to reduce the load.
Unplanned shutdown
An unplanned shutdown of the cache device, for instance from a power interruption,
can cause the P320 device to enter into a metadata rebuild state. Rebuild can take up
to 7 minutes. After the rebuild completes, you must restart XtremSW Cache manually.
XtremSW Cache checks for the occurrence of an unplanned shutdown upon the
execution of the start -cache_dev command.
If an unplanned shutdown is identified, XtremSW Cache sends the following message
to the log:
SfcMtStartCache:: Cache(<cache dev>, sn <serial# string> Rebuild <0/1, 0==the card is
rebuilding> err <SfcMtError_t>
Note: The vfcmt display -serial command is not supported on ESX machines.
Replacing cards
Before replacing an XtremSF card, perform the following steps:
◆ Before changing your system configuration, back up your data.
◆ If your XtremSF card is partitioned and used for storage (DAS), back up this data
before replacing the device.
The procedure for replacing XtremSF cards is documented in the Replacing Cards
technical note on the EMC Support site.
This section describes how to replace a VFCache card. Be sure to review the entire
section before replacing a card.
To replace a card, you must perform the following steps:
◆ “Replacing the physical card” on page 270
◆ “Updating drivers and firmware” on page 272
◆ “Migrating the source devices to the new card” on page 272
To avoid electrical shock, disconnect the computer from the main power and
from any networks before installing the controller card.
2. Remove the cover from the chassis, and remove the old card.
3. Remove the bracket screw, if applicable, or disengage the system retention
mechanism.
4. Unpack the new card and inspect it for damage. Unpack the card in a static-free
environment and follow good antistatic grounding procedures. Remove the card
from the antistatic bag and carefully inspect the device for damage. If you notice
any damage, or if any component is missing, replace with a different card.
5. Replace the mounting bracket (system dependent). If required for your system,
replace the full-length mounting bracket that ships on the card with the shorter
bracket supplied. Save and reuse the two screws that attach the long bracket to
attach the short bracket.
IMPORTANT
Before continuing the installation, it is highly recommended that you record the
following information, found on the blue label that is affixed to the card:
◆ Serial number (SN): ____________________________________
◆ Part number (PN) : ____________________________________
◆ Revision number (REV) : ____________________________________
Recording this information before installing the card into your system can ease future
troubleshooting needs.
6. Insert the new card in the same slot where the old card was mounted (or in
another available PCI Express slot). Press down gently, but firmly, to properly seat
the card in the slot. The following figure illustrates how to insert the card in a PCI
Express slot.
Bracket
Screw
Press
Here
Press
Here
32-bit Slot
(3.3 V Only)
Edge of
Host Board PCI Express x8 Slot
64-bit Slot
3_00200-00 (3.3 V Only)
Note: The card shape, size, and locations of components might vary from this figure.
7. Secure the bracket to the system's chassis. Install the bracket screw, if applicable,
or engage the system retention mechanism to secure the card to the system's
chassis.
8. Replace the system's cover, reconnect the power cord and any network cables.
Turn on the power.
Log files
By default, XtremSW Cache logs errors, warnings, and information messages.
You can increase the log level to debug by setting the environment variable to debug
mode "SFC_LOG_LEVEL=Debug". Note that doing so will affect performance and may
dramatically increase your system log requirements. Set this only on the
recommendation of EMC Technical Support.
You can find the log files in the following locations:
◆ Windows
• C:\ProgramData\EMC\VFC\ECOM_logfile
• C:\ProgramData\EMC\VFC\vfcmtsvc
• C:\ProgramData\EMC\VFC\logs
• C:\Program Files\EMC\VFC\toolkit\prog\log\cimomlog
• C:\Program Files\EMC\VFC\toolkit\prog\log\securitylog
• C:\ProgramData\EMC\VFC\logs - for driver log extract feature logs
• Messages intended for the user are sent to the Windows Application Event log,
together with a unique message ID
◆ Linux
• /opt/ECOM/toolkit/prog/log/VFC_log.txt
• /opt/ECOM/toolkit/prog/log/cimomlog.txt
• /opt/ECOM/toolkit/prog/log/securitylog.txt
• /var/log/messages - for VFC driver messages
◆ ESX
• /scratch/log - for management logs
Error messages
The following table describes common run-time error messages:
Adding a cache cmdAdd:: Cache for /dev/rssdb failed, Error: In this case, Use an existing,
device, for 0xe /dev/rssdb is a supported flash
example, vfcmt SFCMT Add Command error (14): Cache Device non-existent device device.
add is invalid, not connected or discovered yet.
-cache_dev
/dev/rssdb
vfcmt add cmdAdd:: Cache for /dev/sdf failed, Error: 0x7 This error occurs when Use a supported
-cache_dev SFCMT Add Command error (7): Device not the device specified for XtremSW Cache flash
/dev/sdf found or is unknown to VFC the cache device is not device
supported by XtremSW
Cache.
vfcmt add cmdAdd:: Cache for /dev/rssda1 Succeeded, Supported cache sizes Increase the size of
-cache_dev hd=4 are 20GB – 300GB the cache partition.
/dev/rssda1 Proceeding to start cache device
cmdStart:: Cache for /dev/rssda1 failed, Error:
0x26
SFCMT Add command error(38): The size of the
cache device does not meet minimum
requirements.
vfcmt start SFCMT Start command error(7): Device not Here, /dev/rssda1 is Unmount the
-cache_dev found or is unknown to VFC. hosting a mounted filesystem and retry
/dev/rssda1 SFC-Cache Error: XtremSW Cache was unable to filesystem the start command.
gain exclusive access to cache device
Windows event
ID (decimal) Log message Description
8 Device %1 successfully configured Windows specific. This message is logged each time the
XtremSW Cache agent recognizes and configures a disk
device. The %1 is replaced by “harddiskX” where X is
the disk number.
9 Device %1 was not configured Windows specific. This message occurs if the Windows
successfully, due to: %2 XtremSW Cache agent is unable to configure disk device
%1. %2 is replaced with the specific reason in text
string format.
16 XtremSW Cache Agent transition to Logged when the XtremSW Cache agent enters the
running state normal running state.
17 XtremSW Cache Agent transition to Logged when the XtremSW Cache Agent is stopped
stopped state
19 Cache Device %1 successfully added to Logged when a cache device was successfully added.
XtremSW Cache configuration, with %1 refers to the cache device name and %2 is the
handle %2 numeric handle.
20 Cache Device %1 was not added to Logged when an error occurs during the cache add
XtremSW Cache configuration, due to: operation. %1 specifies the cache device name and %2
%2 provides additional error text.
Windows event
ID (decimal) Log message Description
21 Source Device %1 successfully added to Logged when source device %1 has been successfully
XtremSW Cache configuration, with added. %2 is the assigned handle.
handle %2
22 Source Device %1 was not added to Logged when a problem occurs adding a source device.
XtremSW Cache configuration, due to: %2 is replaced with a detailed message.
%2
24 Source Device %1 was not started, due Logged when source device %1 fails to start. %2 is
to: %2 replaced with detailed reason.
26 Cache Device %1 was not started, due Logged when cache device fails to start. %2 contains
to: %2 details.
28 Source Device %1 was not stopped Logged when attempt to stop source device %1 fails.
successfully, due to: %2
30 Cache Device %1 was not stopped, due Logged when attempt to stop cache device failed.
to: %2
35 Disabling I/O for Device %1 succeeded Logged when I/O functionality has been disabled.
37 The configuration for Cache Device %1 %1 is the existing cache device and %2 is the new
successfully migrated to Device %2 cache device.
Windows event
ID (decimal) Log message Description
41 XtremSW Cache Configuration Load Logged when automatic boot configuration starts.
started
43 %1: XtremSW Cache cache device - Logged when the flash device is getting close to running
Approaching End of Specified Lifetime. out of program erase cycles. Prepare to replace the flash
Continued usage may lead to loss of device.
data and/or degraded performance
44 %1: XtremSW Cache cache device - End Logged when the flash device has completely run out of
of Specified Lifetime has been Reached. program erase cycles. The device needs to be replaced.
Any Additional Usage will Invalidate
Warranty and Data Retention
Specification
47 %1: XtremSW Cache device is Now in This message is logged when the flash device exceeds
Decreased Performance Mode due to 100 degrees C.
Thermal Limiting
48 %1: XtremSW Cache cache device is This message is logged when the flash device used for
Now in Thermal Shutdown mode to caching is no longer available. When this event occurs,
Prevent Over Heating all I/O is rerouted directly to the source devices.
If this link is not displayed, either the plug-in is disabled or it was not installed fully.
To correct this, perform one of the following:
◆ In the Feature Manager, enable the plug-in, as described in the following section.
◆ If the plug-in does not appear in the Feature Manager, reinstall it.
Enabling features
You use the Feature Manager to enable, disable, and remove features. The Feature
Manager is available only if there are features installed.
To open the Feature Manager, perform the following steps:
1. From the vSphere client, click Home, and then, from the Solutions and
Applications section, click the EMC VSI icon.
2. Select Feature Manager from the Settings tree menu.
If the activation fails, indicated by an insufficient privileges message, ensure that you
are logged in with a user with the Global:set custom attribute permission
on the selected virtual machine.
If the activation fails after checking permissions, close the vSphere client, then
re-enter it with a user that has the Global:manage custom attributes
permission on the VCenter (for example, an administrator user). This creates the
custom attribute that stores the activation data.
Certificate issues
You may not be able to trust a certificate.
This could be caused by a time gap, where the server that is running the XtremSW
Cache remote agent has a later time then the client that is running the vSphere client.
As the certificate has a start date in the future, you will not be able to accept it.
You can solve this in either of the following ways:
◆ Synchronize the client clock to the server.
◆ Wait to trust the certificate until the client's clock reaches the same time that the
server clock was when the XtremSW Cache remote agent installation took place.
Performance issues
Problem: XtremSW Cache is configured properly, but there is no noticeable
performance boost for the load on the virtual machine.
Solutions:
◆ View the statistics in the XtremSW Cache feature to determine the effectiveness of
XtremSW Cache and the type of load running on the device to be accelerated.
XtremSW Cache is most effective with loads with a high proportion of read vs.
write access.
◆ Ensure that you have provisioned enough space on the cache device to contain
the ‘hot’ working set on the source device to be accelerated.
Usage:
EMC.VFCache.SupportTool.exe [-ovph] [output directory]
You can run the support tool in the following modes:
◆ Normal (no flags)
Output is located in the folder from where the support tool was run.
◆ Output -o
Directs the output to a different folder that you specify.
◆ Verbose -v
Includes additional debugging information.
◆ Prerequisite -p
Checks whether the prerequisites are installed. Results are displayed in the
command line and written to a log file named
EMC.VFCache.SupportTool.Prerequisites.log, in the current
directory.
◆ Help -h
Shows help for the tool.
Log files
The following table describes the log files that are useful for troubleshooting:
The logs generated via the log extract feature are located in the log.zip file located in
C:\ProgramData\EMC\VFC\logs.
The XtremSW Cache Agent certificates are located in the following folder:
%AppData%\EMC\Virtual Storage Integrator\vSphere4\Data
\EMC.VSI.VSphere4.Features.VFCache.Certificates
You can also view the certificate folder via the VSI plug-in. Select Home > EMC VSI >
XtremSW and click Certificates Store.
Note: It is much easier to configure remote connectivity during the installation process
rather than trying to fix it afterwards. For this reason, it is highly recommended to
enable remote monitoring and authentication during the installation process, even if
you do not know if you will be using it.
XtremSW Cache data from a Windows managed system does not appear in
Unisphere
XtremSW Cache data from a Windows managed system does not appear. This can
happen when the following conditions are all met:
◆ The VNX storage to which the managed system is connected can't see the
machine IP.
[HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Polici
es\System]
Note: Setting this value has security implications. It is highly recommended to study
the implications before setting this value.