When evaluating a product, customers typically have a list of evaluation criteria. Among these, support and software upgrades are crucial. The quality of support and the ease of upgrading the software significantly influence the likelihood of converting potential customers into loyal ones.

In most of the sales cycles or deployment services I have been involved in, one of the most common questions and concerns from customers is about the ease of upgrading and the potential downtime. Longer downtimes during upgrades mean lost business opportunities. Therefore, the simplicity of the upgrade process and the duration of any downtime are key factors in their decision-making.

PowerFlex boasts one of the best upgrading processes in the industry. Given our emphasis on PowerFlex’s ability to manage the lifecycle of all components within its architecture, having a robust process in place is essential to support this capability.

PowerFlex adopted the concept of the Release Certification Matrix (RCM) pioneered by DellEMC. They introduced it as part of their lifecycle assurance process for their converged and hyper-converged infrastructure systems, such as Vblock and VxBlock. The RCM was designed to simplify IT management by providing a systematic, certified, and tested upgrade path for various components, ensuring compatibility and stability across their systems.

The Release Certification Matrix (RCM), also known as the Release Compatibility Matrix, is a comprehensive document that outlines the compatibility and certification of various software and hardware components within an IT ecosystem. It serves as a guide for IT professionals to ensure that upgrades, patches, and new installations are applied correctly and efficiently.

Typically, for a data center upgrade, the entire process from planning, remediation, upgrade to post-upgrade verification can range from a few months to over a year, depending on the specific requirements and scale of the data center. With RCM, this process can be as short as a few weeks, as RCMs are factory tested to ensure compatibility and reliability across various components and configurations. This rigorous testing process helps identify and resolve potential issues before the products are released to customers, ensuring a smoother deployment and operation.

In other words, the heavier burden of customer work in terms of upgrade assessment has been taken care of. All the customer needs to do is plan the starting date and the segregation of workload during the upgrade period.

RCM/Intelligent Catalogue (IC)

The Release Certification Matrix (RCM) began as a document outlining the compatibility and reliability of software and hardware to ensure seamless system operation. The PowerFlex support team has taken this a step further. In addition to providing the RCM document, they offer the necessary software in two distinct bundles: the RCM bundle and the Intelligent Catalogue (IC). The RCM bundle is specifically designed for PowerFlex rack, while the IC is tailored for PowerFlex appliance.

Additionally, PowerFlex offers two more consumption models: Custom Nodes and Software-Only. Both involve software deployments, but Custom Nodes come with a pre-validated hardware. PowerFlex Software-Only allows you to install PowerFlex software on any compatible x86 hardware, offering maximum flexibility and customization. Customer has to manage the hardware lifecycle, including updates and maintenance. In contrast, Custom Nodes are pre-validated server building blocks specifically configured for PowerFlex, ensuring compatibility and optimal performance. Since these are discovered as software-only resources, PowerFlex Manager cannot manage the hardware lifecycle for these consumption models, unlike the Rack or Appliance models. PowerFlex Manager can only manage the lifecycle of the PowerFlex software. Firmware updates can then be applied manually using either the RCM bundle or the IC or directly from the hardware vendor.

PowerFlex Manager Intelligence

The PowerFlex manager has a built-in mechanism to detect if the correct version of RCM/IC has been uploaded before initiating the upgrade process. It is always best practice to keep the system as updated as possible. However, this is not always feasible due to customer business operations constraint. Therefore, upgrading from one version to the latest version may require multiple steps. In such scenarios, it is imperative that the customer or field engineer follows the correct sequence to reach the targeted RCM/IC version. If the customer or field engineer uploads the versions in the wrong order, an alert or warning will be triggered as shown, and the upgrade will not proceed until the correct version is uploaded.

Tracking RCM Version

From the picture attached, you will notice that RCM and IC have different ways of tracking their version relative to the PowerFlex OS versions. Example, for IC 45.734.xx, it is referring to PowerFlex version 4.5 release and IC 46.380.xx is referring to PowerFlex version 4.6, so on and so forth.

For RCM, it is slightly different. As you noticed, it is named as 3.8 train, 3.7 train, 3.6 train, etc, etc.

The initial “3” is a layover from the old VCE days when it flagged a PowerFlex RCM (that was our number). The next digit covers the VMware version (vSphere 7 or 8). That’s the “train” which we tied to VMware. The next digit is the M&O feature release version within that train. The last digit is the minor or patch release.

To see what PowerFlex core software versions are included in the RCM, select one of them and details of that RCM bundle will be listed.

However, this pattern of easy differentiation between different version may break in the future. Too early to speculate, will keep the audience update when the times come.

This is sample dashboard summary of how the various RCM train look likes. However, for more in depth information, please contact Dell support.

Initiating PowerFlex Upgrade

To initiate an upgrade process, first note the current RCM/IC version the PowerFlex cluster is running on. Then, determine the target version aiming for. Dell support can then advise on the available upgrade path based on the current environment.

For example, upgrading the PowerFlex appliance from version 4.5 to 4.6 can be achieved in a single hop by moving from 45.374.00 to 46.375.01, as illustrated. However, this might not always be feasible, depending on the current system’s age, since not all customers regularly update their systems. A single hop is the simplest and quickest method, whereas multiple hops are more complex, take longer, and require additional planning. We might not be able to complete all hops within a single maintenance window, and the process may need to be spread over several maintenance windows.

Upgrading PowerFlex Manager

Prior to version 4.x, upgrading a PowerFlex cluster involved upgrading two different VMs: the PowerFlex Manager and the PowerFlex Gateway. Upgrading the PowerFlex Manager was not a prerequisite for upgrading the PowerFlex core, but upgrading the PowerFlex Gateway was mandatory.

In version 4.x, the PowerFlex Gateway is no longer a VM but a service within the PowerFlex Manager. Consequently, the RCM/IC that was previously tied to the PowerFlex Gateway is now tied to a specific version of the PowerFlex Manager. Therefore, if the PowerFlex Manager version is not sufficiently up-to-date, it will not allow you to perform any updates to the PowerFlex core.

The following series of screenshots demonstrates how easy it is to upgrade the management software, i.e., PowerFlex Manager, from 4.5 to 4.6 prior to the actual production upgrade of the compute and storage nodes.

Prep the Environment by Uploading the RCM/IC to PowerFlex Manager

RCM/IC files are quite large, but based on experience, they can typically be imported in less than an hour if done over a local share, compared to using the Secure Connect Gateway, depending on bandwidth. Uploading the RCM/IC is just the beginning; PowerFlex Manager will then need to extract the RCM/IC, create new repositories, and synchronize before it is available for upgrade. Therefore, it is advisable to upload the package in advance, as this can be done at any time without impacting the system or eating into the maintenance window if done during the upgrade windows.

Uploading the RCM or IC bundle will depend on the PowerFlex consumption model that was deployed. In this illustration, it is for a PowerFlex appliance cluster, and the IC bundle was uploaded in this instance, as shown.

The following screenshots demonstrate how easy it is to carry out the entire upload and upgrade process.

Pro and Con of RCM/IC

RCM/IC is a value-added process designed to assist customers who have purchased our products by simplifying the planning and execution of upgrades. Unlike previous experiences, RCM/IC is fully validated and tested in the factory, ensuring it can be deployed to production systems without any compatibility concerns.

Without the RCM process, a typical upgrade would take approximately 3 to 6 months for planning and an additional 3 to 6 months for remediation before the actual upgrade can occur.

However, the main drawback of RCM/IC is its infrequent and lengthy turnaround time, which may not address immediate patches released to fix urgent security vulnerabilities. This often causes significant anxiety among customers, as they are highly discouraged from manually pushing out patches by bypassing PowerFlex Manager, which would render the system non-compliant and unsupported.

Single Component Upgrade (SCU)

As emergency patches become more frequent and normalized, it is no longer feasible to ask customers to wait several weeks for the testing and validation of a new RCM/IC release. To address this issue, a new feature called Single Component Upgrade (SCU) was introduced in version 4.5.

Please note, this feature allows PowerFlex Manager to update one or more components outside of the regular catalog release schedule to address urgent business needs (such as zero-day security patches) in a quick and expedited manner without waiting for in-market releases. It is important to note that minimal or no validation, sanity checks, or risk assessments have been carried out for these updates. Customers must acknowledge and accept the associated responsibilities and risks.

SCU Walkthrough

The following screenshots give a quick demo of how SCU helps to achieve emergency patch to a production environment.

a. IC 45.374.00 is the current complaint version

b. When click on View Bundles, it displays a list of software included in this IC.

c. Clicking on View for the selected item displays with the following and the Modify button enabled.

d. Assuming we have a latest update for iDRAC 7.00.00.00 that needs to be modified.

e. Click on Proceed to continue. Once it is successful, a modified icon will be shown beside the new version.

f. The modified icon is displayed in the bundle that has been modified.

g. The same Modified icon is shown in the IC that had been modified as well.

h. The modified icon is displayed almost everywhere that traced back to the modified iDRAC.

In the next series, we will talk about what are the recent changes to the RCM/IC and its impact to customers.