RE: IOPless machines and multiple CPUs -- MIDRANGE-L

PCIe3 x8 NVMe 1.6 TB SSD NVMe Flash Adapter for IBM i (#EC6V, #EC6U)

The PCIe3 1.6 TB SSD NVMe Adapter (#EC6V) is available for Power S924, S914, and H924 servers, and the PCIe3 LP 1.6 TB SSD NVMe Adapter (#EC6U) is available for the Power E980 server.

The PCIe3 1.6 TB SSD NVMe Adapter:

Features 1.6 TB of low write latency, nonvolatile flash memory on a PCIe Gen3 adapter
Uses NVMe (nonvolatile memory express), which is a high-performance software interface to read or write this flash memory
Adapter can be used in a x8 PCIe Gen3 slot in the system unit
Can provide significantly more read or write IOPS and significantly larger throughput (GB/sec) compared to SAS/SATA SSD
Can be used to satisfy minimum of SSD/DASD and backplane requirements when specified as a load source

At about 8,760 to 17,000 TB of writes to the adapter, it will be at its maximum projected write capability. The nature of the workload has a great impact on the maximum write capacity. If a high percentage of more sequentially oriented writes is used instead of random writes, the maximum write capacity will be closer to the larger value in the range. In the case of a high percentage of random writes, the maximum will be closer to the smaller value in the range. Writes past the adapter's maximum write capacity will continue to work for some period of time, but much more slowly. A Predictive Failure Analysis message will indicate that it is time to replace the adapter if enabled by the system administrator. Customers are recommended to monitor the smart log via their operating system where fuel gauge shows the percentage used.

IBM NVMe adapter failures will be replaced during the standard warranty and maintenance period for adapters that have not reached the maximum number of write cycles. Adapters that reach this limit may fail to operate according to specifications and must be replaced at the client's expense. Data protection is not implemented in the card; protection is provided by OS mirroring.

Features EC6V and EC6U are identical cards except that the tailstock bracket is different. Feature EC6U fits a low-profile PCIe slot, and feature EC6V fits a full-high PCIe slot. For a card with more memory, see features EC6X and EC6W.

Limitations:

The PCIe3 1.6 TB SSD NVMe Adapter is not supported in the PCIe Gen3 I/O drawer. Data protection is not implemented in the card; protection is provided by OS mirroring.
At least one identical first NVMe Adapter pair is required; subsequent NVMe Adapter pairs can be different than the first pair; one NVMe Adapter of different capacity is allowed; NVMe Adapter in pairs is highly recommended.
1.6 TB NVMe Adapter is allowed in maximum of two for a S914 (9009-41A) server with 4-core processor module configuration. Mixing NVMe Adapter for IBM i and SAS drives is not allowed (ten maximum of SAS drives or two maximum of NVMe Adapters for IBM i).

PCIe3 x8 NVMe 3.2 TB SSD NVMe Flash Adapter for IBM i (#EC6X, #EC6W)

The PCIe3 3.2 TB SSD NVMe Adapter (#EC6X) is available for Power S924, S914, and H924 servers, and the PCIe3 LP 3.2 TB SSD NVMe Adapter (#EC6W) is available for the Power E980 server.

The PCIe3 3.2 TB SSD NVMe Adapter:

Features 3.2 TB of low write latency, nonvolatile flash memory on a PCIe Gen3 adapter
Uses NVMe, which is a high-performance software interface to read or write this flash memory
Adapter can be used in a x8 PCIe Gen3 slot in the system unit
Can provide significantly more read or write IOPS and significantly larger throughput (GB/sec) compared to SAS/SATA SSD
Can be used to satisfy minimum of SSD/DASD and backplane requirements when specified as a load source

At about 8,760 to 17,000 TB of writes to the adapter, it will be at its maximum projected write capability. The nature of the workload has a great impact on the maximum write capacity. If a high percentage of more sequentially oriented writes is used instead of random writes, the maximum write capacity will be closer to the larger value in the range. In the case of a high percentage of random writes, the maximum will be closer to the smaller value in the range. Writes past the adapter's maximum write capacity will continue to work for some period of time, but much more slowly. A Predictive Failure Analysis message will indicate that it is time to replace the adapter if enabled by the system administrator. Customers are recommended to monitor the smart log via their operating system where fuel gauge shows the percentage used.

IBM NVMe adapter failures will be replaced during the standard warranty and maintenance period for adapters that have not reached the maximum number of write cycles. Adapters that reach this limit may fail to operate according to specifications and must be replaced at the client's expense. Data protection is not implemented in the card; protection is provided by OS mirroring.

Features EC6X and EC6W are identical cards except that the tailstock bracket is different. Feature EC6W fits a low-profile PCIe slot, and feature EC6X fits a full-high PCIe slot. For a card with more memory, see features EC6Z and EC6Y.

Limitations:

The PCIe3 3.2 TB SSD NVMe Adapter is not supported in the PCIe Gen3 I/O drawer. Data protection is not implemented in the card; protection is provided by OS mirroring.
At least one identical first NVMe Adapter pair is required; subsequent NVMe Adapter pairs can be different than the first pair; one NVMe Adapter of different capacity is allowed; NVMe Adapter in pairs is highly recommended.
3.2 TB NVMe Adapter is not allowed for a S914 (9009-41A) server with 4-core processor module configuration.

PCIe3 x8 NVMe 6.4 TB SSD NVMe Flash Adapter for IBM i (#EC6Z, #EC6Y)

The PCIe3 6.4 TB SSD NVMe Adapter (#EC6Z) is available for Power S924, S914, and H924 servers, and the PCIe3 LP 6.4 TB SSD NVMe Adapter (#EC6Y) is available for the Power E980 server.

The PCIe3 6.4 TB SSD NVMe Adapter:

Features 6.4 TB of low write latency, nonvolatile flash memory on a PCIe Gen3 adapter
Uses NVMe, which is a high-performance software interface to read or write this flash memory
Adapter can be used in a x8 PCIe Gen3 slot in the system unit
Can provide significantly more read or write IOPS and significantly larger throughput (GB/sec) compared to SAS/SATA SSD
Can be used to satisfy minimum of SSD/DASD and backplane requirements when specified as a load source

At about 8,760 to 17,000 TB of writes to the adapter, it will be at its maximum projected write capability. The nature of the workload has a great impact on the maximum write capacity. If a high percentage of more sequentially oriented writes is used instead of random writes, the maximum write capacity will be closer to the larger value in the range. In the case of a high percentage of random writes, the maximum will be closer to the smaller value in the range. Writes past the adapter's maximum write capacity will continue to work for some period of time, but much more slowly. A Predictive Failure Analysis message will indicate that it is time to replace the adapter if enabled by the system administrator. Customers are recommended to monitor the smart log via their operating system where fuel gauge shows the percentage used.

IBM NVMe adapter failures will be replaced during the standard warranty and maintenance period for adapters that have not reached the maximum number of write cycles. Adapters that reach this limit may fail to operate according to specifications and must be replaced at the client's expense. Data protection is not implemented in the card; protection is provided by OS mirroring.

Features EC6Z and EC6Y are identical cards except that the tailstock bracket is different. Feature EC6Y fits a low-profile PCIe slot, and feature EC6Z fits a full-high PCIe slot.

Limitations:

The PCIe3 LP 6.4 TB SSD NVMe Adapter is not supported in the PCIe Gen3 I/O drawer. Data protection is not implemented in the card; protection is provided by OS mirroring.
At least one identical first NVMe Adapter pair is required; subsequent NVMe Adapter pairs can be different than the first pair; one NVMe Adapter of different capacity is allowed; NVMe Adapter in pairs is highly recommended.
6.4 TB NVMe Adapter is not allowed for a S914 (9009-41A) server with 4-core processor module configuration.

Paul

-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Steinmetz, Paul via MIDRANGE-L
Sent: Sunday, July 26, 2020 10:32 PM
To: 'Midrange Systems Technical Discussion' <midrange-l@xxxxxxxxxxxxxxxxxx>; 'Jim Oberholtzer' <midrangel@xxxxxxxxxxxxxxxxx>
Cc: Steinmetz, Paul <PSteinmetz@xxxxxxxxxx>
Subject: RE: IOPless machines and multiple CPUs

Found my answer.

PCIe3 x8 SSD NVMe adapters for IBM i operating system, providing 1.6 TB, 3.2 TB, and 6.4 TB capacity points for Power E980, S914, S924, and H924 servers.

Paul

-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Steinmetz, Paul via MIDRANGE-L
Sent: Sunday, July 26, 2020 9:06 PM
To: 'Jim Oberholtzer' <midrangel@xxxxxxxxxxxxxxxxx>
Cc: Steinmetz, Paul <PSteinmetz@xxxxxxxxxx>; 'Midrange Systems Technical Discussion' <midrange-l@xxxxxxxxxxxxxxxxxx>
Subject: RE: IOPless machines and multiple CPUs

Jim,

Is there an option for NVME Flash without a SAN and VIOS?

Paul

From: Jim Oberholtzer <midrangel@xxxxxxxxxxxxxxxxx>
Sent: Sunday, July 26, 2020 8:53 PM
To: Steinmetz, Paul <PSteinmetz@xxxxxxxxxx>
Cc: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxxxxxxxx>
Subject: Re: IOPless machines and multiple CPUs

Moving to NVMe, that will happen much faster than you may expect. The only thing holding it back in smaller environments is the V5000e does not support it. You need a V5100e and there is a cost difference.

The other thing that may slow it down is upgrading the SAN network to 16 or 32GB from 8GB. Those switches are not cheap.

Oh, and you’ll be running VIOS since the V5100e cannot direct attach to IBM i.

We already have customers doing it now.

As to the move to SSD that wave has crested. Now we don’t configure anything but SSD unless there is a serious need for spinning disk.

On Sun, Jul 26, 2020 at 7:39 PM Steinmetz, Paul <PSteinmetz@xxxxxxxxxx<mailto:PSteinmetz@xxxxxxxxxx>> wrote:
Next question.
NVME Flash storage is much faster than SSD.
How soon do you see people moving to Flash, I know some have already?

Paul

From: Jim Oberholtzer <midrangel@xxxxxxxxxxxxxxxxx<mailto:midrangel@xxxxxxxxxxxxxxxxx>>
Sent: Sunday, July 26, 2020 8:35 PM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxxxxxxxx<mailto:midrange-l@xxxxxxxxxxxxxxxxxx>>
Cc: Steinmetz, Paul <PSteinmetz@xxxxxxxxxx<mailto:PSteinmetz@xxxxxxxxxx>>
Subject: Re: IOPless machines and multiple CPUs

For the host partition for certain that’s true. Might even work for the guests.

Your systems while larger than most certainly is not “huge” where the virtualization is a requirement for HA and redundancy. There are limits in the IBM i hosted environment that do not exist in the external storage/VIOS method. Pick the one that works for your environment.

As we move forward I’m betting hardware that has internal storage is going to get more and more sparse. You can already see this trend with the I/O options available today vs. several years ago.

In any case host the environment you need in the best way that works in your situation. There is no wrong answer in that category.

On Sun, Jul 26, 2020 at 6:29 PM Steinmetz, Paul via MIDRANGE-L <midrange-l@xxxxxxxxxxxxxxxxxx<mailto:midrange-l@xxxxxxxxxxxxxxxxxx>> wrote:
Jim,

Not sure where I read this, but IBM I internal disks (SSD) will out perform VIOS.
VIOS adds overhead and an extra level for the I/O.
Our Production LPAR runs on our host P9, 18 arms, 100% SSD Our R&D LPAR runs as a client off our Production LPAR, 70 Virtual arms, 100% SSD.
R&D performance matches production, better at times.

Paul

-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx<mailto:midrange-l-bounces@xxxxxxxxxxxxxxxxxx>> On Behalf Of Jim Oberholtzer
Sent: Sunday, July 26, 2020 7:18 PM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxxxxxxxx<mailto:midrange-l@xxxxxxxxxxxxxxxxxx>>
Subject: Re: IOPless machines and multiple CPUs

True, VIOS come with PowerVM, and is about the same cost per core as VMWare, or less.

That’s the software. Now get the fiber And Ethernet switches to support the virtual environment and the cost goes up dramatically, so does the complexity.

For true production workloads that have any sensitivity to I/O, or the performance is critical, then VIOS is the way to go. Many IBM i workloads are just fine with IBM i hosting, and that’s far less complex, simple to set up, and easy to maintain.

Jim Oberholtzer
Agile Technology Architects

On Jul 26, 2020, at 12:01 PM, Roberto José Etcheverry Romero <yggdrasil.raiker@xxxxxxxxx<mailto:yggdrasil.raiker@xxxxxxxxx>> wrote:

Configuring an LPAR with more cores than licensed ends up with a
message saying that you are over entitlement but it still uses those cores.
One of the reasons I don't understand i hosting i is the economics of
the exercise. It is a lot more expensive to use IBM i to virtualize
storage/network than the almost free VIOS.
On a big machine you might need more than 2 entire cores for IO and
that gets expensive quickly.

Roberto

On Sun, 26 Jul 2020, 12:39 Patrik Schindler, <poc@xxxxxxxxxx<mailto:poc@xxxxxxxxxx>> wrote:

Hello,

imagine a newer POWER machine with, say, four CPUs. IBM i is licensed
to one CPU, so I guess when you give the LPAR more than one, it won't
be used for running "normal" code. Right?

Does anybody know (and provide evidence) if a second assigned CPU
will be used as "helper CPU" for I/O? Not exactly an IOP, since this
involves drivers and other stuff. My thinking stems from the z-world
where processors can be freely configured to specialized tasks (I/O,
running Linux, running Java, …). The number of CPUs often influences
licensing cost, which probably was one of the reasons to invent this
feature in the first place.

Thanks!

:wq! PoC

PGP-Key: DDD3 4ABF 6413 38DE - https://www.pocnet.net/poc-key.asc

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing list To post a message email:
MIDRANGE-L@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L@xxxxxxxxxxxxxxxxxx>
To subscribe, unsubscribe, or change list options,
visit:
https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%
3a%2f%2flists.midrange.com%2fmailman%2flistinfo%2fmidrange%2dl&umid=7
4b45c41-bd06-4275-95eb-a569440252a4&auth=438b0784514c1757bd202125ca4d
b8b0abdb021e-5265919fd6683b85c59de35bc49034cfbd98a9da
or email:
MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L-request@lists
.midrange.com> Before posting, please take a moment to review the
archives at https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2farchive.midrange.com%2fmidrange%2dl&umid=74b45c41-bd06-4275-95eb-a569440252a4&auth=438b0784514c1757bd202125ca4db8b0abdb021e-bae5db99b4327debaab6f6b080cf5a385440b634.

Please contact
support@xxxxxxxxxxxxxxxxxxxx<mailto:support@xxxxxxxxxxxxxxxxxxxx> for any subscription related questions.

Help support
midrange.com<https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/que
ry?url=http%3a%2f%2fmidrange.com&umid=74b45c41-bd06-4275-95eb-a569440
252a4&auth=438b0784514c1757bd202125ca4db8b0abdb021e-a2284880c3426fbd8
3d47298b511d68277dbc393> by shopping at amazon.com<http://amazon.com>
with our affiliate
link: https://amazon.midrange.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email:
MIDRANGE-L@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L@xxxxxxxxxxxxxxxxxx> To
subscribe, unsubscribe, or change list options,
visit:
https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3
a%2f%2flists.midrange.com%2fmailman%2flistinfo%2fmidrange%2dl&umid=74b
45c41-bd06-4275-95eb-a569440252a4&auth=438b0784514c1757bd202125ca4db8b
0abdb021e-5265919fd6683b85c59de35bc49034cfbd98a9da
or email:
MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L-request@lists.
midrange.com> Before posting, please take a moment to review the
archives at https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2farchive.midrange.com%2fmidrange%2dl&umid=74b45c41-bd06-4275-95eb-a569440252a4&auth=438b0784514c1757bd202125ca4db8b0abdb021e-bae5db99b4327debaab6f6b080cf5a385440b634.

Please contact support@xxxxxxxxxxxxxxxxxxxx<mailto:support@xxxxxxxxxxxxxxxxxxxx> for any subscription related questions.

Help support
midrange.com<https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/quer
y?url=http%3a%2f%2fmidrange.com&umid=74b45c41-bd06-4275-95eb-a56944025
2a4&auth=438b0784514c1757bd202125ca4db8b0abdb021e-a2284880c3426fbd83d4
7298b511d68277dbc393> by shopping at amazon.com<http://amazon.com>
with our affiliate
link: https://amazon.midrange.com