From what I'm reading, if all memory slots are not used, then the full memory bandwidth is not available.
Also, P9 is now using "unbuffered" memory instead of "buffered" memory, which results in a 20% lower bandwidth than P8 systems.
If I go with a S914, I'll probably go with 512gb - 16 EM63 32 GB DDR4 Memory @ 1,179 ea $18,864 maxed, all 16 slots used.
If I go with a S924 instead, I'll probably go with 1 tb - 32 EM63 32 GB DDR4 Memory @ 1,179 ea $37,728 maxed, 32 slots used.
They always say, the more memory the better.
With 1 tb , I could almost keep my entire application in memory.
Amazing how memory prices have dropped, comparing to my previous configs.
Old P5 with 64gb -4 16GB DDR-1 Main Storage 180,000
Cur P7 with 256 gb 8 EM32 32GB (2x16GB) Memory DIMMs, 1066 MHz, 2Gb DDR3 DRAM 51,120
For an S914, desiring 512 gb memory.
1) 16 EM63 32 GB DDR4 Memory @ 1,179 ea $18,864 maxed, all 16 slots used.
2) 8 EM64 64 GB DDR4 Memory @ 4,198 ea $33,584 room for growth, 8 slots used, 8 slots free.
For an S914, desiring 1tb memory
3) 16 EM64 64 GB DDR4 Memory @ 4,198 ea $67,198 maxed, all 16 slots used.
For an S924, desiring 512 gb memory.
4) 32 EM62 16 GB DDR4 Memory @ 619 ea $19,808 maxed, all 32 slots used.
5) 16 EM63 32 GB DDR4 Memory @ 1,179 ea $18,864 room for growth, 16 slots used, 16 slots free.
6) 8 EM64 64 GB DDR4 Memory @ 4,198 ea $33,584 room for growth, 8 slots used, 24 slots free.
7) 4 EM65 128 GB DDR4 Memory @ 9,880 ea $39,520 room for growth, 4 slots used, 28 slots free.
For an S924, desiring 1 tb memory.
8) 32 EM63 32 GB DDR4 Memory @ 1,179 ea $37,728 maxed, 32 slots used.
9) 16 EM64 64 GB DDR4 Memory @ 4,198 ea 67,168 room for growth, 16 slots used, 16 slots free.
10) 8 EM65 128 GB DDR4 Memory @ 9,880 ea 79,040 room for growth, 8 slots used, 24 slots free.
Paul
-----Original Message-----
From: Steinmetz, Paul
Sent: Tuesday, April 03, 2018 12:06 AM
To: 'Midrange Systems Technical Discussion'
Subject: RE: P9 memory performance
Larry,
I found the verbiage, but no specifics.
Still searching.
Obviously, these memory bandwidth figures are based on having all memory slots full and running at the top speed available. If you use slower memory in one socket, they all drop down to the slower speed, lowering bandwidth, and if you don't populate all of the memory slots, the bandwidth drops proportionately there, too.
https://www.itjungle.com/2018/03/05/deal-power9-memory-entry-servers/
We don't know a lot of customers who fill up memory slots to get the full bandwidth of the machines, mainly because customers have been taught to leave a little space in the machines for memory capacity expansion. Memory is so expensive, and workloads so varied in their need for main memory capacity and bandwidth, that it is difficult to give a hard and fast rule. The fact is, customers have to know if they are memory capacity or memory bandwidth constrained, and buy accordingly. Making a mistake in this task could be very costly indeed, both in terms of performance and actual money.
As for memory, the capacity has doubled up to 4 TB maximum across the four sockets, but the shift away from buffered memory means that the memory bandwidth across the sixteen memory slots per socket has dropped. The peak memory bandwidth on the Power S824 was 192 GB/sec, coming in at 173 GB/sec on the STREAM memory bandwidth tests; the Power S924 tops out with 20 percent lower bandwidth at 153 GB/sec peak and with 33.5 percent less on the STREAM test at 115 GB/sec. Part of the problem is that IBM is only supporting 2.13 GHz memory in systems that have ten or more of their memory slots loaded up per socket. Intel "Skylake" Xeon SP systems can have memory running at 2.4 GHz and 2.67 GHz in fully loaded machines, although you have to pay a hell of a processing premium to get machines that support 1.5 TB of memory per socket and therefore give capacity as well as bandwidth on the same order of the Power9 machines. Anyway, a two-socket, top-end Xeon machine has somewhere between 200 GB/sec and 225 GB/sec of sustained memory bandwidth on the STREAM test, which is on par with what IBM is offering. Intel has six memory controllers running a little faster, IBM has eight running a little slower and it all kinda washes out. The big point is that IBM is using the same industry standard memory sticks, so the price differential should disappear and the odds in memory will be even.
https://www.itjungle.com/2018/02/26/inside-ibms-power-s924-power9-entry-system/
https://www.redbooks.ibm.com/redbooks/pdfs/sg248171.pdf
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Understanding%20the%20Benefits%20of%20Dark%20Cores%20and%20Memory
https://pdfs.semanticscholar.org/101a/982b1de9607f131f1d535119bc371e2b312e.pdf
https://openpowerfoundation.org/wp-content/uploads/2016/11/Jeff-Stuecheli-POWER9-chip-technology.pdf
Paul
-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of DrFranken
Sent: Monday, April 02, 2018 5:18 PM
To: Midrange Systems Technical Discussion
Subject: Re: P9 memory performance
Yes maximum performance does occur with all slots full. However I have not seen any statement of by how much. Clearly you wouldn't go get 128GB DIMMs and the minimum number of those and expect that to match the entire server full of 32GB DIMMs but don't know if it's even enough to measure let alone notice.
- Larry "DrFranken" Bolhuis
www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.
On 4/2/2018 4:49 PM, Steinmetz, Paul wrote:
I'm planning on 512 gb for the P9.
Using 8 64 GB EM64.
Leaving slots open for growth.
Somewhere I thought I read that all memory slots should be filled for best memory performance.
Is this correct?
Thank You
_____
Paul Steinmetz
IBM i Systems Administrator
Pencor Services, Inc.
462 Delaware Ave
Palmerton Pa 18071
610-826-9117 work
610-826-9188 fax
610-349-0913 cell
610-377-6012 home
psteinmetz@xxxxxxxxxx
http://www.pencor.com/
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit:
https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at
https://archive.midrange.com/midrange-l.
Please contact support@xxxxxxxxxxxx for any subscription related questions.
Help support midrange.com by shopping at amazon.com with our affiliate link:
http://amzn.to/2dEadiD
midrange.com
