-
Notifications
You must be signed in to change notification settings - Fork 0
Notes About SLES v15 on PowerPC
This page is a collection of my notes about running SLES v15GA on IBM Power9 (P980, specifically) LPARs.
IMPORTANT My observations were originally made circa 2020/2021, and I've been away from that environment for three years now (as of 2024). Certain technical details may have changed (among other things, SLES v15 moved from a v4 to a v5 Linux kernel somewhen around SP2).
My point-of-comparison is, of course, SLES on an x86_64 system. So this page is mainly about the differences in tools and behaviours, as compared to x86_64 installs.
The default SLES install includes powerpc-utils; it delivers a number PPC-specific utilities, including ls-vdev, ls-veth and ls-vscsi. Additionally, SUSE provides the lsvpd package, which includes lsmcode, lsvio, lscfg and lsvpd. These tools are specific to PowerPC environments; some have rough analogs in x86_64 (for example, lsdev and lsscsi), while others (such as lsvio) don't have a direct counterpart.
This is one of the most-reliable ways of checking for PowerPC.
uname -m returns ppc64le, at least on IBM Power9. On IBM Power7, it would return ppc64. This is because IBM Power9 is little-endian and Power7 was big-endian. IBM Power8 could function either way (technically, the PowerPC architecture is bi-endian - this page is about what I've encountered in the products I've used).
IBM PowerPC has no DMI data as the dmidecode executable would understand it, so that executable is not in the SLES on PowerPC distribution. Oddly, the distribution does include python-dmidecode; but I never had time to experiment with it.
This tool did report NVMe drives in the P980 frame that had been assigned to the LPAR; however, it understandably did not "see" the NVMe drives otherwise.
In the environment I inhabited at the time, the only other PCI devices it would find were the Ethernet NICs (which were actually SR-IOV VFs allocated to the LPAR). Interestingly, for the Broadcom boards in the frame, the lspci output was consistent between SLES v12SP4 and SLES v15GA. As one might expect, despite Broadcom OEMing the hardware, the Vital Product Data identified our specific NICs as IBM 00ND464 PCIe3 4-port 10GbE SR Adapter.
I was also used to being able to find MAC address of the NIC as part of the lspci output, at least for most NICs in the x86_64 world. That information never seemed to be visible in the Power9 environment, although I'm not certain if that was related to an LPAR configuration selection or the fact that it was an SR-IOV VF.
First, note that this tool was deprecated in SLES v15GA (and, indeed, in the wider Linux ecosystem); to get it, you have to install the net-tools-deprecated package. The thought of having to use ip made heads explode among the AIX/Tivoli guys, so I installed it rather than deal with their withdrawl symptoms. Since it was deprecated, I didn't do much with it, but the output was essentially the same as you'd see in x86_64.
This generally worked as one would expect to see in the x86_64 world. For me, the most-useful output was to report link state
ethtool <NIC> | grep 'Link detected' | awk '{ print $3 }'
It would also report the driver and version, plus the firmware version of the physical board underlying the SR-IOV NIC, when invoked as ethtool -i <NIC>
Oddly, SLES v15GA loaded a joystick driver (joydev) for the LPAR. The versions of the ibmvscsi and ibmvfc modules were the same as on SLES v12SP4, although the be2net driver was newer.
If you're using SMT8, then lscpu sees that as eight threads per core. The Model Name was reported as POWER9 (architected), altivec supported
This helpfully reported the firmware (aka Licensed Internal Code) version for the Power9 frame, and differentiated between the running code, backup version, and staged.
Similar to the voluminous output of an unbridled dmidecode, this tool will display a lot of data. No, I mean it... a LOT. Unfortunately, where dmidecode -t <TYPE> offers the opportunity to focus the output, lsvpd didn't seem to have a similar feature. That made it difficult to use.