Spectre and Meltdown fixes

This page should give you a short overview of what is currently available to update your Intel based server or Workstation to get as good as possible patched against Spectre ( CVE 2017-5753 (Bounds Check Bypass / Variant 1) and CVE 2017-5715 (Branch Target Injection / Variant 2)) and Meltdown (CVE 2017-5754 (rogue data cache load / Variant 3)).

this update will need a reboot of your server for sure, don't just update and continue to work without rebooting

in general you need to update your kernel to the latest versions provided by your distribution of choice. by now, pretty much any distribution should have released patches.

Here is how it's done…

apt-get update && apt-get upgrade

… in ubuntu and ..

yum  update 

.. in RedHat and CentOS distributions.

this will protect you against Variant 1 and Variant 3 vlunerabilities. In order to also protect against Variant 2, you further need to update the CPU's microcode (basically the CPU's firmware). Instructions on how to do that follow further down on this page

here is a list of links with information about updates available from linux distributions I care most about:

• Debian stretch is fixed as of kernel version 4.9.65-3+deb9u2
• Ubuntu (fixes for all maintained versions available as of Jan. 9th)
• Proxmox VE debian based virtualization environment, fixed for Version 4 and 5 as explained in the linked forum post.
• CentOS (started syncing to mirrors on Jan 4) kernel-3.10.0-693.11.6.el7.x86_64.rpm and related packages fix the bug
• RedHat Enterprise Linux RHEL 7 (released 3.Jan)
• Suse

for further information read those pages or check out the meltdown webpage link section

After you have installed the latest OS updates, your system should be protected against Variant 1 and 3, in order to protect against Variant 2, you also need to install a newer microcode (firmware of the CPU).

There are three ways to get to the latest microcode updates:

Usually the microcode package is also a package in your distributions repository and is updated during a normal os upgrade. However, there where some stability issues with the microcodes released by Intel early this year ( around 18.1.18) so that for example RedHat removed them from their repos.

after that, intel pulled the microcode updates back from their own webpage as well and went back to alpha and beta testing. they have now released a bunch of updated microcodes for their cpu's. generally what we see is, that they start with the newest cpu's and work their way back. I would expect, that the final microcode releases will make it into the respective microcode packages in your linux distribution. the Easiest way is just to run updates regularly and eventually you should get the latest microcode update for your cpu that will enable the Variant 2 fix.

besides waiting for your distro to include the update for you, you can also manually download the latest microcode package from intel (includes microcodes for all intel cpus in one package) and install that on your computer manually. Here is, how that's done:

The latest officially released Microcodes can be found on the intel downloadcenter page. currently this download here is the latest. there should be a banner at the top linking to a newer version, should this link no longer lead to the latest version. So far they have updated the link target together with new releases to always point to the latest release.

1. download the package of your choice to your System:

   cd /root/ 
   wget <url>

2. now move your existing microcode package to another location, so you can move it back in case you have these stability issues that some users had with the new ones. then unpack and load the new microcode:

   cd /lib/firmware/
   mv intel-ucode /root/intel-ucode.old
   tar xvf /root/microcode-20180108.tgz 
   echo 1 > /sys/devices/system/cpu/microcode/reload

you can double check if your microcode was loaded using

dmesg | grep microcode

. you should now be up to date with the latest patches for all three Variants of the Spectre & Meltdown vulnerability.

BIOS packages from the mainboard vendor should also include the latest microcode for the CPU upon release of the BIOS package. sometimes you can get microcode updates earlier through a BIOS update than you can get them through the intel microcode package download. However, if your mainboard producer supplies no mor BIOS updates or if they have a slow release cycle, the intel package might be the faster solution for you. Because Linux only loads the microcode from its own package when it's newer than the version loaded by the bios, a BOIS update that gets you the new Microcode will work even with old Linux Versions that might be out of maintenance. However, if you are using such a Distro, you probably aren't worried too much about security anyway and your system is hopefully only running in a well protected internal network with trusted users.. in that case, don't worry about Meltdown :)

If you do worry about meltdown and want to upgrade the microcode through a new bios, you can find a list of the latest BIOS releases that contain Variant 2 fixes in their included microcode on this Intel-SA-00088 for Intel® Server Boards overview page. The list is updated as soon as new bioses become available.

Intel's Microcode releasenotes are somewhat cryptic. Here is how to read the following lines of the current release notes:

-- Updates upon 20171117 release --
IVT C0		(06-3e-04:ed) 428->42a
SKL-U/Y D0	(06-4e-03:c0) ba->c2
BDW-U/Y E/F	(06-3d-04:c0) 25->28
HSW-ULT Cx/Dx	(06-45-01:72) 20->21
Crystalwell Cx	(06-46-01:32) 17->18
BDW-H E/G	(06-47-01:22) 17->1b
HSX-EX E0	(06-3f-04:80) 0f->10
SKL-H/S R0	(06-5e-03:36) ba->c2
HSW Cx/Dx	(06-3c-03:32) 22->23
HSX C0		(06-3f-02:6f) 3a->3b
BDX-DE V0/V1	(06-56-02:10) 0f->14
BDX-DE V2	(06-56-03:10) 700000d->7000011
KBL-U/Y H0	(06-8e-09:c0) 62->80
KBL Y0 / CFL D0	(06-8e-0a:c0) 70->80
KBL-H/S B0	(06-9e-09:2a) 5e->80
CFL U0		(06-9e-0a:22) 70->80
CFL B0		(06-9e-0b:02) 72->80
SKX H0		(06-55-04:b7) 2000035->200003c
GLK B0		(06-7a-01:01) 1e->22

• – Updates upon 20171117 release – This means, that this package contains the following updates since the last release. so only the microcodes for the cpus mentioned here where updated, everything else is identical to the last release.
• SKL-U/Y D0 (06-4e-03:c0) ba→c2
  • SKL stands for Sky-Lake
  • 06-4e-03 is actually the most useful part, it tells you what cpu that is in cpu-family, model and stepping. you can get this information from /proc/cpuinfo with this command:

    grep -P "^(cpu family)|(model\s*:)|(stepping)" /proc/cpuinfo | tail -3

    . 06 is the family, 4e is the model in HEX format (use google or a scientific calculator to convert if you are lazy :)) and 03 is the stepping.

  • the last part ba→c2 is the relevant part of the version number that changed. For this specific Skylake CPU the Spectre Patch is supposed to be in releases 0xc2 or newer, so this one here contains the patch. Sadly the list with all these releases is under NDA, so i can't share it here. But in general you can expect everything that is released starting with the current package to have the fix in place.
• by the way, 06-4e-03 is also the filename of that microcode.
• rather than browsing through the entire history of the release notes you can also check the version of a specific microcode file using this command: iucode_tool -l intel-ucode/06-4f-01

Should you, for some reason, not be able or willing to run a full update, I have here a minimalistic fix for your centos:

1. download the necessary update packages

   mkdir -p /opt/meltdown
   cd /opt/meltdown
   for p in kernel-abi-whitelists-3.10.0-693.11.6.el7.noarch.rpm kernel-debug-3.10.0-693.11.6.el7.x86_64.rpm kernel-debug-devel-3.10.0-693.11.6.el7.x86_64.rpm kernel-devel-3.10.0-693.11.6.el7.x86_64.rpm kernel-doc-3.10.0-693.11.6.el7.noarch.rpm kernel-headers-3.10.0-693.11.6.el7.x86_64.rpm kernel-tools-3.10.0-693.11.6.el7.x86_64.rpm kernel-tools-libs-3.10.0-693.11.6.el7.x86_64.rpm kernel-tools-libs-devel-3.10.0-693.11.6.el7.x86_64.rpm perf-3.10.0-693.11.6.el7.x86_64.rpm python-perf-3.10.0-693.11.6.el7.x86_64.rpm kernel-3.10.0-693.11.6.el7.src.rpm kernel-3.10.0-693.11.6.el7.x86_64.rpm; do wget http://mirror.centos.org/centos/7.4.1708/updates/x86_64/Packages/$p; done

2. create a repository:

   createrepo .

3. add your repository to yum

   mydir=`pwd`
   
   cat > /etc/yum.repos.d/CentOS-meltdown.repo <<EOF
   # CentOS-meltdown.repo
   #
   # contains minimalistic update to fix meltdown and spectre
   [meltdown-updates]
   name=CentOS-$releasever - Meltdown-Updates
   baseurl=file://$mydir
   gpgcheck=0
   enabled=1
   EOF

4. run the update and reboot the machine:

   yum update
   
   reboot

• https://github.com/speed47/spectre-meltdown-checker

the fix for all this works in a way that it may affect the system performance negatively. Different sources claim different results reaching from no impact at all up to a 30% slowdown. As always, Benchmarks are probably not too representative for your realworld experience. In order to find out what the difference in performance is, you can simply disable the workaround on a patched kernel to run your workload once with and once without the patch.

In CentOS (and probably other linux distributions as well) the workarounds can be enabled or disabled without a reboot using these commands:

echo 0 > /sys/kernel/debug/x86/pti_enabled
echo 0 > /sys/kernel/debug/x86/ibpb_enabled
echo 0 > /sys/kernel/debug/x86/ibrs_enabled
echo 0 > /sys/kernel/debug/x86/retp_enabled

by default all three fixes are enabled, if you want to disable them permanently (=on every boot) you can add these three options to your kernel command line:

noibrs noibpb nopti noretp spectre_v2=off

the last spectre_v2=off is redhat/CentOS specific and might be redundant with the previous ones.