Differences

This shows you the differences between two versions of the page.

--- spectre_and_meltdown_fixes_-_release_dates_for_linux_distros [19.01.2018 15:33] – [Performance Impact] Pascal Suter
+++ spectre_and_meltdown_fixes_-_release_dates_for_linux_distros [24.01.2018 18:19] – [Spectre and Meltdown fixes] Pascal Suter
@@ Line 1: / Line 1: @@
+====== Spectre and Meltdown fixes ======
+This page should give you a short overview of what is currently available to update your Intel based server or Workstation to get as good as possible patched against Spectre ( CVE 2017-5753 (Bounds Check Bypass / Variant 1) and CVE 2017-5715 (Branch Target Injection / Variant 2)) and Meltdown (CVE 2017-5754 (rogue data cache load / Variant 3)).
+===== What's needed? =====
+**this update will need a reboot of your server for sure, don't just update and continue to work without rebooting**
+in general you need to update your kernel to the latest versions provided by your distribution of choice. by now, pretty much  any distribution should have released patches.
+Here is how it's done...
+  apt-get update && apt-get upgrade
+... in ubuntu and ..
+  yum  update
+.. in RedHat and CentOS distributions.
+this will protect you against Variant 1 and Variant 3 vlunerabilities. In order to also protect against Variant 2, you further need to update the CPU's microcode (basically the CPU's firmware). Instructions on how to do that follow further down on this page
+===== Different Distributions and their update status =====
+here is a list of links with information about updates available from linux distributions I care most about:
+  * [[https://security-tracker.debian.org/tracker/CVE-2017-5754|Debian]] stretch is fixed as of kernel version 4.9.65-3**+deb9u2**
+  * [[https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown|Ubuntu]] (fixes for all maintained versions available as of Jan. 9th)
+  * [[https://forum.proxmox.com/threads/meltdown-and-spectre-linux-kernel-fixes.39110/|Proxmox VE]] debian based virtualization environment, fixed for Version 4 and 5 as explained in the linked forum post.
+  * [[https://lists.centos.org/pipermail/centos-announce/2018-January/022696.html|CentOS]] (started syncing to mirrors on Jan 4) kernel-3.10.0-693.11.6.el7.x86_64.rpm and related packages fix the bug
+  * [[https://access.redhat.com/errata/RHSA-2018:0007|RedHat Enterprise Linux RHEL 7]] (released 3.Jan)
+  * [[https://www.suse.com/c/suse-addresses-meltdown-spectre-vulnerabilities/|Suse]]
+for further information read those pages or check out the [[https://meltdownattack.com/#faq-fix|meltdown webpage link section]]
+===== Microcode Update - Yes it's necessary too! =====
+Update 22.1.2017: ** [[https://newsroom.intel.com/news/root-cause-of-reboot-issue-identified-updated-guidance-for-customers-and-partners/|has found the root cause for the latest reboot-issues and revokes all Microcode updates delivered between 01 and 20 January]]. This means, that you should NOT follow the bleow instructions to upgrade your microcode at the moment until intel releases the next version!**
+After you have installed the latest OS updates, your system should be protected against Variant 1 and 3, in order to protect against Variant 2, you also need to install a newer microcode (firmware of the CPU).
+Usually the microcode package is also a package in your distributions repository and is updated during a normal os upgrade. However, there where some [[https://newsroom.intel.com/news/intel-security-issue-update-addressing-reboot-issues/|stability issues]] with the microcodes released by Intel until today (18.1.18) so that for example RedHat removed them from their repos.
+So now you have two options: 1.) wait until a stable microcode update is available and stay vulnerable until then or 2.) install the currently available microcode update and risk having a less stable system. I have to mention, that intel says that only "some configurations" are affected without furhter specifying which configurations they are. I would therefore recommend. to simply try the new microcode and if it does indeed make your system unstable, revert to the currently used microcode.
+Here is, how you can download the microcode package (for all intel processors) and then insert this into your Linux installation for Linux to load the latest microcode.
+The latest officially released Microcodes can be found on the intel downloadcenter page. currently [[https://downloadcenter.intel.com/download/27431/Linux-Processor-Microcode-Data-File?v=t|this download here]] is the latest. there should be a banner at the top linking to a newer version, once released.
+**NOTE: see update above: Intel discurages the use of these microcodes as it seems that they can cause your system to be unstable. Intel in fact changed their recommendations as of Jan. 22 from "ask your vendor to get the latest microcote" to "OMG, don't install the latest microcode!".**
+<del>It seems however, that this package does not include the latest microcodes as they where distributed by RedHat before they removed them from their repos. For example the microcode for the ''Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz'' cpu is still at version ''21'' while the latest supplied by RedHat was ''25''. ''21'' does not yet have the patches for Variant 2. You can download a {{ :meltdown:intel-ucode_rh.tar.gz |copy of the intel-ucode from the removed RedHat Package here}}. </del>
+  - download the package of your choice to your System: <code>
+cd /root/
+wget <url>
+</code>
+  - now move your existing microcode package to another location, so you can move it back in case you have these stability issues that some users had with the new ones. then unpack and load the new microcode: <code>
+cd /lib/firmware/
+mv intel-ucode /root/intel-ucode.old
+tar xvf /root/microcode-20180108.tgz
+echo 1 > /sys/devices/system/cpu/microcode/reload
+</code>
+you can double check if your microcode was loaded using <code>dmesg | grep microcode</code>.
+you should now be up to date with the latest patches for all three Variants of the Spectre & Meltdown vulnerability.
+==== Understanding the release notes ====
+Intel's releasenotes are somewhat cryptic. Here is how to read the following lines of the current release notes: <code>
+-- Updates upon 20171117 release --
+IVT C0		(06-3e-04:ed) 428->42a
+SKL-U/Y D0	(06-4e-03:c0) ba->c2
+BDW-U/Y E/F	(06-3d-04:c0) 25->28
+HSW-ULT Cx/Dx	(06-45-01:72) 20->21
+Crystalwell Cx	(06-46-01:32) 17->18
+BDW-H E/G	(06-47-01:22) 17->1b
+HSX-EX E0	(06-3f-04:80) 0f->10
+SKL-H/S R0	(06-5e-03:36) ba->c2
+HSW Cx/Dx	(06-3c-03:32) 22->23
+HSX C0		(06-3f-02:6f) 3a->3b
+BDX-DE V0/V1	(06-56-02:10) 0f->14
+BDX-DE V2	(06-56-03:10) 700000d->7000011
+KBL-U/Y H0	(06-8e-09:c0) 62->80
+KBL Y0 / CFL D0	(06-8e-0a:c0) 70->80
+KBL-H/S B0	(06-9e-09:2a) 5e->80
+CFL U0		(06-9e-0a:22) 70->80
+CFL B0		(06-9e-0b:02) 72->80
+SKX H0		(06-55-04:b7) 2000035->200003c
+GLK B0		(06-7a-01:01) 1e->22
+</code>
+  * ''-- Updates upon 20171117 release --'' This means, that this package contains the following updates since the last release. so only the microcodes for the cpus mentioned here where updated, everything else is identical to the last release.
+  * ''SKL-U/Y D0	(06-4e-03:c0) ba->c2''
+    * ''SKL'' stands for Sky-Lake
+    * ''06-4e-03'' is actually the most useful part, it tells you what cpu that is in cpu-family, model and stepping. you can get this information from ''/proc/cpuinfo'' with this command: <code>
+grep -P "^(cpu family)|(model\s*:)|(stepping)" /proc/cpuinfo | tail -3
+</code>. ''06'' is the family, ''4e'' is the stepping in HEX format (use google or a scientific calculator to convert if you are lazy :)) and ''03'' is the stepping.
+    * the last part ''ba->c2'' is the relevant part of the version number that changed. For this specific Skylake CPU the Spectre Patch is supposed to be in releases ''0xc2'' or newer, so this one here contains the patch. Sadly the list with all these releases is under NDA, so i can't share it here. But in general you can expect everything that is released starting with the current package to have the fix in place.
+  * by the way, ''06-4e-03'' is also the filename of that microcode.
+===== Minimalistic Fix on CentOS 7.4 =====
+Should you, for some reason, not be able or willing to run a full update, I have here a minimalistic fix for your centos:
+  - download the necessary update packages <code>
+mkdir -p /opt/meltdown
+cd /opt/meltdown
+for p in kernel-abi-whitelists-3.10.0-693.11.6.el7.noarch.rpm kernel-debug-3.10.0-693.11.6.el7.x86_64.rpm kernel-debug-devel-3.10.0-693.11.6.el7.x86_64.rpm kernel-devel-3.10.0-693.11.6.el7.x86_64.rpm kernel-doc-3.10.0-693.11.6.el7.noarch.rpm kernel-headers-3.10.0-693.11.6.el7.x86_64.rpm kernel-tools-3.10.0-693.11.6.el7.x86_64.rpm kernel-tools-libs-3.10.0-693.11.6.el7.x86_64.rpm kernel-tools-libs-devel-3.10.0-693.11.6.el7.x86_64.rpm perf-3.10.0-693.11.6.el7.x86_64.rpm python-perf-3.10.0-693.11.6.el7.x86_64.rpm kernel-3.10.0-693.11.6.el7.src.rpm kernel-3.10.0-693.11.6.el7.x86_64.rpm; do wget http://mirror.centos.org/centos/7.4.1708/updates/x86_64/Packages/$p; done
+</code>
+  - create a repository: <code>createrepo .</code>
+  - add your repository to yum <code>mydir=`pwd`
+cat > /etc/yum.repos.d/CentOS-meltdown.repo <<EOF
+# CentOS-meltdown.repo
+#
+# contains minimalistic update to fix meltdown and spectre
+[meltdown-updates]
+name=CentOS-$releasever - Meltdown-Updates
+baseurl=file://$mydir
+gpgcheck=0
+enabled=1
+EOF
+</code>
+  - run the update and reboot the machine: <code>
+yum update
+reboot
+</code>
+===== Test-Tools =====
+  * [[https://github.com/speed47/spectre-meltdown-checker]]
+===== Performance Impact =====
+the fix for all this works in a way that it may affect the system performance negatively. Different sources claim different results reaching from no impact at all up to a 30% slowdown. As always, Benchmarks are probably not too representative for your realworld experience. In order to find out what the difference in performance is, you can simply disable the workaround on a patched kernel to run your workload once with and once without the patch.
+In CentOS (and probably other linux distributions as well) the workarounds can be enabled or disabled without a reboot using these commands:
+<code>
+echo 0 > /sys/kernel/debug/x86/pti_enabled
+echo 0 > /sys/kernel/debug/x86/ibpb_enabled
+echo 0 > /sys/kernel/debug/x86/ibrs_enabled
+</code>
+by default all three fixes are enabled, if you want to disable them permanently (=on every boot) you can add these three options to your kernel command line:
+<code>
+noibrs noibpb nopti
+</code>