Differences

This shows you the differences between two versions of the page.

--- spectre_and_meltdown_fixes_-_release_dates_for_linux_distros [19.01.2018 15:35] – Pascal Suter
+++ spectre_and_meltdown_fixes_-_release_dates_for_linux_distros [24.01.2018 18:19] – [Spectre and Meltdown fixes] Pascal Suter
@@ Line 1: / Line 1: @@
-====== spectre and Meltdown fixes - release dates for linux distros ======
+====== Spectre and Meltdown fixes ======
-Unfortunately the [[https://meltdownattack.com|Meltdown and Spectre]] vulnerabilities became Public too early. So now we are all keen on getting updates for our servers to work around the respective design issues in modern CPU's ASAP. Originally, the official announcement was planned for Jan. 9, so we can expect that by then most OS's including most Linux Distributions will have a fix in their repos which can simply be installed by running whatever your regular update routine is. That's
+This page should give you a short overview of what is currently available to update your Intel based server or Workstation to get as good as possible patched against Spectre ( CVE 2017-5753 (Bounds Check Bypass / Variant 1) and CVE 2017-5715 (Branch Target Injection / Variant 2)) and Meltdown (CVE 2017-5754 (rogue data cache load / Variant 3)).
+===== What's needed? =====
+**this update will need a reboot of your server for sure, don't just update and continue to work without rebooting**
+in general you need to update your kernel to the latest versions provided by your distribution of choice. by now, pretty much  any distribution should have released patches.
+Here is how it's done...
   apt-get update && apt-get upgrade
-in ubuntu and
+... in ubuntu and ..
   yum  update
-in RedHat and CentOS distributions.
+.. in RedHat and CentOS distributions.
-**this update will need a reboot of your server for sure, don't just update and continue to work without rebooting**
+this will protect you against Variant 1 and Variant 3 vlunerabilities. In order to also protect against Variant 2, you further need to update the CPU's microcode (basically the CPU's firmware). Instructions on how to do that follow further down on this page
-However, now that the news is out, every OS vendor tries to be as fast as possible in pushing out patches, so possibly fixes will be released earlier.
+===== Different Distributions and their update status =====
-here is a list of links with information about updates available from linux distributions i care about:
+here is a list of links with information about updates available from linux distributions I care most about:
   * [[https://security-tracker.debian.org/tracker/CVE-2017-5754|Debian]] stretch is fixed as of kernel version 4.9.65-3**+deb9u2**
   * [[https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown|Ubuntu]] (fixes for all maintained versions available as of Jan. 9th)
@@ Line 19: / Line 26: @@
 for further information read those pages or check out the [[https://meltdownattack.com/#faq-fix|meltdown webpage link section]]
-===== Minimalistic Fix on CentOS 7.4 =====
+===== Microcode Update - Yes it's necessary too! =====
-usually all you need todo today is simply update your linux to the latest patch level and you should be good. as of now (18.1.18) you might need to update the microcode (firmware) for your CPU manually, as some distributions (RHEL for example) have removet the intel firmware from their repos for the moment, due to some [[https://newsroom.intel.com/news/intel-security-issue-update-addressing-reboot-issues/|stability issues]].
+Update 22.1.2017: ** [[https://newsroom.intel.com/news/root-cause-of-reboot-issue-identified-updated-guidance-for-customers-and-partners/|has found the root cause for the latest reboot-issues and revokes all Microcode updates delivered between 01 and 20 January]]. This means, that you should NOT follow the bleow instructions to upgrade your microcode at the moment until intel releases the next version!**
+After you have installed the latest OS updates, your system should be protected against Variant 1 and 3, in order to protect against Variant 2, you also need to install a newer microcode (firmware of the CPU).
+Usually the microcode package is also a package in your distributions repository and is updated during a normal os upgrade. However, there where some [[https://newsroom.intel.com/news/intel-security-issue-update-addressing-reboot-issues/|stability issues]] with the microcodes released by Intel until today (18.1.18) so that for example RedHat removed them from their repos.
+So now you have two options: 1.) wait until a stable microcode update is available and stay vulnerable until then or 2.) install the currently available microcode update and risk having a less stable system. I have to mention, that intel says that only "some configurations" are affected without furhter specifying which configurations they are. I would therefore recommend. to simply try the new microcode and if it does indeed make your system unstable, revert to the currently used microcode.
+Here is, how you can download the microcode package (for all intel processors) and then insert this into your Linux installation for Linux to load the latest microcode.
+The latest officially released Microcodes can be found on the intel downloadcenter page. currently [[https://downloadcenter.intel.com/download/27431/Linux-Processor-Microcode-Data-File?v=t|this download here]] is the latest. there should be a banner at the top linking to a newer version, once released.
+**NOTE: see update above: Intel discurages the use of these microcodes as it seems that they can cause your system to be unstable. Intel in fact changed their recommendations as of Jan. 22 from "ask your vendor to get the latest microcote" to "OMG, don't install the latest microcode!".**
+<del>It seems however, that this package does not include the latest microcodes as they where distributed by RedHat before they removed them from their repos. For example the microcode for the ''Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz'' cpu is still at version ''21'' while the latest supplied by RedHat was ''25''. ''21'' does not yet have the patches for Variant 2. You can download a {{ :meltdown:intel-ucode_rh.tar.gz |copy of the intel-ucode from the removed RedHat Package here}}. </del>
+  - download the package of your choice to your System: <code>
+cd /root/
+wget <url>
+</code>
+  - now move your existing microcode package to another location, so you can move it back in case you have these stability issues that some users had with the new ones. then unpack and load the new microcode: <code>
+cd /lib/firmware/
+mv intel-ucode /root/intel-ucode.old
+tar xvf /root/microcode-20180108.tgz
+echo 1 > /sys/devices/system/cpu/microcode/reload
+</code>
+you can double check if your microcode was loaded using <code>dmesg | grep microcode</code>.
+you should now be up to date with the latest patches for all three Variants of the Spectre & Meltdown vulnerability.
+==== Understanding the release notes ====
+Intel's releasenotes are somewhat cryptic. Here is how to read the following lines of the current release notes: <code>
+-- Updates upon 20171117 release --
+IVT C0		(06-3e-04:ed) 428->42a
+SKL-U/Y D0	(06-4e-03:c0) ba->c2
+BDW-U/Y E/F	(06-3d-04:c0) 25->28
+HSW-ULT Cx/Dx	(06-45-01:72) 20->21
+Crystalwell Cx	(06-46-01:32) 17->18
+BDW-H E/G	(06-47-01:22) 17->1b
+HSX-EX E0	(06-3f-04:80) 0f->10
+SKL-H/S R0	(06-5e-03:36) ba->c2
+HSW Cx/Dx	(06-3c-03:32) 22->23
+HSX C0		(06-3f-02:6f) 3a->3b
+BDX-DE V0/V1	(06-56-02:10) 0f->14
+BDX-DE V2	(06-56-03:10) 700000d->7000011
+KBL-U/Y H0	(06-8e-09:c0) 62->80
+KBL Y0 / CFL D0	(06-8e-0a:c0) 70->80
+KBL-H/S B0	(06-9e-09:2a) 5e->80
+CFL U0		(06-9e-0a:22) 70->80
+CFL B0		(06-9e-0b:02) 72->80
+SKX H0		(06-55-04:b7) 2000035->200003c
+GLK B0		(06-7a-01:01) 1e->22
+</code>
+  * ''-- Updates upon 20171117 release --'' This means, that this package contains the following updates since the last release. so only the microcodes for the cpus mentioned here where updated, everything else is identical to the last release.
+  * ''SKL-U/Y D0	(06-4e-03:c0) ba->c2''
+    * ''SKL'' stands for Sky-Lake
+    * ''06-4e-03'' is actually the most useful part, it tells you what cpu that is in cpu-family, model and stepping. you can get this information from ''/proc/cpuinfo'' with this command: <code>
+grep -P "^(cpu family)|(model\s*:)|(stepping)" /proc/cpuinfo | tail -3
+</code>. ''06'' is the family, ''4e'' is the stepping in HEX format (use google or a scientific calculator to convert if you are lazy :)) and ''03'' is the stepping.
+    * the last part ''ba->c2'' is the relevant part of the version number that changed. For this specific Skylake CPU the Spectre Patch is supposed to be in releases ''0xc2'' or newer, so this one here contains the patch. Sadly the list with all these releases is under NDA, so i can't share it here. But in general you can expect everything that is released starting with the current package to have the fix in place.
+  * by the way, ''06-4e-03'' is also the filename of that microcode.
+===== Minimalistic Fix on CentOS 7.4 =====
 Should you, for some reason, not be able or willing to run a full update, I have here a minimalistic fix for your centos:
@@ Line 48: / Line 113: @@
 reboot
 </code>
-===== Microcode Update - Yes it's necessary too! =====
-Your system should now be patched against Variant 1 and 3, in order to protect against Variant 2, you also need to install a newer microcode. Keep in mind however, that there where some stability issues with the microcodes released until today (18.1.18) so that for example RedHat removed them from their repos. however, this step can be reverted should you have issues with the new microcode (sudden reboots where reported), so if you can afford a few crashes at worst, you should still give it a try and only revert when the system gets too unstable for you to work with:
-  - go to [[https://downloadcenter.intel.com/download/27431/Linux-Processor-Microcode-Data-File?v=t|the intel download center]] and download the latest microcode package.
-  - <code>
-cd /root/
-wget <url copied from downloadcenter>
-</code>
-  - now move your existing microcode package to another location, so you can move it back in case you have these stability issues that some users had with the new ones. then unpack and load the new microcode: <code>
-cd /lib/firmware/
-mv intel-ucode /root/intel-ucode.old
-tar xvf /root/microcode-20180108.tgz
-echo 1 > /sys/devices/system/cpu/microcode/reload
-</code>
-you can double check if your microcode was laded using <code>dmesg | grep microcode</code>.
-you should now up to date with the latest patches for all three Variants of the Spectre & Meltdown vulnerability.
 ===== Test-Tools =====
   * [[https://github.com/speed47/spectre-meltdown-checker]]
 ===== Performance Impact =====
-the fix for all this works in a way that it may affect the system performance negatively. Different sources claim different results reaching from no impact at all up to a 30% slowdown. As always, Benchmarks are probably not too representative for your realworld experience. In order to find out what the difference in performance is, you can simply disable the workaround on a patched kernel to run your workload once with and once without the patch. the workaround can be disabled by passing the boot option ''pti=off''.
+the fix for all this works in a way that it may affect the system performance negatively. Different sources claim different results reaching from no impact at all up to a 30% slowdown. As always, Benchmarks are probably not too representative for your realworld experience. In order to find out what the difference in performance is, you can simply disable the workaround on a patched kernel to run your workload once with and once without the patch.
+In CentOS (and probably other linux distributions as well) the workarounds can be enabled or disabled without a reboot using these commands:
+<code>
+echo 0 > /sys/kernel/debug/x86/pti_enabled
+echo 0 > /sys/kernel/debug/x86/ibpb_enabled
+echo 0 > /sys/kernel/debug/x86/ibrs_enabled
+</code>
+by default all three fixes are enabled, if you want to disable them permanently (=on every boot) you can add these three options to your kernel command line:
+<code>
+noibrs noibpb nopti
+</code>