Upgrading the kernel fails

Question

I currently have kernel version: 4.14.79-v7+ (and I am using Raspbian Stretch)

pi@pi:~ $ uname -a
Linux pi 4.14.79-v7+ #1159 SMP Sun Nov 4 17:50:20 GMT 2018 armv7l GNU/Linux

When I try to update and upgrade my system the packages raspberrypi-bootloader raspberrypi-kernel get upgraded:

root@pi:/etc/kernel/postinst.d# sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  base-files bluez-firmware curl dirmngr gnupg gnupg-agent gpgv hostapd influxdb libc-bin libc-dev-bin libc-l10n libc6 libc6-dbg libc6-dev libcurl3
  libcurl3-gnutls libmosquitto1 libpam-systemd libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc libraspberrypi0 libssl1.0.2 libsystemd0 libudev1
  libwbclient0 libxapian30 locales mosquitto mosquitto-clients multiarch-support openssh-client openssh-server openssh-sftp-server openvpn python3-six
  raspberrypi-bootloader raspberrypi-kernel raspi-config raspi-copies-and-fills samba-common ssh systemd systemd-sysv tzdata udev wireless-regdb
  wpasupplicant
49 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 145 MB of archives.
After this operation, 810 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
...

The upgrade shows no suspicious warnings or errors (I omitted it here, as it is really long) and the returncode of the upgrade command is 0. BUT: The kernel upgrade was not successful as uname still shows the old version number.

pi@pi:~ $ uname -a
Linux pi 4.14.79-v7+ #1159 SMP Sun Nov 4 17:50:20 GMT 2018 armv7l GNU/Linux

But if I have a look e.g. in /lib/modules, there are only folders for the new version

pi@pi:/etc/kernel/postinst.d $ ls -l /lib/modules
total 8
drwxr-xr-x 3 root root 4096 Mar 14 10:47 4.14.98+
drwxr-xr-x 3 root root 4096 Mar 14 10:47 4.14.98-v7+

I am not the first one describing this problem, e.g. in Linux modules mismatch version the same problem occurred. The threads solution (sudo apt-get update; sudo apt-get install --reinstall raspberrypi-bootloader raspberrypi-kernel) did not help me. But another user gave me a hint what might cause the problem: He also had problems because of his modified /boot/cmdline.txt. Mine is also modified, as my system is full-disc encrypted (I enabled the encryption like it was shown in this tutorial).

My /boot/cmdline.txt looks like this: dwc_otg.lpm_enable=0 console=tty1 root=/dev/mapper/sdcard rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cryptdevice=/dev/mmcblk0p2:sdcard

Can anyone help me how to fix my problems, the problem is that after the upgrade I am not able to build my custom initramfs as I am getting the following error because my system is still looking for the old kernel files:

mkinitramfs -o /boot/initramfs.gz
WARNING: missing /lib/modules/4.14.79-v7+
Ensure all necessary drivers are built into the linux image!
depmod: ERROR: could not open directory /lib/modules/4.14.79-v7+: No such file or directory
depmod: FATAL: could not search modules: No such file or directory
depmod: WARNING: could not open /var/tmp/mkinitramfs_ePmCvg/lib/modules/4.14.79-v7+/modules.order: No such file or directory
depmod: WARNING: could not open /var/tmp/mkinitramfs_ePmCvg/lib/modules/4.14.79-v7+/modules.builtin: No such file or directory

This is a huge problem because I use a very custom initramfs and as long as I am not able to build the initramfs my system will not boot correctly after the system upgrade.

@Seamus better not to boot before you exactly know what's going on. It may result in a *kernel panic* or failing to boot. — Ingo, Mar 14 '19 at 14:05
@Ingo: If you say so, but a reboot is required to load the new kernel, and you may see "weird things" when your kernel is out of sync with other updates. I think the reboot advice is commonly offered in this situation. It was given to me - and it worked (I'm still here aren't I ? :) All that said, my comment was a question - not a suggestion, eh? — Seamus, Mar 14 '19 at 14:13
@Seamus And my comment was only a suggestion, not criticism ;-) — Ingo, Mar 14 '19 at 14:26
@Ingo: Oh no... sorry - didn't mean to suggest it was. But we've got to be able to discuss these things... how else to separate truth from fiction? — Seamus, Mar 14 '19 at 15:00
Sorry for my delayed answer. I have a full SD-card image backup, thereby even doing a reboot is okay, as I am able to fully recover the system ;-) — Stefan Wegener, Mar 15 '19 at 09:10

score 2 · Accepted Answer · answered Mar 14 '19 at 15:19

Using an initramfs is problematic because it isn't supported by Raspbian and if the initramfs doesn't match the kernel version it will result in an unbootable system. This can occur when the kernel is updated but not the initramfs. This can also occur if only some modules/drivers are updated, not the whole kernel. On updates you have always to check what was updated and then renew the initramfs manual.

On a kernel update then the corresponding modules are also updated. Raspbian deletes the old modules immediately to save space but the old kernel is still running. The kernel cannot be swapped when it is running. You can only boot with the new kernel. Now you have the situation that you described, old kernel running but new modules installed.

It doesn't matter on reboot if you do not use an initramfs, but with it it's dramatic. The initramfs does not contain the new modules needed for boot up.

You have to make a new initramfs corresponding to the new modules before booting!

rpi ~$ sudo mkinitramfs -o /boot/initramfs.gz 4.14.98-v7+

I have tried to enable automatic update-initramfs. You may have a look at How can I use an init ramdisk (initramfs) on boot up Raspberry Pi?.

Thanks for pointing out that when using mkinitramfs I can specify the version. I should have read the manpage of the command more carefully... Currently I am short of time and probably will be able to test it on Monday. I will tell you, as soon, as I know more. — Stefan Wegener, Mar 15 '19 at 09:25

Upgrading the kernel fails

1 Answers1