Diagnosing Kernel Panics on CentOS 7

Sometimes I have had to diagnose problems with Kernel but it is so sporadic that I always forget the steps to take information to know what the problem comes from, so this is a little guide just to remember what I did the last time.

I also give some advice to prevent what happened to me in the last kernel panic.

When I run “yum update” and this update installs a new kernel version there are some possibilities to generate a kernel panic the next time machine boots, so the first advice would be rebooting the computer when a new kernel is installed and not wait some days later just to check it.

The last time kernel panic was due to /boot partition was full installing the new kernel and the installation process wasn’t end successfully, one option would be I have to control the size of this partition, second option increase boot partition to prevent this problem.

/dev/sda1                239M  197M   25M  89% /boot

Second advice it would be run the clean tasks before running a system update just to release space:

# package-cleanup --oldkernels --count=2

Well once kernel panics to debug the problem it’s needed to remove this kernel parameter (if present): rhgb quiet. This is made by editing the file /etc/default/grub, the original line to edit would be:

GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet"

New line:

GRUB_CMDLINE_LINUX="crashkernel=auto"

Then if you are using Grub2 on CentOS 7 like me you can run the following command:

# grub2-mkconfig -o /boot/grub2/grub.cfg
 Generating grub configuration file ...

The path could be different if you are using an UEFI system read HowTo Grub2 to more information.

The next time you boot with this kernel more log is shown so you can find more accurately the cause of this problem, you also can search the error message on Internet if you haven’t any clue about it.

Apart from removing the mentioned kernel options you add rdshell so a shell is shown if kernel cannot boot properly.

dracut:# 

A common problem when a kernel fails could that is not included in initram file miss some driver to boot properly, like LVM driver when you have root filesystem over LVM, if it is the case you must recreate the initram file:

# dracut -f -v /boot/initramfs-2.6.32-504.el6.x86_64.img 2.6.32-504.el6.x86_64

As a last shot if you like debug core files when kernel panics a core file was generated in / directory so you can use gdb to see where the problem is:

# gdb /boot/vmlinuz-3.10.0-693.11.1.el7.x86_64 core.2912
...
(gdb) bt
#0  0x00007f158da191f7 in raise () from /lib64/libc.so.6
#1  0x00007f158da1a8e8 in abort () from /lib64/libc.so.6
...
(gdb) quit


“Civil disobedience becomes a sacred duty when the state has become lawless or corrupt”
— Mahatma Gandhi

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s