Richard Parmiter

Virtualisation blog and Knowledge Base

  • You are here: 
  • Home
  • Netscaler cannot boot with message “Can’t load Kernel” after failed firmware upgrade

Netscaler cannot boot with message “Can’t load Kernel” after failed firmware upgrade

Posted on January 16th, 2013

Came across this tricky problem today at a client site with some old and unsupported Netscaler devices.

Customer had a failed Netscaler 7000 appliance in an HA pair and got it replaced by Citrix. However, their working Netscaler was running version 8.1 build 56.11 and they wanted to downgrade the firmware on the new appliance to the same version before re-establishing the HA. However the issue experienced would have also occurred for any firmware upgrade as well as downgrade.

For reference the replacement NS7000 had version 9.3 build 56.5cl and eventually was successfully downgraded to v8.1 build 56.11

The following was performed after taking a backup of the replacement netscaler:

  • Located the firmware on the working Netscaler in the /var/nsinstall folder
  • Used WinSCP to copy this locally
  • Copied the firmware to the /var/nsinstall folder on the replacement Netscaler
  • The firmware had already been un-tar’ed in the folder so the firmware and documentation was already available for selection
  • Using the GUI, ran through the upgrade firmware wizard
  • This failed due to a lack of space on the flash drive
  • Rebooted the Netscaler (just to be sure before trying again)

The Netscaler failed to boot.

The blue LCD screen on the front of the device only momentarily showed “booting” and then turned off.

Fortunately I had a Netscaler console cable available, and this was connected to a laptop in the following order:

  • Laptop –> USB to serial cable –> Netscaler serial console cable –> RJ45 to Serial adapter –> Netscaler console port

This enabled me to open a console session to the netscaler using a Putty serial connection (9600, 8, 1, none, none).

the console was displaying a failed boot stating:

can’t load ‘kernel’
can’t load ‘kernel.old’
can’t load ‘kernel.work’
no bootable kernel

It seemed that in the process of the failed firmware upgrade the flash boot disk was amended and the kernel was deleted.

The Citrix forum post (http://forums.citrix.com/thread.jspa?threadID=96524) states that F4 can be pressed during boot to enable boot device selection, and indeed a list was shown for:

1 Netscaler

5 Drive 1

Default: 1

But no matter which keys were pressed, it always attempted to boot from the flash disk.

I eventually resolved this by the following work around at the OK prompt:

set currdev=disk1s1a:

load disk1s1a:/kernel

This is detailed in Citrix forum (http://forums.citrix.com/thread.jspa?threadID=290421) and then boots the Netscaler from the Hard disk. It is then possible to copy the missing kernel back to the flash drive (from the backup made before starting. I made a backup in /var)

cp /var/kernel.gz /flash

Rebooting the device then boots from the kernel on the flash drive as normal – so problem solved!

However, I still needed to downgrade the firmware so this time ran the ./nsinstall command from the cli but again failed due to lack of space. So, copied the /flash/.recovery/hd.recovery.gz file locally and removed from the netscaler. Then performed the firmware installation and said NO to the reboot and manually copied that file back again.

I also manually checked the /flash folder for the new firmware .gz file and it was there as expected.

On rebooting the Netscaler it correctly booted, but didn’t set the NSIP correctly, instead defaulting to the 192.168.100.1 address. Using the console i set NSIP and rebooted again:

  • set ns config -ipaddress x.x.x.x -netmask y.y.y.y     (note x=NSIP & y=netmask)
  • add ns ip z.z.z.z y.y.y.y -type MIP   (note z=MIP IP Address, y=Subnet mask)
  • save ns config
  • reboot

After this final reboot the IP was set correctly and the device was running v8.1 build 56.11 cl as expected.

Hope this helps someone as it took me ages to sort this out!

 

  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Live
  • MySpace
  • Reddit
  • RSS

Filed under Citrix Netscaler |

One Response to “Netscaler cannot boot with message “Can’t load Kernel” after failed firmware upgrade”

  1. Jan C Andersen Says:
    June 2nd, 2016 at 12:27 pm

    Hi
    This really helped me, as my FW upgrade to 11.0-66.11 sort got stuck in the middle of .\installns. I did however still have file access to the system, and because I still could see /flash/kernel.gz and the new /flash/ns-11.0-66.11.gz, I decided to move old FW versions from this directory. To free disk space I also moved old fw from /nsinstall folder and rebooted the Netscaler.
    As I wasn’t quite sure if everything went right with the first upgrade, I decided to run the ./installns once more and then we were good to go

Leave a Reply

*