Xen and serial console over IPMI (SOL)

Recently I had to configure a serial over lan (SOL) console on a bigger Supermicro server (2042G-TRF for those who are interested) that should run Xen. This turned out to be not too easy and several issues had to be resolved.

The first thing I had to do was an upgrade of the BIOS. The original BIOS shared the IPMI IRQ with serial console0. This resulted in scrambled console output, regardless what I tried to do. Current BIOS versions put IPMI on IRQ 5. Before you try anything, make sure that you do not share IRQs between IPMI and a real serial console.

Then I tried to add COM3 to the console list of Xen. Unfortunately, there is no warning that Xen supports only two serial consoles, which are COM1 and COM2. The console just won’t work. Luckily, there is this nice manual to be found at:

/usr/share/doc/packackages/xen/pdf/user.pdf

In there, there are lots of parameters, and also a somewhat sparse description of how to setup the serial console. It turned out, that I had to reconfigure COM2 (likely COM1 would also have worked) to different IO and IRQ. To get the current values of IO-Port and IRQ, either look into the BIOS and write down the values there, or run a default kernel without Xen and run the following command:

dmesg | grep ttyS2

The result should look similar to this:

<6>[   10.232117] 00:09: ttyS2 at I/O 0x3e8 (irq = 5) is a 16550A

This means, we need to set COM2 to IO-Port 0x3e8 and to IRQ 5. The only thing that is now missing is the serial line speed and mode of the serial connection. In my case, I chose a baud rate of 115200,8n1 for the connection.

Now lets put all of this together. First is the grub configuration. This is twofold. Part one is configuring grub in a way that it is also displayed in the serial console. Commonly, this is done in the global section at the beginning of /boot/grub/menu.lst :

serial --unit=2 --speed=115200
terminal --timeout=8 serial console

Unit 2 specifies the third (start counting at 0) serial console.

Part two of the grub configuration affects the Xen section. There, add the console parameters like this:

kernel (hd0,0)/xen.gz console=vga,com2 com2=115200,8n1,0x3e8,5
module (hd0,0)/vmlinuz-2.6.32.43-0.13-xen root=/dev/md1 console=tty0 console=xvc0,115200

In my case, we have root on a mirrored raid, you just have to add the console parameters and com2 parameter to the configuration file. Note that the last console that is in the module line is considered the system console by the kernel.

To make all of this permanent and survive the next kernel update, also add the parameters to /etc/sysconfig/bootloader:

XEN_KERNEL_APPEND="console=tty0 console=xvc0,115200"
XEN_APPEND="console=vga,com2 com2=115200,8n1,0x3e8,5"

Again, your options may vary.

The last configuration to be made is activating a getty for xvc0. This is accomplished in /etc/inittab. Search for a line starting with ‘#S0’. There add a line like the following:

S0:12345:respawn:/sbin/agetty -L 115200 xvc0 vt102

After doing all of this, you are ready to test your SOL console. For me, the following command works nicely:

ipmitool -I lanplus -H <IPMI-IP-Adress> -U <IPMI-User> sol activate
Advertisements
This entry was posted in IPMI, openSUSE, SOL, Xen. Bookmark the permalink.

3 Responses to Xen and serial console over IPMI (SOL)

  1. Miles Fidelman says:

    Have you had any luck getting your IPMI sol console to get into the BIOS on your Supermicro box? I have a new box, and at this point:
    – I can connect to the IPMI board
    – I can power cycle via IPMI
    – I can connect via SOL
    – at the moment, I have things set to boot via LAN, and I can see the console messages from the Intel boot agent (on the SOL session), but…
    – I can’t seem to interrupt the hardware boot and get into the BIOS over the SOL terminal
    Any thoughts or suggestions?

    Thanks!

    • Miles Fidelman says:

      Nevermind – looks like hitting lots of CRs gets the box’s attention, then a DELETE key works.

      • azouhr says:

        I never tried this. I always used this IPMIView tool from Supermicro which provides a KVM console also for linux. However, it is interesting to read that hitting lots of CRs also does the trick to get into the BIOS over SOL.

        Thanks!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s