IBR Server Management

Remote management of our servers is mainly based on

• SSH to access running operation systems (hostname.ibr.cs.tu-bs.de),
• IPMI to access management controllers (hostname.mgmt.ibr.cs.tu-bs.de), this includes power control and Serial-over-LAN (SOL) console access
• iDRAC to access DELL management controllers via Web (https://hostname.mgmt.ibr.cs.tu-bs.de)
• Libvirt to manage our KVM virtual machines on kvm1..kvm4

Machines (physical or virtual) may be maintained

• only by IBR admins. These hosts may be attached to the VLAN ibr-core and be regarded "trusted" in some sense, e.g. as NFSv3 clients with less strict authentication policies.
• by IBR fellows. These hosts are usually attached to the VLAN ibr-misc, but never to ibr-core. Usually, IBR admins should also be given root access, e.g. by ~root/.ssh/authorized_keys.

IPMI

The tool ibr-ipmi allows all users to read some information, e.g.:
ibr-ipmi knecht power status

Host supervisors have some more priviledges, e.g.:
ibr-ipmi knecht power on

Host supervisors can use IPMI "serial-over-lan" (SOL) to connect to the serial console:
ibr-console x12
Of course, this requires the host to have getty running on the serial line and the BIOS to be configured for console redirection to SOL.

The following examples require IPMI access credentials and permissions, i.e. the authorization of ibr-ipmi does not work for them. Look for IPMI documentation to get more details.

Get a list of available IPMI commands:
ipmitool -I lanplus -H cloud1.mgmt.ibr.cs.tu-bs.de -U root -f ~/.ipmipw

Power status:
ipmitool -I lanplus -H cloud1.mgmt.ibr.cs.tu-bs.de -U root -f ~/.ipmipw power status

Get a serial console connection (SOL):
ipmi-console -h x11.mgmt.ibr.cs.tu-bs.de -u root -P

SOL requires BIOS settings to connect COM1 to the BMC and some more IPMI settings:
ipmitool sol set privilege-level admin 1
ipmitool sol set non-volatile-bit-rate 115.2 1
ipmitool sol set volatile-bit-rate 115.2 1
ipmitool sol set force-encryption true 1
ipmitool sol set force-authentication true 1
ipmitool sol set enabled true 1
ipmitool sol payload enable 1

A symlink to TEMPLATE in /ibr/adm/fai/config/files/etc/init/ttyS0.conf may be used on FAI-maintained hosts to establish a serial console login.

IPMI BMC configuration

root@x1:~# ipmitool lan print
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD 
Auth Type Enable        : Callback : MD2 MD5 
                        : User     : MD2 MD5 
                        : Operator : MD2 MD5 
                        : Admin    : MD2 MD5 
                        : OEM      : 
IP Address Source       : DHCP Address
IP Address              : 10.9.34.178
Subnet Mask             : 255.255.0.0
MAC Address             : 90:b1:1c:17:2f:78
SNMP Community String   : public
IP Header               : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Gratituous ARP Intrvl   : 2.0 seconds
Default Gateway IP      : 10.9.0.1
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14
Cipher Suite Priv Max   : Xaaaaaaaaaaaaaa
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM

root@x1:~# ipmitool lan set 1 vlan id 140  ## (only if VLAN on common eth0)

root@x1:~# ipmitool lan set 1 ipsrc dhcp

root@x1:~# ipmitool lan set 1 access on

root@x1:~# ipmitool user list 1
ID  Name	     Callin  Link Auth	IPMI Msg   Channel Priv Limit
2   root             true    true       true       ADMINISTRATOR

root@x1:~# ipmitool user set password 2
Password for user 2: XXXXX

root@x1:~# ipmitool user test 2 16
Password for user 2: XXXXX
Success

root@x1:~# ipmitool user priv 2 4 1

root@x1:~# ipmitool user enable 2

root@x1:~# ipmitool channel info 1
Channel 0x1 info:
  Channel Medium Type   : 802.3 LAN
  Channel Protocol Type : IPMB-1.0
  Session Support       : multi-session
  Active Session Count  : 0
  Protocol Vendor ID    : 7154
  Volatile(active) Settings
    Alerting            : disabled
    Per-message Auth    : disabled
    User Level Auth     : disabled
    Access Mode         : always available
  Non-Volatile Settings
    Alerting            : disabled
    Per-message Auth    : disabled
    User Level Auth     : disabled
    Access Mode         : always available

root@x1:~# ipmitool channel setaccess 1 2 callin=on ipmi=on link=on privilege=4

--- for IPMI prometheus/grafana monitoring:

root@x1:~# ipmitool user set username 3 monitoring

root@x1:~# ipmitool user set password 3
Password for user 3: XXXXX

root@x1:~# ipmitool user priv 3 2 1

root@x1:~# ipmitool user enable 3

root@x1:~# ipmitool channel setaccess 1 3 callin=on ipmi=on link=on privilege=2

    

BIOS settings

IBM x3650