Managing VMware ESX Direct User Interface (DCUI)

Introduction

Version: vSphere 5.5

The Direct Console User Interface (DCUI) is the front-end management system that allows for some basic configuration changes and troubleshooting options should the VMware ESX host become unmanagable via conventional tools such as the vSphere Client or vCenter.

Screen Shot 2013-10-21 at 13.24.32.png

Typical administration tasks include:

  • Reset ‘root’ password
  • Configure Lockdown mode
  • Configure, Restart, Test and Restore the VMware ESX Management Network
  • Restart Management Agents
  • Configure Keyboard
  • Troubleshoot
  • View System Logs
  • Reset System Configuration (Factory Reset)
  • Shutdown/Restart the VMware ESX Host

Most actions are carried out by using [F2] on the keyboard or [F11] confirm changes, along with typical options such as [Y] and [N] to various system prompts. Before carrying out any task you will be required to supply the ‘root’ password. However, the first law of security is to secure the physical server – so take care to ensure your access to ILO/RAC/BMC interfaces are properly secured. Although the VMware ESX host can be rebooted from the DCUI this is regarded as an action of last resort. If the VMware ESX hosts has running VMs these will crash, and may or may not be restarted on other hosts depending on whether they are part of a cluster.

Reset ‘root’ password

You may need to reset the ‘root’ password because an inappropriate one was initially assigned, or it has become disclosed to individuals who should be denied access. There are easier ways to change the root password using the vSphere Client, and if you need to change the ‘root’ account password for many hosts (say on a quarterly basis) you might find VMware’s PowerCLI and other scripting automation tools are a better approach.

Whilst special characters are supported with the ‘root’ passwords there have been reported cases of certain character types causing a problem. Generally, if you stick with alphanumerics and common characters such as !@$%&*()}{|”:?><,./\’;][ then you should have no problem. Avoid special characters which are region or culturally specific. Indeed there have been cases were even very common characters such as the $ dollar sign have caused problems. So tread carefully.

1. Open a console window to the physical VMware ESX hosts

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Select “Configure Password” and press [ENTER]

4. Supply the old password together with two copies of the new password, and press [ENTER]

Screen Shot 2013-10-21 at 13.46.41.png

NOTE: To carry out this same task in VMware PowerShell first authenticate to the VMware ESX host, and use the “Set-VMHostAccount”

Reset ‘root’ password (PowerCLI)

Screen Shot 2013-10-21 at 14.01.16.png – Resetting the Root Password in PowerCLI for Single ESX host

Connect-VIServer $HostName -User root -password Password1
Set-VMHostAccount -UserAccount root -password P@ssw0rd  

Screen Shot 2013-10-21 at 14.01.16.png – Resetting the Root Password in PowerCLI for Multiple ESX hosts

In this case the .PS1 script attaches to vSphere and dumps a list of ESX hosts, and then is disconnected. It then logs into each ESX host as root/Password1, and then uses Set-VMHostAccount to reset the password to P@ssw0rd. Note – neither of these passwords are very good!

Write-Host "Connecting to vCenter" 
Connect-VIServer "vcnyc.corp.com" -user administrator -password Password1 
$VMHosts = Get-VMHost | Sort-Object Name 
Disconnect-VIServer -Confirm:$False  
ForEach ($VMHost in $VMHosts) 
{ 
$HostName = $VMHost.Name 
Connect-VIServer $HostName -User root -password Password1
Set-VMHostAccount -UserAccount root -password P@ssw0rd  
Disconnect-VIServer -Confirm:$False
}
Configure Lockdown mode

Lockdown mode is a method of restricting direct access to the VMware ESX host using the vSphere Client. It is enabled when adding the VMware ESX host to vCenter. Without “Lockdown Mode” enabled an administrator can bypass the vCenter management server – and modify advanced settings on the host. This can bypass methods of generating audit trails, and centralized management. Enabling the “Lockdown Mode” prevents this from happening.

Screen Shot 2013-11-02 at 03.20.56.png Note: Screen grab taken from the vCenter Web Client as an ESX host is being added into the Inventory.

Once enabled attempts to connect directly to the host using the legacy vSphere Client will result in an error message indicating a lack of privileges even when the ‘root’ account is being used. Even if the SSH Protocol is enabled on the host, although PuTTy and SSH client sessions could be established. Logons would fail because of lockdown mode.

Screen Shot 2013-12-04 at 11.35.34.png

Screen Shot 2013-12-04 at 11.39.25.png

If a case does arise where direct host access is required using the vSphere Client or PuTTy – then the mode can be disabled from the DCUI.

1. Open a console window to the physical VMware ESX hosts

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Select Configure Lockdown mode, and use the [Spacebar] to toggle the option off/on

Screen Shot 2013-12-04 at 11.46.58.png]

Configure, Restart, Test and Restore the VMware ESX Management Network

The installation of VMware ESX defaults to assigning the first network card discovered to the management network, and configuring the host for a DHCP assigned address. This might be unsuccessful depending on how the physical server is patched to the switch, and whether a DHCP server is present on the network. Additionally, the physical switch maybe configured for VLANs. The default installation of VMware ESX does not allow for the setting of VLAN Tags until after the vmkernel has been loaded, and the DCUI enabled.

Almost all of the changes made in the “Configure Management Network” require a restart of the networking for the management of the ESX host. This is something you will prompted to do when ever you exit these pages.

Screen Shot 2013-10-21 at 16.35.33.png

The following instructions are illustration of changing the default physical NIC, setting the VLAN value and configuring a static IP options.

Network Card Assignment

1. Open a console window to the physical VMware ESX hosts

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Configure Management Network” and press [ENTER]

4. Select “Network Adapters” and press [ENTER]

5. Using the [SPACEBAR] and cursor keys you can select network interfaces. In the screen grab below two physical NICs (vmnic0 and vmnic1) have been allocated to the VMware ESX host. This will automatically offer out of the box load-balancing and redundancy.

Screen Shot 2013-10-21 at 15.31.36.png

NOTE: If you do add two network cards in this manner the second interface is adds as “Standby” adapter to the Standard vSwitch that is created during the installation. This may not be the SysAdmin’s desired configuration – and you may wish to handle this in a different manner.

Screen Shot 2013-10-22 at 11.50.49.png


VLAN Configuration

1. Open a console window to the physical VMware ESX hosts

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Configure Management Network” and press [ENTER]

4. Select “VLAN (Optional)” and press [ENTER]

5. Type in the VLAN ID value, and press [ENTER]

Screen Shot 2013-10-21 at 15.43.16.png


IP Configuration

1. Open a console window to the physical VMware ESX hosts

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Configure Management Network” and press [ENTER]

4. Scroll down and select “IP Configuration” and press [ENTER]

Screen Shot 2013-10-21 at 16.20.11.png

5. Complete the IP configuration as it befits your local network requirements:

Screen Shot 2013-10-21 at 16.20.11.png

6. The DNS settings can be modified by selecting “DNS Configuration” and press [ENTER]. The DNS Configuration allows for the setting of a primary and secondary DNS server, together with the short “hostname”. The fully-qualified domain name (FQDN) is completed by configuring the “DNS Suffix” options

Screen Shot 2013-10-21 at 16.34.22.png

Screen Shot 2013-10-21 at 16.35.09.png

7. The IP Configuration can be tested using the “Test Management Network” options. This allows the SysAdmin to test communication to the router (if present) and the DNS servers on the network – as well as confirming the hostname is resolvable to via DNS.

Screen Shot 2013-10-21 at 17.12.41.png

Screen Shot 2013-10-21 at 17.13.00.png

IMPORTANT: Although the test here passed on the hostname, the test merely checks to see if the hostname is present on the DNS server. It does not verify if the hostname (or ANAME record) is valid or pointing at the correct address. As such this means you could still have incorrect entries in the DNS database. It’s recommend to use a utility like nslookup to confirm that both forward and reverse DNS looks resolve to the correct name.

Screen Shot 2013-10-21 at 19.52.01.png


Restore Network Configuration

Restoring the network configuration is quite a dangerous option if not used correctly. It has the potential to reset the network to such a state that you will not be able to communicate to the VMware ESX host without resorting to the DCUI to resume communication. It also has the possibility of disconnecting virtual machines (VMs) that are running on the VMware ESX host. Additionally, it has the ability to remove standard and distributed virtual switches (vSwitch) from the host in event that these have become broken on the host beyond repair.

CAUTION: As such you should approach these options with extreme care.

1. Open a console window to the physical VMware ESX hosts

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Network Restore Options” and press [ENTER]

4. Select the Restore option required, and press [ENTER]. In the following case the option “Restore Network Settings” was selected.

Screen Shot 2013-10-21 at 17.24.49.png

Configure Keyboard

Whilst the VMware ESX host keyboard settings can be configured during installation, it possible to modify this after the installation itself.

1. Open a console window to the physical VMware ESX host

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Configure Keyboard” and press [ENTER]

4. Use the cursor keys to highlight the preferred language, and then spacebar to select the new keyboard type. Press [ENTER] to make the change.

Screen Shot 2013-10-21 at 19.02.41.png

Troubleshooting Options
Restart Management Agents

In the early days of VMware ESX occasionally the host would appear as being “disconnected” in the management system of vCenter. Although the host has a “WatchDog” service designed to restart the core management agent, this would be unsuccessful. In recent years these random disconnections have been resolved – and its now highly unusual for an VMware ESX host to enter a disconnected state. Nowadays, if this happens its is more normally another cause such as an IP conflict, or the host being rebooted in non-authorised manner or some type of hardware failure. Nonetheless, the option to restart management agents does exist in the DCUI. If you do use this option you will need to be patient as it can take time for other systems to “retry” the connection and reconnect to the host.

1. Open a console window to the physical VMware ESX host

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Troubleshoot Options” and press [ENTER]

4. Use the cursor keys to scroll down to “Restart Management Agents” and press [ENTER]. In the following page not only do you have the option to simply restart the management agents, but also collect extra troubleshooting information. Notice the warning about this disconnects all existing remote management software.

Screen Shot 2013-10-21 at 17.42.29.png

Enabling ESXi Shell and SSH together with Timeout Values

It is possible to get true command-line access to the VMware ESX host. This can be either by the “ESXi Shell” normally access via the ILO/RAC/BMC card or using the Secure Shell protocol (SSH) commonly access on TCP port 22 using a SSH client like PuTTy. In addition to these options being enabled they can be enabled for a designated period as well, to allow temporary console access. This is prevents the need to have protocols like SSH enabled all the time, which could be regarded by some as a security weakness. If you do have a require to permanently enabled SSH access this can be done from the “Security Profile” on the VMware ESX host either with the vSphere Client or using vCenter with the Web-Client.

Important: If you intend to set timeout values you must set these before enabling the ESXi Shell and/or SSH.

1. Open a console window to the physical VMware ESX host

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Troubleshoot Options” and press [ENTER]

4. Use the cursor keys to select “Modify ESXi Shell and SSH Timeouts”, and press [ENTER]. Configure the durations for the “Availability Timeout” and “Idle Timeout”. A zero value can be specified which indicates that sessions never expire.

Screen Shot 2013-10-21 at 18.34.07.png

5. Next we can Enable the ESXi Shell and SSH. These a toggle options where pressing [ENTER] switches the option from “Enable…” to “Disable…”

Screen Shot 2013-10-21 at 18.38.06.png

6. Accessing the ESXi Shell requires exiting the DCUI back to the main screen and then pressing [ALT+F1] on the keyboard. The keystroke [ALT+F2] will toggle the SysAdmin back to the DCUI. Typing the command ‘exit’ at the ESXi Shell prompt logs the SysAdmin out of the environment.

Screen Shot 2013-10-21 at 18.44.31.png

7. Accessing the ESXi SSH Service requires a SSH Client. For Windows systems the most popular is the free PuTTy tool. Linux and Apple support their own native SSH command-line utilities.

Screen Shot 2013-10-21 at 18.53.19.png

Unsupported Tip: The DCUI is a process like any other on the VMware ESXi host. It is possible to access the DCUI from a SSH session using PuTTy. This is not a support usage, and it runs the risk of disconnecting the very SSH session that allows it work. The DCUI is accessed from the SSH session by typing the command “dcui” and the SysAdmin can exit the shell using the keystroke [CTRL+C]

Screen Shot 2013-10-21 at 18.56.06.png

View System Logs

There many ways of viewing and gathering the system logs from a VMware ESX host. Viewing them via the DCUI is perhaps least friendly method but it is possible.

1. Open a console window to the physical VMware ESX host

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “View System Logs” and press [ENTER]

4. Pressing a number on the keyboard from 1-6 will allow you to view the system logs, and [Q] on the keyboard will quit the log view, and return the SysAdmin back to the DCUI screen.

Screen Shot 2013-10-21 at 19.18.40.png

Screen Shot 2013-10-21 at 19.19.13.png

Reset System Configuration (Factory Reset)

A “Reset System Configuration” (or more commonly referred to as a ‘factory reset’) reconfigures the ESX host back to its initial installation. This achieve by maintain various system states between reboots. Before issuing a “Reset System Configuration” its is recommended to carry out a manual backup of the VMware ESX host. This can be done using the command vicfg-cfgbackup or PowerCLI.

IMPORTANT: A reset of the VMware ESX host also resets the root password back to being blank. As consequence all previous passwords including the one configured at the installation are lost.

1. Open a console window to the physical VMware ESX host

2. Press [F2] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 13.44.10.png

3. Use the cursor keys to scroll down and select “Reset System Configuration” and press [ENTER]. Next press [F11] to confirm you wish to carry-out the reset, followed by [ENTER] to confirm a reboot of the system.

Screen Shot 2013-10-21 at 19.30.00.png

Screen Shot 2013-10-21 at 14.01.16.png

Get-VMHostFirmware -VMHost esx01nyc.corp.com -BackupConfiguration -DestinationPath C:\

Set-VMHostFirmware -ResetToDefaults

Note: This will back up the VMware ESX host called “esx01nyc.corp.com”. After the backup has completed a zip file in the .tgz format will be created called C:\configBundle-esx01nyc.corp.com.tgz. The cmdlet “Set-VMHostFirmware” has the capacity to send the reset process to the host as well.

Shutdown/Restart the VMware ESX Host

There are many ways to shutdown or reboot the VMware ESX host. By far the most appropriate method would be use vCenter “maintenance mode” which in conjunction with the VMotion and the Distributed Resource Schedule (DRS) feature successfully evacuate all the VMs from the host, before a shutdown or reboot instruction is given. You should exhaust all reasonable efforts to gain control over the the VMware ESX host to carry out a graceful outage of the host. Only use the power button or the shutdown/restart functionality of the DCUI if you have no other option.

1. Open a console window to the physical VMware ESX host

2. Press [F12] on the keyboard, and supply your current ‘root’ logon password and press [ENTER]

Screen Shot 2013-10-21 at 20.08.11.png

3. Pressing [F2] on the keyboard will trigger a shutdown, whereas pressing [F11] will trigger a reboot. Before using the [F2] ensure you have suitable access to trigger a power on of the physical server!

Screen Shot 2013-10-21 at 14.01.16.png

Set-VMHost esx01nyc.corp.com -State maintenance

Restart-VMHost -vmhost esx01nyc.corp.com

Note: Once a host is in maintenance mode it remains in this mode even after a reboot. An VMware ESX in maintenance mode cannot power on a VM, nor have VMs moved to it.

Source: http://www.mikelaverick.com/wiki/index.php?title=Installing_VMware_ESX

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s