Tutorial: ESX 4.0 Syslog Configuration

This tutorial is intended to explain how to configure syslog on both ESX and ESXi to point to a remote syslog server (for this tutorial I will be using vSphere, but the concepts apply to earlier versions as well as many*nix distros).  In both cases the configuration is fairly simple, but can be confusing.  I also wanted to include a sample configuration that I use for syslog-ng which is an excellent logging platform for syslog.

ESX

First, I have a single host with ESX 4.0 installed and configured with a service console IP address.  I have connected to this IP address with putty over SSH.  Once logged in to the ESX server, there are just a few steps that need to be taken. Also, I use the VI editor – check on google for more information on VI and its commands. I am going to assume that readers posses a basic knowledge of this.

1. First, edit the syslog configuration file (/etc/syslog.conf) by typing:

vi /etc/syslog.conf

The display will look similar to the following:

Click To Enlarge

I added a line at the end of this file that has the following contents:

*.debug                                      @<ip of syslog server>

What this will do is send syslog messages from all facilities and levels to the IP address of the server following the @ symbol.  One caveat with this line is that the space between the *.debug and the @ symbol must be the tab character.  If not the syslog daemon will simply drop the line.  If you desire more than one destination, just add another line with a different host after the @ symbol.

2. Next we need to restart the syslog service, to do that enter one of the following:

service syslog restart

or

/etc/init.d/sysog restart

The display should look similar to the following:

Click To Enlarge

If there are any errors at this point, troubleshoot accordingly.

3. Next we need to open firewall ports for the syslog traffic.  We are not able to do this through the service console, so we must do this at the command line.  It is a pretty simple command and if it is successful, it won’t produce any output.

esxcfg-firewall –openPort 514,udp,out,”Syslog”

Click To Enlarge

4. Next we need to reload firewall rules on the server and then verify that the rule was added.  For this, we need to type in two commands. The first one reloads the firewall (this will not produce any output if it is successful).  The second one will query the firewall for all active rules and will produce a lot of output. The two commands are:

esxcfg-firewall -l
esxcfg-firewall -q

Click To Enlarge

The output outlined in red indicates the rule that we added.  This change is not visible in the GUI – the only way to verify these changes is via the command line.

ESXi

This configuration is much simpler – you only need to change one variable in the advanced settings.  One downside is that ESXi is limited to one destination server However, there are other ways to get around this (via syslog relays).

In ESXi the following configuration can be found by clicking on your server, then navigating to the configuration tab and then clicking on “Advanced Settings” under the Software Settings section.  Once that window is open, there is a syslog section near the bottom.  Expand that section and click on “Remote” and fill in an IP address or hostname in the box for the variable ‘Syslog.Remote.Hostname” and click save.  Screenshot:

Click To Enlarge

Bonus

I wanted to include a sample syslog-ng config section for syslog-ng which is what I use to collect my syslogs and sort them accordingly.  Syslog-ng is awesome because it allows much greater flexibility in the processing and archiving of logs.  The extended configuration of syslog-ng is a topic for another post, but here is the configuration I use for most hosts:

Click To Enlarge

This configuration separates each server into an individual file by day according to the time from the source server (in this case the ESX server).  This gives file names like “vm0.09.02.2009.log”, which is great for log rotation.  Also note the “S_” infront of each macro; this tells syslog-ng to use the source information instead of re-writing the logs to the timezone of the syslog-ng server.  It is also possible to filter based on syslog facility, but I will cover that in another post dealing with the analysis of ESX logs.

Update: Videos!

I am going to try and include videos of these tutorials, here is my first go at it so please be kind!

ESX:

Tutorial: 802.3ad Port Configuration with ESX

Just wanted to post a quick post about how to configure 802.3ad port consolidation with a Cisco switch and VMware ESX (vSphere was used for this example).  I was using an HP DL380 with 2 onboard NIC and 2 24 port Cisco 3750G connected with stackwise cables.

Switch Configuration:

…
interface GigabitEthernet1/0/14
description ESX NIC 2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
end
…
interface GigabitEthernet2/0/14
description ESX NIC 2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
end
…
interface Port-channel1
description ESX PortChannel
switchport trunk encapsulation dot1q
switchport mode trunk
end
…

What I did was group two gigabit ports (1/0/14 and 2/0/14) into one port channel group (1).  Then I applied the trunk settings to the port channel instead of the individual ports.  As for the ESX side, I configured the virtual switch with more than one active adapter and set the “load balancing” to “Route based on IP hash.”

If this set up would not have been using a trunk, the following configuration would have been used:

…
interface Port-channel1
description ESX PortChannel
switchport mode access
switchport access vlan <vlan for port group>
end
…

This configuration has seemed to be fairly reliable. I plan to do some load testing to investigate how balanced the load really is. I will post a follow-up if there are any interesting findings.