Ideas on the creation of a distributed Operating System that is shared by everyone in the world. This environment would have enough flexibilty to allow anybody to do anything from anywhere. The point where ideas come to life and technology is no longer in the forefront, but simply the back end of a globally shared and agreed upon internetwork.
Skyrails is a social network (or any graph really) visualization system. It has a built in programming language for processing (as far as visualization attributes goes) the graph and its attributes. The system is not only aimed at expert users though, because through the scripting languages menus can be built and the system can be used by any users. The main distinguishing point of the system comes from the built in scripting language, the added flexibility of how to represent attributes (nodes can be binded to planes and spheres based on their attributes) and the scriptability of the user interface system.
Eve 3D
This is a network vizualization project that looked like it had potential, but dies on the vine a few years ago.
In the effort to create a modern network management interface that is both scalable, robust and open source, I have started looking at the current state of tools for log and realtime analyses. I wanted to find some resources that we could use for visualizing data with new methods.
I am a very visual learner and thus I can get a lot more out of visual data sets than I could out of text logs or numeric analysis. Metaphores and high level overviews provide more detail in interpretation than would the RAW data. Another example of this would be machine language versus high level language and object oriented programming. I am sure that if you have the in depth knowledge of protocols, and can do real time packet analyses with insight into the way that many different client / server / peer environments should act and react, then the vizualization is all done in your head.
For the rest of use meer mortals, we need the tools.
Logstalgia (aka ApachePong) replays or streams a standard website access log (eg access.log) as a retro arcade game-like simulation.
LILO is the boot loader (LILO=Linux Loader) used for ESX server (remember the service console is based on a modified version of Red Hat Linux 7.2). Lilo.conf is the text file to describe the OS to boot. This text file is compiled into a binary file that is actually used by LILO. If you are troubleshooting the APPEND line, then use vmkpcidivy tool. You should not have to revert to manually editing this file. If you ever do edit this file, then you need to compile it with lilo –c to write the binary file.
Append=
The pci device mask specified in the append line of lilo.conf is actually an include, not a mask out. The important thing to remember is the append line defines the hardware visible to the service console. Don’t replace LILO with another boot loader, e.g. GRUB. ESX manages allocation of PCI devices between service console and VM kernel with the expectation of LILO.
You can also view PCI device allocation using the MUI, found under Startup Options
Alternatively, you can use the legacy web interface using the URL
LILO loads the service console kernel, e.g. /boot/initrd-2.4.9-vmnix2.img This kernel obeys what is stored in the file /etc/inittab
/etc/inittab
This file is read by the init process and specifies the run level to be used by the service console. The line that states the run level will look something like this.
id:3:initdefault:
The init process then works through the start up scripts in the appropriate diectory. For run level 3, this directory would be
/etc/rc.d/rc3.d
The file also starts up the virtual terminals on the ESX server mingetty tty2 through mingetty tty5. The mingetty process is minimum getty.
/etc/rc.d/rc.local
A text file used most commonly when we need to share the service console NIC with the VM kernel. This may be required in say a blade server which has only 2 NICs, and we need to dedicate 1 NIC to Vmotion.
insmod vmxnet_console devName=vmnic0 ifup eth0
/etc/rc.d/rc2.d
Contains start-up scripts for run level 2
/etc/rc.d/rc3.d
Contains start-up scripts (logical links) for run level 3. Run level 3 is used most of the time as it is command line full multi-user mode. The ones we are interested in are shown below:
If we wanted to change a service so that it is enabled for a particular run level, then we can use chkconfig –level.
chkconfig --level 1 ntpd on
The above command would turn on ntpd for run level 1, this would not affect the run levels that ntpd was already set for. So in this example, the ntpd run levels would be
ntpd 0:off 1:on 2:on 3:on 4:on 5:off 6:off
service --status-all
This lists all the service daemons and their status. We can find running services by looking for the running status
service –-status-all |grep running
would produce an output similar to the following:
crond (pid 1423) is running
httpd (pid 1486 1482 1479) is running
syslogd (pid 1136) is running
sshd (pid 1208) is running
To avoid unnecessarily rebooting an ESX server after making certain configuration changes, we can frequently just restart the appropriate daemon. For example we could restart the Apache web server for the MUI with the command:
service httpd.vmware restart
and we can also check a named service running status with
service httpd.vmware status
S12syslogd
The centralised logging system. When ESX is running, both the service console and VMkernel log messages through it.
/etc/ssh
Secure shell settings
Quick way to restart the ssh daemon. Use full path to do this.
/etc/init.d/sshd restart
/etc/ssh/ssh_config
Text file with the configuration of SSH client.
/etc/ssh/sshd_config
Text file with the configuration file for SSH daemon. An important setting in this file is PermitRootLogin=Yes/No. You can quickly check this with a grep on the file.
grep permit /etc/ssh/sshd_config
~/.ssh/known_hosts
A text file that stores the RSA keys for known hosts. If we rebuild a host and then reconnect to it over SSH we may be prevented if the known_hosts file has cached the old key. I think SCP (secure copy) uses RSA keys as well as SSH sessions. Note this file is maintained on a per-user basis.
/etc/xinetd.conf
The config file for Xinetd, the network services daemon? The daemon itself is stored in /usr/sbin/xinetd This launches the daemons that are bound to it on demand
vmware-authd
This is the authentication daemon. This daemon authenticates users of the management interface (MUI) and remote consoles using the username/password database defined in /etc/passwd. This service binds via the xinetd daemon and so the configuration file that specifies the listening port is in
/etc/xinetd.d/vmware-authd
This text file contains the settings for the VMware remote access authentication daemon. This file specifies the TCP:902 port used by remote console. If this port was changed here, it must also be changed in the file /etc/vmware/config.
If we wanted to add Kerberos off-box authentication for MUI access, then its in the pluggable authentication module configuration file that corresponds to this daemon that we would make a change. This file is found at
/etc/pam.d/vmware-authd
We would need to change the current "auth required" to "auth sufficient" and add a last line of "auth required" using the Kerberos authentication module.
S90vmware
This script performs the following actions:
vmnix driver Loaded by modprobe vmnixmod.o
VMkernel Loaded by vmkloader
Logger
VMkdump Any dump will be copied to /root
Starts VMs Performed by vmware-serverd
vmware-serverd
This daemon runs on demand in the service console to provide information to any VMware service that needs it. This performs actions in the service console on behalf of the VMware Remote Console and the web based MUI. It is started at boot time to do any VM autostarts. This process is replaced with vmware-ccagent if Virtual Center is installed.
S91httpd.vmware
This script starts the Apache web server which provides the ESX Server MUI. Configuration is stored in
/usr/lib/vmware-mui/apache/conf/httpd.conf
This process communicates with vmware-serverd for backend data. Remember a refresh in the browser is only a refresh to Apache, to get new data, click on the refresh button to get new kernel data. Remember if the httpd.vmware service starts and then stops immediately, check your service console disk space.
vmware-snmpd
vmware-snmptrap
/usr/sbin/vmkstatus
This is a perl script which runs every x seconds on tty1.
lsmod
Lists drivers loaded for the service console linux. Remember this command differs from the vmkload_mod utility which shows the modules loaded for the VMkernel.
The same information can be found by cat /proc/modules
Starting & Stopping ESX Server
init 0
Instructing a halt
init 1
Instructing run level 1
esx 1
If this is entered at the LILO boot prompt we can get a root shell. We are instructing the vmnix kernel to execute at run level 1 (single user mode).
esx 3
Again at the LILO boot prompt, this time, we are instructing the vmnix kernel to execute at run level 3 (the default). If we suspected the run level was wrong we can use this to get back up and running without having to revert to booting Linux on its own.
linux rescue
Used when you boot the ESX server with a Red Hat Linux CD
chroot
Change root directory to new directory specified as a parameter
shutdown
Brings down the system in a secure way.
shutdown -h now Halt after shutdown
shutdown -r now Restart after shutdown
linux -s
At the LILO boot loader, the default options are
esx Normal ESX boot
linux Linux SMP kernel, no VMkernel load
linux-up Linux Uni-processor kernel, no VMkernel load
If we use the cursor key at the LILO screen to select one of the three default choices, the boot prompt (displayed below the menu) changes to reflect this. This allows us to augment the boot command with an option switch.
boot: linux –s
In this case, the –s instructs Linux to boot in single user mode. A critical security point here is that in single user mode, Linux automatically logs on as root! Once in single user mode if we wish to continue into multi-user mode then we type either exit or CTRL-D
RPM Packages
rpm
As ESX service console is based on modified Red Hat Linux, we can use the RPM package installation method.
rpm -qa
Networking
/etc/modules.conf
Identifies interface eth0 and aliases
alias eth0 e1000
alias eth1 e1000
options e1000 Speed=100
Ifup eth0
Brings interface “eth0” up
Ifdown eth0
Takes interface “eth0” down.
Ifconfig
Prints a list of the network interfaces. A quick way of getting IP and MAC of interfaces Can also be used to bring interfaces up or down
ifconfig eth0 up
ifconfig lo down
mii-tool
Interface tool. Doesn’t work correctly with some network cards, including Intel 1000 Pro copper NICs.
service network restart
Great way to do orderly restart of vmnix network configuration. Another way would be to do
ifdown eth0; ifup eth0
/etc/hosts
Host lookup file, just like windows. The ESX server MUST have an entry for itself in its own HOSTS file.
/etc/resolv.conf
The DNS name server settings, This file does not need to be present if you are not using DNS.
/etc/sysconfig/network
Text file containing hostname and default gateway.
/etc/sysconfig/network-scripts/ifcfg-eth0
IP, mask and device name. The text file contains IP configuration for interface eth0, which is reserved for the service console.
route
Modify or print routing table
route Prints routing table
route del –net default Deletes the default gateway
route add –net default gw w.x.y.z Adds a new default gateway
netconfig
Network configuration setup program
/etc/issue
File which shows ESX and vmnix version
VMware ESX Server 2.1.2
Kernel 2.4.9-vmnix2 on an i686
uname -a
Outputs details of the server as shown
Linux esx1.taupoconsulting.net 2.4.9-vmnix2 #1 Fri Aug 6 04:38:44 PDT 2004 i686
/etc/fstab
Local and remote file systems which are mounted at boot. This file is explained in more detail in the mount section.
/proc/net/NICfamily/eth0.info
A text file that can be checked to see what the service console NIC is doing. The speed of the console NIC specified in the file modules.conf can be confirmed by this file.
User Administration
id
Displays the UID for the currently logged on user or if supplied with a parameter can be used to display the UID of a named user.
passwd
Used to change the logged on users password or for a named account
passwd
Adds a user to /etc/passwd
By default a user added with default options has a group created of the same name. We can add the user to another group, e.g. have a group membership of say ESXUsers
useradd alistair –G vmsqladmins
The above command adds a user called alistair whose primary group (-g) is alistair and other group (-G) membership is vmsqladmins
We can add additional parameters to more fully specify the account.
In the above example the users’ primary group is Finance and the shell is specified. In this case the shell is /bin/false which is a bogus shell which would prevent logon as this user. By default in the service console vmnix, this would be /bin/bash for the Bourne-again shell. It appears the only other shell shipped with vmnix is csh (the C shell).
groupadd
Adds a group to /etc/group
groupadd esxadmins
In the above example, a new group called esxadmins is created and therefore a new line appears in /etc/group.
gpasswd
The best tool for modifying the /etc/group file. The following command adds greg to the ESXUsers secondary group.
gpasswd –a greg ESXUsers
Group removal is simple with the –d switch:
gpasswd –d tony ESXUsers
useradd
This command is used to modify a user. Be very careful with this command as when used with –G to set the users group membership, it is not adding the user to a group but is actually setting the list of secondary groups a user belongs to. Therefore in the following example if bill had secondary group list of ESXUsers and SQLadmin, then after entering:
usermod –G ali bill
then bill would only have a secondary group of ali and nothing else!
groupmod
Modify a group
su
Switch user, super user and used without parameters expects root. If we restrict root from logging in over SSH, then we force remote users to authenticate as themselves and then SU to run privileged commands if need be, thus leaving a decent audit trail.
sudo
Allows delegation of administration in terms of certain commands that normally only a particular user can execute (usually root).
visudo
This is just the "vi" text editor, but it automatically opens the /etc/sudoers file. The point of visudo is to ensure we always edit the right file as the location of the sudoers file differs between nix distributions, but this command is constant and will utilise the right sudoers file for the distro being used.
/etc/sudoers
The text file that contains the sudo users and the rules that apply to them.
This may look like a list of users, but it is groups. As the service console (vmnix) is a modified version of Red Hat Linux, the Linux security configuration is the same as Red Hat. One feature of Red Hat no found in all Linux distributions is that of the user private group (UPG). Whenever you create a user, a group of the same name is created also and the user is made a member. The format of the file is:
groupname:x:user1,user2
so when we see groups like JohnSmith:x:513 we can assume the 513 is the UID for the user JohnSmith and this is his UPG.
/etc/passwd
List of security users. When we add a user with either the MUI or a command line tool such as useradd, we are adding to this text file.
Skeleton directory – new home directories take copies of the files stored here.
/var/log
This directory stores key log files for both the service console and the VMkernel. Of note are the vmkernel, vmkwarning & messages file logs. These logs can be viewed with the more, cat, head and tail command line tools.
We can also access these logs via the MUI via the following link in the Options tab.
If you use the sudo tool to run a command under a different security context then the log file /var/log/secure will contain the audit trail for such activity.
Check the file /etc/syslog.conf for logging settings.
netstat
Shows currently active network connections
lsof
List open files Pipe the results into grep to check for open ports
lsof |grep IPv4.\*LISTEN
pam
Pluggable Authentication Module
free -m
Shows free memory in the service console. The "-m" switch specifies to display the results in megabytes.
fdisk
Disk partitioning tool
du
Disk usage. Great for finding out which folders are using disk space
du –h /home/ali/vmare
du –h ~
du –s summary
df -h
Print disk partitions (with human readable switch)
vdf -h
Print disk partitions with knowledge of VMFS partitions (type FB) (with human readable switch)
dd
Disk dump utility common to Linux. Copy a file while converting and formatting. This can be a quick and dirty way of making an ISO. This could be done in the service console with
dd if=/dev/cdrom of=/vmimages/new.iso bs=32k
This tool could also be used to go from ASCII to EBCDIC etc.
This tool can be used to create an additional swap file. For example, if we did not allocate a big enough swap partition for the service console during ESX installation, we can create one now in a file of 64MB.
dd if=/dev/zero of=/swapfile bs=1M count=64
If we did add a swap file, we would need to make sure it is started when ESX starts. Therefore, an entry in /etc/fstab would be needed as this file describes the local and remote file systems to mount at boot. The total amount of service console swap space is the sum of the swap partition and any swap files that are active.
mkswap
A command that must be run against a newly created swap file in order to activate it. Think of creating a swap file with dd like creating a partition, then mkswap is like formatting that partition. The swapon command then turns it on when needed.
swapon
Enables swap file for service console
swapoff
Disables swap file for service console
/proc/swaps
A text file that can be checked to see what swap the service console is using. The output contains a priority which shows which swap device will be used first before the other(s). Useful to determine if swap space is getting used and if there is more than 1 swap. Remember this is vmnix (service console) swap, not VMkernel. The VMkernel swap is in one or more files on a VMFS volume (hence the strong recommendation that even when using a SAN, a vmfs volume is created on direct attached storage to allow local swap).
Filename Type Size Used Priority
/dev/sda3 partition 554232 0 -1
/swapfile file 65528 0 -2
Basic File Commands
touch
When used with a non-existent filename, this tool creates an empty file of name filename. However, this can be used to touch an existing file and update its last modified or last accessed attributes. This could be scripted if required. Watch for running touch against any file stored on a VMFS as there appears to be a problem. Remember that not all Linux tools are modified for VMFS awareness. The VMFS is not an ext3 partition but is a mount point, it’s the VMkernel that accesses any VMFS volumes.
cat > file
Enter text at command line and then CTRL-D to write to file.
echo blah > file
Writes the text following echo command to file. This could be good for quickly creating files
Another great use of this technique is to make changes to the ESX server configuration via the /proc hierarchy, e.g. changing the number of shares for a VM
echo 2500 > /proc/vmware/vm/nnn/cpu/shares
would change the VM CPU shares to 2500. However such a change would only exist for the duration of the world created for that VM. After the VM is powered off this in memory structure is lost. To make such a change persistent, we would need to add the line
sched.cpu.shares = "2476"
to the VMX file of the virtual machine.
head filename
By default, the head command prints the first 10 lines of the specified file. We can choose how many lines we want instead of 10 by specifying the –n switch. This is good for looking at the file /proc/vmware/vmhba:x:x:x/0:0 with the –n 22 switch. Also good for using with the file command to determine whether a virtual disk is in ESX format or COW format.
head server.dsk | file -
The “–“ is crucial to making the above command work. For an ESX virtual disk we would expect to see something like
standard input: x86 boot sector
tail filename
Prints the last 10 lines of the specified file. Just like the head command, there is a –n switch that can be specified to list the last n lines of the named file.
sort
Can alphabetically or numerically sort redirected command output or files.
sort fruit
or to sort a basic score sheet
sort –g –k 2 scores.txt
grep
Group regular expression, used to string search the files or command outputs. You can use grep –i to indicate search with case insensitive.
find
The find utility is used much in the same way as many Windows people used the DIR command. If you know roughly what files you are looking for, then this is the tool. The ls tool simply lists, whereas the find tool will find according to one or more criteria, a common one being find files modified in the last day using the –mtime switch as shown in the table.
–mount used to ensure it doesn't traverse to remote file systems
-size obvious
-mtime modified in the last n*24
-mmin modified in the last n minutes
-ls use output format as if ls were used
-name name the file you are looking for (you just don’t know where it is!)
find –mmin -30 files modified in last 30 minutes
find –mtime 1 files modified in last 24 hours
find –size +10000 files in excess of 10,000 bytes
find –mount –size +10000 -ls files on non-remote file system
find –name “hosts” -ls file called hosts
vi
Standard UNIX text editor
I
ESC
:wq
:q!
nano
Another text editor, more friendly but you should use –w to avoid word wrap.
setup
Allows changing of NIC, region, firewall, mouse, keyboard
ls -a
List files in a directory including hidden (also known as dot files due to their prefix) files.
ls -dl */
List directories in long format (does not display files). Could add as a shell alias, say lsd.
ll
Same as ls –al
less
Scrollable command line, great for piping large output into
more
Exactly the same as DOS and Windows, also great for piping large output into.
chown
Changes file ownership. If only 1 user name is specified then the user ownership is set only and the group ownership if left unchanged as shown in the solaris.vmx example below. However rather than having to use chown and then chgrp straight after it, you can set user and group ownership in one operation by specifying the username(s) separated by a colon as in the netware5.vmx example shown.
#chown ali solaris.vmx
#chown ali:ali netware5.vmx
chgrp
Changes the group owner for a file, leaving the user owner unchanged.
#chgrp ali file.txt
chmod
Changes file permission. We can use either letters or numeric equivalency when setting permissions. We set permission for 3 principals, the user, the group and others (ugo). If you are from a Windows background then don’t confuse “o” with owner.
#chmod u+rx,g+r,o+r file.txt
Note that using + or – indicates we are adding to or removing from existing permission. If we wish to reset the permission we use “=” to explicitly set the object permissions.
#chmod u=rx,g=r,o=r file.txt yields r-xr—-r--
Sometimes you will see a chmod using “a” to specify all (user, group & other), so we could quickly set read permissions by
#chmod a-wx,a+r file.txt yields r—-r-—r--
A more common way to set permission is using chmod is using numeric equivalent values (4,2,1 for r,w,x) and permutations thereof.
chmod 777 windows2k.vmx set perms to rwxrwxrwx chmod 754 windows2k.vmx set perms to rwxr-xr-- (default)
Watch for chmod commands with 4 digits, e.g. chmod 0754. This refers to additional attributes
Sticky bit SUID (Set User ID) SGID (Set Group ID)
The sticky bit can be set on executables which tells Linux to keep the application in memory. The reason for this is to improve load times for other users who wish to run the same executable. This relates to the multi-user nature of UNIX/Linux. Given the speed of memory and disk access nowadays the need to keep applications in memory is much less important and so the sticky bit needed so much.
chmod
Set User ID is used on an executable so that when it is run, it is run under the security context of the file owner not the current user. That means in case I have an application whose owner is ' root ' and it has its SUID bit set, then when I run this application as a normal user, that application would still run as root. Since the SUID bit tells Linux that the the User ID root is set for this application and whenever this application executes it must execute as if root was executing it (since root owns this file).
Set Group ID. Just like SUID, setting the SGID bit for a file sets your group ID to the file's group while the file is executing
To set any of these 3 attributes, we use a 4th digit preceding the usual 3 used with chmod. That digit is set using the following:
4 Set user ID (s) 2 Set group ID 1 Set sticky bit (t)
So if we want to set a file with permission rwxr-xr-x and set the user ID bit we could use
chmod 4755
umask
Another permissions feature you may encounter is that of umask.
VMware Tools
vmkpcidivy
A VMware tool. Used to divide up the PCI resources between the service console and VMs or if they are shared. E.g. 1 NIC to SC, 1 SCSI adapter to share between SC and VMs, 1 SCSI adapter to VMs. This tool is stored in /usr/sbin/vmkpcidivy
This is a command line tool which asks a series of questions. Use with the –i switch for interactive.
This is also used to refreshnames and –q vmhba_devs
For example, if I had a SAN LUN of vmhba1:0:25 and lets say I removed the VMFS from this LUN and wished to use it from the service console, I’d refresh names and then query the vmhba_devs to find out what device name the service console was going to use for this LUN, e.g. /dev/sda.
A very useful feature of this tool is the ability to create a new profile. This adds a new boot option to the LILO boot menu that will have its own allocation of memory and PCI devices. If you are unsure about the changes you are making, then create a new profile e.g. esx (modified)
vmkfstools
The vmkfstools utility is the tool for managing virtual disks. Remember that to copy into a VMFS could have an adverse affect on other VMs with virtual disks on the same LUN. We always want to avoid using copy to populate a VMFS. Copy operations will update the volume in 16k blocks causing unnecessary SCSI reservations to update vmfs metadata.
The switches that can be used with the command are listed below:
-i to import a virtual disk to VMFS
-eto export a virtual disk from VMFS
-sto re-scan for new LUNs
-S to set vmfs metadata volume label
-X 6000M ./file.dskto extend an existing DSK to 6GB
-c 4000M ./file.dskto create a new empty virtual disk
-C to create a new vmfs volume
-l vmfsnameto list virtual disks on specified vmfs
-F to set the access mode e.g. public/shared
-k to create a VMkernel swap file
-w to activate a VMkernel swap file
-y to deactivate a VMkernel swap file
-T to convert a vmfs1 volume to vmfs2
Remember that the vmfs parameter always goes last on this command parameter set for vmkfstools. This can be confusing for the beginner as the source and target order is different for imports and exports.
If we want to simply list the files on a vmfs volumes we use the -l switch.
vmkfstools –l /vmfs/vmhba0:0:0:8
or if we wish to use the volume label;
vmkfstools –l <vmfs-metadatalabel>
would produce an output similar to the following
Name: VMFS2-VOL1 (public)
Capacity 129465874944 (123461 file blocks * 1048576) avail
Permission Uid Gid Attr Bytes (Blocks) Last Modified Filename
rw------- 0 0 swap 2146435072 ( 2047) Nov 18 18:25 Swap.vswp
rw------- 500 500 disk 4194304000 ( 4000) Nov 16 14:12 VM1.dsk
rw------- 500 500 disk 6291456000 ( 6000) Nov 23 22:19 VM2.dsk
rw------- 500 500 disk 2621440000 ( 2500) Nov 17 23:09 VM3.dsk
rw------- 500 500 disk 4194304000 ( 4000) Nov 24 18:11 VM4.dsk
This command would create a new virtual disk (monolithic) on the specified vmfs volume. Remember it is always better to use the VMFS name as this will not change even if your hardware does.
To import a virtual disk into the VMFS we use vmkfstools with the -i switch. This will take a virtual disk in COW format into monolithic format without causing excessive SCSI reservations on the LUN holding the target VMFS
This is used to manage the VM kernel core dump partition. We can change the partition used if required. This tool is also needed if the core dump partition had been removed because ESX expects it to be there when starting up, so we need to tell ESX that it has gone.
vmkdump -q Query the VM kernel for which partition it will use
vmkdump -p vmhba0:0:0:3 Set VM kernel dump partition to partition 3
vmkdump –p none:0:0:0 Set VM kernel dump partition to none
Remember the vmkcore partition does not have a mount point in the service console and is not specified as ext3. We can use the fdisk -l command to view where the core dump partition is in relation to the disk layout.
vmkload_mod
Allows viewing with the –l switch, loaded and unloaded VMkernel modules. This command differs from lsmod which lists the modules loaded for the service console. This is a very good way of differentiating what modules the kernel is using versus the ones used by vmnix.
Note, the -l parameter can also be specified as --list
Devices
/etc/modules.conf
Not only lists eth0 (as discussed in network section) but in fact this text file sometimes describes the devices that are assigned to the service console by vmkpcidivy.
modinfo
This tool takes a service console driver and displays the options it supports. For example
modinfo e1000
Would produce a list of flow control settings for the Intel gigabit NIC.
modinfo cciss
Would produce the file details and version of the HP Smart Array controller.
lspci
Great tool for listing pci devices. Could be used to demo what the VM is presenting to the guest OS.
lspci –v verbose output
lspci –t print tree of PCI devices (use –tv for verbose tree)
You may wish to examine /proc/pci also in order to correctly identify PCI devices and their slot configurations. One point to note is that when you are faced with PCI slot numbers is that not all hardware vendors number their slots in a straight forward left to right configuration. Make sure you know your slot numbers and their layout!
lsusb
A tool to list USB devices. Gives out way more info that is actually required. Remember that USB devices cannot be presented to virtual machines in ESX Server. If you wish to use a USB device in ESX, then you will have to use a USB over IP device and install the appropriate driver software into your guest OS for this. The most common USB over IP device is companyX
kudzu
A Red Hat tool to detect and configure hardware.
/etc/vmware/netmap.conf
This text file maps ESX virtual switch names to device names. It is a network map config file as opposed to the more generic devices map config file (devnames.conf).
Remember that if a virtual switch has no physical adapters, then it is vmnet_x. If a virtual switch has only 1 physical adapter is is vmnic_x. If there are two or more physical adapters assigned to a virtual switch, then the device is bondx.
A bond can be in one of three modes, out-mac (default), out-ip and standby
out-mac A VM virtual NIC is assigned to a pNIC in the bond and it uses only that
out-ip A VM TCP conversation is placed on an available pNIC
standby A VM will only use one NIC until a failure, then the other is used
/etc/vmware/devnames.conf
This text file maps device names (example above) to modules and their PCI addresses. Note that the devnames.conf file contains SCSI devices and NIC devices.
002:14.0 megaraid vmhba0
005:04.1 nic vmnic0
011:07.0 nic vmnic1
012:08.0 nic vmnic2
/etc/vmware/vmware-devices.map
Appears to be like a hardware compatibility list. Watch out for creating your own device map, devices.local The /etc/vmware/vmware-devices.map file contains a list of devices supported by ESX Server. This release includes support for a local version of this file, /etc/vmware/vmware-devices.map.local. Modify the vmware-devices.map.local to select different device drivers. This file is not modified during an ESX Server upgrade, preserving your customizations. The vmware-devices.map.local is read when the VMkernel is loaded:
Any changes to the vmware-devices.map.local file require a reboot, or at least an unload/reload of the VMkernel to take effect.
Entries in the vmware-devices.map.local files are used in addition to the entries in the vmware-devices.map file. The vmware-devices.map.local file does not need to mirror the vmware-devices.map file.
Any vmware-devices.map.local file entries that correspond to the vmware-devices.map file entries supercede the vmware-devices.map file entries.
/etc/vmware/config
/etc/vmware/hwconfig
Loads of information in this text file. Useful for finding which nic is in which team. If using alongside devnames.conf and netmap.conf use the following command
grep –i vmnic /etc/vmware/hwconfig
/dev/fd0
How to address the floppy disk drive
/dev/sda
SCSI device
/proc/vmware/pci
Text file with discovered PCI devices
Bus:Sl.F Vend:Dvid Subv:Subd Type Vendor ISA/irq/Vec P M Module Name Spawned bus
000:00.0 8086:3590 1028:016e Host/PCI Intel C
000:02.0 8086:3595 0000:0000 PCI/PCI Intel 001 C
000:03.0 8086:3596 0000:0000 PCI/PCI Intel 004 C
000:04.0 8086:3597 0000:0000 PCI/PCI Intel 007 C
000:05.0 8086:3598 0000:0000 PCI/PCI Intel 010 C
000:06.0 8086:3599 0000:0000 PCI/PCI Intel 013 C
000:29.0 8086:24d2 1028:016e USB Intel 11/ 16/0x69 A C
000:29.1 8086:24d4 1028:016e USB Intel 10/ 19/0x71 B C
000:29.2 8086:24d7 1028:016e USB Intel 7/ 18/0x79 C C
000:29.7 8086:24dd 1028:016e USB Intel 3/ 23/0x81 D C
000:30.0 8086:244e 0000:0000 PCI/PCI Intel 016 C
000:31.0 8086:24d0 0000:0000 PCI/ISA Intel C
000:31.1 8086:24db 1028:016e IDE Intel 0/ / A C
001:00.0 8086:0330 0000:0000 PCI/PCI Intel 002 C
001:00.2 8086:0332 0000:0000 PCI/PCI Intel 003 C
002:14.0 1028:0013 1028:016e RAID Dell 7/ 38/0x91 A S vmhba0 004:00.0 8086:0329 0000:0000 PCI/PCI Intel 005 C
004:00.2 8086:032a 0000:0000 PCI/PCI Intel 006 C
005:04.0 8086:1010 8086:1012 Ethernet Intel 7/ 58/0x99 A C 005:04.1 8086:1010 8086:1012 Ethernet Intel 10/ 59/0xa1 B V e1000 vmnic0
010:00.0 8086:0329 0000:0000 PCI/PCI Intel 011 C
010:00.2 8086:032a 0000:0000 PCI/PCI Intel 012 C
011:07.0 8086:1076 1028:016d Ethernet Intel 11/ 72/0xa9 A V e1000 vmnic1
012:08.0 8086:1076 1028:016d Ethernet Intel 11/ 73/0xb1 A V e1000 vmnic2
016:13.0 1002:5159 1028:016e Display ATI 7/ 18/0x79 A C
findnic
This tool is also known as “The VMkernel Network Card Locator”. It locates a physical NIC in an ESX server by using pings. It is left to the operator to check which NIC is sending the echo requests by either unplugging network cables or inspecting flickering lights on the NIC or the switch. The useful feature of this tool is we can ping based on the device name that the VMkernel uses to access the NIC. We should remember that each physical NIC in an ESX server that is allocated to the VMkernel does not itself have an IP address, therefore in to perform an ICMP echo request, we need to temporarily give that NIC an IP; this is specified as the first IP parameter in the command, the second IP parameter being the ICMP destination.
findnic –i 5 vmnic2 10.0.0.1 192.168.1.3
The above command will send ICMP echo requests to 192.168.1.3 every 5 seconds. We could also use the –f switch which would flood ping.
File Systems
procfs
The volatile /proc directory hierarchy can be treated as a file system but is held in RAM.
/proc/vmware/sched/cpu
A text file snapshot of cpu scheduling. You could increase CPU shares using this method
echo 10000 > /proc/vmware/vm//cpu/shares
watch
This is a fantastic utility that polls whatever command you supply it with an displays a running changing status. For example, we could use the command
watch cat /proc/vmware/mem
to obtain a dynamic view of memory usage by VMware. If you are viewing lots of output but can't see what is actually changing we use the -d parameter to specify differences, thus highlighting changes between refreshes.
watch –d cat /proc/vmware/mem
To specify a different refresh rate, we use the --interval=n parameter
smbclient
This is a redirector type tool to view and connect to SMB (Microsoft networking) hosts. Before updating the /etc/fstab file with remote file system information, check first using smbclient that the share is visible. The following was produced with
Sharename Type Comment
--------- ---- -------
E$ Disk Default share
Microsoft Disk
IPC$ IPC Remote IPC
D$ Disk Default share
NETLOGON Disk Logon server share
REMINST Disk Remote Installation Share
ADMIN$ Disk Remote Admin
SYSVOL Disk Logon server share
VPLOGON Disk Symantec AntiVirus
C$ Disk Default share
VPHOME Disk Symantec AntiVirus
To create a mount point to a Microsoft share is very straightforward. Remember, we are allowing the service console to access a remote file system. This is not related to what virtual machines are doing. Further, we need to be careful if we are attempting to do any file operations due to potential limits with 2GB file sizes.
1. Add entry to local hosts (/etc/hosts) for MS host
2. Create directory where you wish the mount point with mkdir
3. Run smbclient –I -U user –L NetBIOSComputername to check you can see the shares
4. Add an entry to /etc/fstab
5. Mount the remote file system with mount /mountdir
6. Change directory into mount point and the directory will no longer be empty but will appear just like a mapped drive.
Alternatively, if you just want to map temporarily to a Microsoft host and not have to modify fstab, then use smbclient interactively as shown:
Network File System (NFS) is provided by rpc.nfsd and would normally be launched by an nfs script in /etc/rc.d. To start using NFS to mount directories on other ESX or Linux servers, we can use the following steps:
1. Change the ESX Server which is to be the NFS server to use medium security (using the MUI is easiest for this)
2. Add an entry to the file /etc/exports on the NFS server by either using vi text editor (/vmimages *) or use the exportfs command
3. Check the NFS client is not excluded by the server file /etc/hosts.deny
4. Up to you if you explicitly allow the NFS client by adding the NFS client to /etc/hosts.allow on the NFS server
5. Start the NFS server daemons
a. /etc/rc.d/init.d/service portmap start b. /etc/rc.d/init.d/service nfslock start c. /etc/rc.d/init.d/service nfs start
6. Start the NFS client daemons
a. /etc/rc.d/init.d/service netfs start b. /etc/rc.d/init.d/service portmap start
7. Mount remote export directory from the client with the command
a. mount –t nfs server:/export localdir/localmount
showmount
This command is used by a NFS client to see what directories are being exported by a NFS server.
showmount –e nfsserver
This command can be specified with the hostname name or IP address of the NFS server holding the exported directories.
exportfs
The exportfs command allows you to selectively export or unexport directories without restarting the various NFS services.
NIS
NIS stands for Network Information Service. Ensures numeric owner IDs are unique across the organisation. This is because numeric owner IDs are used in NFS, so we are often going to have mistaken identity as used 515 on the nfs client will not be the same as user 515 on the nfs server.
vmware-mount.pl
Just like the Windows utility to mount virtual disks when they are powered off so you can check what’s in them. This will mount ext3 and vfat as read/write but NTFS as read only. To find out what file systems are in the virtual disk, use the –p switch.
vmware-mount.pl –p /vmfs/VMFS-VOL1/win2k3.dsk
--------------------------------------------
VMware for Linux - Virtual Hard Disk Mounter Version: 1.0 build-9638\nCopyright 1998 VMware, Inc. All rights reserved. -- VMware Confidential
--------------------------------------------
Nr Start Size Type Id Sytem
-- ---------- ---------- ---- -- ------------------------
1 63 8160957 BIOS 7 HPFS/NTFS
If we actually want to mount a partition then we need to be specific and create a directory (or use an existing) to be our mount point.
This command will tie up the console window hence you’ll need to spawn a new window first to navigate to /myntfs to view the contents.
mount
Mount file system This command can be used with the –t switch to specify file system type, e.g. nfs, smbfs or iso9660
mount -t nfs [-o options] esx4:/vmimages /root/other_server_vmimages
umount
Un-mount file system. Note it is u-mount and not unmount!
smbmount
A mount –t smbfs passes control across to this utility. We can use this utility directly if we prefer for mounting SMB host file systems.
/etc/fstab
If we have smb mount points permanently, then this file could end up with user credentials in it. This file is readable by everyone so this is not good. We can place the credentials for the smbmount in a hidden secured file in our home folder eg. /root/.smbcreds
A hidden file that extends shell script for the BASH shell. This is found in the users home folder.
/etc/bashrc
System-wide bash shell settings
clear
Equivalent of CLS command in MS-DOS and in Windows command prompt.
sum
Prints the 16-bit checksum and size of the specified file.
md5sum
Prints the MD5 checksum for the file supplied as a command parameter. VMware publish md5 checksums for all downloads to allow the end user to confirm that the file had not be corrupted in download.
cal
Display calendar for current month -3 last, current and next month 200x display year cal 2005 | less to get scrollable calendar
/etc/vmware/vm-list
Auto generated text file listing the VMs on the server. This file should not be manually edited, should match up with output of vmware-cmd –l.
Process Management
PID
PID stands for Process ID. Every running process has a process ID that is valid for the length of process execution.
PPID
PPID stands for Parent Process ID, which is the PID of the process that launched that process.
is useful as the “w” indicates wide format, so we can see the full directory path to the vmx file.
pstree -ph
Show process tree with PIDs and ancestors
esxtop
VMware version of process viewer top
renice
Change process priority. If there were many VMs running concurrently, it may be necessary to temporarily raise the priority of the MUI in order to complete administrative tasks
#ps –eaf |grep vmware
#renice –p #renice –p
To reset the PID of these processes back to their defaults, use renice again to set the priority to zero.
pidof
Finds the PID of a named process.
#pidof vmware-authd
&
We can configure processes to run in the background by adding a “&” suffix when launching the program.
#sleep 10 &
jobs
List the process jobs running in the background.
SAN & Disk Management
Disk.MaxLUN=8
This is the default setting, which scans 0 to 7 on start up. Remember if we what to scan up to LUN N we must set Disk.MaxLUN to N+1. This setting is found in the /etc/vmware/vmkconfig file but should be modified via the MUI (Options tab, Advanced Settings).
Disk.MaskLUNs
This parameter controls LUN visibility, again this is in the /etc/vmware/vmkconfig file. LUN masking is only supported on fibre channel HBAs. This overrides the Disk.MaxLUN setting.
vmhba0:0:4,6-255 would scan 0,1,2,3,5 i.e. skip 4 and skip 6 through 255 vmhba0:0:3,4,9-255 would scan 0,1,2,5,6,7,8 i.e. skip 3 & 4 and skip 9 through 255
/etc/vmware/vmkconfig
This is a text file which stores vmkernel configuration, like an INI file or registry key. The important fact to note is that this file does not exist until you make an edit away from the default setting.
vmkfstools –s vmhba0
Used to re-scan for new LUNs on specified host bus adapter.
proc/vmware/scsi
WWN
SAN devices are identified by a world wide name, a unique 64-bit address. Remember we can use the perl script wwpn.pl to determine quickly what the WWN is for the installed FC hba.
LUN
A LUN identifies individual units of storage behind a SCSI ID. A LUN could be a single disk or a RAID5 volume.
Zoning
Zoning is either hard (switch port) or soft (WWN controlled)
LUN Masking
LUN Masking is a disk array feature that controls which LUNs are presented to which WWNs
/proc/scsi/driver/number
The WWN for an adapter would be found in this file.
Disk.SupportSparseLUN
The setting Disk.SupportSparseLUN should =1 as LUNs may be discontiguous. For example if there are disk volumes at LUNs 0,1,2 and 6 then we want to be sure that after LUN 2, the VMkernel storage driver does not stop scanning. We want the LUN scanning to reach the last LUN specified in DiskMaxLUN parameter, regardless of whether the visible LUN numbers are contiguous or not.
Disk.RetryUnitAttention=1
This setting tells ESX server to retry SCSI commands as vendor specific status codes may have been received and ESX may think the volume is present but not accessible when in fact its just a message that cache has been upgraded.
Performance
Free
A simple utility to display free memory in the service console.
esxtop
This is the VMware version of top and provides CPU, memory and disk configuration realtime information just like top does, but this time we only see the data relating to the VMkernel so we see worlds instead of processes as resource consumers.
top
Shows the running processes in the service console and lists the top consumers of CPU time.
iostat
vmstat
Don’t get confused by this command, it is a Linux command, not a VMware command. This is meant to view Linux processes, memory and paging. We generally use vmstat with 2 numeric parameters, the first parameter is how frequently the tool should run (specified in seconds). The second parameter specifies how many times the tool should run e.g.
vmstat 3 10
would run the tool every 3 seconds for 10 times and then exit.
man cpu
Great info on using the /proc/vmware file system to change share allocation and processor affinity.
/proc/vmware
In a subdirectory called /vm there are subdirectories for each vm labelled by number How do you find out what number corresponds to what VM?
grep –i worldid= ~ali/vmware/ISAserver/vmware.log
Some other examples of great info that can be extracted are outlined in the following table:
/proc/vmware/vm/139/cpu affinity 0,1,2,3
/proc/vmware/vm/139/cpu shares 1000
/proc/vmware/vm/139/cpu status
/proc/vmware/vm/138/cpu hyperthreading
/proc/vmware/sched/cpu
vmkusage
Generates web page usage reports. Was not automatically installed in previous versions of ESX
vmkusagectl install to install the utilisation web pages & setup a cron job vmkusage –graph to generate graph images vmkusage –regroove to wipe the db and start stats again
Not sure if we need to be logged in for this to work, but vmkusage does appear to be required. We get loads of output on this page, similar to running command line tools. This is part of what looks like the legacy interface to ESX server, i.e. it doesn’t look as cool as the MUI of ESX 2.x.
crontab
Lists the cron jobs scheduled for the user when used with the –l (list) switch.
SNMP
snmpwalk
Used to walkthrough SNMP mibs. -M – use MIBSDIR -m all use mibs list instead of default mibs list
Trap Receiver
A utility to display SNMP traps. MIBs can be loaded into this.
Perl Scripts
vmsnap.pl
Backup script to backup a virtual disk whilst the virtual machine is still running. This is achieved by the script creating a REDO file and if necessary a REDO file of a REDO file; thus reducing the changing data and allowing a straight copy of the non-changing file.
vmres.pl
This script is the restore tool for vmsnap.pl
wwpn.pl
Used to determine the world wide name (WWN) for hba.
vmkstatus.pl
This Perl script runs on tty1 of the ESX server.
pbind.pl
Create or delete HBA persistent bindings
vmware-mount.pl
This is used to mount a virtual disk in the service console while the virtual machine is not running. For example, if a configuration file in the guest OS virtual disk is preventing a virtual machine from booting correctly, you could
An event like a complete data center power failure is something you never want to experience. Having recently gone through one I thought I would share some lessons learned from it.
This particular data center had a full UPS (uninterruptible power supply) system and backup diesel generator when a routine battery maintenance performed on the UPS shorted some circuits causing power loss to the entire data center. This event made me realize that a little preparedness can go along way in getting servers and virtual machines (VMs) back online after a power failure.
First and foremost, the DNS (Domain Name System) is probably the most important service in your data center. Most servers and workstations use DNS names instead of IP addresses to communicate with each other. Without DNS, servers can’t get to anything by hostname and will effectively be isolated from each other. Most administrators are used to using DNS names, so when DNS is not available they usually do not know the IP addresses of the server and subsequently can’t connect to them. So it is a good idea to have a hard copy of all your servers and their IP addresses somewhere in your data center for you to reference when DNS is not available.
Virtual servers can be even more problematic. If you have all your DNS servers virtualized which cannot be started because of network or shared storage issues, you can run into problems starting other servers and services that rely on DNS. Consider having at least one physical DNS server or having one or two DNS servers running on local storage instead of shared storage.
Another helpful insight: Make sure you know command line procedures for administration on your host servers. You may not be able to connect to your host via a graphic user interface (GUI) until certain systems are up so the command line can be your only way to check the host server health and perform VM operations. Again, it helps to have paper documentation of the host command line utilities and their syntaxes.
Finally you want to make sure you start your servers back up in the proper order due to dependencies that certain servers and applications have. Obviously, with the network unavailable, not much is going to function properly. The storage-area network (SAN) is also critical for your host servers that utilize shared storage for VMs. Windows servers also take a very long time to boot if a DNS server and domain controller are not available when they are starting.
Below is a general order for restarting your servers and applications.
Network SAN DNS servers DHCP servers Database servers Application/Web servers
Should you assign a virtual machine (VM) more than one virtual processor or not? It’s common for admins to configure virtual symmetric multiprocessing, or VMs with multiple CPUs, whether it is needed or not.The decision to use more then one virtual processor in a VM should be based on an actual requirement by the applications installed on the VM and not simply because two processors are better then one. Many physical servers commonly have multiple CPUs regardless if the applications running require them. While being wasteful of server resources, this does not negatively impact a physical server but most VMs will usually run better with one virtual processor and can actually run slower when more than one is assigned to it.
The reason for this is the hypervisor’s CPU scheduler must find simultaneous cores available equal to the number assigned to the VM. So a four VCPU VM will need to have four free cores available on the host for every CPU request that is made by the VM. If there are not four cores available because other VMs are using them then the VM must wait until the cores become available. Single VCPU VMs have a much easier time because they only need there to be a single core available for the scheduler to process CPU requests for it.
Here are some tips on assigning VCPUs to VMs:
Limit the number of VSMP VMs on your hosts. The less you have, the better your VMs will perform. Assign a VM multiple VCPUs only if you are running an application that requires it and will make use of them. Don’t assign a VM the same amount of VCPUs as your host system has total cores available. If you are going to use VSMP have at least twice (preferably three or four times) the number of cores available on your host system then that of your VM with the most VCPUs. So if you have a four VCPU VM, have at least eight cores available on your host server and preferably 16. If you are converting a multi-CPU physical Windows server to a single VCPU VM, make sure you change the HAL from multiprocessor to uniprocessor. Don’t use CPU affinity as it restricts the scheduler and makes it harder to process CPU requests. The scheduler is very good at what it does, so let it do its job.
VMware finally made the move that everybody predicted and was awaited for a long time: releasing its hypervisor for free.
During the Q2 2008 earnings call the company announced that before the end of July (the planned date is July 28) it will release the Update 2 for VMware Infrastructure 3.5 and that will give away the lightweight edition of the product, ESX 3i, for free.
The new 3i edition was introduced at the end of the last year. It doesn’t change the characteristics of the hypervisor but fundamentally chances a part of its architecture, dropping the Console Operating System (COS) and reducing the overall system footprint to 32Mb. The change allowed several OEMs to preload this edition of the hypervisor into their servers, through internal USB keys, Solid-State Drives (SSD) or hidden partition in the primary hard drives.
virtualization.info has learned some additional details about the move:
VI 3.5 and ESX 3i will continue to share a the large majority of the code base (so there will be an ESX 3i Update 2). There will be no delays on the release of both products.
ESX 3i will continue to have the same features that has today, without additional limitations
ESX 3i will continue to have the same APIs, allowing anybody to develop free or commercial alternatives to VirtualCenter (despite some features like VMotion cannot be replicated because of the VMware SDK limitations).
VMware will not require the purchase of any software & support subscription to access the product. Customer will be able to get the code without any restriction.
The customers that purchased the current version of ESX 3i directly from the VMware online store will be eligible for a rebate.
The ESX version that includes the Console Operating System (COS) will not be faded out (at least in the short timeframe). Most VMware customers are currently using that version and the company will support them for a long time.
VMware Server will not be faded out (as many could suppose). The company still sees the product as a valuable proposition for a different kind of audience.
At first sight this move seems a clear attempt to recover the terrible stock performance that VMware is suffering from the departure of its former CEO Diane Greene. But virtualization.info has learned that the plan was not developed by the new CEO Paul Maritz, but by Greene herself several months ago.
How the free ESX will impact the market?
First of all, the competition will move on the virtual infrastructure management (with a special focus on the storage) and automation (for hosted desktops, virtual labs, VM lifecycle, autonomous computing, cloud computing) areas, turning the hypervisor as a commodity, embedded in the operating system or not. Any vendor that will not adopt the same strategy (like Citrix or Virtual Iron) will have a hard time to justify the price of its solution.
Secondarily, the product will finally boost the diffusion of VMware technologies in the SMB market. This will seriously disturb the Microsoft effort in this space and all the other virtualization vendors that have a clear focus on the small companies (like Virtual Iron or Parallels).
Last but not least, the sales channel will become critical to win the competition for most customers. When multiple market leaders have a similar proposition in terms of price and feature set, the experience in managing the channel does the real difference. VMware didn’t seem able to compete with Microsoft and Citrix here, but the new CEO certainly has the knowledge to correct the mistakes of the past.
OpenNebula transforms a physical cluster into a flexible virtual infrastructure which dynamically adapts to the changing demands of a service workload. OpenNebula leverages existing virtualization platforms to create a new virtualization layer between the service and the physical infrastructure. This new layer supports the execution of the services on a physical cluster, extending the benefits of VMMs (Virtual Machine Monitors) from a single physical resource to a cluster of resources. OpenNebula effectively decouples a server (deployed as a pre-configured VM) not only from the physical infrastructure but also from its physical location.
Centralized management of a pool of VMs and physical resources
Balance of workload to improve efficiency and utilization.
Server consolidation to a reduced number of physical systems, so reducing space, administration effort, power and cooling requirements or supporting the shutdown of systems without interfering workload.
Dynamic resizing of the physical infrastructure by adding new hosts.
Dynamic cluster partitioning to execute different services.
Support for heterogeneous workloads with multiple (even conflicting) software requirements, allowing the execution of software with strict requirements as jobs that will only run with a specific version of a library or legacy application execution.
On-demand provision of VMs to meet the demands of the service end-users.
Additionally, the virtualized system infrastructure scales all the benefits from the hypervisor layer, such as protection against malicious use of resources or malware, protecting long running jobs from system failures, the ability to reproduce situations… The Grid and Virtualization Use Cases working draft by the OGF Grid and Virtualization Working Group provides a description of virtualization capabilities and use cases.
The OpenNebula Virtual Infrastructure Engine differentiates from existing VM managers in its highly modular and open architecture designed to meet the requirements of cluster administrators. The last version supports Xen and KVM virtualization platforms to provide the following features and capabilities:
Efficient Resource Management. The scheduling module provides a generic framework to build any capacity provision policy. The default scheduling policy determines the best host to start a VM according to requirement and rank expressions consisting on infrastructure parameters. Support for advance reservation of capacity is provided through the Haizea VM-based lease manager.
Powerful Interface. The OpenNebula Engine provides a powerful API and CLI for monitoring and controlling VMs and physical resources.
3rd Party Software Integration. OpenNebula interface allows its integration with third-party tools, such as job managers, service adapters, VM image managers…; so providing a complete solution for the deployment of flexible and efficient virtual infrastructures.
Open and Flexible Architecture. Hypervisor access is performed through customizable programs, thus enabling administrators to tailor OpenNebula's behavior by adding new infrastructure metrics and parameters or even supporting new Hypervisors.
Failure Tolerance. OpenNebula uses a persistent database backend to store host and VM information. In case of a local failure, the daemon can be restarted and all the running VMs recovered. All the OpenNebula components have been designed to be independent so a failure of one of them does not put the system at risk.
On-demand Scale out of Service Workloads. OpenNebula provides plugins to access Amazon EC2 to supplement local resources with cloud resources to satisfy peak or fluctuating demands.
Ease of Installation and Administration. OpenNebula is installed on a UNIX cluster front-end without requiring the installation of new services in the remote resources, apart from the VM monitor.
Open Source Software. OpenNebula is open source software released under Apache license v2.0 and support is provided through a mailing list.
OpenNebula is an Active Project. As engine for the dynamic management of VMs, OpenNebula is being enhanced in the context of the RESERVOIR project (EU grant agreement 215605), the European initiative in Cloud Computing, to address the requirements of several business use cases.
Front-end on UNIX, and its XML-RPC API allows remote access from any system
VM Load Balancing
The scheduling module provides a generic framework to build any policy for capacity provision. The default scheduling policy determines the best host to start a VM according to requirement and rank expressions consisting on infrastructure parameters. Support for advance reservation of capacity is provided through the Haizea VM-based lease manager
Windows Server 2008 Hyper-V, hypervisor-based server virtualization technology, allows you to make the best use of your server hardware investments by consolidating multiple server worklaods as separate virtual machines (VMs) running on a single physical machine. With Hyper-V, you can also efficiently run multiple different operating systems concurrently, on a single server, and fully leverage the power of x64 computing.
When you’re planning a virtualization infrastructure with Hyper-V, please be sure to stay within the supported limits below. In addition, the limits discussed in this document are highly dependent on the underlying hardware server configuration.
· Hyper-V requires an x64 processor with Hardware-assisted virtualization. This is available in processors that include a virtualization option; specifically, Intel VT or AMD Virtualization.
· Hardware Data Execution Protection (DEP) is also required and must be enabled. Specifically, you must enable Intel XD bit (execute disable bit) or AMD NX bit (no execute bit).
· Physical Processor support: Hyper-V is supported on systems with up to 16 logical processors; A logical processor can be a core or a hyper-thread. Examples include:
o Single processor/Dual core system = 2 logical processors
o Single processor/Quad core system = 4 logical processors
o Dual processor/Dual core system= 4 logical processors
o Dual processor/Quad core system=8 logical processors
o Quad processor/Dual core system= 8 logical processors
o Quad processor/Dual core, hyper-threaded systems=16 logical processors
o Quad processor/Quad core systems=16 logical processors
· Virtual Processor support: Up to 4 virtual processors per virtual machine
Note: Hyper-V may run on systems with greater than 16 logical processors; however these configurations are not supported.
2. Storage Area Networks (SANs): iSCSI, Fiber Channel, SAS
3. Network Attached Storage (NAS)
IMPORTANT: Although the I/O performance of physical SCSI and IDE can differ significantly, this is not true for the virtualized SCSI and IDE devices in Hyper-V. Hyper-V IDE and SCSI storage devices both offer equally fast high I/O performance when integration services are installed in the guest operating system.
· Maximum number of virtual CD/DVDs: By default, each virtual machine is provided one virtual CD/DVD. However, each virtual machine can support up to 3 virtual DVD drives per virtual machine (on the virtual IDE bus only) as one IDE device must be used as a virtual boot device.
· Pass-through CD-DVD:
o For security purposes, only one virtual machine can access a physical CD/DVD at a time.
· Virtual CD/DVD:
o Hyper-V supports reading virtual CD/DVD .ISO files
· Each virtual machine supports up to 2 virtualized COM ports that can be used to communicate with a physical server (local or remote) via a serial named pipe.
For more information about the Microsoft support policy on non-Microsoft virtualization hardware, click the following article number to view the article in the Microsoft Knowledge Base:
897615 (http://support.microsoft.com/kb/897615/) Support policy for Microsoft software running in non-Microsoft hardware virtualization software
This article lists the server and client guest operating systems that are supported on a Hyper-V virtual machine host that is running on a Windows Server 2008-based computer.
Note Virtual machines are configured to use one or two virtual processors.
•
Windows Vista Business x86 with Service Pack 1
•
Windows Vista Enterprise x86 with Service Pack 1
•
Windows Vista Ultimate x86 with Service Pack 1
Windows Vista, x64-based editions
Note Virtual machines are configured to use one or two virtual processors.
•
Windows Vista Business x64 with Service Pack 1
•
Windows Vista Enterprise x64 with Service Pack 1
•
Windows Vista Ultimate x64 with Service Pack 1
Windows XP Professional, x86-based editions
•
Windows XP Professional x86 with Service Pack 3
Note Virtual machines are configured to use one or two virtual processors.
•
Windows XP Professional x86 with Service Pack 2
Note Virtual machines are configured to use one virtual processor.
Windows XP Professional, x64-based editions
Note Virtual machines are configured to use one or two virtual processors.
•
Windows XP Professional x64 with Service Pack 2
How To install Hyper-V RTM on Server Core
Whenever I hear Server Core and Hyper-V used in one sentence I get a big smile on my face. Here's how to install a freshly created Server Core installation with Hyper-V RTM in six easy to follow steps on your Server Core console:
Step 1
Choose a good box to use. The processor should be a recent x86-64 one, like an AMD64 Athlon64, an AMD64 Opteron, or an Intel Core Duo processor. The motherboard and BIOS should be equipped with Hardware-assisted Virtualization and Hardware-enforced DEP.
Consult with your hardware vendor for suitable boxes before purchasing a suitable box.
Tip! If you have a box you think is suitable download and run SecurAble, a tool by Gibson Research Corporation. It will not only show whether your hardware meets the requirements, but also whether Hardware Assisted Virtualization and Hardware-enforced DEP are enabled in BIOS.
Step 2
Install a 64bit version of Windows Server 2008 on your machine. When prompted to install a specific version choose to install it Server Core instead of a full installation. On first boot log in as Administrator with a blank password. Change the password afterwards to a password that complies with the password complexity requirements. Configure your Server Core installation with your preferred settings. Configure the time, time zone and time synchronization (using control.exe timedate.cpl), set the hostname (using netdom.exe), settle licensing (using slmgr.vbs), configure IP settings (using netsh.exe) and make sure you're making backups of your installation (using wbadmin.exe). If you want you can enable Windows Update en Remote Desktop (using wsregedit.wsf)
Install the update package for Microsoft Hyper-V RTM, which is mentioned in Microsoft Knowledgebase article 950050. You can do this in two ways, depending on your hurry:
Download the standalone update(s) and install manually
Download the update(s) through Automatic Updates
Note: For localized installations of Hyper-V please take a look at Microsoft Knowledgebase article 951636. Download and apply the package when appropriate.
Download and install the update(s) manually
On another computer than your Server Core box you can download the 64bit version of the update for Windows Server 2008 x64. You can get the *.msu file onto your Server Core installation using a USB stick, a network share, a FTP server on your network or burn it on CD or DVD media. You can run the update with the following command:
wusa.exe Windows6.0-KB950050-x64.msu
In the Windows Update Standalone Installer screen click OK to install the update.
In the screen Installation complete press the Restart Now button. After the restart log in as Administrator again.
Download through Automatic Updates
You can alternatively use Windows Update or Windows Server Update Services to automatically get the update installed on your Server Core box. On a freshly configured Server Core box run the following commands to enable Automatic Updates:
cd C:\Windows\System32 cscript SCregEdit.wsf /AU 4 net stop wuauserv net start wuauserv
Afterwards you can make your Server Core box check for updates using the following command:
wuauclt.exe /detectnow
Your Server Core box will detect and download updates and will schedule to install them at the pre-configured time. On a default installation this is 3:00 AM. Remember: You can use Group Policies and Windows Server Update Services to change this behavior.
To check whether your Server Core box correctly detected the updates it needs take a look at the C:\Windows\WindowsUpdate.log file using Notepad.exe. Most recent activity is added at the end of the log file.
Step 4
Check the correct installation of the update. You don't want to end up with having to redo your virtual machines, so it's important to check the update was installed successfully. To check the correct installation of update packages run the following command:
wmic qfe list /format:list
One of the updates in the list should be the update with HotFixID KB950050. For localized installations also check for the update with HotFixID KB951636.
Step 5
Install the Hyper-V role on your Server Core box, using the following command:
start /w ocsetup.exe Microsoft-Hyper-V
Note: The ocsetup.exe command is case-sensitive. Type the role exactly as mentioned above.
The Windows package manager will report the system must be restarted to enable the changes. Press Yes to let the package manager restart the system.
Step 6
Check the correct installation of the Hyper-V Server Role using oclist.exe. After your Server Core box restarted log in as Administrator. I use the following command to report only the installed roles and features: (which prevents scrolling)
Congratulations! After performing the previous six steps you ended up with a Server Core Hyper-V server. Using the command line on your Server Core box, the Hyper-V Manager from a Windows Vista (SP1) or Full installation of Windows Server 2008 or through Powershell you can now begin managing Hyper-V.
VMware and Cisco just published a valuable 90-pages paper which provides a detailed comparison between the virtual networking and the physical one (doesn't matter if it's focused on Cisco equipment and capabilities).
If virtual networking confuses you or if you have to configure a particularly complex network layout this guide is absolutely worth checking.
I am looking at a way to automate the P2V and V2V process of VMware converter. A couple of products that I am looking at using are AutoIT and Autohotkey both of which seem to do the job.
I will figure out the best way to create a simple exe that can the be a scheduled task that does the conversion.
I will then use the resulting exe files to perform scheduled backups of existing systems. And for restoration or syncing, mounting of these resulting vmdk files and syncronization with the source.
Just one more week until BriForum 2009!
-
I can't believe that it's here already. BriForum will take place the week
after next, from July 21-23 in Chicago. I was going to write about how this
year ...
Back to the top
