Wednesday, December 24, 2008

Boot charts for 2008.11 and snv101

Bootchart is a great tool for performance analysis and visualization of the Linux boot process. Eric Schrock and Dan Price worked on a Bootchart port using DTrace but I can't find any recipes on their blogs.
I've viewed bootchart source files for two days and by means of several hacks managed to receive boot charts for OpenSolaris. First of all I need to get uptime values in bootchart format (this is /proc/uptime content on linux). I've written the small utility (upt.c):

#include <sys/times.h>
#include <limits.h>
#include <dat/dat_platform_specific.h>

main {
struct tms ts;
clock_t t = times(&ts);

//we need jiffies from uptime value for bootchart
printf("%u\n",(unsigned long) ((DAT_UINT64) t*100 /CLK_TCK));
}

Now the main script for logging (/boot/bootchartd):

#!/usr/bin/bash

PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/gnu/bin:$PATH"

# mounting dir for logging
mount -F tmpfs -o size=32m swap /var/log/bootchart

# start time hack for java parser
echo "001" > /var/log/bootchart/ps.log
echo " PID PPID S %CPU COMMAND" >> /var/log/bootchart/ps.log
echo " 1 0 S 0.0 kernel" >> /var/log/bootchart/ps.log
echo >> /var/log/bootchart/ps.log

while true; do
# uptime jiffies
uptime=`/sbin/upt`
echo $uptime
# Log the command output and removing pid 0 for preventing "No processes found" error
eval "ps -e -o pid,ppid,s,pcpu,comm"|grep -v sched
echo
/usr/gnu/bin/sleep 0.2
done >> /var/log/bootchart/ps.log

Let’s create directory /var/log/bootchart:

mkdir /var/log/bootchart

Next I move /sbin/init to /bin:

mv /sbin/init /bin

and create a new init:

cat /sbin/init

#!/bin/sh

/boot/bootchartd &
exec /bin/init

Now it's possible to reboot. After booting I write a header file:

echo "title = Boot chart for $( hostname | sed q ) ($( date ))" > /var/log/bootchart/header
echo "system.uname = $( uname -srvm | sed q )" >> /var/log/bootchart/header
if [ -f /etc/release ]; then
echo "system.release = $( sed q /etc/release )" >> /var/log/bootchart/header
else
echo "system.release = $( sed 's/\\.//g;q' /etc/issue )" >> /var/log/bootchart/header
fi

and create boot chart:

java -jar bootchart.jar

Parsing /var/log/bootchart
Wrote image: ./bootchart.png

This was the first boot chart created for OpenSolaris 2008.11 (Intel CPU 2.26GHz 512mb RAM). No disk and cpu utilization information yet:


Boot chart for OpenSolaris 2008.11
Here the boot chart for snv101:


Boot chart for snv101

Monday, December 22, 2008

Installing pkg without pkg

Unfortunately I can't install some IPS packages without dependences. Christopher Bergström has written a small shell script for sunstudio installation. Some time I've tested it and have created the ruby version for installation of any package. This script does not process "driver" tag in manifest but in the rest is suitable for installation and a bit more exact than shell script. You can view the source code (getpkg.rb) here.
Sample usage (I want to install SUNWipkg to /opt/ipkg):
# getpkg inst_dir manifest_url 
getpkg.rb /opt/ipkg http://pkg.opensolaris.org/release/manifest/0/SUNWipkg%400.5.11%2C5.11-0.101%3A20081104T032047Z 

Monday, December 15, 2008

Preparing small miniroot with zfs and ssh

This is instructions for building smallest OpenSolaris miniroot (standalone RAM) with ssh and zfs.
I use the receptions familiar to funs of minimization according to Jim Connors's framework. But now we will use IPS for installation of packages and we should delete files of some packages manually ("--install" or "--uninstall --without-deps" not yet implemented in pkg).
Well, I'm running Solaris Express and will use "/opt/test" as working dir. Here are the packages for small miniroot (/opt/pkg.list):
SUNWcsd
SUNWcs
SUNWcakr
SUNWcar
SUNWckr
SUNWcnetr
SUNWcsl
SUNWesu
SUNWgss
SUNWgssc
SUNWkey
SUNWkvm
SUNWlibms
SUNWlibsasl
SUNWloc
SUNWlxml
SUNWnfsc
SUNWnfsckr
SUNWopenssl
SUNWos86r
SUNWpr
SUNWpsdcr
SUNWpsdir
SUNWrmodr
SUNWbip
SUNWsmapi
SUNWssh
SUNWsshcu
SUNWsshd
SUNWtecla
SUNWtls
SUNWtoo
SUNWusb
SUNWusbs
SUNWzfs
SUNWzfskr
SUNWzlib
SUNWzone
SUNWbash
SUNWrtls

As you can see, I have Realteck network adapter (SUNWrtls). Let's go:
# working dir
export ROOTDIR=/opt/test

# create the packaging db (pkg already installed on Nevada)
/usr/bin/pkg image-create -F -a opensolaris.org=http://pkg.opensolaris.org ${ROOTDIR}

cd ${ROOTDIR}
for pkg in `cat ../pkg.list`; do
/usr/bin/pkg install ${pkg}
done

But after that we have some additional packages because of dependences. Now we need to remove these unneeded packages (/opt/rm.txt) :
SUNWaudd
SUNWgccruntime
SUNWinstall-libs
SUNWmd
SUNWpicl
SUNWpool
SUNWwbsup

I cannot delete some packages and will use "pkg contents" command:
for pkg in `cat ../rm.txt`; do
pkg contents -m $pkg|grep file|sed -e 's/.*path=//; s/pkg.csize.*//'|\
awk '{print "./"$1}';
pkg contents -m $pkg|grep link|sed -e 's/.*path=//; s/target.*//'|\
awk '{print "./"$1}';
done > files.tmp

We need "libmeta.so" library for devfsadm:
cat files.tmp|sed 's/.*libmeta.*//g' > files.list

Now I delete packages files:
for file in `cat ../files.list `; do rm -rf $file; done

Basic miniroot configuration:
cd ${ROOTDIR}

# Copy usr/sfw/lib/libwrap.so.1 to usr/lib
cp ${ROOTDIR}/lib/svc/seed/global.db ${ROOTDIR}/etc/svc/repository.db

rm ${ROOTDIR}/usr/lib/libcrypto.so*
cp ${ROOTDIR}/usr/sfw/lib/libcrypto.so.0.9.8 ${ROOTDIR}/usr/lib


# Fix etc/vfstab
echo "/devices/ramdisk:a - / ufs - no nologging" >> ${ROOTDIR}/etc/vfstab

# Set the timezone
echo "TZ=US/Eastern" > ${ROOTDIR}/etc/TIMEZONE

# Set hostname
echo "small" > ${ROOTDIR}/etc/nodename

# Create /etc/inet/hosts and /etc/inet/ipnodes file
echo "127.0.0.1 localhost loghost" > ${ROOTDIR}/etc/inet/hosts
echo "192.168.1.1 small" >> ${ROOTDIR}/etc/inet/hosts
echo "::1 localhost loghost" > ${ROOTDIR}/etc/inet/ipnodes
echo "127.0.0.1 localhost loghost" >> ${ROOTDIR}/etc/inet/ipnodes
echo "192.168.1.1 small" >> ${ROOTDIR}/etc/inet/ipnodes

# Set nsswitch.conf
cp ${ROOTDIR}/etc/nsswitch.files ${ROOTDIR}/etc/nsswitch.conf

# Set IP Address
echo "small" > ${ROOTDIR}/etc/hostname.rtls0

# Set up default route
echo "192.168.1.200" > ${ROOTDIR}/etc/defaultrouter

# Set the environment variables for svccfg.
SVCCFG_DTD=${ROOTDIR}/usr/share/lib/xml/dtd/service_bundle.dtd.1
SVCCFG_REPOSITORY=${ROOTDIR}/etc/svc/repository.db
SVCCFG=/usr/sbin/svccfg

export SVCCFG_DTD SVCCFG_REPOSITORY SVCCFG

${SVCCFG} import ${ROOTDIR}/var/svc/manifest/milestone/sysconfig.xml

# turnoff boot-archive, manifest-import
${SVCCFG} -s system/boot-archive setprop start/exec=:true
${SVCCFG} -s system/manifest-import setprop start/exec=:true
${SVCCFG} -s network/rpc/bind delpg sysidtool
${SVCCFG} -s system/metainit:default setprop general/enabled=false

# Here's what needed to add ssh
${SVCCFG} import ${ROOTDIR}/var/svc/manifest/system/cryptosvc.xml
${SVCCFG} -s system/cryptosvc:default setprop general/enabled=true

${SVCCFG} import ${ROOTDIR}/var/svc/manifest/network/inetd.xml
${SVCCFG} -s network/inetd:default setprop general/enabled=true

${SVCCFG} import ${ROOTDIR}/var/svc/manifest/network/rpc/gss.xml
${SVCCFG} -s network/rpc/gss:default setprop general/enabled=true

${SVCCFG} import ${ROOTDIR}/var/svc/manifest/network/ssh.xml
${SVCCFG} -s network/ssh:default setprop general/enabled=true

# Creating RSA/DSA keys for ssh
/usr/bin/ssh-keygen -q -f ${ROOTDIR}/etc/ssh/ssh_host_rsa_key -t rsa -N ''
/usr/bin/ssh-keygen -q -f ${ROOTDIR}/etc/ssh/ssh_host_dsa_key -t dsa -N ''

# Edit etc/ssh/sshd_config to allow ssh to root account
ex -s ${ROOTDIR}/etc/ssh/sshd_config > /dev/null <<> /dev/null

# Remove packaging, xpg4, sfw - you can remove more but be carefully
rm -rf var/sadm/* var/pkg/*
rm -rf usr/xpg4 usr/sfw

Now I prepare a miniroot image:
# we need freespace ~10mb
size=`/usr/bin/du -ks ${ROOTDIR}|/usr/bin/nawk '{print $1+10240}'`

/usr/sbin/mkfile ${size}k /opt/miniroot
lofidev=`/usr/sbin/lofiadm -a /opt/miniroot`

# get the miniroot raw device and create a filesystem on it
rlofidev=`echo $lofidev |sed s/lofi/rlofi/`
/usr/sbin/newfs -m 0 $rlofidev

# mount ufs filesystem
/usr/sbin/mount -o nologging $lofidev /mnt

/usr/bin/find .  /usr/bin/cpio -pdum /mnt
/usr/sbin/umount /mnt

/usr/sbin/lofiadm -d /opt/miniroot
/usr/bin/7za a -tgzip -mx=9 /opt/miniroot.gz /opt/miniroot

Done. Now I have a small OpenSolaris 2008.11 miniroot with ZFS and SSH (32601K gzipped - I've also removed a number of unnecessary binaries from SUNWcs). It's possible to make an image even less.

Thursday, December 11, 2008

Creating 90mb LiveCD using Distribution Constructor

Continuing a theme, it's possible to create a 90mb iso (minimal working LiveCD without Gnome and X) using the Distribution Constructor - command-line tool for building preconfigured, bootable OpenSolaris images. In the OpenSolaris 2008.11 release, this tool takes an XML manifest file as input, and builds an ISO image that is based on the parameters specified in the manifest file. I'm running Solaris Express and will use old-format configuration file.
I have added additional packages (SUNWgzip, SUNWgrub, SUNWslim-utils) in my short list (see post of December 08). Note: I need only "rtls" driver for this test so you should add packages for your network adapters. Besides, I use checkpoints as it will be necessary to change a script "/lib/svc/method/live-fs-root".
I have downloaded Distribution Constructor kit:
# hg clone ssh://anon@hg.opensolaris.org/hg/caiman/distro_constructor

I only have changed the list of packages and use my own checkpoint in "build_dist.bash":
# ./build_dist.bash small.conf

After installation of packages I have a pause (thanks to checkpoint). Now it's necessary to remove string "/usr/sbin/set_lang" in "/lib/svc/method/live-fs-root" and set "system/coreadm enabled=false" in generic_live.xml (otherwise there will be an errors after booting). It's only first test, you can change more. Then I finish creating the image. In DC output there will be some errors - the majority of them concerns Gnome which is not present.
I boot LiveCD in virtual machine - at first sight it works perfectly .
The test is fulfilled: for my list of packages the final iso has the size 90mb (but you can add any packages which are necessary for you).

Tuesday, December 9, 2008

OpenSolaris: life without Gnome

I have console OpenSolaris installed (see my post of December 08). But I have forgotten to install the most valuable package:
pkg install SUNWzfs-auto-snapshot

Now I can install X:
pkg install SUNWxorg-server
pkg install SUNWxorg-xsvc
pkg install SUNWxorg-agp
pkg install SUNWxorg-graphics-ddx
pkg install SUNWxorg-client-programs
pkg install SUNWxwfsw
pkg install SUNWxwopt
pkg install FSWxorg-fonts

After installation I run "xinit" and get xterm. However it would be better to install some minimalistic window manager, for example, dwm.
pkg install SUNWwget
wget http://code.suckless.org/dl/dwm/dwm-5.3.1.tar.gz

Gcc,binutils,gmake and headers are required:
pkg install SUNWxorg-headers
pkg install SUNWgmake
pkg install SUNWbinutils
pkg install SUNWgcc

I love vim. Also I need gzip:
pkg install SUNWvim
pkg install SUNWgzip


If you plan to compile some other apps, I advice to install substudio:
pkg install sunstudioexpress

Next I unpack archive, correct config.h and compile dwm. It's possible to start dwm xsession from smf. You can look how it's made in BeleniX or MilaX.


As a result I got system with short boot time and an easy and fast window manager.
At the end I need usual user:
mkdir /home
useradd -d /home/me -m -s /bin/ksh -c "System Developer" me
passwd me

Monday, December 8, 2008

Minimal OpenSolaris 2008.11 Install

Using instructions from Edward Pilatowicz' blog I got small working OpenSolaris 2008.11 in the console version: without Xorg (and Gnome of course).
First of all the list of the core packages for normal work in the console is necessary. My computer has Realtek ethernet adapter and I also have added a SUNWrtls package. Here's example list (pkg.txt):
SUNWcsd
SUNWcs
SUNWzone
SUNWcsl
SUNWlibsasl
SUNWlibms
SUNWpr
SUNWlxml
SUNWzlib
SUNWtls
SUNWopenssl
SUNWpool
SUNWzfs
SUNWsmapi
SUNWtecla
SUNWckr
SUNWpicl
SUNWmd
SUNWinstall-libs
SUNWwbsup
SUNWgccruntime
SUNWcar
SUNWcakr
SUNWusbs
SUNWusb
SUNWaudd
SUNWkvm
SUNWos86r
SUNWrmodr
SUNWpsdcr
SUNWpsdir
SUNWcnetr
SUNWesu
SUNWkey
SUNWnfsckr
SUNWnfsc
SUNWgss
SUNWgssc
SUNWbip
SUNWbash
SUNWloc
SUNWsshcu
SUNWsshd
SUNWssh
SUNWtoo
SUNWzfskr
SUNWipf
SUNWrtls
SUNWipkg
SUNWpython-pyopenssl
SUNWpython-cherrypy
SUNWadmr
SUNWadmap
SUNWadmlib-sysid
SUNWPython
SUNWbzip
SUNWxwrtl
SUNWTk
SUNWTcl
SUNWlexpt
SUNWperl584core

As last versions of packages have dependences, for example SUNWcs is installing SUNWgccruntime, SUNWPython is installing packages SUNWTk and SUNWTcl. But finally I got installed system with size about 280 mb - it's a smaller than 3 Gb;)
I use the same commands as in Edward's blog (I'm running Solaris Express):
zpool create -f rpool c1t1d0s0
zfs set compression=on rpool
zfs create -p rpool/ROOT/indy/opt
zfs set canmount=noauto rpool/ROOT/indy
zfs set canmount=noauto rpool/ROOT/indy/opt
zfs set mountpoint=legacy rpool/ROOT/indy
zfs set mountpoint=legacy rpool/ROOT/indy/opt

# we're going to do our install in /a
PKG_IMAGE=/a; export PKG_IMAGE

# mount our zpool on /a
mkdir -p $PKG_IMAGE
mount -F zfs rpool/ROOT/indy $PKG_IMAGE
mkdir -p $PKG_IMAGE/opt
mount -F zfs rpool/ROOT/indy/opt $PKG_IMAGE/opt

For Solaris Express you must install pkg (not required if you're booting from OpenSolaris or MilaX LiveCD):
hg clone ssh://anon@hg.opensolaris.org/hg/pkg/gate
cd gate/src
make install

# create the basic opensolaris install image..
pkg image-create -F -a opensolaris.org=http://pkg.opensolaris.org $PKG_IMAGE
pkg refresh


Note: I use here list of basic packages - pkg.txt (see above):
for pkg in `cat pkg.txt`; do pkg install $pkg; done

Next:
# seed the initial smf repository
cp $PKG_IMAGE/lib/svc/seed/global.db $PKG_IMAGE/etc/svc/repository.db
chmod 0600 $PKG_IMAGE/etc/svc/repository.db
chown root:sys $PKG_IMAGE/etc/svc/repository.db

# setup smf profiles
ln -s ns_files.xml $PKG_IMAGE/var/svc/profile/name_service.xml
ln -s generic_limited_net.xml $PKG_IMAGE/var/svc/profile/generic.xml
ln -s inetd_generic.xml $PKG_IMAGE/var/svc/profile/inetd_services.xml
ln -s platform_none.xml $PKG_IMAGE/var/svc/profile/platform.xml

# mark the new system image as uninstalled
sysidconfig -b $PKG_IMAGE -a /lib/svc/method/sshd
touch $PKG_IMAGE/etc/.UNCONFIGURED

# configure our new /etc/vfstab
printf "rpool/ROOT/indy -\t/\t\tzfs\t-\tno\t-\n" >> $PKG_IMAGE/etc/vfstab
chmod a+r $PKG_IMAGE/etc/vfstab

# turn off root as a role
printf "/^root::::type=role;\ns/^root::::type=role;/root::::/\nw" |\
ed -s $PKG_IMAGE/etc/user_attr

# delete the "jack" user
printf "/^jack:/d\nw" | ed -s $PKG_IMAGE/etc/passwd
chmod u+w $PKG_IMAGE/etc/shadow
printf "/^jack:/d\nw" | ed -s $PKG_IMAGE/etc/shadow
chmod u-w $PKG_IMAGE/etc/shadow

# copy of my existing host sshd keys
cp -p /etc/ssh/*key* $PKG_IMAGE/etc/ssh

# configure /dev in the new image
devfsadm -r $PKG_IMAGE
ln -s ../devices/pseudo/sysmsg@0:msglog $PKG_IMAGE/dev/msglog

# update the boot archive in the new image
bootadm update-archive -R $PKG_IMAGE

# update to the latest version of grub (this command generated
# some errors which i ignored).
$PKG_IMAGE/boot/solaris/bin/update_grub -R $PKG_IMAGE

# create an informational grub menu in the install image that
# points us to the real grub menu.
cat <<-eof> $PKG_IMAGE/boot/grub/menu.lst

#########################################################################
#                                                                       #
# For zfs root, menu.lst has moved to /rpool/boot/grub/menu.lst.        #
#                                                                       #
#########################################################################
EOF

# create the new real grub menu
cat <<-eof> /rpool/boot/grub/menu.lst
default 0
timeout 10
splashimage /boot/grub/splash.xpm.gz

title Solaris 2008.11 console
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/indy
kernel\$ /platform/i86pc/kernel/\$ISADIR/unix -k -B \$ZFS-BOOTFS,console=ttya
module\$ /platform/i86pc/\$ISADIR/boot_archive

EOF

# make the grub menu files readable by everyone.
chmod a+r $PKG_IMAGE/boot/grub/menu.lst
chmod a+r /rpool/boot/grub/menu.lst

# setup /etc/bootsign so that grub can find this zpool
mkdir -p /rpool/etc
echo pool_rpool > /rpool/etc/bootsign

umount $PKG_IMAGE/opt
umount $PKG_IMAGE
reboot


After reboot you get the console version of OpenSolaris with which you can do all that you want. Next you can install only those packages which are necessary for you. For example it's useful to owners of old notebooks. This installation can be made from an OpenSolaris LiveCD in text mode or with help of a MilaX LiveCD (where's ipkg is included).

How to determine PXE mac address when booting illumos via PXELinux/iPXE

In illumos, if you need to determine the interface which was used for booting via PXE then it's possible to use "boot-mac" pro...