The perfect linux audio setup - ALSA with late 2010 mac mini (macmini4,1)

One of the greatest additions to the unibody model mac mini is the addition of an HDMI output (through the nvidia MCP89).
Not only does this allow connecting another HD screen, but it marks the inclusion of HD multi channel audio. Not only audio over HDMI is supported but also the mini-DP output received the welcomed update. And when you thought it can't get any better - each has its own audio device (analog = 0, DP = 7, HDMI = 8). Why is this such an improvement over the optical S/PDIF in the previous models? Several reasons:

1. The obvious - 8ch LPCM > Dolby 5.1
   
2. One less cable to the receiver :)
3. I can now use the dual analog/digital output as an analog output (because I have a multi zone receiver and the second zone can only use analog sources).
   

First off - getting the HDMI audio to work in linux
Kernel support - need to set SND_HDA_CODEC CIRRUS and SND_HDA_CODEC_HDMI (I'm using 2.6.38, it used to be SND_HDA_CODEC_NVHDMI).
This however is not enough on the mac mini because I get write errors when I try to open the hdmi device. Quite a bit of web research has revealed the following changes are needed:

1. Load the module with the options: snd_hda_intel.enable_msi=0 snd_hda_intel.probe_mask=0xffff,0xfff2 snd_hda_intel.model=mbp55

2. Use /etc/asound.conf and /usr/share/alsa/cards/HDA-Intel.conf from this post. This is supposed to fix the switched channels, but I'm not sure it's really required (looks a bit too complex).

This gets you basic but working setup :)

Now for the fun stuff
A few things are missing in the basic setup. First off, the HDMI output volume is fixed and the mixer doesn't affect it. I haven't figured out how to get the alsa mixer settings right. So, we'll use the alsa softvol plugin (with some degradation of quality I guess...). Also, the output needs to be dmix'ed so multiple applications can use it at the same time. And lastly, I want the audio to be distributed to all outputs simultaneously so I can hear my music on my home theater (HDMI), monitor attached headphones (DP) and in my bedroom (analog). This is done using the alsa multi plugin.

Here is my .asoundrc

# dmix the hdmi output
pcm.!hdmi {
 type dmix
 ipc_key 1024
 ipc_key_add_uid false
 ipc_perm 0660
 slave {
     pcm "hw:0,8"
     rate 48000
     channels 8
     period_time 0
     period_size 1024
     buffer_time 0
     buffer_size 4096
 }
}

# dmix the dp output
pcm.dp {
 type dmix
 ipc_key 1025
 ipc_key_add_uid false
 ipc_perm 0660
 slave {
     pcm "hw:0,7"
     rate 48000
     channels 2
     period_time 0
     period_size 1024
     buffer_time 0
     buffer_size 4096
 }
}

# dmix the analog output
pcm.analog {
 type dmix
 ipc_key 1026
 ipc_key_add_uid false
 ipc_perm 0660
 slave {
     pcm "hw:0,0"
     rate 48000
     channels 2
     period_time 0
     period_size 1024
     buffer_time 0
     buffer_size 4096
 }
}

pcm.softvol {
 type softvol
 slave {
     pcm  "all"
 }
 control {
     name "PCM"
     card 0
 }
}

pcm.!default {
 type plug
 slave.pcm "softvol"
}

pcm.all {
 type plug
 slave.pcm "multi"
 ttable.0.0 1.0
 ttable.1.1 1.0
 ttable.2.2 1.0
 ttable.3.3 1.0
 ttable.4.4 1.0
 ttable.5.5 1.0
 ttable.6.6 1.0
 ttable.7.7 1.0
 ttable.0.8 1.0
 ttable.1.9 1.0
 ttable.0.10 1.0
 ttable.1.11 1.0
}

# distribute audio to both analog and hdmi outputs
pcm.multi {
 type multi
 slaves.a.pcm "hdmi"
 slaves.a.channels 8
 slaves.b.pcm "analog"
 slaves.b.channels 2
 slaves.c.pcm "dp"
 slaves.c.channels 2
 bindings.0.slave a
 bindings.0.channel 0
 bindings.1.slave a
 bindings.1.channel 1
 bindings.2.slave a
 bindings.2.channel 2
 bindings.3.slave a
 bindings.3.channel 3
 bindings.4.slave a
 bindings.4.channel 4
 bindings.5.slave a
 bindings.5.channel 5
 bindings.6.slave a
 bindings.6.channel 6
 bindings.7.slave a
 bindings.7.channel 7
 bindings.8.slave b
 bindings.8.channel 0
 bindings.9.slave b
 bindings.9.channel 1
 bindings.10.slave c
 bindings.10.channel 0
 bindings.11.slave c
 bindings.11.channel 1
}

Note that the DP output is set to 2ch 48KHz because it is connected to my monitor and it seems that it is limited in its capabilities (it only supports headphones anyway). This also means all other outputs need to be set to 48KHz otherwise you get errors from the multi-device pcm (it seems that all slaves used by the multi pcm need to have the same parameters). If the DP output is disabled I can use 96KHz or even 192KHz...

Next Steps...
I need to try out pulseaudio again. It might reduce the amount of alsa plugins in use. I also need to figure out how to record audio (for icecast).

Sources:
http://wiki.xbmc.org/?title=HOW-TO_set_up_HDMI_audio_on_nVidia_GeForce_G210,_GT220,_or_GT240
http://forum.xbmc.org/showthread.php?t=74778

Workaround for blank unlock dialog in kde with cairo-dock running

Just execute the following after cairo-dock is running:

xprop -root -remove RGB_DEFAULT_MAP

The following wrapper scripts takes care of it automatically:

#!/bin/sh
/usr/bin/cairo-dock &
sleep 5
xprop -root -remove RGB_DEFAULT_MAP

Sources:
http://bugreports.qt.nokia.com/browse/QTBUG-15092
https://bugs.kde.org/show_bug.cgi?id=262064

Using github and layman for my own gentoo overlay

Since I almost always have a local portage overlay in any gentoo machine I create, I've started to maintain in github.

Now, originally I just pulled the repository to /usr/local/portage.git on each machine and added the dir to PORTDIR_OVERLAY in make.conf.

But since I already have other third party overlays and maintain them using layman, I though why not use that for my own overlay as well?

So all is required is to create a custom repositories xml file and point to it from the layman config.

The repostiories xml (this one is using https since I can't access git from work - the down side is that it ask for password on every layman sync, but git://github.com/asssaf/portage.git or git@github.com:asssaf/portage.git can be used where this is not a problem):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE repositories SYSTEM "/dtd/repositories.dtd">
<repositories xmlns="" version="1.0">
  <repo quality="experimental" status="unofficial">
    <name><![CDATA[mygithub]]></name>
    <description><![CDATA[mygithub]]></description>
    <homepage>https://github.com/asssaf/portage</homepage>
    <owner>
      <email></email>
      <name><![CDATA[Assaf]]></name>
    </owner>
    <source type="git">https://asssaf@github.com/asssaf/portage.git</source>
    <feed></feed>
  </repo>
</repositories>

And in /etc/layman/layman.cfg, add the line in bold:

overlays  : http://www.gentoo.org/proj/en/overlays/repositories.xml
            file:///etc/layman/localrepositories.xml

The overlay will be available for addition using 'layman -a mygithub' and pulled into (by default) /var/lib/layman/mygithub.

Bluetooth network connection to the Palm Pre from a linux machine

The Palm Pre in its current version has a very limited bluetooth stack. It only allows outgoing connections to audio devices. No serial port, PAN or even sending contacts to another phone.

It does however support incoming PAN connections.
This is how I do it.

1. First, pair the phone with your machine using your favorite gui (bluedevil, blueman) or command line utility.

2. Set up IP configuration for the bnep0 interface
In gentoo this is done in the /etc/conf.d/net file. I just needed to add the following line:

config_bnep0="10.1.1.61 netmask 255.255.255.0 broadcast 10.1.1.255"

3. Firewall - set it up to allow traffic. Also need to set up masquerading/proxyarp for returning traffic.

4. Start a connection:

pand --connect AA:BB:CC:DD:EE:FF --role PANU --nodetach --master

Where AA:BB:CC:DD:EE:FF is the mac of the palm pre bluetooth interface

This will bring up the bnep0 interface and allow connecting to the pre via ssh, etc.

Known issues:
1. Most WebOS programs will not be aware of this connection (you won't be able to use the browser). But most terminal operations will be.
1. DHCP, DNS: The DHCP configuration is geared towards a mobile wifi hotspot and is kind of screwed up (or at least I don't understand it) for this purpose.
2. Proxy ARP / IP bridging mode instead of NAT (TODO)

Tomcat and Logging

Tomcat is a very popular java application server. It is very light weight and easy to use. It's derivative, tcserver, is also very nice. It has all the goodness of tomcat, and on top of that it gives you the feel of a real J2EE app server (tomcat is only java servlet compliant).

Logging from your webapp in tomcat is a piece of cake. For example, if you like to use SLF4J logging API together with logback as the backend, you just drop the jars in the webapp's WEB-INF/lib and the configuration (logback.groovy) in WEB-INF/classes and you're good to go.

You will notice, however, that something else is eating away your disk space beside your webapp's logs. Tomcat has its own logging. It writes several files, has its own logging configuration (and logging technology) and even does its own rotation for (only) some of the files. A log rotation scheme that can't be easily disabled and might not fit your global rotation scheme.

By default tomcat uses juli, which is based on the standard java.util.logging with some enhancements for supporting per-webapp configuration. This only matters to those that don't have their own logging set up inside their webapp. Also, the default configuration is quite surprising. It sets up several files for different categories that may or may not interest you.
There's:

manager.log and host-manager.log for the respective webapps (which you may not want to have deployed, but the empty log files will still be created).
localhost.log will have message about servlets coming up and down.
catalina.log will contain mostly internal tomcat messages.
Some logging messages will be sent to stdout and get mixed with actual direct writes to stdout/stderr as well as with the output from scripts that start and stop tomcat. If you redirect this to a file (catalina.out) it is not rotated and can grow very big over time. Some messages are send to both catalina.log and catalina out, which is confusing if not just redundant.

All the files created (this doesn't include the console output) use the juli file handler that by default adds a date stamp to the file name (for rolling). It is not configurable! You simply can't remove it (you can just change the prefix and suffix but there will always be a date in the middle).

These are all the options I can think about (after scouring the net for solutions):
* Replace tomcat logging with log4j or logback (requires dropping the jars in the bin dir and changing the startup script to include them in the classpath)
* Replace the juli FileHandler with the standard java.util.logging FileHandler and have only a single handler for everything, since we don't need the per-webapp logging (we set that up within the webapp)
* Use SocketHandler to write to syslog instead of directly into a file
* Use MemoryHandle since most of these message can be ignored until we actually have a problem
* Use nop logging configuration, but again we lose troubleshooting information

In any case setting the sallowOutput option to true in the context configuration file will catch some messages written to stdout/stderr and redirect them to the webapp logger. It doesn't catch all messages though (I think ones that are written early).

For rotation I want to use logrotate so it is possible to:
* Use SocketHandler to write to syslog, and syslog can be told to reopen its log files
* Enhance the logging framework to listen to a signal to reopen its files
* Use cronolog (but again we lose control of the file naming since it adds a date suffix - I want to use numbers)
* Use the (nonatomic) copytruncate option in logrotate but risk losing some messages

In the end I decided to do the following:
* Comment out everything it tomcat's logging.properties except a single catch-all FileHandler, using the default java.util.logging FileHandler instead of juli's. This includes disabling the ConsoleHandler (I don't need it since I'm running tomcat in a server and never interactively)
* Rotate the created files (the single log and the stdout redirected to a file) using logrotate with the copytruncate flag (this is not safe for the stdout file, but I don't expect it to grow much now that it doesn't include any of tomcat's own logging)
* Enabled the swallowOutput flag
* Use syslog for all the logback logs and rotate with logrotate them without copytruncate

Restoring x server zap functionality

The xorg x server on gentoo has the ctrl-alt-backspace disabled by default. I really like that functionality (especially when seeing that there is a lot of memory in use when nothing is running, after a very long uptime).

Apparently you should use kernel SAK which is superior (alt-break-K if you have magic sysrq configured in the kernel). For some reason it is not recommended to use this to SAK, so a new key sequence should be defined for it. A good place to do it in gentoo is in /etc/conf.d/local.start, and since ctrl-alt-backspace is available, it can be used for this purpose (key code 14):

# Set ctrl-alt-backspace to SAK
echo "control alt keycode 14 = SAK" | /usr/bin/loadkeys

Optionally restore the key map in /etc/conf.d/local.stop:

# restore keymap to default
/usr/bin/loadkeys --default

It is also a good place to set a replacement key sequence for reboot, if ctrl-alt-del is disabled in /etc/inittab. For example ctrl-alt-break (key code 119):

echo "control alt keycode 119 = Boot" | /usr/bin/loadkeys

Gradle ebuild

Latest ebuild can be found in github: https://github.com/asssaf/portage/tree/master/dev-java/gradle-bin

Pretty ugly (uses the bundled jars instead of installed ones), but it works.

# dev-java/gradle-bin-0.9_rc1.ebuild

# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI="2"

inherit java-pkg-2

MY_PN=${PN%%-bin}
#MY_PV=${PV/_pre/-preview-}
MY_PV=${PV/_rc/-rc-}
MY_P="${MY_PN}-${MY_PV}"

DESCRIPTION="Build Tool for Java"
SRC_URI="http://dist.codehaus.org/${MY_PN}/${MY_P}-bin.zip"
HOMEPAGE="http://www.gradle.org/"
LICENSE="Apache-2.0"
SLOT="0"
KEYWORDS="~x86"

RDEPEND=">=virtual/jdk-1.5"

IUSE=""

S="${WORKDIR}/${MY_P}"


src_unpack() {
unpack ${A}
}


# TODO we should use jars from packages, instead of what is bundled
src_install() {
#TODO set GRADLE_HOME
#local jar JARS="gradle-core gradle-open-api gradle-ui gradle-wrapper"

local gradle_home="${ROOT}/usr/share/${PN}"

insinto "${gradle_home}"
#doins gradle-imports || die "Failed to install files"

cd lib
for jar in *.jar; do
 java-pkg_newjar ${jar} ${jar}
done

insinto "${gradle_home}/lib/plugins"
doins plugins/*

#sed -i "s/-${PV}.jar/.jar/" bin/gradle || die "Failed to patch launcher script"
#dobin bin/gradle || die "Failed to copy launcher script"
java-pkg_dolauncher "gradle" --main org.gradle.launcher.GradleMain --java_args "-Dgradle.home=${gradle_home}"
}

Update (2-Aug-10): I'm getting a NoClassDefError when trying to run JUnit tests. I've switched to the stable 0.8, which seems to work fine (just renamed the ebuild).

Update (27-Aug-10): I found a workaround for this issue. Adding the following lines to the build.gradle file fixes it:

test.bootstrapClasspath('/opt/sun-jdk-1.6.0.20/jre/lib/rt.jar')
test.bootstrapClasspath('/usr/share/gradle-bin/lib/commons-lang-2.5.jar')

Also updated the ebuild to 0.9_rc1.

Update (3-Nov-10): I've added additional enhancements to the ebuild but decided that keeping the blog up to date is the wrong way to go about it so I'm doing all ebuild maintenance in github from now on. My portage repository is here (and this ebuild is under dev-java/gradle-bin).

My /boot kernel management (2)

In the last post I described my /boot kernel scheme and eselect-based management tool. Another piece of that is the script to actually install a new kernel to that fancy structure.

Here it is:

#!/bin/sh

#
# "make install" script for i386 architecture
#
# Arguments:
#   $1 - kernel version
#   $2 - kernel image file
#   $3 - kernel map file
#   $4 - default install path (blank if root directory)
#

KERNEL_VERSION="$1"
KERNEL_IMAGE="$2"
KERNEL_MAP="$3"
BASE_INSTALL_PATH="$4"
KERNEL_BASE="$BASE_INSTALL_PATH/installkernel"

die () {
 ERR="$1"
 shift

 if [ -z "$1" ]
 then
  echo Aborting 1>&2
 else
  echo $* 1>&2
 fi

 exit $ERR
}


verify () {
 if [ ! -f "$1" ]; then
  echo ""                                                   1>&2
  echo " *** Missing file: $1"                              1>&2
  echo ' *** You need to run "make" before "make install".' 1>&2
  echo ""                                                   1>&2
  die 1
 fi
}

# Make sure the files actually exist
verify "$KERNEL_IMAGE"
verify "$KERNEL_MAP"

# check if the version already exists in /boot and find the appropriate p-level
KERNEL_BASE_DIR="$KERNEL_BASE/$KERNEL_VERSION"
KERNEL_PLEVEL="0"

while [ -e "$KERNEL_BASE_DIR/p$KERNEL_PLEVEL" ]
do
 KERNEL_PLEVEL="$(expr $KERNEL_PLEVEL + 1)"
done

INSTALL_PATH="$KERNEL_BASE_DIR/p$KERNEL_PLEVEL"

# backup old modules if needed
#TODO warn before overwriting existing backup?
#TODO check if the previous plevel is identical?

if [ $KERNEL_PLEVEL != 0 ]
then
 OLD_PLEVEL="$(expr $KERNEL_PLEVEL - 1)"
 MODULES_BACKUP_FILE="/lib/modules/$KERNEL_VERSION-p$OLD_PLEVEL.tgz"
 echo "Creating modules backup for previous kernel: $MODULES_BACKUP_FILE"
 tar czf "$MODULES_BACKUP_FILE" -C /lib/modules $KERNEL_VERSION || die 4
fi


# make the new kernel dir
echo "Creating new kernel dir: $INSTALL_PATH"
mkdir -p $INSTALL_PATH || die 2

# copy the file to the new kernel dir
echo "Copying kernel files"
cp "$KERNEL_IMAGE" $INSTALL_PATH/ || die 3
cp "$KERNEL_MAP" $INSTALL_PATH/ || die 3
cp .config $INSTALL_PATH/ || die 3

#TODO run eselect and make modules_install?
echo ""
echo "Done. You should probably do the following now:"
echo "1. eselect the new kernel"
echo "2. run make modules_install"
echo "3. run modules-rebuild"
echo "4. update the ChangeLog"
echo ""

In order to hook this to the standard make install command run at kernel build, this needs to be placed in /sbin/installkernel.
Alas, that position is already taken by a file belonging to the debianutils package. Arrgghhh!!
I opened a bug (and provided a patch) to make this optional.

For now the ebuild for installing the script (myinstallkernel-0.1.ebuild) must block debianutils:

EAPI="2"

LICENSE="GPL"
SLOT="0"
KEYWORDS="x86"
IUSE="+symlink"

DEPEND="symlink? ( sys-apps/debianutils[-installkernel] )"

src_install() {
 newsbin ${FILESDIR}/${P} ${PN}

 if use symlink; then
  dosym /usr/sbin/${PN} /sbin/installkernel
 fi

 insinto /usr/share/eselect/modules
 newins ${FILESDIR}/mykernel.eselect-${PV} mykernel.eselect
}

Update 03-Jun-2010:
1. Updated myinstallkernel script to version 0.3.
2. The bug wasn't accepted by gentoo, but they suggested using CONFIG_PROTECT="/sbin/installkernel" in /etc/make.conf. This kinda works, so I've removed the block from the ebuild.

My /boot kernel management

Some linux distribution simply put the kernel image at /boot/vmlinuz and maybe a backup version as /boot/vmlinuz.old. Same goes for a System.map file.

I prefer to store multiple kernel versions in a hierarchical structure with symlink pointing to the current kernel (which are in turn referenced by grub's menu).

For example, my current kernel is minigen32 (mini since it's my mac mini kernel, gen for gentoo and 32... you can guess :) ).

Under /boot/kernels I have a subdirectory for minigen32 and under that a subdir for each kernel version. I also put another subdir level for local build number, since I might have several builds of the same kernel version.

So I have:
/boot/kernels/minigen32/2.6.32-gentoo-r7/p0
/boot/kernels/minigen32/2.6.32-gentoo-r7/p1

etc., etc.

Each such directory contains the same files. The good 'ol bzImage, System.map and .config.

Then I have symlinks for stable, testing, current, previous etc. pointing to these subdirs, and a simple grub file can be written (kernel /testing/bzImage). When I want to switch kernels I just need to update the symlinks. No need to touch the grub configuration (unless of course I need to change the kernel command line).

Using gentoo's handy eselect utility i've built a module to handle this scheme for me.

$ eselect mykernel list
Available kernel symlink targets:
  [1]   /boot/kernels/minigen32/2.6.32-gentoo-r7/p0  stable
  [2]   /boot/kernels/minigen32/2.6.32-gentoo-r7/p1  testing
  [3]   /boot/kernels/minigen32/2.6.32-gentoo-r7/p2  

$ eselect mykernel set testing 3
$ eselect mykernel set testing 2

$ eselect mykernel list
Available kernel symlink targets:
  [1]   /boot/kernels/minigen32/2.6.32-gentoo-r7/p0  
  [2]   /boot/kernels/minigen32/2.6.32-gentoo-r7/p1  stable
  [3]   /boot/kernels/minigen32/2.6.32-gentoo-r7/p2  testing

Code for the module (just drop it in ~/.eselect/modules/mykernel.eselect):

DESCRIPTION="Manage the /boot kernel symlinks"
VERSION="0.3"

local BASE_DIR="/boot"
local KERNELS_BASE="$BASE_DIR/kernels"
local TAGS=( stable testing )

# find a list of kernel symlink targets
find_targets() {
 local f
 for f in $KERNELS_BASE/*/*/p* ; do
  #[[ -d ${f} ]] && basename "${f}"
  #[[ -d ${f} ]] && echo "${f#${BASE_DIR}/}"
  [[ -d ${f} ]] && echo "${f}"
 done 
}


### show action ###

describe_show() {
 echo "Show the current tagged kernels"
}

do_show() {
 local TAGS_TO_SHOW=( "${TAGS[@]}" )
 has $1 ${TAGS[@]} && TAGS_TO_SHOW=( $1 )
 
 for TAG in ${TAGS_TO_SHOW[@]}
 do
  my_show $TAG
 done
}

my_show() {
 local TAG=$1
      
 write_list_start "Current ${TAG} kernel"
       if [[ -L $BASE_DIR/${TAG} ]] 
 then 
  write_kv_list_entry $BASE_DIR/$(readlink "${BASE_DIR}/${TAG}") ""
 else
  write_kv_list_entry "(unset)" ""
 fi
}

### list action ###

describe_list() {
 echo "List Available Kernels in $BASE_DIR"
}

do_list() {
 local i j SYMLINKS TAG targets=( $(find_targets) )
 write_list_start "Available kernel symlink targets:"

 for (( j = 0; j < ${#TAGS[@]}; j++ ))
 do
  [[ -L $BASE_DIR/${TAGS[$j]} ]] && SYMLINKS[$j]=$BASE_DIR/$(readlink "${BASE_DIR}/${TAGS[$j]}")
 done

 for (( i = 0; i < ${#targets[@]}; i++ ))
 do
  local mark=""

  for (( j = 0; j < ${#TAGS[@]}; j++ ))
  do
   if [[ ${targets[${i}]} == ${SYMLINKS[$j]} ]]
   then
    mark="${mark} $(highlight ${TAGS[$j]})"
   fi 
  done  

  targets[${i}]="${targets[${i}]} ${mark}"
 done

 write_numbered_list -m "(none found)" "${targets[@]}"
}

### set action ###

describe_set() {
 echo "Tag a kernel"
}

do_set() {
 local usage="Usage [${TAGS[@]}] [kernel]" tag=$1 target=$2 symlink
 [[ ${#} != 2 ]] && die -q ${usage}
 has $tag ${TAGS[@]} || die -q ${usage}
 
 symlink=$BASE_DIR/$tag
 if [[ -L "${symlink}" ]] ; then
  set_symlink "${target}" "${symlink}"  || die -q "Couldn't set a new symlink"

 elif [[ -e "${symlink}" ]] ; then
  die -q "Target file already exists and is not a symlink: ${symlink}"

 else
  set_symlink "${target}" "${symlink}" || die -q "Couldn't set a new symlink"
 fi 
}


set_symlink() {
 local target=${1} symlink=${2}
 if is_number "${target}" ; then
  targets=( $(find_targets) )
  target=${targets[$(( ${target} - 1 ))]}
 fi
 if [[ -z ${target} ]] ; then
  die -q "Target \"${1}\" doesn't appear to be valid!"
 elif [[ -d "${target}" ]] ; then
  local sym_dir=$(dirname ${symlink})
  if [[ ! -d ${sym_dir} ]]; then
   mkdir -p ${sym_dir} || die -q "Could not create ${my_dir}"
  fi
  ln -snf "${target#${BASE_DIR}/}" "${symlink}"
 else
  die -q "Target \"${1}\" doesn't appear to be valid!"
 fi
}

Update 3-Jun-2010: Updated eselect module to version 0.3

Revised highlighting script to use sed = full regex support

I had a TODO in the header of my bash highlighting script (hl) for a long time to add support for case insensitive highlighting and regex support, for a very long time.

Today I finally fixed it by changing the script to use sed.

Example usage:

$ glxinfo | hl GL_ARB_multitexture\|GL_EXT_framebuffer_object
        \|GL_ARB_shader_objects\|GL_ARB_shading_language_100
        \|GL_ARB_fragment_shader

The script:

#!/bin/sh
# Highlight specified pattern

RED=`echo -en '\e[31m'`
GREEN=`echo -en '\e[32m'`
YELLOW=`echo -en '\e[93m'`
NORMAL=`echo -en '\e[00m'`

usage () {
        echo "Usage: `basename $0` [red | green | yellow] str"
        exit 1
}


[ "$1" ] || usage

if [ -n "$2" ]
then
        case $1 in
        red) COLOR="$RED";;
        green) COLOR="$GREEN";;
        yellow) COLOR="$YELLOW";;
        *) usage;;
        esac

        shift
else
        COLOR="$RED"
fi

STR="$1"

sed -E "s/($STR)/$COLOR\1$NORMAL/g"

SSL with multiple hostnames

I noticed today that my SNI setup in apache is not working reliably. Apparently it takes the localhost certificate no matter how I am connecting. This seems to related to the content of /etc/hosts on the server, which I don't want to mess around with just to get SNI working.

Fortunately both IE and FF support an extension allowing multiple hostnames to be specified in the certificate in addition to the CN (reportedly Java does not support this extension, will need to check). The extension (Subject Alternate Names) is described in RFC 5280.

I've done the following to generate such certificates with openssl:

First, we need to add the extension support in openssl.cnf:

(not sure this is required, supposedly there's a security risk) in the [CA_default] section add:

copy_extensions = copy

in the [req] section add:

req_extensions = v3_req

in the [v3_req] section:

subjectAltName = IP:1.2.3.4,IP:127.0.0.1,DNS:myhost.acme.com,DNS:myhost,DNS:localhost.localdomain,DNS:localhost

Now we can generate a request, careful to include the config file so the extensions are included:

openssl req -config /etc/ssl/openssl.cnf -new -out multi.csr -key server.key

And sign it, again careful to include the config file

openssl x509 -extfile /etc/ssl/openssl.cnf -extensions v3_req -req -days 365 -in /tmp/multi.csr -CA ca.crt -CAkey ca.key -set_serial x -out multi.crt

And that's about it!

Archive portage logs

Just dump the following script in /etc/cron.daily to compress old portage logs and delete ancient ones, without using logrotate or tmpwatch/tmpreaper. After a year of running lappy, my /var/log/portage dir got to almost 1GB.

#!/bin/sh

## remove really old logs (180 days)
find /var/log/portage -name '*.log*' -mtime +180 -print0 | xargs --null --no-run-if-empty rm

## compress big logs (over 10 KB)
find /var/log/portage -name '*.log' -mtime +3 -size +10k -print0 | xargs --null --no-run-if-empty gzip

Magic SysRq Ehancement (3)

It would be nice to take the sysrq-visual and add it as a kernel patch. I thought it would be nice to see the leds blinking while the emergency sync is running, because many times you can't know for sure when it's done.

Looking at the code, there is a call emergency_sync(), which is performed asynchronously. It prints a debug message to when it's done, but there doesn't seem to be a nice and clean way to be notified of the completion. I need to figure out which is better, calling a different sync method (synchronous), or adding a notification mechanism to the sync operation (patching the kernel code in a pretty critical area.)

Oracle Listener Configuration Revisited

Configuring the oracle 10.2 listener correctly might not be as easy as it seems.
First off, Oracle has the ability to automagically configure everything without any configuration file. The defaults should usually suffice, but if you want to make some changes it can get a bit confusing.

When the oracle service starts (and more importantly, the listener) it says to where it is logging and where it is taking the listener parameters from (for example $ORACLE_HOME/network/admin/listener.ora). Now, if for any reason Oracle can't find or read this file it will happily continue with it's automatic configuration but will do so silently, while still printing the configuration file name!

This is where it stumped me. I made some changes to that file (changed tracing level) but it didn't make any effect. Later I found the reason was that the file was owned by root, while the listener was running as oracle and had no read permission to the file. It took a while to figure that out.

Anyway, back to the problems I was trying to solve. Namely, the occasional hangs at oracle startup/shutdown. I found out that oracle (again, actually the listener) always tries to resolve the machine hostname (probably from gethostname(), dunno) and connect to the result (in netstat I saw connections to ports 199 and 1521). If there is a bogus resolution (put something unreachable in /etc/hosts), the listener process will hang for the machine's defined connect timeout (usually 2 or 5 minutes).
It seems this happens regardless of what you put down in the (HOST = x.x.x) in the listener.ora file. I tried putting down 0.0.0.0, localhost and 127.0.0.1, but in all cases it still tried resolving the hostname.

The lesson here, boys and girls, is that you must make sure that your hostname always resolves to something reachable. E.g. in /etc/hosts you should have:

127.0.0.1 myhost

or

0.0.0.0 myhost

Fair enough. But there are a few additional things that can be done. First, the old oracle init script (very old, probably taken from RedHat 9) calls lsnrctl before calling dbstart and dbshut.
I noticed that these scripts already handle the lsnrctl, so I removed the additional calls from the init script. The hang is gone! Actually the hang is still happening, but since the oracle scripts start lsnrctl in the background, it doesn't block the oracle init script anymore.

Also a warning message in the listener.log after the timeout expires drew my suspicion to the ONS ("subscription to the node down event is still pending"). I found a couple of solutions online, one is to add SUBSCRIBE_FOR_NODE_DOWN_EVENT_{listener_name} = OFF. That took care of the warning message and some of the connections (the one to 1521 is still there) but it still hangs.

Finally I also commented out the cfgHostname call in the init script since all that crap is not really needed. It's easier to set the HOST in listener.ora to something constant such as 0.0.0.0 or localhost than changing the file and moving directories around everytime the hostname/ip changes.

Magic SysRq enhancement (2)

I've rewrote this module using workqueue instead of timers and it works perfectly now on blackbird (to be tested on lappy soon).

An additional feature is that I now flip the leds twice so the original state is restored and also the blink is not missed if all leds were originally on.

Code is here

Magic SysRq enhancement

I really love the Magic SysRq key. It's always nice to know that you can do some emergency actions even when the system is 99% locked up. What's my favorite sequence? Probably 'subo'. I'm still figuring it out.

One of my beefs with MSRK is that sometimes I can't be sure if it is responding (am I pressing the right sequence?). This usually happens when I get a crash while in X, not seeing the console.
I wrote a small kernel module to add a new action to the Magic SysRq key. The new action's, "Visualize", only purpose is to give back a cue that it's listening.

I copied from some kernel hacking example code for blinking the keyboard leds (should be possible if the kernel still reads commands from the keyboard) and for beeping (although I haven't got that working yet).

First try is using timers ('add_timer'). This seems to be working nicely on lappy but inconsistently on blackbird (sometimes the leds don't light up and sometime they light up but don't go off). I'm pretty sure that the timer function gets called because I can see the printk's. I guess this means that the IOCTL is failing. I'll try replacing the timer with work queue, see if that helps.

P.S.: Another difference between lappy and blackbird: On blackbird the 'v' from the Alt-SysRq-V sequence gets swallowed, whily on lappy it gets printed. Strange.