/var/run/mysqld is removed after each reboot?

After recent upgrade on my Gentoo box I've accidently got /var/run/mysqld directory removed. The socket path pointed there, and, of course, mysqld denied to launch. The same thing happened some time ago on Ubuntu and openSUSE. I've no clue what makes it(mysqld? rc script?) remove the directory. But I've got at least two simple fixes for that and wanna share it in this post.

Fix #1

Just change path to the socket in /etc/mysql/my.cnf and get rid of these upgrade issues:

socket = /tmp/mysqld.sock

I have it in client and mysqld sections.

Fix #2

Change the runlevel script. In Gentoo it looks like the following:

checkconfig() {
    if [ ! -d /var/run/mysqld ] ; then
        checkpath -d -m 755 -o mysql:mysql /var/run/mysqld
    fi
    if [ -z "${MYSQL_PERMS_OK}" ] && [ "$(stat -c %a /var/run/mysqld)" != "755" ] ; then
        ewarn "mysqld run dir is not world readable, you should reset the perms:"
        ewarn "chmod 755 /var/run/mysqld"
        ewarn "To disable this warning, set 'MYSQL_PERMS_OK' in /etc/conf.d/mysql"
    fi
}
start() {
    checkconfig
    # ...
}

You should have got it. checkconfig is called on at the very beginning of /etc/init.d/mysql start handler.

That's it. I hope this saves someone's time.

Installing ImageMagick with DejaVu support

I've had troubles with DejaVu fonts in ImageMagick. Almost everything with -font 'DejaVu-Sans-Condensed' failed. Well, it was searching for the font, consuming much of CPU resources, and used default one afterwards. In this post I'll show how to compile ImageMagick from source with DejaVu support. Althouth it may seem trivial, the process may sometimes be tricky.

First off, install FreeType dev package. It should be called libfreetype*-dev or so. For instance, in Debian:

$ sudo apt-get install libfreetype6-dev

Make sure you have DejaVu font installed. Package name should be named somewhat like 'ttf-dejavu'. In Debian you can install it as follows:

$ sudo apt-get install ttf-dejavu-core ttf-dejavu ttf-dejavu-extra

Download the source

Download tarball from http://www.imagemagick.org/script/download.php and unpack it.

Find font directories

Figure out the main directory with fonts and a directory with DejaVu fonts, and specify them in configuration:

$ ./configure --with-freetype=yes --with-djvu=yes \
--with-fontpath=/usr/share/fonts/truetype \
--with-dejavu-font-dir=/usr/share/fonts/truetype/ttf-dejavu 

Compile and install:

$ make
$ sudo make install 
# or 
# sudo make checkinstall
# whatever preferred/appropriate.

ldconfig

I had to make the follownig for my /usr/local prefix:

$ sudo ldconfig

Likely, you should do it also.

Verify

Verify that ImageMagick recognizes DejaVu:

$ identify -list font | grep -i dejavu

This should output a list with installed DejaVu fonts.

I hope this will be helpful for someone.

eio - a new PHP extension for asynchronous POSIX I/O

Eio is a new PECL extension for asynchronous POSIX I/O.

Eio provides interface for the libeio library. It means that each POSIX call runs in a separate thread. It is important to aware that the order of ungrouped requests is not precisely known. However, libeio, as well, as eio, supports complex call creation to accumulate a set of regular eio_* requests in a single group request. This allows to create a group request ex. to open, read and close a file.

Currently Windows platforms are not supported.

Any extension-specific suggestions are welcome :)

Creating new Vim filetype for server logs

In this post I'll show how to create a simple Vim syntax script to highlight source of common error and access logs.

First make sure you have the following folders and files:

~/.vimrc		- the main config
~/.vim/ftdetect		- directory with filetype scripts
~/.vim/syntax		- directory with specific syntax scripts

If something missing, you can copy it from $VIMRUNTIME directory. Find block starting with

if has("autocmd")

and add the following line there:

au BufRead,BufNewFile *.log set filetype=error_log

Create filetype script in ~/.vim/ftdetect/error_log.vim:

if did_filetype()	" filetype already set..
	finish		" ..don't do these checks
endif


au BufRead,BufNewFile *.error.log set filetype=error_log
au BufRead,BufNewFile *.access.log set filetype=error_log

Finally you need syntax script itself, ~/.vim/syntax/error_log.vim:

" Vim syntax file
" Language:    error_log
" Maintainer:  Ruslan Osmanov <rrosmanov at gmail dot com>
" Last Change: Thu Sep  1 15:30:52 MSD 2011
" Version:     1.0

if version < 600
  syntax clear
elseif exists("b:current_syntax")
  finish
endif

" Always ignore case
syn case ignore

" General keywords which don't fall into other categories
syn keyword error_logKeyword 			error warning notice 

" Special values
syn keyword error_logSpecial         	referer client

" Strings (single- and double-quote)
syn region error_logString           start=+"+  skip=+\\\\\|\\"+  end=+"+
syn region error_logString           start=+'+  skip=+\\\\\|\\'+  end=+'+

" Numbers and hexidecimal values
syn match error_logNumber            "-\=\<[0-9]*\>"
syn match error_logNumber            "-\=\<[0-9]*\.[0-9]*\>"
syn match error_logNumber            "-\=\<[0-9][0-9]*e[+-]\=[0-9]*\>"
syn match error_logNumber            "-\=\<[0-9]*\.[0-9]*e[+-]\=[0-9]*\>"
syn match error_logNumber            "\<0x[abcdefABCDEF0-9]*\>"

" IP addresses
syn match error_logIP            	"-\=\<[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\>"

syn region error_logOperator         start="referrer\:" end="" contains=ALL

" URI
syn match error_logURI            	"http\:\/\/[^ ]*"

" Date 
syn match error_logDate 			"\[[0-9]*\/[a-zA-Z]\{3\}\/\d\{4\}:\d\{2\}:\d\{2\}:\d\{2\} +\d\{4\}\]"
syn match error_logDate				"\[[a-zA-Z]\{3\} [a-zA-Z]\{3\} \d\{2\} \d\{2\}:\d\{2\}:\d\{2\} \d\{4\}\]"

" Domain
syn match error_logDomain            "\(www\.\)\?[a-z0-9\-\_]*\.\(ru\|com\|by\|uz\|ua\|kz\|su\|net\|org\)"

" Define the default highlighting.
" For version 5.7 and earlier: only when not done already
" For version 5.8 and later: only when an item doesn't have highlighting yet
if version >= 508 || !exists("did_error_log_syn_inits")
	if version < 508
		let did_error_log_syn_inits = 1
		command -nargs=+ HiLink hi link <args>
	else
		command -nargs=+ HiLink hi def link <args>
	endif

	HiLink error_logKeyword         Statement
	HiLink error_logIP				Type
	HiLink error_logSpecial  	    Special
	HiLink error_logString          String
	HiLink error_logNumber          Number
	HiLink error_logDate			Comment 
	HiLink error_logURI				Underlined 
	HiLink error_logDomain			Underlined 

	delcommand HiLink
endif

let b:current_syntax = "error_log"
" vim:sw=4:

Colorized svn status

I post some shell scripts here to output svn status command output colorized.

Bash Script

#!/bin/bash - 

set -o nounset                              # Treat unset variables as an error

txtblk='\e[0;30m' # Black - Regular
txtred='\e[0;31m' # Red
txtgrn='\e[0;32m' # Green
txtylw='\e[0;33m' # Yellow
txtblu='\e[0;34m' # Blue
txtpur='\e[0;35m' # Purple
txtcyn='\e[0;36m' # Cyan
txtwht='\e[0;37m' # White
bldblk='\e[1;30m' # Black - Bold
bldred='\e[1;31m' # Red
bldgrn='\e[1;32m' # Green
bldylw='\e[1;33m' # Yellow
bldblu='\e[1;34m' # Blue
bldpur='\e[1;35m' # Purple
bldcyn='\e[1;36m' # Cyan
bldwht='\e[1;37m' # White
unkblk='\e[4;30m' # Black - Underline
undred='\e[4;31m' # Red
undgrn='\e[4;32m' # Green
undylw='\e[4;33m' # Yellow
undblu='\e[4;34m' # Blue
undpur='\e[4;35m' # Purple
undcyn='\e[4;36m' # Cyan
undwht='\e[4;37m' # White
bakblk='\e[40m'   # Black - Background
bakred='\e[41m'   # Red
badgrn='\e[42m'   # Green
bakylw='\e[43m'   # Yellow
bakblu='\e[44m'   # Blue
bakpur='\e[45m'   # Purple
bakcyn='\e[46m'   # Cyan
bakwht='\e[47m'   # White
txtrst='\e[0m'    # Text Reset

RESULT="`svn st`"
echo "$RESULT" | while read LINE
do
 case "${LINE:0:1}" in
  'M')
   echo -e "$txtylw$LINE$txtrst"
   ;;
  'X')
   echo -e "$bakpur$LINE$txtrst"
   ;;
  '?')
   echo -e "$txtgrn$LINE$txtrst"
   ;;
  'D')
   echo -e "$bakylw$LINE$txtrst"
   ;;
  'I')
   echo -e "$txtblu$LINE$txtrst"
   ;;
  *)
   echo -e "$LINE";
   ;;
 esac
done

AWK Script

#!/usr/bin/awk -f
BEGIN {
  cmd = "svn st";
  while (cmd | getline) {
    char = substr($1, 0, 1);
    if (char == "M") {
      print "\033[36m(", $1, ")    ", $2, $3, "\033[0m"; 
    } else if (char == "X") {  
    print "\033[35m(", $1, ")    ", $2, "\033[0m"; 
    } else if (char  == "A") {  
      print "\033[32m(", $1, ")    ",  $2, $3, "\033[0m"; 
    } else if (char == "?") { 
      print "\033[1;31m(", $1, ")    ", $2, $3, "\033[0m"; 
    } else if (length($1) == 1 ) {
      print $1,"      " $2; 
    } else if ($1 == "---") { # changelists etc.
      print "\033[0;40m", $1, $2, $3, $4, "\033[0m"; 
    } else {
      print $0;
    }
  }
}

Mapping special characters in Sphinx configuration

Sphinx sometimes assumes some characters as word separators. For instance, letters 'e' and 'ё' are kinda similar in Russian. Some print issues even replace the latter with the former. However, Sphinx assumes 'ё' a word separator. To prevent it, one should search for Unicode code points and append mapping to charset_table index config:

charset_type  = utf-8
charset_table  = 0..9, A..Z->a..z, _, a..z, U+A8->U+E5, U+B8->U+E5, U+410..U+42F->U+430..U+44F, U+430..U+44F, \
U+0451->U+0435

Here U+0451->U+0435 maps 'ё'(U+0451) to 'e'(U+0435). The Unicode code points could be found here.

References

• UTF-8 encoding table and Unicode characters, http://www.utf8-chartable.de/unicode-utf8-table.pl
• Sphinx charset_table config setting

How to create a Sphinx wordform dictionary

To build a wordform file for Sphinx you may use some kind of spellchecker dictionary like myspell, ispell, pspell, aspell. Let's make a wordform file for Russian language from myspell-russian package in openSUSE.

To install myspell-russian:

$ sudo zypper in myspell-russian
$ rpm -ql myspell-russian
/usr/share/doc/packages/myspell-russian
/usr/share/doc/packages/myspell-russian/descr_en.txt
/usr/share/doc/packages/myspell-russian/descr_ru.txt
/usr/share/doc/packages/myspell-russian/description.xml
/usr/share/doc/packages/myspell-russian/dictionaries.xcu
/usr/share/doc/packages/myspell-russian/icon.png
/usr/share/doc/packages/myspell-russian/licence.txt
/usr/share/myspell
/usr/share/myspell/ru_RU.aff
/usr/share/myspell/ru_RU.dic

Build wordforms:

$ spelldump /usr/share/myspell/ru_RU.dic /usr/share/myspell/ru_RU.aff wordforms_myspell_ru_RU.txt
spelldump, an ispell dictionary dumper

Loading dictionary...
Loading affix file...
Using MySpell affix file format
Dictionary words processed: 146265

Detect encoding:

$ cat wordforms_myspell_ru_RU.txt | enca -L ru 
KOI8-R Cyrillic
  LF line terminators

Конвертируем в UTF-8:

$ iconv -f KOI8-R -t UTF-8 -o wordforms_myspell_ru_RU_UTF8.txt wordforms_myspell_ru_RU.txt

Finally, in sphinx.conf set:

wordforms     = /path/to/wordforms_ru_RU_UTF8.txt

See also

• http://freshmeat.net/projects/enca/
• http://sphinxsearch.com/docs/manual-0.9.9.html#conf-wordforms