Software Engineer's Notes: 2011

15 December 2011

Installing ImageMagick with DejaVu support

I've had troubles with DejaVu fonts in ImageMagick. Almost everything with -font 'DejaVu-Sans-Condensed' failed. Well, it was searching for the font, consuming much of CPU resources, and used default one afterwards. In this post I'll show how to compile ImageMagick from source with DejaVu support. Althouth it may seem trivial, the process may sometimes be tricky.

First off, install FreeType dev package. It should be called libfreetype*-dev or so. For instance, in Debian:

$ sudo apt-get install libfreetype6-dev

Make sure you have DejaVu font installed. Package name should be named somewhat like 'ttf-dejavu'. In Debian you can install it as follows:

$ sudo apt-get install ttf-dejavu-core ttf-dejavu ttf-dejavu-extra

Download the source

Download tarball from http://www.imagemagick.org/script/download.php and unpack it.

Find font directories

Figure out the main directory with fonts and a directory with DejaVu fonts, and specify them in configuration:

$ ./configure --with-freetype=yes --with-djvu=yes \
--with-fontpath=/usr/share/fonts/truetype \
--with-dejavu-font-dir=/usr/share/fonts/truetype/ttf-dejavu

Compile and install:

$ make
$ sudo make install 
# or 
# sudo make checkinstall
# whatever preferred/appropriate.

ldconfig

I had to make the follownig for my /usr/local prefix:

$ sudo ldconfig

Likely, you should do it also.

Verify

Verify that ImageMagick recognizes DejaVu:

$ identify -list font | grep -i dejavu

This should output a list with installed DejaVu fonts.

I hope this will be helpful for someone.

11 October 2011

eio - a new PHP extension for asynchronous POSIX I/O

Eio is a new PECL extension for asynchronous POSIX I/O.

Eio provides interface for the libeio library. It means that each POSIX call runs in a separate thread. It is important to aware that the order of ungrouped requests is not precisely known. However, libeio, as well, as eio, supports complex call creation to accumulate a set of regular eio_* requests in a single group request. This allows to create a group request ex. to open, read and close a file.

Currently Windows platforms are not supported.

Any extension-specific suggestions are welcome :)

01 September 2011

Creating new Vim filetype for server logs

In this post I'll show how to create a simple Vim syntax script to highlight source of common error and access logs.

First make sure you have the following folders and files:

~/.vimrc		- the main config
~/.vim/ftdetect		- directory with filetype scripts
~/.vim/syntax		- directory with specific syntax scripts

If something missing, you can copy it from $VIMRUNTIME directory. Find block starting with

if has("autocmd")

and add the following line there:

au BufRead,BufNewFile *.log set filetype=error_log

Create filetype script in ~/.vim/ftdetect/error_log.vim:

if did_filetype() " filetype already set.. finish " ..don't do these checks endif

au BufRead,BufNewFile *.error.log set filetype=error_log au BufRead,BufNewFile *.access.log set filetype=error_log

Finally you need syntax script itself, ~/.vim/syntax/error_log.vim:

" Vim syntax file
" Language:    error_log
" Maintainer:  Ruslan Osmanov <rrosmanov at gmail dot com>
" Last Change: Thu Sep  1 15:30:52 MSD 2011
" Version:     1.0

if version < 600
  syntax clear
elseif exists("b:current_syntax")
  finish
endif

" Always ignore case
syn case ignore

" General keywords which don't fall into other categories
syn keyword error_logKeyword 			error warning notice 

" Special values
syn keyword error_logSpecial         	referer client

" Strings (single- and double-quote)
syn region error_logString           start=+"+  skip=+\\\\\|\\"+  end=+"+
syn region error_logString           start=+'+  skip=+\\\\\|\\'+  end=+'+

" Numbers and hexidecimal values
syn match error_logNumber            "-\=\<[0-9]*\>"
syn match error_logNumber            "-\=\<[0-9]*\.[0-9]*\>"
syn match error_logNumber            "-\=\<[0-9][0-9]*e[+-]\=[0-9]*\>"
syn match error_logNumber            "-\=\<[0-9]*\.[0-9]*e[+-]\=[0-9]*\>"
syn match error_logNumber            "\<0x[abcdefABCDEF0-9]*\>"

" IP addresses
syn match error_logIP            	"-\=\<[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\>"

syn region error_logOperator         start="referrer\:" end="" contains=ALL

" URI
syn match error_logURI            	"http\:\/\/[^ ]*"

" Date 
syn match error_logDate 			"\[[0-9]*\/[a-zA-Z]\{3\}\/\d\{4\}:\d\{2\}:\d\{2\}:\d\{2\} +\d\{4\}\]"
syn match error_logDate				"\[[a-zA-Z]\{3\} [a-zA-Z]\{3\} \d\{2\} \d\{2\}:\d\{2\}:\d\{2\} \d\{4\}\]"

" Domain
syn match error_logDomain            "\(www\.\)\?[a-z0-9\-\_]*\.\(ru\|com\|by\|uz\|ua\|kz\|su\|net\|org\)"

" Define the default highlighting.
" For version 5.7 and earlier: only when not done already
" For version 5.8 and later: only when an item doesn't have highlighting yet
if version >= 508 || !exists("did_error_log_syn_inits")
	if version < 508
		let did_error_log_syn_inits = 1
		command -nargs=+ HiLink hi link <args>
	else
		command -nargs=+ HiLink hi def link <args>
	endif

	HiLink error_logKeyword         Statement
	HiLink error_logIP				Type
	HiLink error_logSpecial  	    Special
	HiLink error_logString          String
	HiLink error_logNumber          Number
	HiLink error_logDate			Comment 
	HiLink error_logURI				Underlined 
	HiLink error_logDomain			Underlined 

	delcommand HiLink
endif

let b:current_syntax = "error_log"
" vim:sw=4:

03 August 2011

Colorized svn status

I post some shell scripts here to output svn status command output colorized.

Bash Script

#!/bin/bash - 

set -o nounset                              # Treat unset variables as an error

txtblk='\e[0;30m' # Black - Regular
txtred='\e[0;31m' # Red
txtgrn='\e[0;32m' # Green
txtylw='\e[0;33m' # Yellow
txtblu='\e[0;34m' # Blue
txtpur='\e[0;35m' # Purple
txtcyn='\e[0;36m' # Cyan
txtwht='\e[0;37m' # White
bldblk='\e[1;30m' # Black - Bold
bldred='\e[1;31m' # Red
bldgrn='\e[1;32m' # Green
bldylw='\e[1;33m' # Yellow
bldblu='\e[1;34m' # Blue
bldpur='\e[1;35m' # Purple
bldcyn='\e[1;36m' # Cyan
bldwht='\e[1;37m' # White
unkblk='\e[4;30m' # Black - Underline
undred='\e[4;31m' # Red
undgrn='\e[4;32m' # Green
undylw='\e[4;33m' # Yellow
undblu='\e[4;34m' # Blue
undpur='\e[4;35m' # Purple
undcyn='\e[4;36m' # Cyan
undwht='\e[4;37m' # White
bakblk='\e[40m'   # Black - Background
bakred='\e[41m'   # Red
badgrn='\e[42m'   # Green
bakylw='\e[43m'   # Yellow
bakblu='\e[44m'   # Blue
bakpur='\e[45m'   # Purple
bakcyn='\e[46m'   # Cyan
bakwht='\e[47m'   # White
txtrst='\e[0m'    # Text Reset

RESULT="`svn st`"
echo "$RESULT" | while read LINE
do
 case "${LINE:0:1}" in
  'M')
   echo -e "$txtylw$LINE$txtrst"
   ;;
  'X')
   echo -e "$bakpur$LINE$txtrst"
   ;;
  '?')
   echo -e "$txtgrn$LINE$txtrst"
   ;;
  'D')
   echo -e "$bakylw$LINE$txtrst"
   ;;
  'I')
   echo -e "$txtblu$LINE$txtrst"
   ;;
  *)
   echo -e "$LINE";
   ;;
 esac
done

AWK Script

#!/usr/bin/awk -f
BEGIN {
  cmd = "svn st";
  while (cmd | getline) {
    char = substr($1, 0, 1);
    if (char == "M") {
      print "\033[36m(", $1, ")    ", $2, $3, "\033[0m"; 
    } else if (char == "X") {  
    print "\033[35m(", $1, ")    ", $2, "\033[0m"; 
    } else if (char  == "A") {  
      print "\033[32m(", $1, ")    ",  $2, $3, "\033[0m"; 
    } else if (char == "?") { 
      print "\033[1;31m(", $1, ")    ", $2, $3, "\033[0m"; 
    } else if (length($1) == 1 ) {
      print $1,"      " $2; 
    } else if ($1 == "---") { # changelists etc.
      print "\033[0;40m", $1, $2, $3, $4, "\033[0m"; 
    } else {
      print $0;
    }
  }
}

29 July 2011

Mapping special characters in Sphinx configuration

Sphinx sometimes assumes some characters as word separators. For instance, letters 'e' and 'ё' are kinda similar in Russian. Some print issues even replace the latter with the former. However, Sphinx assumes 'ё' a word separator. To prevent it, one should search for Unicode code points and append mapping to charset_table index config:

charset_type  = utf-8
charset_table  = 0..9, A..Z->a..z, _, a..z, U+A8->U+E5, U+B8->U+E5, U+410..U+42F->U+430..U+44F, U+430..U+44F, \
U+0451->U+0435

Here U+0451->U+0435 maps 'ё'(U+0451) to 'e'(U+0435). The Unicode code points could be found here.

References

16 June 2011

How to create a Sphinx wordform dictionary

To build a wordform file for Sphinx you may use some kind of spellchecker dictionary like myspell, ispell, pspell, aspell. Let's make a wordform file for Russian language from myspell-russian package in openSUSE.

To install myspell-russian:

$ sudo zypper in myspell-russian
$ rpm -ql myspell-russian
/usr/share/doc/packages/myspell-russian
/usr/share/doc/packages/myspell-russian/descr_en.txt
/usr/share/doc/packages/myspell-russian/descr_ru.txt
/usr/share/doc/packages/myspell-russian/description.xml
/usr/share/doc/packages/myspell-russian/dictionaries.xcu
/usr/share/doc/packages/myspell-russian/icon.png
/usr/share/doc/packages/myspell-russian/licence.txt
/usr/share/myspell
/usr/share/myspell/ru_RU.aff
/usr/share/myspell/ru_RU.dic

Build wordforms:

$ spelldump /usr/share/myspell/ru_RU.dic /usr/share/myspell/ru_RU.aff wordforms_myspell_ru_RU.txt
spelldump, an ispell dictionary dumper

Loading dictionary...
Loading affix file...
Using MySpell affix file format
Dictionary words processed: 146265

Detect encoding:

$ cat wordforms_myspell_ru_RU.txt | enca -L ru 
KOI8-R Cyrillic
  LF line terminators

Конвертируем в UTF-8:

$ iconv -f KOI8-R -t UTF-8 -o wordforms_myspell_ru_RU_UTF8.txt wordforms_myspell_ru_RU.txt

Finally, in sphinx.conf set:

wordforms     = /path/to/wordforms_ru_RU_UTF8.txt

23 April 2011

Strange C sizeof(struct) results?

If you wonder, why sizeof operator returns values bigger than you expect, you probably have to learn about compiler data structure alignment(as I had to:). Here I post a simple program demonstrating the case.

main.c

#include <stdio.h>
#include <string.h>

typedef struct 
#ifdef PACKED 
__attribute__ ((packed)) 
#endif
{
    char *str;      // 8
    char ch ;       // 1
    int a;          // 4
    int b;          // 4
    int c;          // 4
    // Packed total size should be: 21
    // Aligned total size should be: 24
} test_t;

static void print_sizes(test_t *t)
{
    printf("DATA TYPE SIZES\n");
    printf("char*: %ld\n", sizeof(char*));
    printf("char: %ld\n", sizeof(char));
    printf("int: %ld\n", sizeof(int));

    printf("\nSIZE OF *t: %ld\n", sizeof(*t));
}

int main (int argc, char const* argv[]) {
    test_t t;
    t.str = "test string"; 

    print_sizes(&t);

    return 0;
}

makefile

ifeq ($(PACKED), true)
    PACK=-DPACKED
else
    PACK=
endif

ifdef OUT
    OUTFILE=$(OUT)
else
    OUTFILE=test
endif

CC=gcc
CFLAGS=-Wall -c $(PACK)

all: test

test: main.o
    $(CC) main.o -o $(OUT) 

main.o: main.c
    $(CC) $(CFLAGS) main.c -o main.o

clean: 
    rm -f *.o *~

Compile versions with packed and padded memory:

$ make PACKED=true OUT=test_packed
$ make OUT=test_padded

and test them:

$ ./test_packed
DATA TYPE SIZES
char*: 8
char: 1
int: 4

SIZE OF *t: 21

$ ./test_padded
DATA TYPE SIZES
char*: 8
char: 1
int: 4

SIZE OF *t: 24

So one should consider a space-time tradeoff for every particular program.

I'd be glad, if it helps someone.

19 April 2011

How to add new highlight mode in GEdit

Here I show common steps to create custom language for GtkSourceView 2.0(GEdit uses it to display code). Then we'll create MBox language and special highlighting for it.

Common steps

1. Make directories for language definition and for new MIME:

$ mkdir -p ~/.local/share/gtksourceview-2.0/language-specs/
$ mkdir -p  ~/.local/share/mime/packages/

2. Create new MIME definition in ~/.local/share/mime/packages/

3. Create .lang file in ~/.local/share/gtksourceview-2.0/language-specs/

4. Update MIME database

$ update-mime-database ~/.local/share/mime

New language for MBox

STEP - 2: New MIME
Actually it should be already in /usr/share/mime/application/mbox.xml. But one would probably override it:

$ cat > ~/.local/share/mime/packages/mbox.xml
<!--
Author:     Ruslan Osmanov
Date:       Tue Apr 19 05:12:58 UTC 2011
Version:    1.0
-->
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
    <mime-type type="application/mbox">
        <comment>MBox Source</comment>
        <magic priority="50">
            <match type="string" offset="0" value="&gt;"/>
        </magic>
        <glob pattern="*.mbox"/>
    </mime-type>
</mime-info>
^D

^D stands for Ctrl + D shortcut.

STEP - 3: New Language Definition

The GNOME Language Definition Reference 2.0[1] and Language Definition v2.0 Tutorial [2] are good start points.
Syntax highlighting styles could be found in /usr/share/gtksourceview-2.0/styles/.

This info is quite enought to implement a new language:

$ cat > ~/.local/share/gtksourceview-2.0/language-specs/mbox.lang
<?xml version="1.0" encoding="UTF-8"?>
<!--
Author: Ruslan Osmanov 
Date:   Mon Apr 18 17:08:52 UTC 2011
Version:   1.0
MBox language spec
-->
<language id="mbox" _name="MBox" version="2.0" _section="Email">
  <metadata>
 <property name="mimetypes">text/x-mbox;application/mbox</property>
 <property name="globs">*.mbox</property>
 <property name="line-comment-start">&gt;</property>
  </metadata>
  <styles>
 <style id="keyword" _name="Keyword" map-to="def:keyword"/>
 <style id="underlined" _name="Underlined" map-to="def:underlined"/>
 <style id="comment" _name="Comment" map-to="def:comment"/>
 <style id="string" _name="A string" map-to="def:string"/>
  </styles>
  <definitions>
 <context id="mbox">
   <include>
  <context id="keyword" style-ref="keyword">
    <prefix>^</prefix>
    <suffix>( |\:)</suffix>
    <keyword>(From|To|Subject)</keyword>
    <!-- Russian -->
    <keyword>(От|Кому|Тема)</keyword>
  </context>
  <context id="link" style-ref="underlined">
   <keyword>\b(ftp|http|https)\:\/\/.*\b</keyword> 
   <keyword>mailto\:.+\@[a-z]+</keyword> 
  </context>
  <context id="comment" style-ref="comment">
    <start>^&gt;</start>
    <end>$</end>
    <include>
   <context ref="link"/>
    </include>
  </context>
  <context id="comment-multiline" style-ref="comment">
    <start>^--</start>
    <include>
   <context ref="def:in-comment"/>
    </include>
  </context>
  <context id="string" end-at-line-end="true" style-ref="string">
    <start>"</start>
    <end>"</end>
  </context>

   </include>
 </context>
  </definitions>
</language>
<!--vim: set ft=xml:-->

^D

Note, ^D stands for Ctrl + D shortcut.

STEP - 4. Update MIME database

$ update-mime-database ~/.local/share/mime

Voila! You now have new MBox language in View - Highlight Mode - Email section.

References

08 April 2011

Change indentation: spaces to tabs

To convert space indentation in significant number of files, you may use the
following script.

#!/bin/bash - 
#===============================================================================
#          FILE:  fixtab.sh
#         USAGE:  ./fixtab.sh <directory>
#   DESCRIPTION:  Convert expanded tabs into tabs with 4 space width
#       OPTIONS: $1 directory
#        AUTHOR: Ruslan Osmanov 
#       CREATED: 04/08/2011 03:26:00 PM UZT
#===============================================================================
set -o nounset                              # Treat unset variables as an error

DIR=$1

function usage(){
 echo "bash $0 <directory>"
}

[[ ! -d $DIR ]] && usage

# Correct existing modelines I used to include modelines beginning from set
# expandtab, so this worked for me. If you don't use modelines, comment it out 
find $DIR -name *.php -exec \
 sed -i -e 's%# vim: set expandtab:%# vim: set noet:%g' {} \;

# Retab & format source code
find $DIR -name *.php -exec \
 vim -u NONE \
 -c 'set ft=php' \
 -c 'set shiftwidth=4' \
 -c 'set tabstop=4' \
 -c 'set noexpandtab!' \
 -c 'set noet' \
 -c 'retab!' \
 -c 'bufdo! "execute normal gg=G"' \
 -c wq \
 {} \;

For example:

$ bash ./fixtab.sh /srv/www/myproject

I would be glad, if this helped somebody.

21 March 2011

When middle click doesn't work in Vim

I mean the Linux feature to use mouse middle click to paste selected text. In vim(not gvim) it sometimes doesn't work. Here I post how to fix it.

In ~/.vimrc find line like

set mouse=a

and change it to

set mouse=r

That's it!

20 March 2011

RPM for nethogs in openSUSE

Nethogs is a 'top' for network connections. It seems, RPMs are no longer supported(At least homepage links for RPM don't work). I had some troubles making an RPM for in in openSUSE. So I'd share a workaround for it.

Download lastest(for now) release tarball and uncompress it:

$ cd /tmp && wget 'http://sourceforge.net/projects/nethogs/files/nethogs/0.7/nethogs-0.7.0.tar.gz/download'
$ tar -xzvf nethogs-0.7.0.tar.gz

Now make

$ cd nethogs
$ make

You may try running

$ sudo checkinstall -R

but if you are at openSUSE box as me, you'd probably get error that says you didn't specify filename in
'install -d -m 755'... If so, edit ./Makefile:
1) find line:

sbin  := $(DESTDIR)/sbin

2) append the folliwing line after that:

bin  := $(DESTDIR)/bin

That's it! Yoy may now run

$ sudo checkinstall -R

and follow checkinstall instuctions. Everything should be OK now.

Good luck.

UPDATE
You might have to install checkinstall:

$ sudo zipper in checkinstall

Create RPM from source in openSUSE easily. Checkinstall

Being new to openSUSE, I searched for a took "like" checkinstall in Debian. But
openSUSE already has it! :) I'll show how to use it.

Firstly, install it:

$ sudo zypper in checkinstall

Now the whole process mainly looks like the following.

$ ./configure && make
######################################################################
# or 
# qmake 
# or whatever appopriate before doing make install.
######################################################################
$ sudo /usr/sbin/checkinstall
######################################################################
# ... checkinstall advices some settings
######################################################################
$ rpm -i /path/to/generated/rpm

Now let's build an RPM for KeePassX password manager:

$ cd /tmp
$ mkdir keepasx
$ cd keepasx
$ svn co https://keepassx.svn.sourceforge.net/svnroot/keepassx/trunk
$ cd trunk
$ qmake

######################################################################
# To install qmake:
# $ sudo zypper in libqt4-devel
######################################################################
$ sudo /usr/sbin/checkinstall
######################################################################
# Now I had some problems with uncreated directories... Had to issue:
# $ sudo mkdir /usr/share/keepassx /usr/share/mimelnk
# and repeat 
# $ sudo /usr/sbin/checkinstall
######################################################################

Now checkinstall should generate file like

$ /usr/src/packages/RPMS/x86_64/keepassx-0.4.3-1.x86_64.rpm

So may now install the RPM normally:

$ sudo rpm -i /usr/src/packages/RPMS/x86_64/keepassx-0.4.3-1.x86_64.rpm

14 March 2011

How to enable color prompt in byobu

Byobu sets $TERM environment variable to screen-bce, screen.linux, or something beginning with screen(at least currently in Ubuntu 10.10). And if you have your bash prompt normally colorized, it probably won't work in byobu without some changes in ~/.bashrc.

To enable color prompt in boybu find out what's the value of $TERM within byobu:

$ echo $TERM

screen-bce

Now in ~/.bashrc find lines like

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm*) color_prompt=yes;;

and append the following line there:

    screen*) color_prompt=yes;;

Then find lines like:

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"

and change them to:

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*|screen*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"

Here is my ~/.bashrc:

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
[ -z "$PS1" ] && return

# don't put duplicate lines in the history. See bash(1) for more options
# don't overwrite GNU Midnight Commander's setting of `ignorespace'.
#export HISTCONTROL=$HISTCONTROL${HISTCONTROL+,}ignoredups
# ... or force ignoredups and ignorespace
export HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
export HISTSIZE=1000
export HISTFILESIZE=3000

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm*) color_prompt=yes;;
    screen*) color_prompt=yes;;
esac

# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
 # We have color support; assume it's compliant with Ecma-48
 # (ISO/IEC-6429). (Lack of such support is extremely rare, and such
 # a case would tend to support setf rather than setaf.)
 color_prompt=yes
    else
 color_prompt=
    fi
fi
if [[ "$color_prompt" = yes && "$USER" = root ]]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]: \[\033[01;34m\]\w\[\033[00m\] \$ '
elif [[ "$color_prompt" = yes ]]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[00;33m\]\u@\h\[\033[00m\]: \[\033[01;34m\]\w\[\033[00m\] \$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w \$ '
fi
unset color_prompt force_color_prompt

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*|screen*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

# Alias definitions.
if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
fi

export EDITOR='vim'
export GREP_COLOR=';33'
#export PATH=$PATH:/usr/local/mysql/bin:/usr/local/mysql/sbin

11 March 2011

Change wallpaper periodically in Ubuntu

Thanks to UbuntuForums.org, I made a simple cron job changing desktop wallpaper depending on time.

crontab

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
@hourly /share/scripts/bash/change_wallpaper.sh

change_wallpaper.sh

#!/bin/bash -
# Get the pid of nautilus
nautilus_pid=$(pgrep -u $LOGNAME -n nautilus)
# If nautilus isn't running, just exit silently
if [ -z "$nautilus_pid" ]; then
    exit 0
fi
# Grab the DBUS_SESSION_BUS_ADDRESS variable from nautilus's environment
eval $(tr '\0' '\n' < /proc/$nautilus_pid/environ | grep '^DBUS_SESSION_BUS_ADDRESS=')
# Check that we actually found it
if [ -z "$DBUS_SESSION_BUS_ADDRESS" ]; then
    echo "Failed to find bus address" >&2
    exit 1
fi
# export it so that child processes will inherit it
export DBUS_SESSION_BUS_ADDRESS
HOUR=$(date +%H)
PICDIR="/usr/share/backgrounds/"
#echo "HOUR: $HOUR"
#echo "PICDIR: $PICDIR"
case $HOUR in
21|22|23|00|01|02|03|04|05|06)
/usr/bin/gconftool-2 --type string --set  /desktop/gnome/background/picture_filename $PICDIR"NATURE-ZenLizard.png"
;;
12|13|14|15|16|17|18|19|20)
/usr/bin/gconftool-2 --type string --set  /desktop/gnome/background/picture_filename $PICDIR"Fern_by_aalex04.jpg" 
;;
09|10|11)
/usr/bin/gconftool-2 --type string --set  /desktop/gnome/background/picture_filename $PICDIR"Bon-Echo_wallpaper.jpg"
;;
*)
/usr/bin/gconftool-2 --type string --set  /desktop/gnome/background/picture_filename $PICDIR"Ropey_Photo_by_Bob_Farrell.jpg"
;;
esac

I wonder, why it didn't work without DBUS_SESSION_BUS_ADDRESS detection. However, it works now. I'd be glad, if it helps someone.

10 March 2011

Simple CLI tool to escape HTML entities

Following scripts read content from stdin and output html-escaped string.

PHP

#!/usr/bin/env php
<?php
/* htmlescape.php */
echo str_replace("\n", "<br />\n", htmlspecialchars( file_get_contents('php://stdin')));
?>

Python

#!/usr/bin/python
# htmlescape.py
import cgi, sys
print cgi.escape(sys.stdin.read())

Usage

$ php htmlescape.php < file.html

$ python htmlescape.py < file.html

or for executables:

$ cat file.html | htmlescape.py
$ cat file.html | htmlescape.php

It also works like that:

$ ./htmlescape.py 
if (a > b && c < d) func();                                       
^D
if (a &gt; b &amp;&amp; c &lt; d) func();

^D is Ctrl+D shortcut.
I used it in writing this blog ;)

09 March 2011

Copy SVN working copy changes to a folder

To copy changed files and folders to a temporary folder, I use the following bash script:

#!/bin/bash - 
#          FILE:  cpsvnst.sh
#         USAGE:  ./cpsvnst.sh source_dir destination_dir
#   DESCRIPTION:  Copy changed files and folders from SVN working copy to a temp. dir.
#        AUTHOR: Ruslan Osmanov 
#       CREATED: 11/25/2010 01:37:18 PM UZT

set -o nounset                              # Treat unset variables as an error

# Displays the message, usage info and exits with error code 1
function my_usage()
{
    msg=$1
    [[ $msg ]] && echo $msg
    echo "Usage:
    ./$0 source_dir destination_dir
    source_dir          Source directory
    destination_dir         Destination directory"
    exit 1
}

# Returns dir name with trailing slash
function my_get_dirname()
{
    dir=$1
    if [[ ${dir:${#dir}-1:1} != '/' ]]; then 
        dir=$dir"/"
    fi
    echo $dir
}

src_dir=`my_get_dirname $1`
dst_dir=`my_get_dirname $2`
verbose=1

# Validate args
if [[ ! -d $src_dir ]]; then 
    usage "'$src_dir' is not a directory"
elif [[ ! -d $dst_dir  ]]; then 
    usage "'$dst_dir' is not a directory"
fi

# Remember current dir 
dir=`pwd`
cd $src_dir

# Loop through files and folders 
svn st | awk '{ print $2 }' | while read F
do
    if [[ $F != 'framework' ]]; then
        # Create directory, if not exists
        d=`dirname "$dst_dir$F"`
        if [[ ! -d $d ]]; then 
            mkdir -p "$d"
        fi

        # Copy file or directory
        if [[ -f $F ]]; then
            [[ $verbose = 1 ]] && echo "FILE $F"
            cp -f "$F" "$dst_dir$F"
        elif [[ -d $F ]]; then
            [[ $verbose = 1 ]] && echo "DIR $F"
            [[ ! -d "$dst_dir$F" ]] && mkdir -p "$dst_dir$F"
            cp -rf $F $dst_dir$F/../
        fi
    fi
done

# Go to the initial dir 
cd $dir

I'd be glad, if it helped someone.

Snippets in Vim

For common programming snippets in Vim install snipMate plugin.

SnipMate supports autoit, cpp, c, html, javascript, java, mako, objc, perl, php, python, ruby, sh, _, snippet, tcl, tex, vim and zsh file types by default. Default completion is <tab> key. I found it useful to change <tab> to ex. Ctrl+J:
in ~/.vim/after/plugin/snipMate.vim changed

ino <silent> <tab> <c-r>=TriggerSnippet()<cr>
snor <silent> <tab> <esc>i<right><c-r>=TriggerSnippet()<cr>

ino <c-j> <c-r>=TriggerSnippet()<cr>
snor <c-j> <esc>i<right><c-r>=TriggerSnippet()<cr>

To use multiple filetypes(as well, as corresponding snippets), you can use whether autoCmd in ~/.vimrc like:

au BufRead *.php set ft=php.html
au BufNewFile *.php set ft=php.html

or modeline like:

/* vim: set ft=html.php: */

Really nice plug-in with intellectual features. E.g. in an HTML file you type:

doctype<c-j>

,
and it suggests to choose one of 7 DOCTYPEs:

HTML 4.01 Strict
HTML 4.01 Transitional
HTML 5
XHTML 1.0 Frameset 
etc.

Enjoy!

UPDATE
Some extra snippets could be found here.

To create your own snippet clone a .snippet file in ~/.vim/snippets/ and modify it on your needs. For instance, I've created this one:

$ cat ~/.vim/snippets/sql/s3_localized.snippet
INSERT INTO \`s3_localized\` (\`localized_code\`) VALUES ('${1:SOMETHING}'); 
SET @ins_id:=(SELECT LAST_INSERT_ID());
INSERT INTO \`s3_localized_text\` (\`localized_id\`, \`lang_id\`, \`localized_text\`)
VALUES 
(@ins_id, 1, '${2:rus}'),
(@ins_id, 2, '${3:eng}'),
(@ins_id, 3, '$3'),
(@ins_id, 4, '$3'),
(@ins_id, 5, '$3'),
(@ins_id, 6, '$3'),
(@ins_id, 7, '$3');

It generates

INSERT INTO `s3_localized` (`localized_code`) VALUES ('SOMETHING'); 
SET @ins_id:=(SELECT LAST_INSERT_ID());
INSERT INTO `s3_localized_text` (`localized_id`, `lang_id`, `localized_text`)
VALUES 
(@ins_id, 1, 'rus'),
(@ins_id, 2, 'eng'),
(@ins_id, 3, 'eng'),
(@ins_id, 4, 'eng'),
(@ins_id, 5, 'eng'),
(@ins_id, 6, 'eng'),
(@ins_id, 7, 'eng');

Ctrl+J moves to the next variable declared in snippet. $-references allow change repeated strings simultaneously(as 'eng' above)

08 March 2011

Cyrillic-to-English key mapping in Vim and Vimperator

To avoid switching between keyboard layouts in navigation, visual mode etc. within a mixed Cyrillic/English text append the following to your ~/.vimrc:


set langmap=ФИСВУАПРШОЛДЬТЩЗЙКЫЕГМЦЧНЯ;ABCDEFGHIJKLMNOPQRSTUVWXYZ,фисвуапршолдьтщзйкыегмцчня;abcdefghijklmnopqrstuvwxyz

Similar mapping works for Vimperator. If you use Vimperator, append the following whether to ~/.vimperatorrc, or ~/.vimperatorrc.local(where appropriate):


map пе gt
map пЕ gT

map Ф A
map И B
map С C
map В D
map У E
map А F
map П G
map Р H
map Ш I
map О J
map Л K
map Д L
map Ь M
map Т N
map Щ O
map З P
map Й Q
map К R
map Ы S
map Е T
map Г U
map М V
map Ц W
map Ч X
map Н Y
map Я Z
map ф a
map и b
map с c
map в d
map у e
map а f
map п g
map р h
map ш i
map о j
map л k
map д l
map ь m
map т n
map щ o
map з p
map й q
map к r
map ы s
map е t
map г u
map м v
map ц w
map ч x
map н y
map я z

08 February 2011

Export all Tomboy notes into HTML/XHTML

Just wrote Python script to export all the Tomboy notes into an HTML file. The following script connects Tomboy via DBus Interface, creates XHTML file and converts it to HTML by means of an XSL. Conversion powered by xsltproc utility.

tomboy2html.py

#!/usr/bin/python
# Copyright (C) 2011 - Ruslan Osmanov <rrosmanov@gmail.com>
# 
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
# See <http://www.gnu.org/licenses/>
# 
##
# @file tomboy2html.py
# @brief Export all Tomboy notes in XHTML/HTML file(s)
# @author Ruslan Osmanov
# @version 1.0
# @date 08.02.2011
# @details xsltproc utility required
# @copyright Copyright (C) 2011 - Ruslan Osmanov

import dbus, dbus.glib
try:
  import gobject
except ImportError:
  # gobject functions are moved to dbus.glib in this Python version
  pass
import os, sys, getopt
import re

def usage():
  print __file__ + " [OPTIONS]"
  print """Options:
-h, --help    Display help
-p, --prefix    Optional. Output filename prefix. Default: notes
-x, --xhtml   Optional. Generate XHTML file also. Default: off 
-s, --xsl   Optional. XSL for XML-HTML conversion. Default: tomboy-notes.xsl 
-d, --debug   Optional. Debug mode. Default: off 
"""

def main(argv):
  debug_mode = False
  prefix = 'notes'
  xhtml = False
  xsl_filename = os.path.dirname(__file__) + '/tomboy-notes.xsl'

  # Get CLI options
  try:
    opts, args = getopt.getopt(argv[1:], "s:p:xhd", 
        ("xsl=", "prefix=", "debug", "xhtml", "help"))

  except getopt.GetoptError:
    usage()
    sys.exit(2)

  # Save CLI options
  for o, a in opts:
    if o in ('-h', '--help'):
      usage();
      sys.exit();

    elif o in ('-x', '--xhtml'):
      xhtml = True

    elif o in ('-d', '--debug'):
      debug_mode = True

    elif o in ("-p", "--prefix"):
      prefix = os.path.expanduser(a)

    elif o in ("-s", "--xsl"):
      xsl_filename = os.path.expanduser(a)

  xhtml_filename = prefix + '.xhtml'
  html_filename = prefix + '.html'
  if False == os.path.exists(xsl_filename):
    print "XSL file '"+xsl_filename+"' doesn't exist"
    sys.exit(2)

  if debug_mode:
    print "xhtml =", xhtml_filename, "\nhtml =", html_filename
    print "\nxsl =", xsl_filename

  # Access the Tomboy remote control interface
  bus = dbus.SessionBus()
  obj = bus.get_object("org.gnome.Tomboy", "/org/gnome/Tomboy/RemoteControl")
  tomboy = dbus.Interface(obj, "org.gnome.Tomboy.RemoteControl")

  # Get note URIs
  all_notes = tomboy.ListAllNotes()

  # Write each note XML to the XHTML file
  f = open(xhtml_filename, "w")
  f.write('<?xml version="1.0" encoding="utf-8"?>' + 
      '<?xml-stylesheet type="text/xsl" href="' +xsl_filename+'"?>'
      '<notes>')
  for n in all_notes:
    if (debug_mode):
      print "NOTE '"+n+"'"
    xml = re.sub(r'<\?xml[^<]*\?>', '', tomboy.GetNoteCompleteXml(n))
    xml = re.sub(r'\&\#x?.*\;', '', xml)
    f.write(unicode(xml).encode("utf-8"))

  f.write('</notes>')
  f.close()

  # Generate HTML
  cmd = "xsltproc -o '%(html)s' '%(xsl)s' '%(xhtml)s'" % \
      {'html': html_filename.replace("'", "\\'"), 
          'xsl': xsl_filename, 
          'xhtml': xhtml_filename}
      if debug_mode:
        print cmd
  os.system(cmd)

  if xhtml == False:
    os.unlink(xhtml_filename)

if __name__ == "__main__":
    main(sys.argv)

tomboy-notes.xsl

<?xml version='1.0' encoding="utf-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:tomboy="http://beatniksoftware.com/tomboy"
xmlns:notes="http://beatniksoftware.com/tomboy/notes"
xmlns:size="http://beatniksoftware.com/tomboy/size"
xmlns:link="http://beatniksoftware.com/tomboy/link"
version='1.0'>

<xsl:output method="html" indent="no" />
<xsl:preserve-space elements="*" />

<xsl:param name="font" />
<xsl:param name="export-linked" />
<xsl:param name="export-linked-all" />
<xsl:param name="root-note" />

<xsl:param name="newline" select="'&#xA;'" />

<xsl:template match="/">
<html>
<head>
    <title>Tomboy Notes Export</title>
    <style type="text/css">/*<![CDATA[*/
        body { <xsl:value-of select="$font" /> }
        h1 { 
            font-size: xx-large;
            font-weight: bold;
            border-bottom: 1px solid black; 
        }
        div.note {
            position: relative;
            display: block;
            padding: 5pt;
            margin: 5pt;
            white-space: -moz-pre-wrap; /* Mozilla */
            white-space: -pre-wrap; /* Opera 4 - 6 */
            white-space: -o-pre-wrap; /* Opera 7 */
            white-space: pre-wrap; /* CSS3 */
            word-wrap: break-word; /* IE 5.5+ */ 
        }
        table.note-info{
            margin: 20px 0 5px;
            font-size:xx-small;
            border-width: 0 1px 1px 0;
            border-style: solid;
        }
        table.note-info td{
            padding: 2px 5px;
            border-width: 1px 0 0 1px;
            border-style: solid;
        }
        /*//]]>*/
    </style>
</head>
<body>
    <xsl:for-each select="notes/tomboy:note">
    <div class="note">
        <xsl:apply-templates select="tomboy:text | tomboy:title"/>

        <table class="note-info" boder="0" cellpadding="0" cellspacing="0"
            summary="Note Info">
            <tr>
                <td>Last updated:</td>    
                <td><xsl:value-of select="tomboy:last-change-date" /></td>
            </tr>    
            <tr>
                <td>Tags:</td>    
                <td>
                    <xsl:for-each select="tomboy:tags">
                    <div><xsl:value-of select="tomboy:tag"/></div>    
                    </xsl:for-each>
                </td>
            </tr>    
        </table>
    </div>
    </xsl:for-each>
</body>
</html>
</xsl:template>



<xsl:template match="text()">
    
    <xsl:value-of select="."/>
</xsl:template>

<xsl:template match="child::tomboy:text">
    <xsl:apply-templates select="node()"/>
</xsl:template>

<xsl:template match="tomboy:title">
    <h1><xsl:value-of select="text()"/></h1>
</xsl:template>
<xsl:template match="tomboy:bold">
    <strong><xsl:apply-templates select="node()"/></strong>
</xsl:template>
<xsl:template match="tomboy:italic">
    <i><xsl:apply-templates select="node()"/></i>
</xsl:template>
<xsl:template match="tomboy:monospace">
    <code><xsl:apply-templates select="node()"/></code>
</xsl:template>

<xsl:template match="tomboy:strikethrough">
<strike><xsl:apply-templates select="node()"/></strike>
</xsl:template>

<xsl:template match="tomboy:highlight">
<span style="background:yellow"><xsl:apply-templates select="node()"/></span>
</xsl:template>

<xsl:template match="tomboy:datetime">
<span style="font-style:italic;font-size:small;color:#888A85">
<xsl:apply-templates select="node()"/>
</span>
</xsl:template>

<xsl:template match="size:small">
<span style="font-size:small"><xsl:apply-templates select="node()"/></span>
</xsl:template>

<xsl:template match="size:large">
<span style="font-size:large"><xsl:apply-templates select="node()"/></span>
</xsl:template>

<xsl:template match="size:huge">
<span style="font-size:xx-large"><xsl:apply-templates select="node()"/></span>
</xsl:template>

<xsl:template match="link:broken">
<span style="color:#555753;text-decoration:underline">
<xsl:value-of select="node()"/>
</span>
</xsl:template>
<xsl:template match="link:internal">
<a style="color:#204A87" href="#{node()}">
<xsl:value-of select="node()"/>
</a>
</xsl:template>

<xsl:template match="link:url">
<a style="color:#3465A4" href="{node()}"><xsl:value-of select="node()"/></a>
</xsl:template>

<xsl:template match="tomboy:list">
<ul>
<xsl:apply-templates select="tomboy:list-item" />
</ul>
</xsl:template>

<xsl:template match="tomboy:list-item">
<li>
<xsl:if test="normalize-space(text()) = ''">
<xsl:attribute name="style">list-style-type: none</xsl:attribute>
</xsl:if>
<xsl:attribute name="dir">
<xsl:value-of select="@dir"/>
</xsl:attribute>
<xsl:apply-templates select="node()" />
</li>
</xsl:template>

<xsl:template match="//tomboy:x | //tomboy:y | //tomboy:width | //tomboy:height | //tomboy:cursor-position | //tomboy:create-date | tomboy:last-change-date | tomboy:open-on-startup | tomboy:last-metadata-change-date | tomboy:tags">
<xsl:comment>literal</xsl:comment>
</xsl:template>

</xsl:stylesheet>

Usage

I saved the files here in ~/scripts/python/tomboy2html/. Thus, to save all the Tomboy notes in all_notes.xhtml and all_notes.html, I can issue the following:

$ chmod +x ~/scripts/python/tomboy2html/tomboy2html.py
$ ~/scripts/python/tomboy2html/tomboy2html.py -p "all_notes" -x -s \
~/scripts/python/tomboy2html/tomboy-notes.xsl

It creates all_notes.xhtml and all_notes.html files in current directory.

Update 09 February 2011

Clone the Git repo:

$ git clone git://github.com/rosmanov/Tomboy2HTML.git

Subscribe to: Posts ( Atom )

15 December 2011

Download the source

Find font directories

Compile and install:

ldconfig

Verify

11 October 2011

01 September 2011

03 August 2011

Bash Script

AWK Script

29 July 2011

References

16 June 2011

See also

23 April 2011

main.c

makefile

19 April 2011

Common steps

New language for MBox

References

08 April 2011

21 March 2011

20 March 2011

14 March 2011

11 March 2011

crontab

change_wallpaper.sh

10 March 2011

PHP

Python

Usage

09 March 2011

08 March 2011

08 February 2011

tomboy2html.py

tomboy-notes.xsl

Usage

Update 09 February 2011