10. heinäkuuta 2009

Current state of lportal database functionality

The git version of lportal is having progress in supporting MySQL and Oracle.

I am using Liferay Portal 5.2.3 SQL Scripts as a standard data set. Unfortunately it is too sparse, and most of the failures are flunk since not enough test data is available.

This is the current state of tests for MySQL and PostgreSQL:

lportal: version 1.2.0_git
lportal: using MySQL adapter
lportal: using default Liferay schema build 5201, version 5.2.1
Loaded suite /usr/lib64/ruby/gems/1.8/gems/rake-0.8.4/lib/rake/rake_test_loader
Started
..FF....FFFFFFFFFFFF...FF...F......................FFF.EEF.E.EF..F.FFF............................
..FFFFFFFFFF...FFFFFFFFF.....FEFEFF.FF.FFFF......F........F.........FFFFFFF.....FFFF.FF..FFFF..F.E
FF..FFF..F.....FFFFFFFFFFFFFFFFFFFFFFFFEEFFFFFFFFFFFFF
Finished in 10.139719 seconds.
250 tests, 6123 assertions, 118 failures, 9 errors



lportal: version 1.2.0_git
lportal: using PostgreSQL adapter
lportal: using default Liferay schema build 5201, version 5.2.1
Loaded suite /usr/lib64/ruby/gems/1.8/gems/rake-0.8.4/lib/rake/rake_test_loader
Started
..FF....FFFFFFFFFFFF...FF...F......................FFF.EE..E.E....................................
..FFFFFFFFFF...FFFFFFFFF..................F......F........EE........EEEEEEE.....FFFF.EF........E.E
....FFF........FFFFFFFFFFFFFFFFFFFFFFFFEEFFFFFFFFFFFFF
Finished in 56.998852 seconds.
250 tests, 9772 assertions, 86 failures, 18 errors

9. heinäkuuta 2009

Oracle 10g on Ubuntu 9.04 x86_64

I installed Oracle 10g for testing a particular piece of code. My development environment is:

$ uname -om ; lsb_release -a|grep ^De

x86_64 GNU/Linux

Description: Ubuntu 9.04

and before installing I read some horror stories of people trying to run Oracle on Debian and/or x86_64. These were old posts, so I went on to try it out. In the end I had no hitches and it just works. :)

After downloading the deb package, I installed some core libraries someone recommended. I have no idea if all of these are required (why gcc/make when this is a binary package?) but for the reference, this is what I installed before the oracle deb. BTW, that blog has good tuning tips.

sudo apt-get install gcc libaio1 lesstif2 lesstif2-dev make libc6 libc6-i386 libc6-dev-i386 libstdc++5 lib32stdc++6 lib32z1 ia32-libs

Oracle was a breeze to install:

$ dpkg -i --force-architecture oracle-xe-universal_10.2.0.1-1.0_i386.deb
dpkg - warning, overriding problem because --force enabled:
package architecture (i386) does not match system (amd64)
(Reading database ... 234278 files and directories currently installed.)
Unpacking oracle-xe-universal (from oracle-xe-universal_10.2.0.1-1.0_i386.deb) ...
Setting up oracle-xe-universal (10.2.0.1-1.0) ...
update-rc.d: warning: /etc/init.d/oracle-xe missing LSB information
update-rc.d: see
Executing Post-install steps...
-e You must run '/etc/init.d/oracle-xe configure' as the root user to configure the database.


$ /etc/init.d/oracle-xe configure

Oracle Database 10g Express Edition Configuration
-------------------------------------------------
This will configure on-boot properties of Oracle Database 10g Express
Edition. The following questions will determine whether the database should
be starting upon system boot, the ports it will use, and the passwords that
will be used for database accounts. Press to accept the defaults.
Ctrl-C will abort.

Specify the HTTP port that will be used for Oracle Application Express [8080]:8888

Specify a port that will be used for the database listener [1521]:

Specify a password to be used for database accounts. Note that the same
password will be used for SYS and SYSTEM. Oracle recommends the use of
different passwords for each database account. This can be done after
initial configuration:
Confirm the password:

Do you want Oracle Database 10g Express Edition to be started on boot (y/n) [y]:

Starting Oracle Net Listener...Done
Configuring Database...Done
Starting Oracle Database 10g Express Edition Instance...Done
Installation Completed Successfully.
To access the Database Home Page go to "http://127.0.0.1:8888/apex"


Apex works so I am happy, but how about a command line console?
Ah, there's SQL*Plus.

I needed to set up some environment variables (note the unorthodox install location of the binary),
which I based on the packaged shell script "/usr/lib/oracle/xe/app/oracle/product/10.2.0/server/bin/oracle_env.sh"

I put this in /etc/environment:

export ORACLE_HOME="/usr/lib/oracle/xe/app/oracle/product/10.2.0/server"
PATH="${PATH}:${ORACLE_HOME}/bin"
export ORACLE_SID=XE


Testing the admin account, and checking the platform. Oracle is running in 32-bit mode, while the platform is 64-bit.

$ sqlplus SYS AS SYSDBA

SQL*Plus: Release 10.2.0.1.0 - Production on Thu Jul 9 15:10:54 2009

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Enter password:

Connected to:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production

SQL> select platform_name from v$database;

PLATFORM_NAME
--------------------------------------------------------------------------------
Linux IA (32-bit)



The SQL*Plus help files need to be manually loaded:

cd /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/sqlplus/admin/help
sqlplus SYS AS SYSDBA @helpdrop.sql
sqlplus SYS AS SYSDBA @hlpbld.sql helpus.sql


In conclusion, I was happy to get it running without running a virtual Centos server. =)

7. toukokuuta 2009

.vimrc + Python indentation

I had two separate projects (vcpynet-client and MXManager) which belong together. MXManager is a GUI in PyQt4 for vcpynet-client (which is now renamed to mx-download). That is worth another post.

I needed to re-indent the two codes to have the same indentation style. In Ruby I tend to use 2 spaces. In Python it feels a bit more natural to use 4 spaces. These files used both styles which is ok as long as the indentation is strict in each separate file.

Re-indenting in Vim should be simple. I defined these in my .vimrc:

set tabstop=4
set softtabstop=4
set shiftwidth=4
set shiftround
set expandtab
set autoindent
set paste


Vim re-indents the file in normal mode by ”gg=G”.

Whoops, didn't work on my files. Guess I need to be more strict. I break some lines in a bit unorthodox fashion but Python 2.5 seems to accept it. Vim doesn't.

Sed accomplised it, so for the record:

sed -i 's/\ \ /\ \ \ \ /g'

4. toukokuuta 2009

NSLU2 + UTF8 ♥

NSLU2 aka ”slug” is a small consumer-grade embedded ARM device with USB 2.0 + 100M ethernet that can run Debian. I bought mine a few years ago for ~ 100 €. It is not very fast but rather handy as a 24/7 online shell. It is extremely silent and low-power; there are no fans whatsoever and it consumes only ~ 5 W with a flash drive as the operating system boot device.

Not until today I needed UTF-8 support on the device. I hadn't even noticed that the install didn't include all the necessary locales. So..


nslu ~ 11:55:19 Mon May 04
$ sudo dpkg-reconfigure locales
Generating locales (this might take a while)...
en_GB.ISO-8859-1... done
en_GB.ISO-8859-15... done
en_GB.UTF-8... done
en_US.ISO-8859-1... done
en_US.ISO-8859-15... done
en_US.UTF-8... done
fi_FI.ISO-8859-1... done
fi_FI.UTF-8... done
fi_FI.ISO-8859-15@euro... done
Generation complete.
nslu ~ 13:41:54 Mon May 04


This took almost two hours on the ”slug”, whereas on a standard modern desktop PC it is done in a matter of a few minutes. The next step is to configure the proper keyboard layout.


$ sudo apt-get install console-data console-tools debconf
$ sudo dpkg-reconfigure console-data


Finally, I defined the default system-wide locales:


# echo 'LC_ALL="fi_FI.UTF-8"' > /etc/default/locale

18. huhtikuuta 2009

Sofar and Roswell

Roswell was a place where an UFO crashed at 1947. Then the US government covered it up with a story of a crashed weather balloon. The X-files was a TV series based on this urban legend that then grow out of proportion. The series is very enjoyable and interesting to watch.

The ”factual” story behind Roswell is quite as amazing, as told by a brilliant physicist and Berkeley professor Richard A. Muller. The Berkeley university has made a wonderful policy of uploading complete courses online available under a Creative Commons license! I love especially the Muller lectures, some of which are also available on YouTube. He makes each lecture very interesting and educational and can explain the physics in an easily understandable way. He doesn't go into the deep mathematics, but rather explains qualitative facts of nature and makes all subjects connect to real life.

The Sofar bomb, actually, is a mind-blower. This item was used in WW2 as an SOS device for the pilots who flew over the Pacific Ocean. They threw an hollow sphere into the ocean and the navy could approximate their location and come to rescue. Those who have seen the Hunt for Red October know about the sound channel. In this channel the sound waves bend and reach high ranges because the channel reduces the scattering. The same phenomenon appears also in the atmosphere. This is what the Roswell weather balloon was measuring. Cold war signals from USSR. So when the balloon crashed, the ”flying discs” they had collected were in fact circular microphones.

What was the saucer then?

3. huhtikuuta 2009

Bringing lportal to level with MySQL

A few people have asked me to fix lportal to support MySQL. Why it doesn't yet, is because the different database schemas vary slightly.

Whereas in PostgreSQL the "Journal::ArticleResource" model uses table "journalarticleresource", with the primary key of "articleresourceid", in MySQL the respective values are "JournalArticleResource" and "articleResourceId".

Basically I have the solution worked out, but not all models are yet up-to-date. Currently the unit tests are horrible:

lportal: version 1.2.0_git
lportal: using MySQL adapter
lportal: detected Liferay schema build 5201, version 5.2.1
Loaded suite /usr/lib64/ruby/gems/1.8/gems/rake-0.8.4/lib/rake/rake_test_loader
Started
..FFEEEEEEEEEEEEE.F.....F.....F...EEEEEE.EEEEEEEEEEEEEEEEEEEEE....EEEEEEEEEE...EEEEEEEEEEEEEEEEEEEEEEEEEE....EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
Finished in 0.630093 seconds.

2. huhtikuuta 2009

Streaming from webcam

A little while ago I purchased a cheap webcam: Logitech QuickCam E3500.
It works in Linux with the in-kernel 'uvcvideo' driver (plug-and-play) and I was interested to tinker with live video stream.

It is very simple to view (a) or record (b) a video stream;

(a) mplayer tv://
(b) mencoder -quiet tv:// -tv noaudio -ovc lavc -o "webcam-$(date "+%Y%m%d %H:%M").avi"


Streaming live video to the internet is a bit more complicated. At first I wanted to use FlowPlayer, a neat Flash program that has elaborate JavaScript controls, to view the stream using RTMP stream, which is a proprietary protocol.

Setting up a RTMP server on the other hand seemed to be impossoble to setup in the time that I was going to invest. There seemed to be only few options; the Apple Darwin Streaming Server and Red5. In short, Darwin did not compile and Red5 was way too much work to configure.

Away with RTMP - behold, VLC to the rescue!

vlc -I dummy --no-audio --no-sout-audio v4l2:///dev/video0:width=320:height=240 \
--sout='#transcode{venc=ffmpeg,vcodec=x264,vb=256,vt=128}:std{access=mmsh,mux=asfh,dst=:8080/stream.asf}'

MMS, WTF?! A deprecated proprietary protocol and ASF container! Oh well, this was the first (and worst) simple streaming method I came across, and thought it would be worth a try. This seems to be a popular way of doing HTML embedded video.


My internet connection is slow so I opted for the x264 codec, which by the way, is not supported by IE or WMP, which kind of cancels out the theoretical advantage of MMS that is works on the regular Windows user. With Firefox + VLC Mozilla plugin this seems to actually work. IE users can display the stream in VLC.


<OBJECT ID="MediaPlayer" WIDTH="640" HEIGHT="480"
CLASSID="CLSID:22D6f312-B0F6-11D0-94AB-0080C74C7E95"
STANDBY="Loading Windows Media Player components..."
TYPE="application/x-oleobject"
CODEBASE="http://activex.microsoft.com/activex/controls/mplayer/en/nsmp2inf.cab#Version=6,4,7,1112">
<PARAM name="autoStart" value="True">
<PARAM name='showControls' value="False">
<PARAM name="filename" value="mms://:8080/stream.asf">
<EMBED TYPE="application/x-mplayer2"
SRC="mmsh://:8080/stream.asf"
NAME="MediaPlayer"
WIDTH=640
HEIGHT=480>
</EMBED>
</OBJECT>