Jasmine Guard for CoffeeScript rocks on Rails 3

CoffeeScript sure is interesting and I had a very positive experience trying it with Jasmine BDD testing kit. With Guard autocompiling the .js files, it becomes a charm on Rails 3.

Jasmine BDD in CoffeeScript on Vimeo.

This setup guide from pivotallabs.com coupled with this updated gist you will have the infrastructure set up in no time. Any change to the CoffeeScript source or spec files should trigger the Guard to recompile the JavaScript.

To compile "bare" js see this tip to add :bare => true to your Guardfile.

Start guard and jasmine servers and browse to localhost:8888 to run the test suite.

$ guard
$ rake jasmine

Here is a good example of writing a CoffeeScript spec. More spec examples will eventually sprout up, undeniably. And naturally, the Jasmine wiki is a good source of information.

Here's one example: timer.coffee

class Timer
  constructor: (@timer_json) ->
    @start_date = new Date(@timer_json.start_date)
    @end_date = new Date(@timer_json.end_date)
    [@start_hours, @start_minutes] = @timer_json.start_at.split(":")
    [@end_hours, @end_minutes] = @timer_json.end_at.split(":")

And the corresponding test: timer_spec.coffee

describe 'timer-parser', ->

  timer_data = {
    start_date: "2011/06/20", end_date: "2011/06/25",
    start_at: "12:00", end_at: "04:04"
  }

  it 'should parse properly', ->
    timer_json = JSON.parse JSON.stringify timer_data
    timer = new Timer(timer_json)
    expect(timer.start_date).toEqual new Date "2011/06/20"
    expect(timer.end_date).toEqual new Date "2011/06/25"
    expect(timer.start_hours).toEqual "12"
    expect(timer.start_minutes).toEqual "00"
    expect(timer.end_hours).toEqual "04"
    expect(timer.end_minutes).toEqual "04"

Writing JavaScript in 2011 - catching up with the latest trends

JavaScript has definitely bloomed during the last few years. I do remember the first time I heard of JavaScript in the late 90s - I wrote a small script that showed a random image (out of a static array) on a static HTML page that I tested in Netscape and IE5-or-so. Today people run JavaScript in top- and mid-range mobile devices, on their web browsers, on servers, on the Gnome3 desktop.

JavaScript is the only language for building an application for all major "smart" mobile devices from within a singular codebase. That, already, interests me enough to give a closer look on some modern JavaScript tools - and write my observations about them.

Foremost, testing JavaScript was something I had completely neglected before. If I was to write serious applications using JavaScript, I'd absolutely needed a way to write unit tests. Secondly, I wanted to have a look at CoffeeScript, which has had a lot of positive attention lately.

I started out by digging into the recent archives of JavaScriptWeekly newsletter. Out of a good number of testing tools, I selected to try out JsTestDriver. It is compact, requiring only a single jar file and a configuration file. The jar contains a small web server that is ran on the source code directory, and exposes a specific url that should be opened in the browsers that will run the JavaScript code to be tested. Cool! If you consider to try out JsTestDriver yourself, I recommend you checkout their SVN repository and compile the jar yourself - the jar file given in the example is outdated and does not contain all the features that the guides may refer to - for example, mocking a piece of HTML that the JavaScript modifies.

Mocking AJAX was out of my scope at this stage, for that you may find this article to be helpful.

For learning to use JsTestDriver, I chose to write a simple slideshow application that uses CSS3 transformations to change the pictures. The required JavaScript for this seemed simple enough, and I wanted to learn some new CSS tricks as well. ;) So, this is "bébé", written in a few days using jQuery-powered CoffeeScript. I started out by getting the basics of CoffeeScript, having it to load the images onto the page, and then wrote tests to see the pictures were actually loaded. Testing the user interface seems to be very difficult, due to differences in how different browsers behave under JsTestDriver. I set the picture position to absolute, since elsehow I could not get the CSS transition to appear smooth (and have the image to disappear without hiding it with JavaScript). CoffeeScript is pretty amazing, and works really well with jQuery. :)

The code for this, along with the JsTestDriver tests, can be found on Github. This seems to work reasonably well in Firefox 4, Chrome, Opera 11, Safari (and Mobile Safari on iOS) and Android browsers. Firefox 3 behaves correctly, it just doesn't show the transition effect. IE fails miserably, never showing anything except the first image. :/

Here are some nice pointers to related articles:

• A Python programmer’s first impression of CoffeeScript
• Small demonstration of coffeescript with jQuery UI in a Rails app
• CoffeeScript with WebSQL and jQuery
• jQuery Test-Driven Development
• Automating JavaScript Testing with QUnit
• Testing Backbone applications with Jasmine and Sinon

..and now for something different ..
PC emulator in the browser running Linux on JavaScript

HTML5 native audio support comparison

Here are the results of a quick test how various browsers on OS X 10.6 (and iPhone 3GS) play HTML5 <audio>.

UPDATE 25.5.2011: I updated the data and found that the audio support has become better in many browsers. I left in old data for comparison.

Browser / OS	mp3	wav	ogg
Android 2.2	X	X	X
Chrome 9.0.597	ok	X1	ok
Chrome 11	ok	X1	ok
Firefox 3.6.13	X	ok	X2
Firefox 4.0.1	X	ok	ok
iOS 4.2	ok	X	X
Opera 11.1	X	ok3	ok
Safari 5.0.3	ok	X	X
Safari 5.0.5	ok	ok3	X

• ¹    plays, but is buggy
• ²    hangs on load
• ³    is able to seek!

Likewise with <video>, there is a need to encode to many formats to support all browsers. I have no data of IE9. Unfortunately Android 2.2 cannot play any audio type at all! This is supposed to be fixed in Android 2.3. See this link for more information on HTML5 audio on Android. Also, Opera 11.1 could play ogg files, but I noticed that when they were encoded in higher bitrates, the playback stopped in the middle of the track, systematically. This seems to be fixed in Opera 11.11. Chrome buggyness on wav files appears upon seeking- the play stops and cannot be started again without reloading the page.

The tests were conducted by creating a very simple HTML5 document with this content:

      <!DOCTYPE html>
      <html>
      mp3: <audio src="test.mp3" controls></audio>
      wav: <audio src="test.wav" controls></audio>
      ogg: <audio src="02.ogg" controls></audio>
      </html>
    

The files were hosted (locally) from the same directory as the html document by ad-hoc Python web server:

      python -m SimpleHTTPServer
    

Audio CD mastering with open source tools

You want to burn an audio CD from a bunch of audio files from various sources, not direct CD rips? You can use open source tools to check the properties of the tracks and prepare them properly to avoid potential pitfalls and ensure a better listening experience.

The tools used in this memo:

• cuetools
• libsndfile
• normalize-audio
• shntool
• SoX
• wavbreaker

Some of these do have overlapping functionality. With the exception of wavbreaker (needs ALSA/OSS), all of them are available on both OS X and Linux.

Installation on Ubuntu:

  $ sudo apt-get install sndfile-programs sox normalize-audio wavbreaker shntool cuetools

Please note that the following changes to wav files are destructive so be sure to operate on copies of the originals.

Normalize

Check if the tracks need normalization.

  $ sndfile-info *.wav | egrep 'File|Signal Max'

    File : 01.wav
    Signal Max  : 16462 (-5.98 dB)
    File : 02.wav
    Signal Max  : 19946 (-4.31 dB)
    File : 03.wav
    Signal Max  : 24188 (-2.64 dB)
    File : 04.wav
    Signal Max  : 12092 (-8.66 dB)
    File : 05-06.wav
    Signal Max  : 16781 (-5.81 dB)
    File : 07.wav
    Signal Max  : 32768 (0.00 dB)
    File : 08-09.wav
    Signal Max  : 27442 (-1.54 dB)

So there is one with a really loud peak (07.wav) and a few bit quieter tracks. This is what normalize-audio suggests to do (gain column):

  $ normalize-audio -n *.wav

      level        peak         gain
    -12,6922dBFS -5,9791dBFS  0,6922dB   01.wav
    -14,3614dBFS -4,3119dBFS  2,3614dB   02.wav
    -15,2635dBFS -2,6370dBFS  3,2635dB   03.wav
    -22,4749dBFS -8,6588dBFS  10,4749dB  04.wav
    -15,9599dBFS -5,8126dBFS  3,9599dB   05-06.wav
    -13,3062dBFS 0,0000dBFS   1,3062dB   07.wav
    -13,3358dBFS -1,5407dBFS  1,3358dB   08-09.wav

To go with automatic settings:

  $ normalize-audio *.wav

    Applying adjustment of 0,69dB to 01.wav...
    Applying adjustment of 2,36dB to 02.wav...
    Applying adjustment of 3,26dB to 03.wav...
    Applying adjustment of 10,47dB to 04.wav...
    Applying adjustment of 3,96dB to 05-06.wav...
    Applying adjustment of 1,31dB to 07.wav...
    Applying adjustment of 1,34dB to 08-09.wav...

I really didn't like what this did to track 04.wav, so I reverted it back from the backups and adjusted a smaller gain manually:

  $ normalize-audio -g 2,5dB 04.wav 

    Applying adjustment of 2,500000dB...
     04.wav            100% done, ETA 00:00:00 (batch 100% done, ETA 00:00:00)

See normalize-audio --help for more controls. Normalization does not adjust signal-to-noise ratio, but trust your ears and speakers when adjusting volume. The desired peak level is within range -1,0 .. -6,0 dB. See Wikipedia.

Resample to Red Book CD standard

Convert the tracks to CD audio format (stereo, 16 bit, 44100 Hz). First check their current sample rate:

  $ sndfile-info *.wav | egrep 'File| Channels| Sample Rate| Bit Width'

    File : 01.wav
      Channels      : 2
      Sample Rate   : 48000
      Bit Width     : 16
    File : 02.wav
      Channels      : 2
      Sample Rate   : 48000
      Bit Width     : 16
    File : 03.wav
      Channels      : 2
      Sample Rate   : 48000
      Bit Width     : 16
    File : 04.wav
      Channels      : 2
      Sample Rate   : 44100
      Bit Width     : 16
    File : 05-06.wav
      Channels      : 2
      Sample Rate   : 48000
      Bit Width     : 16
    File : 07.wav
      Channels      : 2
      Sample Rate   : 44100
      Bit Width     : 16
    File : 08-09.wav
      Channels      : 2
      Sample Rate   : 44100
      Bit Width     : 16

Here I have a bunch of files in incompatible sampling rate. SoX can do many things to audio files, including resampling.

  $ sox inputfile.wav -r 44100 -b 16 -c 2 outputfile.wav

I converted the 48 kHz files with this ad-hoc loop:

  $ IFS=$'\n' ;\
    for f in $(\
      sndfile-info *.wav |\
      egrep 'File|^Sample Rate' |\
      grep -B 1 48000 |\
      awk -F' : ' /File/{'print $2'}); do \
        sox "${f}" -r 44100 -b 16 -c 2 "${f}-44k1.wav" && rm -f "${f}" ; done

Eliminate sector boundary errors (SBE)

To make sure there won't be any clicks between tracks, when the audio is continuous over a track change segue, you need to ensure the proper length of each track. On disc, tracks begin at full sectors, and a single sector length is 1/75 sec. If the wav of previous track is just slightly too short, the cd burner will insert some silence where audio is missing, to fill out the sector. This is enough to cause an annoying click in some cd players.

shntool is a great tool for extracting information and manipulating audio files. I will not give an example here, but see the output from shntool info *.wav to examine the properties of your wave files.

Examine the output of the shntool len command to discover problems:

  $ shntool len *.wav

     length     expanded size    cdr  WAVE problems  fmt   ratio  filename
      3:43.65       39490508 B   -b-   --   -----    wav  1.0000  01.wav-44k1.wav
      5:40.02       59979756 B   -b-   --   -----    wav  1.0000  02.wav-44k1.wav
      4:26.66       47076708 B   -b-   --   -----    wav  1.0000  03.wav-44k1.wav
      2:22.08       25067660 B   ---   --   -----    wav  1.0000  04.wav
      5:53.48       62382468 B   -b-   --   -----    wav  1.0000  05-06.wav-44k1.wav
      4:36.65       48838628 B   -b-   --   -----    wav  1.0000  07.wav
      7:39.68       81128396 B   -b-   --   -----    wav  1.0000  08-09.wav
     34:23.22      363964124 B                            1.0000  (7 files)

The b in the cdr column indicates a SBE. The hyphen indicates that everything is OK.

I wrote a small script called sbeok to check wav for SBEs. You can either use the script or shntool.

  $ sbeok *.wav

    01.wav-44k1.wav: FAILED, missing 1968 bytes
    02.wav-44k1.wav: FAILED, missing 992 bytes
    03.wav-44k1.wav: FAILED, missing 968 bytes
    04.wav: OK
    05-06.wav-44k1.wav: FAILED, missing 2024 bytes
    07.wav: FAILED, missing 696 bytes
    08-09.wav: FAILED, missing 1536 bytes

Seems all but track 04.wav has improper length and a small amount of silence would be added to the end of these tracks while burning the cd. Additionally, there are two files: 05-06.wav and 08-09.wav that need to be cut and they have music during the segue.

The brute force approach to recalculate sector boundaries is to merge all wavs into one and manually insert the track markers. This has the benefit that you don't need to cut tracks in previous steps of your workflow, but to leave it until last.

Merge all tracks together with shnjoin.

  $ shnjoin *.wav

    Joining [01.wav-44k1.wav] (3:43.65) --> [joined.wav] (34:23.22) : 100% OK
    Joining [02.wav-44k1.wav] (5:40.02) --> [joined.wav] (34:23.22) : 100% OK
    Joining [03.wav-44k1.wav] (4:26.66) --> [joined.wav] (34:23.22) : 100% OK
    Joining [04.wav] (2:22.08) --> [joined.wav] (34:23.22) : 100% OK
    Joining [05-06.wav-44k1.wav] (5:53.48) --> [joined.wav] (34:23.22) : 100% OK
    Joining [07.wav] (4:36.65) --> [joined.wav] (34:23.22) : 100% OK
    Joining [08-09.wav] (7:39.68) --> [joined.wav] (34:23.22) : 100% OK
    Post-padded output file with 1128 zero-bytes.

Use wavbreaker to set track split markers.

  $ wavbreaker joined.wav

This opens up a friendly GUI to insert your track marks. Seek to proper position in the topmost waveform view, and adjust the exact cut point from the second-to-top view. Click "Add" to enter a marker .. the interface is very easy to use. You could split the tracks from within wavbreaker, but once you have the marker file, it is possible to script the whole process. "Export to TOC" into file master.toc and exit the program.

Use shnsplit with cuebreakpoints to split the files based on the TOC. Write the master CD cast to directory "master":

  $ mkdir master
  $ cuebreakpoints -i toc master.toc | shnsplit -t %n -d master joined.wav

    Splitting [joined.wav] (34:23.22) --> [master/01.wav] (3:44.44) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/02.wav] (5:40.54) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/03.wav] (4:25.48) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/04.wav] (2:23.67) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/05.wav] (3:23.38) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/06.wav] (2:28.05) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/07.wav] (4:36.48) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/08.wav] (1:18.24) : 100% OK
    Splitting [joined.wav] (34:23.22) --> [master/09.wav] (6:21.69) : 100% OK

How about those sector sizes now?

  $ sbeok master/*.wav

    master/01.wav: OK
    master/02.wav: OK
    master/03.wav: OK
    master/04.wav: OK
    master/05.wav: OK
    master/06.wav: OK
    master/07.wav: OK
    master/08.wav: OK
    master/09.wav: OK

  $ shntool len master/*.wav

    length     expanded size    cdr  WAVE problems  fmt   ratio  filename
     3:44.44       39617132 B   ---   --   -----    wav  1.0000  master/01.wav
     5:40.22       60027788 B   ---   --   -----    wav  1.0000  master/02.wav
     4:26.05       46934204 B   ---   --   -----    wav  1.0000  master/03.wav
     2:23.67       25382828 B   ---   --   -----    wav  1.0000  master/04.wav
     3:21.15       35491724 B   ---   --   -----    wav  1.0000  master/05.wav
     2:30.28       26525900 B   ---   --   -----    wav  1.0000  master/06.wav
     4:36.48       48799340 B   ---   --   -----    wav  1.0000  master/07.wav
     1:18.24       13815692 B   ---   --   -----    wav  1.0000  master/08.wav
     6:21.69       67370732 B   ---   --   -----    wav  1.0000  master/09.wav
    34:23.22      363965340 B                            1.0000  (9 files)

Looks good. How are the volume levels?

    File : master/01.wav
    Signal Max  : 22620 (-3.22 dB)
    File : master/02.wav
    Signal Max  : 28166 (-1.31 dB)
    File : master/03.wav
    Signal Max  : 29384 (-0.95 dB)
    File : master/04.wav
    Signal Max  : 16501 (-5.96 dB)
    File : master/05.wav
    Signal Max  : 26729 (-1.77 dB)
    File : master/06.wav
    Signal Max  : 20980 (-3.87 dB)
    File : master/07.wav
    Signal Max  : 29205 (-1.00 dB)
    File : master/08.wav
    Signal Max  : 17139 (-5.63 dB)
    File : master/09.wav
    Signal Max  : 27442 (-1.54 dB)

All of them are within acceptable range, and the volume is perceived to be sensible acroll the tracks during playback. The perceived volume depends on the dynamic range of the audio - along with the equipment and the listener self as well.

Burn to CD

Since you're already on the command line, on Linux you can use cdrecord to burn the CD.

  $ cdrecord -v speed=1 dev=0,0,0 -dao -audio master/*.wav

Recipe of request-response over AMQP with Python and Java

This is a recipe to setup synchronized messaging over AMQP. The recipe source zip file contains an implementation of a Python backend broker. Along comes example Python and Java clients that send messages over AMQP to the running broker(s). The broker load can be balanced over to several processes. RabbitMQ can automagically balance the incoming requests to multiple consumers. This setup has been deployed successfully, achieving decent performance (we have benchmarked only the private implementation, our throughput is approx. 30 msgs/sec. with three backend processes).

Good presentations on concurrent messaging and AMQP are available from:

• http://www.rabbitmq.com/resources/erlang-exchange-talk-final/ex.html
• http://www.igvita.com/2009/04/06/henry-ford-event-driven-architecture/
• http://kallistec.com/2009/05/15/basic-topic-publishing-with-amqp/
• http://hopper.squarespace.com/blog/2008/10/6/multithreading-amqp-clients.html
• http://www.slideshare.net/old_sound/interoperability-with-rabbitmq

AMQP can be used as a message bus between various system components in different languages maintained by different teams.

Here are the bare bones of this recipe:

• create a unique identifier (qid) for the request
• create a new consumer and bind it to a temporary response queue
• publish the message into the durable request queue
• wait for the response on the response queue
• close response channel

Topic exchange is the only exchange type that honors routing keys, which are essential for pairing request and response together, using the unique identifier in routing key.

Unzip the recipe (or clone the recipe repository from GitHub). Install RabbitMQ (default settings are ok), and install Python modules 'amqplib' and 'carrot' (plus 'jsonrpclib' for the tests).

Launch the RabbitMQ server.

sudo rabbitmq-server

Start the example backend broker:

cd py-src
python example_broker.py

This will launch a daemon that listens for messages on a specific AMQP channel. A "request" is a thin JSON wrapper. If the request includes a request id (qid), the daemon will return the current time over the AMQP. If the request is not identified, the time will be printed instead.

Run the example requests in Python and Java, while the example_broker is running:

cd py-src
python example_request.py

Output:

Response from AMQP: 
{u'msg': u'Sat Jan 29 20:43:36 2011'}

Run the same requests from Java:

cd java-src
java -cp .:build:lib/json.jar:lib/rabbitmq-client-1.8.1.jar:lib/commons-logging.jar:lib/commons-io.jar \
  example.ExampleRequest

Output:

29.1.2011 21:11:01 recipe.amqpbus.AMQPRequestResponseImpl newConnection
INFO: ExampleBroker connected to RabbitMQ @ 127.0.0.1:5672/
29.1.2011 21:11:01 recipe.amqpbus.AMQPPublisher send
INFO: publishing query to ExampleBroker_req with binding key: example.request
{"q":{"q":"hello AMQP backend, I am Java"}}
-----------------------------------------
29.1.2011 21:11:01 recipe.amqpbus.AMQPPublisher send
INFO: publishing query to ExampleBroker_req with binding key: example.request.q538
{"q":{"q":"time can you get me please?"},"qid":"q538"}
29.1.2011 21:11:01 recipe.amqpbus.AMQPConsumer receive
INFO: amq.gen-dyK/PZQn/brNl9jTR/tlAg== received message: 
{"msg": "Sat Jan 29 21:11:01 2011"}
Response from AMQP: {"msg":"Sat Jan 29 21:11:01 2011"}

The example_broker output should print the events when it receives the requests: Output:

  consumer received message: 
  {'exchange': u'TestExchange', 
    'consumer_tag': u'__main__.ExampleBroker-e7175ec0-cd1f-4d2e-973b-1eeb42d4071d', 
    'routing_key': u'example.request.*', 
    'redelivered': False, 
    'delivery_tag': 2, 
    'channel': <amqplib.client_0_8.channel.Channel object at 0x1007bbad0>}
  -------------------
  received request 355:
  what time is it?
  -------------------
  response to TestExchange with routing_key: example.response.355, message: 
  {"msg": "Sat Jan 29 20:47:59 2011"}
  using channel_id: 3
  Channel open
  Closed channel #3

You should be able to pick up the ingredients and integrate them to your codebase as you see fit.