Archive for December, 2010

End of year roundup

30 Dec

Its been a busy 2010 for the MusicNet project and we’ve made great progress. Our Alignment Tool is now becoming more mature and the latest code is showing significant increases in task speed, making the workflow much more efficient. We expect to be able to release Beta 2 in the new year, so keep checking back for more details.

Alignment Progress (Work Package 4)

The performance and usability improvements to our Alignment Tool (Work Package 2) have had a dramatic effect on our overall alignment progress. We are now at 56% complete, which places us firmly on target to complete the entire dataset before the proposed deadline (end of March 2011).

Codex/User Portal (Work Package 6.2)

Work has also begun on the MusicNet Codex, which aims to be a single source of search for Musicologists to find information and links into our datapartners catalogs. Although this is in the very early beta stages, it is functional and we are adding more composers as and when they are aligned.

Visit the beta of the MusicNet Codex:

The Codex publicly demonstrates for the first time the outputs of the Alignment Tool and shows the integration with the LinkedBrainz project (read about our meetup with the LinkedBrainz project).

Please feel free to leave any feedback on the Codex in the comments.

Linked Data (Work Package 5)

Work is underway to convert the output of the Alignment Tool into usable Linked Data. In the New Year we plan to release our proposed URI Scheme (Work Package 3) and also expose an early version of our data/alignment as Linked Data.


Performance & Usability Improvements

09 Dec

Lately we’ve been working hard to improve the workflow of our Alignment Tool. Based on the real user experience of the musicologist using the tool daily, we were able to implement some simple performance and UX (User Experience) updates that have had a dramatic effect on efficiency.

The improvements we’ve made and the effect they’ve had on the workflow are outlined below – although some of these may seem simple, it’s only in hindsight and after real-world user testing that the need for such tweaks becomes apparent.

Starting Point

When we released Beta 1 of our tool, we ran some benchmarks to get a sense of how quickly the alignment task could be achieved. Initially we found that the rate at which you could create verified matches was: 135/hr.

Performance Improvements, Phase 1

Alphabetically auto-sort newly created groups

In Beta 1, newly created groups were pushed to the bottom of the groups list so as to reduce the need to re-sort and re-render a potentially large list – a procedure that typically doesn’t perform well in a Javascript environment.  However, this is problematic from a UX perspective: after creating a new group, the alphabetical sorting of the grouped item column is corrupted, which makes comparing its contents to the alphabetically sorted ungrouped column more time consuming.

Happily, recent improvements to Javascript engines and the uptake of ECMAScript 5 functions, such as Array.sort(callback), allow the browser itself to perform the re-ordering rather than us coding it in a Javascript routine. By altering the behaviour of the alignment tool so that groups added to the grouped item column are now listed in their correct alphabetical position, we were able to improve the user experience and remove the difficulty in comparing the contents of the ungrouped and grouped item columns. In testing the change we were unable to make the browser stall or freeze during the re-sort/re-render and did not notice a reduction in interface speed or responsiveness.

Scroll to newly created groups

In Beta 1 this action was the default as we knew that the new group would be at the bottom of the list. In changing to the re-sorting model we needed to work out where the newly created group resided in the list and then scroll the list to make sure the element was visible in the lists viewport.

As it turns out this was quite a simple process, the server returns the ID of the newly created group which allows us to find the element on the DOM after its been sorted. We can then work out how much to scroll the list based on the newly created group elements offsetTop property.

We also added a small Javascript ‘blink’ animation to draw the users attention to the newly created group.

Highlight List Items

When examining the associated metadata in the right hand metadata view pane for a group that has been suggested by the system, sometimes items’ metadata might be ordered differently to how items were listed in the column. Using the Beta 1 code this posed a problem if the entries labels were all identical, as it meant there was no way to identify which metadata belonged to which list item.

To solve this we added a hover effect in the metadata view pane which highlighted the associated list item, allowing for much quicker and accurate removal of single items.

Phase 1 Improvements led to a new match rate of 169/hr (25% improvement on Beta 1)

Performance Improvements, Phase 2

Fix Diacritics

A lot of the tooling we used to generate the datafiles we required as input to Alignment Tool didn’t seem to handle diacritics as well as expected. Specifically all the composers we had imported from the Grove database seemed to have escape characters placed in front of any diacritic. The diacritic remained in tact but there were just extra characters in the string.

We programmatically removed these characters to aid readability during the alignment process.

Create a Merge function

One feature we were missing in Beta 1 that we anticipated we might need was the ability to merge two or more groups into one. The most common use cases where this was required are (i) where the system generates two different groups for the same composer based on two re-occuring variations in name usage, or (ii) where the user creates a new group for ungrouped items, before realising that a suitable group for these items already existed.

This function has now been added and can be found in the Alignment Tool SVN repository on Google Code.

Phase 2 Improvements led to a new rate of 279/hr (106% improvement on Beta 1)