[Le Site]

The LilyPond Report #18

Valentin Villenave — 2010-05-12T17:04:30Z

Welcome to this eighteenth issue of the LilyPond Report !

This issue will be filled with emotion and coolness, paper bags and zigzag-ending staves, plus the usual Frogs and Bugs. As always, you are warmly invited to post comments at the bottom of the page, or even contribute to the LilyPond Report's next issues, by contacting Valentin or Graham.[sommaire]

Editorial

(by Valentin Villenave)

Greetings,

yesterday I was interviewed by a student who's writing a thesis about Free Software. Explaining why one contributes to Free Software is not always easy, particularly in a world where we were led, for the past two-and-a-half centuries, to believe that working has to be boring, unpleasant, and rewarded only with money. (Yes, that was my social-alienation-theory of the week, bear with me. :) )

The LilyPond project is the first (and only) Free Software project I've been involved with, so I probably lack comparison material, but nevertheless I have the feeling that our community has a very specific charm : perhaps because we're all musicians ? Although I've never met so many people both talented and smart, what strikes me everyday is the dedication and patience of these people — of course, there are many ways of being kind, some of which may take a while to get used to...

Why contribute to LilyPond ? Personally speaking, I began contributing as a way of showing my gratitude to these aforementioned people. I continued contributing because this gratitude somehow turned into friendship (with a hint of pride : hey, I'm hanging out with the cool kids now !). And eventually, at some points I stopped contributing, mainly out of guilt for not being able to be as clever, brave and competent as the rest of us.

Of course, I could hardly speak for other contributors. But how come that leaving this project is so hard, even seemingly impossible, to leave the LilyPond project for good ?

There's an untranslatable French expression for these exceptionally fine dishes or wines one can't stop asking to be served : "it tastes like revenez-y". Surely, we must do something right, don't we ? :-)

Release news

(by Graham Percival)

The current stable release is 2.12.3-2 ; all normal users should be
engraving with this version. We have no plans on making any more
2.12 releases.

The current development release is 2.13.20. This version has 10
Critical issues, with patches pending for 2 of them. The release
is slightly closer than it was when the previous Report was
published, but not much. In particular, issue 989 (ensure that no
information is only in the regtests) has received almost no
attention. This is saddening, since it's the easiest issue for
normal users to help with.

Oh well ; if users don't particularly want 2.14 any time soon, then
I shouldn't get worked up about it either.

In short

(by Valentin Villenave)

– Hey, look : Wikipedia has updated its webdesign before us ! Is it a success ? Well, let's have a look at the talk page :

Leave it alone I like it the way it is. don't mess with it. WKIPEDIA IS NOT SO GOOD AS IT WAS wikipedia has grown dull -thus people are not attacted to it.i wish you should improve it,

Is that the kind of reactions we have to expect when we finally release our own new design ?

– Hey, look : the new (lovely) companion of Doctor Who

Ouverture pour deux pianos, sur le nom d'Anne-Marie de Lavilléon-Verdier

Licence CC-by-sa © Valentin Villenave, 2004-2010

is named... Amy Pond. Sounds familiar ?

(And, no, don't ask how comes a French LilyPonder knows about the very latest episodes of an UK-based TV show. Just, don't.)

– Andrew Hawryluk has established an interesting comparison of LilyPond's font vs other music fonts. Nice !

– fred@ is no longer with us. This mysterious contributor, whom very little is known about, had signed every commit from our (dark, ancient) pre-git times ; these commits are now properly attributed. Goodbye fred@ !

– So let me get this right : it took us three days, and more than twenty mails, to come up with a name for the bug-handling team : the bug squad !

Well done.

Nobody else had ever, ever thought of it :-)

Conference sightings !

(by Valentin Villenave)

Reinhold appeared at the Linux Audio Conference in Utrecht /
Netherlands, presenting two papers :

- R. Kainhofer : A MusicXML Test Suite and a Discussion of Issues
  in MusicXML
- R. Kainhofer : OrchestralLily : A Package for Professional Music
  Publishing with LilyPond and LaTeX

As Reinhold told us :

Both are connected with LilyPond, the first one since the test
suite was originally written as regtests for the musicxml2ly
converter, but has now reached a state where it is useful also for
several other projects. Also, the suite is still in lilypond's git
master in the directory input/regression/musicxml2ly.

And the second paper obviously presents my orchestrallily package
and shows off some beautiful scores produced with lilypond.

Both papers can be downloaded from my homepage.

Graham Percival will be at the international Libre
Software Meeting 2010 in Bordeaux, France, running from 6 to 11 July 2010. John Mandereau, Valentin Villenave, Nicolas Sceaux and Francisco Vila will be attending as well ; Graham will be giving a lecture entitled : Sustainability in Free Software — Developers as a non-renewable resource. Here's the pitch :

The time and energy which developers spend on open-source projects is not an infinite resource. Developer effort can stall due to external demands on their time (such as family, career, or health), but also due to internal factors (such as a loss of motivation or interest). Long-term projects (5+ years old) should try to engage in sustaindable development practices. How can we retain developer interest ? How can we prepare for the inevitable loss of developers ? How can we train the next generation of developers ?

The LilyPond community definitely can relate to that : for the past couple of years, perhaps our main goal has been to make sure that the LilyPond project can go on as the original lead developers have less and less time to invest in it.

Graham has been at the heart of this perpetuation process : in 2008 with the Grand Documentation Project, that aimed to involve as many people as possible to write better and more maintainable documentation ; in 2009 with the Frogs, a bug-fixing task-force magically turning normal people into full-blown, hardcore developers (at least that's what it looked like from the outside :) ), and in 2010 with the Grand LilyPond Syntax Stabilization Project, that will hopefully bring us a stable, reliable syntax for the next decade or so.

Wow.

Whilst "developers" certainly are a "non-renewable resource", we do need more people like Graham, who are vital to a Free Software project even though they may not write a single line of code code (actually he does, but you get the point).

Paper-Baggers

(by Graham Percival)

When young children are embarrassed, many hide their faces —
trying to hide from the outside world. Well, many LilyPond
Developers are still children at heart, and wear brown paper bags
over their heads when they screw up.

"Ballade pour un mois de mai", pour flûte et piano

Licence CC-by-sa © Valentin Villenave, 2003-2010

22 March 2010,
Han-Wen

Neil wrote :
> Your first patch from the set you've pushed breaks the
> positioning of fetaText glyphs, since they now all have logical
> extents in the Y axis (rather than ink extents).

Ouch - can someone pass me the brown-paper bag ?

27 March 2010,
Jan

Subject : hold your horses, new repo #3 ETA 1500 CET

Han-Wen, you're ready to hand me the brown paper bag ?

29 March 2010,
Graham

The attached patch, plus the attached brown-paper-bag-for-Graham
discussion on IRC, points to a solution.

IRC discussion :

(16:02:12) janneke: import langdefs; for lang in langdefs.LANGUAGES: ... (16:02:41) gperciva: File "../lilypond/scripts/build/create-weblinks-itexi.py", line 11, in  (16:02:41) gperciva: import langdefs (16:02:41) gperciva: ImportError: No module named langdefs (16:02:48) janneke: try: (16:02:59) gperciva: remember, we DO NOT COMPILE SCRIPTS ON LILYPOND.ORG (16:03:11) janneke: PYTHONPATH=python python scripts/build/create-weblinks-itexi.py (16:03:28) gperciva: oh hey, that works (16:03:31) janneke: YES I BLOODY KNOW THAT - LEARN ABOUT LINUX ALREADY!!! (16:03:46) janneke: peace, love etc. (16:03:48) ***janneke is off

Post-scriptum (in a paper bag)
(by Valentin Villenave)

Actually, a few days before this IRC discussion with Jan, Graham himself had gone berserk towards your truly. That was my first instant-message discussion with him ever, and it only took him a mere five minutes to end up giving me the advice to screw something in something (can't remember the exact wording)... Nice :-)

Of course, this kind of paper-bag moments Graham was referring to above is by no means a developers' privilege. Simple users can experience it to, maybe even more so. I still have a very vivid memory of one of my most shameful moments in the LilyPond community (there were plenty, of course) : in 2006, I was just beginning to get involved in the community, it was the first time in my life that I subscribed to a mailing list (which, incidentally, is how I learned English), and I had this glorious feeling that, hey, this is where the cool kids are, with their clever remarks and witty jokes, let's become one of the cool kids, let's find something funny to say and make everybody laugh !

Needless to say, that was an utter disaster. Perhaps the most pathetic part is, this didn't even cure me from my chronic overenthusiastic hey-let's-make-everybody-laugh syndrome.

And what about you, dear readers ? Have you ever experienced such a awkward gee-what-was-I-thinking-when-I-posted-this moment ? If so, please let us know and perhaps we can laugh about it today, and therefore make you feel better ! :)

(OK. Now, where is this brown paper bag ?)

The no-we-don't-hate-Flash-videos of the Report

(by Valentin Villenave)

A few days ago, our statistics expert Francisco Vila has updated his visualization of LilyPond's source code contributions. Looking at it made me go through bewilderment, happiness and sadness ; I wonder if it will give you the same kind of mixed feelings, depending on your own past with LilyPond — and your amount of sensitivity of course :)

That was our no-we-don't-hate-Flash-videos part of the week ; for those of us who don't have Flash, here's a nice trick to use VLC instead.

News from the Frog Pond

(by Graham Percival)

The Frogs are ordinary LilyPond users who have chosen to get
involved in their favorite software's development. Fixing bugs,
implementing new features, documenting the source code : there's a
lot to be done, but most importantly : this is a chance for
everyone to learn more about LilyPond, about Free Software, about
programming… and to have fun. If you're curious about any of it,
then the word is : Join the Frogs !

In the past two months, the Frogs have begun investigating our
eventual switch to guile 2.0 (which could dramatically speed up
LilyPond), trying to make \thumb behave like fingerings instead of articulations, and trying to improve program output messages.

Honorable mention goes to our new patch-tracking worker, who has
uncovered a number of patches that I'd completely forgotten about.
It's sad to think of how much work has been lost in the hustle and
bustle of lilypond development ; I'm very happy to see a dedicated
helper tracking patches !

The Snippet of the Report

(by Valentin Villenave)

Graham was speaking of "work that has been lost" ; here's a very cool snippet that I have just saved from the darkness of forgotten, abyssal eons (namely January 2009) :

This snippet was written by Mark Polesky (now one of our most talented Frogs) to demonstrate the ability to have irregular end of staves, an irregularity that can even be emphasized with a zigzag line.

That's one enjoyable addition for my personal "hey, I had no idea Lily could do that" collection" ! :)

Granted, I was the one who was supposed to add it in the first place (January 2009 was the time where I had to officially deliver the score of my opera to the opera house, maybe that explains why I dropped the ball on this one). I'm glad that it all ended well in this case, but if you do remember of another snippet that has gone lost, please do feel free to nag me with it ! :)

Bug Report of the Report

(by Graham Percival)

Our favorite bug exchange came from a suggestion to allow 3 and 5 as durations. But what would 3 mean ?

Trevor Skeggs suggested a dotted 4. since 3 is mid-way between 2
and 4. Boris Shingarov thought that since 2 was half a whole note
and 4 was a quarter, 3 should be a third (i.e. a triplet minim /
half note). However, David Kastrup neatly resolved the
discussion :

That view is far too mathematical. 1, 2, 4, 8, 16, 32, 64 are
symbols for particular note lengths with a visible representation,
without numeric interpretation. 3, 5, 6, 7... don't have visual
representations associated with them. One of the strengths of
Lilypond is that you should be able to quote reasonably simple
input to a musician without Lilypond knowledge, and he'll be able
to figure out what you mean.

3 and its ilk don't really fit into that category. Too clever.

Three cheers for keeping LilyPond "non-clever" !

That concludes the eighteenth issue of The LilyPond Report. Next
installment is expected on July 1st, Canada Day !

Cheers,
Graham Percival & Valentin Villenave

The LilyPond Report #17

Valentin Villenave — 2010-03-01T12:22:56Z

Welcome to this seventeenth issue of the LilyPond Report!

Yay, the Report is back, with a new team! It has been said that two heads are better than one — does it apply to newsletters as well? Read on and let us know! In this issue we'll talk about websites and poetry, frogs and bugs, not to mention an extensive review of the Frescobaldi editor!
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

Editorial

At last, the Report is back! Hopefully for long, since it is now being handled by two editors, namely yours truly and... (wait for it...) Graham Percival himself!

When he's not living without the Bunnies in Glasgow, Graham `I'll be gone in a month' Percival is still on board, grumpier as ever. As a result, you can expect the new Report to have a very different tone than it used to have in the past!

Our goal is to publish this newsletter on a bi-monthly basis (that is, every two months, not twice a month). Is it reasonably achievable? Well, it depends.

If we try and keep up with the mailing lists and the LilyPond community, then two months is just ridiculously too long: that amounts to four development releases, dozens of major decisions, hundreds of bug reports, thousands of (often interesting) discussions... On the other hand, if we manage to keep our heads above the water and only mention carefully selected items, then the Report might give an interesting perspective, less directly concerned with the community's everyday life.

But most of all, as I repeatedly said for the past couple of years, the Report needs your contributions! I know I haven't been quite reliable in running it, but it's somehow a pity that the first person who finally stepped up and offered to give me a hand is also the one who's desperately been trying to leave the LilyPond project for the past three years,

Nevertheless, as Graham said:

A very grumpy Report may or may not be funny; on the other hand, maybe if there's 1 or 2 very grumpy reports, people will get motivated to write something, if only to make it more enthusiastic!

Cheers,
Valentin.

Behold our new website!

Affaire étrangère — partition d'orchestre

The new LilyPond website is nearing completion. If you haven't taken a look at it yet, try reading it now!

(Note that the translation infrastructure is not yet completed for the
new website.)

If you have any design suggestions, it's not too late to make
changes — get in touch with us.

Release news

The current stable release is 2.12.3-2; all normal users should be
engraving with this version. We have no plans on making any more
2.12 releases.

The current unstable release is 2.13.14. This version has 14
known Critical issues, with probably twice that number of
unknown critical problems. We do not recommend that normal users
engrave with it.

A common question in open-source projects is "when will the next
stable version be released?"; the typical answer is "when it's
ready". LilyPond is no exception: 2.14 will be out when the
number of Critical issues reaches 0, and stays at 0 for
two weeks. When will that happen? Well, it will happen when
these issues are resolved. Items are resolved by contributors
working on them. The more work people do, the quicker issues get
resolved.

Unfortunately, most of the current Critical issues require
attention from experienced developers; any helpful users trying to
jump in right now would only delay matters. However, issue
989 (ensure that no information is only in the regtests) could
benefit from helpful users.

What does this entail? Well, you need to look at all the lilypond
syntax inside a regression test (a short piece of testing code).
Then you need to check that this syntax is included in the
documentation. In most cases, you can just use your general
knowledge of lilypond — the docs obviously explains
cis'4., so you don't need to literally find each
portion of that syntax in the docs. However, if the regtest uses
little-known syntax or new features, the documentation might not
reflect this. We have a large number of regtests, so it would be
great if we could divide them between 5-10 people!

(Please feel free to contact Valentin if you're interested.)

— Graham.

News from the Frog Pond

The Frogs are ordinary LilyPond users who have chosen to get
involved in their favorite software's development. Fixing bugs,
implementing new features, documenting the source code: there's a
lot to be done, but most importantly: this is a chance for
everyone to learn more about LilyPond, about Free Software, about
programming... and to have fun. If you're curious about any of it,
then the word is: Join the Frogs!

Affaire étrangère — partition piano/voix

(Cette version inclut les didascalies du compositeur.)

In the past several months, most Frog activity has been concerned with guitars.

Back in 2009, our Italian contributor (and Free Software activist) Federico Bruni noted on his blog how fast LilyPond's support for tablatures was improving:

Now also the modern musician who needs tablature will be able to use LilyPond easily and enjoy the good-looking of TabStaff.
I guess this will open the doors of LilyPond to many new users.
[...]
Marc Hohl, an expert user of the LilyPond community, committed himself to creating a configuration file which allowed any tablature user to get the desired output without being forced to tweak the source file each time. Since the last spring he has submitted his changes to the users' testing and expert developers' assessment, until a great result was achieved.

I do thank a lot Marc Hohl for the great work and the patience shown in answering the questions and requests from users, who often ask for the most weird things ;-).

By the way, there is now a specific mailing list for tablature users: http://lists.lilynet.net/tablatures/

And yet, the situation has improved again since this blog post: for instance, hammeron and pulloff are now supported as well, and bends implementation is on its way, thanks to the huge amount of work provided by Marc.

Ian Hulin has also fixed an old (and quite annoying) bug about tuplet brackets.

More importantly, the Frogs have spent a lot of time discussing
LilyPond architecture, debugging techniques, improving the
Contributor's Guide, and generally laying the foundation for
future contributors. Judging from the number of posts that I
don't understand, the Frogs are definitely learning advanced parts
of LilyPond programming!

— Graham.

LilyPond's companions

Reviewing Frescobaldi
by Valentin Villenave.

For several months, I have been feeling unhappy, uncomfortable, frustrated. Then a couple of weeks ago, I finally realized what felt wrong, deeply down in my heart: I wasn't using the KDE4 desktop environment anymore! With the recent release of KDE4.4, I decided to get rid of all things GTK-ish alltogether (bye-bye LXDE, XFCE, GNOME, IceWM, Fluxbox...) and found myself with a slick, responsive, glamourous desktop environment again: then I realized what I had been missing in my life.

Feeling lighthearted again, I decided that the time had come for something I hd been meaning to do for a long time already: give Frescobaldi a proper review!

- History

Frescobaldi is a LilyPond-oriented text editor for KDE4. It has originally been intended as a plugin for the Kate editor, for KDE3 then for KDE4 when it became widely used. Frescobaldi is now developed as an independent program, albeit deeply integrated into the KDE desktop environment. It has its own website, in English and Dutch; Frescobaldi itself is well localized, and supports no less than ten languages!

- Author

Much like LilyPondTool, the other easy-LilyPond-editing environment of choice, Frescobaldi is essentially a one-man work. Wilbert Berendsen is a well-known contributor of our project, and is responsible for the Dutch LilyPond community. Wilbert is also an organist, teacher, editor, composer, conductor, performer (if I understand well, he'll be giving a concert in a few days). His website is quite interesting, and contains some scores and recordings (I particularly like this short piece by Louis Vierne in Ogg Vorbis, too bad he hasn't specified a license or I'd have embedded it here...).

- Development

Unlike LilyPondTool, where many features and graphical widgets had to be implemented (almost) from scratch, Frescobaldi uses the KDE libraries, glued together in Python using PyKDE. As a result, it has a very nice, professional look (did I mention how fond I am of KDE4 ?).

This is particularly visible when comparing, for instance, Frescobaldi:

... with the Kate editor:

As you can see, the text-editing part looks basically the same.

- Installation

From what I can tell, Frescobaldi is relatively well-known in the KDE community (with an 85% rating!); as a result it is available in most major distributions and you may not have to compile it from source (which is, by the way, not a particularly pleasant experience for any KDE app, as it requires installing hundreds of megabytes of -devel packages, plus cmake and the like).

Interestingly, the Frescobaldi packagers for my distribution (Funda Wang & Frederik Himpe) didn't specify LilyPond as a dependency, and that was a smart move: this leaves me free to download and install whatever LilyPond version I want, be it the stable, development branch, or even a home-compiled git source.

A couple of clicks is all it took for me to find myself with Frescobaldi installed, and listed in my menu under "Sound&Video" (go figure). The very last version (1.0.2) had been released just a few days ago, and yet my distribution's repository had already been updated.

- Interface

By default, Frescobaldi opens with a very minimal text area. When you're used to KDE, your first reaction will probably be to look at the vertical tabs on the left and right borders, that actually are retractable panels.

The left panel gives access to a number of articulation marks:

On the right, you may notice a "LilyPond" button. Clicking on it immediately opens two things : a terminal emulator at the bottom, and a PDF-preview panel on the right. KDE's PDF engine is used here (like in the Okular reader), whereas LilyPondTool has to rely on the (er, much perfectible) JPedal library. One minor downside though: there aren't any buttons/icons/toolbars in this PDF-preview panel, so quite a few functions are missing — plus, you better have a scrollwheel if you want to zoom it...

Unlike in jEdit/LilyPondTool, you do not have to save your source code first (and carefully give it a .ly extension). Frescobaldi is obviously meant to work "out of the box", and does indeed a great job at it: similarly, I did not have to manually specify the full path to the LilyPond binary.

At the bottom of the window, a few tabs (that I missed at first) allow you to open a terminal (just like in Dolphin,, KDE's file manager), consult the log (if any), and... record music using Rumor!

État des lieux des sujets de discussion des jeunes de 16 à 25 ans sur le Web

(Relevés bruts)

Unfortunately, I did not have the necessary tools to properly test this feature. But its interface sure looks attractive.

- Documentation

The right panel contains another tab: that's the online documentation browser. Where LilyPondTool embeds its own copy of the docs (with a very useful documentation browser that includes a search function), Frescobaldi, once again, takes advantage of the KDE libraries by embedding a web browser (the same engine as Konqueror).

This might come as a disadvantage though, as you will need an active connection to be able to browse the docs. On the plus side, this ensures that you will always have access to the latest docs build. But then again, there's still room for improvement: the browser points to the stable Documentation by default (which might be fine), but surprisingly enough, users are not offered with an option to change the URL (for example to use the latest development documentation, or an offline doc-build instead).

Another annoyance, that Frescobaldi is certainly not to blame for, is that the language detection didn't work, so foreign users may have to use the documentation and website in English by default.

Frescobaldi itself comes with its own offline handbook, like LilyPondTool (though Frescobaldi's may be a little more newbie-oriented).

- Text editing

As we've seen, Frescobaldi relies on KDE's Katepart component, and therefore has a smooth and pleasant look-and-feel when it comes to source-code editing. Blocks folding/nesting is beautifully indicated with a color gradient:

Affaire étrangère — code source LilyPond

Pour adapter la partition ou la découper différemment, vous pouvez consulter librement son code source, le télécharger, et le compiler au moyen du logiciel GNU LilyPond.
N'hésitez pas à proposer des changements, corrections, améliorations ou autres "patches" !

Unlike in LilyPondTool, not only { ... } blocks are indented, but << ... >> blocks as well:

Dossier de présentation (2006)

Context-sensitive autocompletion is well supported, which is a plus with regard to LilyPondTool: here are a few examples...

Dossier pédagogique de l'Opéra de Montpellier

Programme vendu lors des représentations

Livret de l'opéra

(version presque définitive...)

État des lieux des sujets de discussion des jeunes de 16 à 25 ans sur le Web

Commande du Conseil Général de l'Essonne, juin 2010.
© V. Villenave 2010, licence GNU FDL.

Scheme syntax-highlighting is simpler, but better-handled than in LilyPondTool (where, for instance, Scheme keywords aren't highlighted only in Scheme blocks). Still, I somehow prefer LilyPondTool's indenting and folding when it comes to Scheme code (but it's probably a matter of habit — and taste: Kate's brownish coloring of Scheme isn't very pleasant).

Like LilyPondTool, Frescobaldi features customizable code expansions:

État des lieux des sujets de discussion des jeunes de 16 à 25 ans sur le Web

(Source au format LibreOffice.)
© V. Villenave 2010, licence GNU FDL.

- Wizards

Frescobaldi includes a Score Creation Wizard, that very much resembles LilyPondTool's... but with a twist.

The first window looks exactly like LilyPondTool's... but if you look closely you'll notice that the preview "image" (that is precisely not an image, in Frescobaldi's case) is better centered... and even localized! How nice.

The usual list of instruments (aka "let's pretend we're Sibelius for a while").
Where LilyPondTool sports a three-pane interface, Fresco only uses two (with submenus), and adds specific options on the left.

Here's the twist: once you have chosen your options, you are presented with a preview of your settings! I have to confess that at this point, my jaw dropped and the only thing I could say for a while was :

woooooooow...

- Gizmos

We already mentioned the "Insertion" panel that allows to quickly add articulations to your music. This certainly looks nice, but one could argue that having to use the mouse to add articulations is hardly quicker than using the keyboard. Most users would probably prefer to assign keyboard shortcuts to these functions. LilyPondTool has also such buttons, but in a drop-down submenus, which is less convenient but preferable from a screen-real-estate point of view.

In this "quick insertion" panel, I was surprised not to find a way to quickly add slurs, phrasing slurs, or manual beams. This seems way more useful than adding \rheel or \lheel indications — but perhaps it's because the author is an organist :)

Finally, let's note that this feature is not context-sensitive. It is smart enough to leave rests alone and deal with compound durations such as cis2*5/8, but doesn't identify comments or \lyricmode blocks, nor does it take the notenames language into account.

There are a couple of nice and convenient unique tools. For instance, in the "Pitch" submenu, an option allows you to change the language used throughout your source code. Nothing extraordinary, but it's welcome nevertheless.

Another one, possibly my favorite: a well thought-out wizard allows you to easily create blank staff paper!

Impromptu pour violon et piano

Licence CC-by-sa © Valentin Villenave, 2004-2010

Once again, there's even a preview:

- Conclusion

Such features make Frescobaldi not just a nice toy to play with, but a considerate editor that has obviously been designed by someone who actually writes music. Wilbert has cleverly used all the power and flexibility of KDE4, and that amounts to a very slick software with an irresistible "wow" effect. I did have seen some screenshots on the website, I had even contributed to Frescobaldi translation, and as someone who's generally used to testing bleeding-edge software, I really wasn't expecting to be impressed. Yet here I am.

One can but regret that this beautiful software actually targets a very narrow range of users: people who run KDE4 (that's between a third and a half of all GNU/Linux users). For these users, Frescobaldi is a treat already; for example, I could perfectly imagine a classroom full of Free-Software-driven laptops running Frescobaldi. Unfortunately it's a far cry from supporting people who need such software most: that is, Microsoft Windows ~~hostages~~ users.

KDE4 is known to be theoretically portable to this system, so one could imagine building Frescobaldi for Windows. Unfortunately things are a bit complex: from QT4 that run natively without a glitch on Windows (SMPlayer), to KDE4-tied apps that require hundreds of megabytes of dependencies (Amarok), I'm afraid we're not there yet.

One last word about Frescobaldi compared to jEdit/LilyPondTool. As you may have noted, I've been familiar for a long time with this later environment. Does this review of Frescobaldi make me want to switch? Well, although I do have considered it at some point, I think I am going to stay faithful to my ugly/bloated/Java editor of choice.

Not because of the numerous features it has, which Frescobaldi does not (reverse point-and-click, real-time syntax parsing, virtual piano, MIDI player, offline docs, PDF ruler, etc.) — I have hardly ever used these features, and do not see the need for them.
Not because it's multi-platform: from now on I won't bother running anything else than KDE4 GNU/Linux. Not even because I designed its new icons :)

But more for a sentimental reason: LilyPondTool and I go way back, and I have seen its progress, I have seen Java become Free, I have seen Bertalan add new features one after another, fixing bugs, in a much less developer-friendly context than what KDE4 provides. I have taught generations of students how to install and use it, I've even made video tutorials about it.

Just like the old laptop I'm using to write this review: it's ugly, tired, I've changed almost every parts of it myself over the years. But while I can appreciate a beautiful and powerful computer whenever I see one, it's the story behind it that counts. That being said, I hope this review will give some readers the curiosity to give Frescobaldi a try, and, who knows, perhaps they'll make their own story with it — and if you do, please do tell us: we all love stories!

— Valentin.

Bug Report of the Report

Our favorite bug report from these two months came from Roman
Stawski:

 This short polyphony employs A trivial customised Voice but the lyrics ignore the first note in the score -- that's not the behaviour of choice.

As it happens, it turned out to be another instance of an existing
issue. But the creative poem was definitely appreciated!

— Graham.

... Aaand this concludes the seventeenth issue of The LilyPond Report.
The next instalment will be published on Saturday, May 2010 the 3rd.

Cheers,
Graham Percival & Valentin Villenave.

The LilyPond Report #16

Valentin Villenave — 2010-03-01T11:51:55Z

Welcome to this sixteenth issue of the LilyPond Report!

While today's instalment certainly took its time, here it is at last, with many guests and contributions that will definitely make it up. Before reading, here's a little game for you: Who said...
– "There's tons of things that I don't truly care about"?
– "LilyPond is just too flexible"?
– "Approach a computer and anything may cause a problem"?
Read on for the answers.
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

Editorial

Greetings,
why do you use LilyPond?, asked our new Release Meister Graham 'Grumpy' Percival in a recent discussion. Before the discussion went adrift (interestingly teaching us how an Elvis Presley song can be public domain), a few interesting questions were raised:
Graham:*nobody* knows *anything* about
the non-English forums for discussing lilypond? really?!?

Well, the Report begs to differ:
– LilyPond Spanish community
– LilyPond German community
– LilyPond Dutch community
– LilyPond French community
– LilyPond Portuguese/Brazilian community
– To be completed...

Why do we use LilyPond? Well, the Report has clearly enough demonstrated in the past why anybody sane enough would use it over of any other music notation software, so we believe this point has been made.
The real question would be: why do people keep hanging around even when they no longer use LilyPond? Of all the people who discuss on our mailing lists, contributors, developers, "Frogs" etc., many do not write any music: as he repeatedly said, Graham Percival himself hasn't been writing music for years; your editor has not written a single note in eight months... And yet, here we (still) are.

The way I see it, a Free Software project is not a product. It is a bunch of people, and with regards to LilyPond, a bunch of people who are both brilliant and friendly. Hence, the great high-quality software, that is merely a "byproduct" of our community.

Oh, wait. In a few weeks from now, there will be yet another compelling reason to use LilyPond: our brand new website.

And some of us are already thinking ahead. Last time we discussed the possibility of a future LilyPond 3.0 version; well, Graham — again — has begun thinking about it.

His plan? Making sure that the LilyPond syntax is one-hundred-percent consistent and safely upgradeable. This project is codenamed 'LSD', or 'GLISS', or whatever funny acronym you may come up with.

In a few weeks, you too will be able to help design what the next major LilyPond version will look like!

The LilyPond companion of the Week

After almost two years of development, the new version of the LilyPondTool plugin is finally out!

This Java-based plugin will turn any installation of the jEdit code editor, on any recent operating system, into a powerful LilyPond integrated development environnment — shorter: writing LilyPond scores has never been easier.

The LilyPond Report has proposed LilyPondTool's only developer, Bertalan Fodor, to tell us the story behind his project.

Some words about LilyPondTool

by Bertalan Fodor

It was in 2003. I just started looking at LilyPond. While I found it quite promising, its documentation was not very usable at the time, because searching was very bad, you had to have multiple many MB files open at the same time and searching all the way through to get the necessary information.
So I decided to convert the documentation to a more practical format, JavaHelp, and that was the birth of LilyPondTool.

Why Java? Because using this I can make my program available on virtually every platform. Actually I've always found strange that many free projects are telling much about "freedom as in free speech", but they mandate the OS. That's not freedom. LilyPond even at those times were an exception. They spent a lot of time providing binaries for Windows. But at that time this needed Cygwin which needed special command calls and so, so I made LilyPondTool help with this.

Then I always had problems about understanding \override, and finding out which property to set, so I implemented the \override autocomplete and so I could understand.

This happened just before my marriage in 2004. In 2005, LilyPondTool already was a quite feature rich editing tool, having many useful things at hand. But at the end of 2005, just after the birth of my first son, the most revolutionary step happened. The motivation was that, I too often made the following mistake: c.4 instead of c4. So I decided to implement an almost full LilyPond parser in Java. It was not perfect, but most errors are correctly found while typing. I think it is the most important feature of LilyPondTool.

The next big step one year later was the integrated PDF viewer and the ruler. That's again a unique feature.

Then, after the birth of my second son in 2007, I again had some time. I started to play with integrating a Scheme system in LilyPondTool, that could provide really real parsing of LilyPond input. Soon I found that you can't parse LilyPond input fully without running LilyPond fully. It is just too flexible. So I pended the project, and instead asked the community what features they'd like the most. And so this 2009 release will become the second most important release, because it contains all more complicated feature requests: the virtual piano and the dockable pdf viewer now pushes LilyPondTool to a new level of usability.

Actually my favorite feature is reverse point-and-click. It came from a feature request on LilyPondTool's SourceForge page. I think it is the feature that makes LilyPondTool really unique and fun.

What comes next? I really want to do the "almost full" parser.
– First I will change the parser engine in LilyPond to CUP instead of ANTLR, because that uses the same approach to parser generation. (LALR instead of LL(*))
– Then I will include Julie (my Guile-compatible Scheme project) in LilyPondTool. Now it will be based on Sisc. It wouldn't provide full interpretation of everything (that would need reimplementing a lot of LilyPond in Java instead of C++), but could provide quite useful features (autocompletion and instant syntax checking in Scheme code for example)

My real problem with developing LilyPondTool is that I don't have time to use it as I rarely use LilyPond. Fortunately my fellow users test my half-broken releases to polish them to perfectness...

There is one more little thing. I think it would be good to provide a download link to jEdit/LilyPondTool from the home page. Unfortunately this suggestion is still ignored, I don't know why. It's going to go only into the "alternate editors" section. But it should be an Officially Recommended Editor, and not just an alternative to the crappy editors included with LilyPond.

Many thanks to Bertalan for this contribution.

The Statistics of the Week

In the previous issue we begun looking at some statistics by Francisco Vila about the LilyPond project. In this second instalment, he provided us with two graphs:

The first one shows the evolution of the LilyPond installer size over the years. In green, Windows installers; in red, "shar" installers for Linux-x86.

I think it is funny that some sizes were crossed in 2.8 as polyphony
voices. These data are grabbed from the download page by a local PHP
script which retrieves real byte sizes from the links, not rounded Mb
sizes that appear in the web.

The overall size of LilyPond's installers has been steadily increasing. That may or may not be a good thing: it may imply that LilyPond is getting more and more powerful — and indeed, there are quite a few things you can do now in LilyPond which you couldn't several years ago —; but it could also mean that the development quality is decreasing, with less optimised code, for instance.

Of course, we're also putting a great deal of effort in making sure this will not happen. This even led our lead developer Han-Wen to complain about a possible shift in focus:

I am somewhat disappointed that a lot of the latest
lilypond efforts seem to be centered around janitorial work. While
janitorial work is often useful and a good way to introduce yourself
to a code base, it should not become the focus of either development
or discussion about development.

The mailing lists of the week

Quite interestingly, Han-Wen mentioned the way discussions about development should be handled. Indeed, there has been a tremendous amount of activity these past months, and as a result, the traffic on our mailing lists has recently impressively increased:

on the LilyPond-user list...

and even more so on the LilyPond-devel list.

While more people and more discussions might be a good thing, it also implies less intelligibility. Therefore Graham suggested that we could use some additional mailing lists, in addition to our -user, -devel and bug- list. For instance, he suggested a proposals mailing list, that could be useful to discuss long-term plans.

Well, one of the good things with having an informal community website such as LilyNet is that adding new ressources is quick and cheap, and can easily be reverted if the idea eventually doesn't work. In this regard, I started creating a few low-traffic mailing lists, designed for people who have to discuss something specific that doesn't really belong either on -user or -devel.

These new lists may now be found on lists.lilynet.net; as of today these include frogs, midi, proposals, tablatures and translations. While this initiative hasn't been officially announced anywhere, it has so far proved quite useful for some contributors, whether they want to keep informed of the translation status or improve LilyPond's support for guitare tablatures — without having to cope with the huge volume of data that's posted everyday on our main mailing lists.

The LilyPond-related-thingy-you'll-never-understand... of the week

If you have been following some discussions lately on the developer's list, you may have noticed a three-letters acronym: GUB.

The so-called Grand Unified Builder — not to be confounded with GRUB, the GRand Unified Bootloader that helps boot your operating system, was created by LilyPond's authors (Han-Wen and Jan) as a side-project. GUB, which has recently reached version 3, is...

Well, what is it actually?

Jan Nieuwenhuizen — Perhaps I can help you with that.

The LilyPond Report — Oh, hi Jan! Sure, I was getting a bit lost here. So, what is this thing called GUB and what does it do?

J. N. — GUB makes the work that the LilyPond developers do available for users: it produces LilyPond installers for all types of computers.

GUB also reduces the frequency, duration and intensity of the developer's or release manager's headaches, as it is an automated system. With a one button press, the release manager can produce up-to-date installers, straight from the latest development version, for all types of computers. So, all users are treated equal.

Having frequent releases that an ordinary, non-programmer user can use and evaluate, speeds up the feedback loop, and thus makes steering development more effective and agile.

L. R. — I understand that it is all about portability. Was it important to you that LilyPond could be installed on different operating systems?

J. N. — Ethically, yes. Han-Wen and I started LilyPond with the intention of providing beautiful and free music notation for everyone. Of course that means: users of any type of computer. Hey, I have even run it on my n770 cellphone!

L. R. — GUB was written for LilyPond only, but could it be used for other cross-platform Free software projects?

J. N. — Yes. It currently supports a minimal set of dependencies to build a few projects such as LilyPond or Denemo (about 180 dependencies/libraries are supported). It can build binary installers of your project for Windows, Linux
(also 64 bit) and MacOS X (also ppc) and FreeBSD, from the very latest sources, straight from GIT or SVN. A very light set of dependencies is required to run GUB and compile LilyPond, most everything is included in GUB [notable exceptions: perl, texlive].

It is quite dependable; the builds can be reproduced by using checksummed,
rather picky Python build scripts that bomb out on errors by default.

However, please note that:
– GUB does not provide binary packages
– GUB has a possibly fatal design flaw: it does not use a chroot to do the builds. This was intentional, it does not require ROOT, it seemed easier to access the build system. However, this means that it *cannot* produce binary packages for the native build tools.

Also, this means that in GUB there is a difference between a native linux-x86 build tool and a cross compiled linux-x86 tool, say tools::libtool and linux-x86::libtool. This is another fatal flaw, it means that a package provided by the cross build specification is not automatically available as a build tool.

– GUB is a new, standalone, mostly unsupported mini source-based distribution. It *should* have been built on .deb packages. Now, GUB users/developers have to maintain packages themselves and cannot steal/share the work from/with Debian developers. :-(

L. R. — You mentioned that GUB supported other projects, can you elaborate on that?

J. N. — As a pet project,

I added Inkscape to see if GUB would be able to handle gtk/Xorg dependent projects. I wanted to announce it, but then found Inkscape *did* provide binary linux packages. Recently I resurrected building Inkscape with GUB and wanted to
announce it to the Inscape developers, only to find
that all linux gtk+-based packages are broken. I'm still planning
to do that, but the outcome (if/when) is unknown.

In October 2007, I started working on the Novell-funded version of OpenOffice.org, Go-ooo. OO.o has always been available for windows and mac, but the current builds use the proprietary Microsoft Visual C++ environment to provide Windows binaries. So we'd like to cross build it. So far, the OOo/go-oo mingw build produces an installer, but does not run yet.

It is now being absorbed by a Google Summer of Code project and further developed in the suse build system. If that runs, and if/when I/someone finds some time, GUB could easily be fixed to produce working OO.o mingw installers, but not sure who'd use that.

L. R. — Wow, really? Inkscape and OpenOffice? Jan, you're so like a rockstar to me right now...

J. N. — Er, keep it real. Both OOo and inkscape are not used/blessed by the project and currently do not/hardly produce anything usable.

L. R. — Is it conceivable to use GUB for any software?

J. N. — Yes.

L. R. — Really? Aren't there some downsides? Hard-coded stuff, unportable requirements that may cause a problem?

J. N. — Approach a computer and anything may cause a problem :-)

(Many thanks to Jan "Rockstar" Nieuwenhuizen for his time.)

It has to be noted that, for the very first time in LilyPond history, GUB has (reportedly) been mastered by mortal human beings. As a result, the last few downloadable releases were built by Graham Percival (and others managed to get it working too.

Er, immediately afterwards it did get broken for the past three months. But do keep faith.

The Postcard of the Week

As you may have noticed, a strange disease tends to affect overly-dedicated LilyPond contributors: the main symptom of this (as of yet little known) illness is that they simply can't stay in one single country for more than a few months. For example, our Translations Meister John Mandereau has recently decided to move to Pisa, Italy — but the most affected must be our beloved contributor Graham "Grumpy" Percival, whose disease led to move from Canada to Malaysia, then back to Canada, then in a few days... to Scotland.

Fortunately, this also gave him a chance to send us a new Postcard...

Reducing Inefficiency
by Graham Percival

I stopped using lilypond 4-5 years ago. Not as part of a huge
switch to a different music typesetter — rather, I finished my
university studies in composition, and nobody was playing my
pieces. Oh, amateur musicians quite enjoyed my works, but I
wasn't finding any interest from the academic community. So I
moved on to other fields, eventually ending up in the emerging
field of computer-assisted music education.

So why am I still doing lilypond development? Well, there's a
number of reasons. Fondness for open source, personal
friendships, adding material to my CV... but the biggest reason is
my distaste for inefficiency.

My first steps in LilyPond development were directly fueled by
this: I saw people asking the same questions over and over. They
were answered politely (it generally wasn't me answering them :),
but this struck me as inefficient. Answering an email might take
5-10 minutes while improving the documentation could take 30-60
minutes... but by the time you had answered 6 emails, it would
have been better to improve the documentation.

However, improving the documentation wasn't trivial: it took me
two weeks to figure out how to begin fixing typos. I had to learn
CVS, configure, install a ton of dependencies, diff, etc. They
were all tools that served me well in later years, but they were a
fairly large barrier to contributing. It didn't help that I was
very shy about asking for help.

When I decided to stop doing documentation work, I still
remembered the initial discouragement, so I started the Grand
Documentation Project. The stated goals were to clean up a large
portion of the documentation, but the unofficial goal was to train
a group of people to replace me. In the beginning, I would take
care of all the technical details (source management, diffs,
making sure the texinfo files compiled, etc), and if they seemed
serious about long-term documentation contributions, I would
gradually wean them off my assistance.

Later on, I noticed that potential programmers couldn't figure out
how to get started, and the existing programmers had learned that
many well-intentioned offers of programing never pan out when they
have to actually do work. As a result, existing programmers
didn't spend much time discussing potential programmers. In most
cases, this saved the community time, but I'm sure that some of
those potential programmers _would_ have been great contributors
if they had been mentored. I therefore started the Contributor's
Guide, as a combination of help and warning to anybody thinking
about getting involved. If they were serious, they could read how
to get started. If they weren't serious, they would get
discouraged before anybody invested time mentoring them.

I'm fond of the phrase "in a democracy, we receive the government
we deserve" (the quotation has been ascribed to a number of
people). I like to apply it to lilypond: "in an open-source
project, the community receives the program / bugs / documentation
that they deserve". If a user truly wants something done —
explaining something better in the docs, fixing a bug, making a
flashier website — then they can help do it.

Of course, what if they want a bug fixed, but don't know how to
program? I have three answers.

First, in keeping with my theme of altering political phrases, I
employ the term "trickle-up development" (coming from
"trickle-down economics). The idea is this: even if you don't
know anything about programming, you can help doing other tasks.
This means that the other developers don't need to do these tasks
themselves, which means they have more time to spend on the tasks
which you can't do.

For example, my next task after sending this email is to handle a
complaint that our direction-specific documentation isn't clear.
One user had difficulty figuring out that \slurUp affected all
future slurs, while ^( affected just one slur. So I need to read
NR 1.3.2 Slurs and NR 5.4.2 Direction and placement, figure out
how it could be explained better, and write the text and/or
lilypond examples.

This isn't hard — I'm willing to bet that almost all readers of
the Report are capable of explaining the difference. And at least
21 people are capable of modifying the docs accordingly, because
that's how many people contributed to GDP. But nobody else
investigated this "mundane, routine" issue, so I'm going to do it.
After that, I'm going to investigate/document/fix some problems in
the release process — I'd like to make a new 2.12 release that
has a working GUI for OSX 10.5. If somebody else had done the
documentation issue, I wouldn't need to do it, so I would be
working on that problem now. The connection between user
documentation and better releases might not be obvious, but it's
there!

The second way that users can fix bugs when they don't know how to
program is simple: learn how to program. Don't claim that you
can't learn anything — if you're alive, you can learn. If you
truly care about some issue, then you'll spend the time to learn
how to fix it.

I'm not blaming you if you *don't* truly care about beamlets,
Gregorian notation, or whatever the bug is. There's tons of
things that I don't truly care about! But I don't claim that such
bugfixes or new features are truly important to me.

The third way is a combination of the above two points: take care
tasks so that other developers don't need to do them, but always
keep trying to do tasks that are slightly more complicated than
you can currently handle. Learn how scheme tweaks work by writing
documentation about scheme! Improve your knowledge of lilypond
fundamentals by editing the tutorial for beginners! Increase your
scheme proficiency by creating a neat tweak, then try fixing a bug
that uses the same kinds of scheme constructs!

Of course, this third way is slightly dangerous: as you work on
more complicated things, you'll want to stop working on the
simpler things. But if those simple things are "daily
maintenance" tasks, then somebody's gotta do them. So either you
get stuck doing mundane tasks, or you recruit new contributor(s)
to do those easy tasks, allowing you to concentrate on more
complicated issues.

Say... anybody want to learn how to write documentation for
lilypond?

... Aaand this concludes the sixteenth issue of The LilyPond Report.

Cheers,
Valentin Villenave

The LilyPond Report #15

Valentin Villenave — 2010-03-01T11:50:02Z

Welcome to this fifteenth issue of the LilyPond Report!

Where do you go when you're looking for LilyPond scores? Are we soon going to see a LilyPond 3.x series? How _not_ to print music with LilyPond? And why use the Internet to turn on a lamp? These are some of the questions that will be addressed in today's issue of the Report.
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

Editorial

Greetings,
at last, a new Report; as you may have noticed, I am currently (unfortunately) unable to publish this column on a weekly (if at all regular) basis, so it's more of a Debian-like "release-when-ready" style. As I said many times, my main problem as an editor is not to find something worth writing about, but about making choices in the never-decreasing number of interesting things that happen every day, every week, within our community and in the Free Software world.

But even when you've decided to deal with a specific topic, finding a proper way to do so is hardly the easy part. Sometimes the material is there, for instance on the mailing lists archives, and it is just a matter of patience to find it. Sometimes there isn't enough material, and you have to investigate a little bit. And sometimes, well, sometimes the hardest part is to put things together with some sort of a logical sense. This week's article about LilyPond scores wasn't a easy one to write in this regard.

In the former instalment, we talked about the new Frogs team; well they now have a dedicated homepage on LilyNet, where you can read (and subscribe to) their mailing list. Please also have a look at their place on the Wiki, which Patrick McCarty has been busy setting up. On the upcoming issues, we will have a closer look at this team and what they do.

Eventually, Francisco Vila has begun to provide the Report with interesting statistics about the LilyPond project; we'll have a look at that today and in several issues to come.

What's up with LilyPond (-scores)?

Recently, Wilbert Berendsen shared with us a score he has just engraved, introducing the source code repository he has created for the occasion at Google Code.

The .ly files live in the lilymusic googlecode repository, a repo which
I'm btw happy to share with anybody wanting to write good LilyPond scores (of public domain or otherwise Free music) under free licenes and svn version control.

This led Jay Anderson to make a short list of known LilyPond scores source-code repositories:

– Wilbert's

– A lesser known Google Code project, from a Spanish-speaking contributor whom I do not know,
– Jay's own repo,
– Sven Axelsson's repo (mostly bagpipe music),
– Xavier Shay's repository (a few jazz scores),
– your editor's work,
– last but not least, Master Nicolas Sceaux's repository of authoritative editions of baroque music.

Of course, such repositories are meant to host only source code. But a number of free online music libraries include LilyPond scores. On these websites, you can download scores as compiled PDF files, as well as the sources.

– the Mutopia project immediately comes to mind. It contains only LilyPond-engraved scores, all released under specific Free licences.

– the International Music Scores Library Project (IMSLP) has been going through a lot, but is now back online. Is contains mostly scanned scores, but you may find some LilyPond scores. And these may include source files.

– the Werner Icking Music Archive (WIMA) has many PDF downloadable scores. Some of these have been engraved using LilyPond. Some may come with their source files.

–

the Gutenberg project now has its own music library. Whether it will contain many LilyPond scores or not remains to be seen.

– the Choral Public Domain Library (CPDL), as its name states, is about vocal music. Even though it is not limited to LilyPond scores, several members of our community do contribute to this library on a regular basis, including Andrew Hawryluk, Peter Chubb, Reinhold Kainhofer or Laura Conrad. Laura has contributed 133 scores as of this day, and she explained why she'd rather post these scores on CPDL than on Mutopia:

for a particular piece of music, I think it makes more sense to
either have it on my own website or to contribute it to a more central
place where people are looking for music.
In other words, I don't expect people to look for a madrigal typeset in
lilypond; I expect them to look for a three-part madrigal, or a madrigal
by Thomas Weelkes, and mutopia doesn't particularly have a reputation in
the madrigal-performing community as a place to go (unlike Werner
Icking or CPDL).

Of course, many LilyPonders do publish their scores on their personal websites, and it would be absolutely impossible to reference these here (if at all). It allows them (like Nicolas Sceaux) to offer their scores either as compiled PDF or source archives.

Source code repositories, online music libraries, and personal websites: quite a lot of resources to look at, and one can wonder if things couldn't be made simpler. In different ways.

– The one website to rule them all approach. According to Hajo Dezelski (and he certainly has a point), this website should be Mutopia:

I ask myself again why do I have
to add another link to my list, when we have a working repository for
music: Mutopia.

Following closely discussions on this mailing-list you will notice
that tons of scores are engraved with Lilypond. (Applause!) And most
of this music seems to be without any copyright restrictions. But I
can seldom find them in Mutopia.[...] Why is Mutopia not the official music code repository for Lilypond?

The upside: well, one single place, wouldn't that be nice...
The downsides:

- What do we do with already existing external resources?
- How do we handle syntax evolution?
- What about licensing constraints?

– The let's make something new and better approach. For instance, Mike Blackstock is aiming to open a new Wiki, that (unlike our current Wiki) would be interfaced with LilyPond and designed to host scores.

The upside: wiki-like editable scores! How cool is that?
The downsides:

- On the server side, it requires some serious computing power.
- On the implementation side, it requires a crew of skilled web-developers.
- Whether a wiki-interfaced LilyPond can deal with complex source code (with includes, lots of Scheme, etc) in a safe way has yet to be proven.

– The geeky web 2.5.0 shiny venture capital newfangled interactive web my-face-space-book stuff approach (Graham K. Percival). I said earlier that referencing all personal websites/repositories/music libraries that contain LilyPond scores is a nearly impossible task. However, there may be a way to achieve it: by offering a central, widely-visible place where the LilyPond community can unite their strengths. For example, the IMSLP has an impressive list of free online music libraries.

There's more to it. Through a customized Google search box, through RSS feeds aggregation, JavaScript embedding etc, we could conceive a simple webpage where people could, at a glance, see the latest Mutopia/CPDL additions, as well as a preview of Nicolas Sceaux's latest score (yeah, that's three times I mention him, what do you know, I'm a big fan), search for a specific score in all LilyPond resources known to man, and so on.

In a similar way, I can imagine a source code repository that would be interfaced with both SVN and GIT, and that would automatically retrieve the source code from given external repositories.

The point with this latter way would be to leave resources were they are, but to reference, syndicate them, and therefore give them visibility, consistency, and ease of use.

News from the Free world (and Lily)

So, more than two years after the last stable version, LilyPond 2.12 was released on Christmas Eve. While we had been looking forward to this for months, it still came as a surprise — which some of us weren't happy about, because of unresolved bugs or documentation issues.

Such controversies aren't unusual in the Free world; one can think, for

instance, about the KDE 4 quarrel. Moreover, Reinhold Kainhofer made an interesting point:

As things are currently in LilyPond, I don't think that the term "stable
version" makes much sense. All our releases seem to me like snapshots of the
current development version at some random times

This, in turn, brings up the question of the version numbering model.

For ages, LilyPond used the same numbering model as the Linux kernel: odd-numbered development releases, even-numbered stable releases. This model has one big downside, that we just experienced: it tends to make the development cycle really long, and at some point the 'stable vs unstable' question becomes irrelevant.

Projet Pinocchio - La partition

Cette partition a été compilée sous forme de fichier PDF, prêt à être imprimé et déchiffré !

Projet Pinocchio - La partition

Cette partition a été compilée sous forme de fichier PDF, prêt à être imprimé et déchiffré !

Therefore, the Linux kernel has dropped this model, just issuing one release after another, exactly in the same way that Reinhold suggested. Recently, Linus Torvalds even considered entirely dropping the 2.6.xx.x naming model, since nothing compelled him anymore to switch from 2.6 to 2.8, or even from 2.* to 3.*...

While it is unlikely that we'll see a Linux 3.0 version any decade soon, LWN's Grumpy Editor has predicted that the 2.6.* numbering scheme will still go on, and make it at least to 2.6.33 by next year.

And what about Lily?

Well, it seems that, after all, we're not far away from the Linux situation. Like the Linux kernel, LilyPond has moved from a small 0.x project to a bigger community-driven 1.x project with more developers; then some major internal changes (particularly introducing the Guile interpreter) have led to a new 2.x series... And now?

In August 2004, Han-Wen announced:

LilyPond development is gearing up towards a new, better, funkier
glitzier stable release, and that is LilyPond 3.0.

In February, he was still referring to LilyPond 3.0. A few months later [1], it became 2.10 instead. And then, 2.12, 2.14, etc...

So, are we stuck in 2.* numbers? I'm not sure. I think it's a matter of generation. LilyPond started approximatively at the same time as other GNU parts, such as Glib/GTK or the Gimp, that are still at version 2.* as well. One could also mention the Blender project [2], that it now at... version 2.49!

Not only does that mean that LilyPond is as active as other successful free software projects, but it also implies that, even as a "niche" project, we're still part of something bigger. And this "bigger picture" is moving forward: fohttp://news.lilynet.net/ecrire/?exec=articles_edit&id_article=109r instance, the

GNU project's General Public License has reached version 3 less than two years ago; it has also recently been decided that GNOME 2.30 will be renamed GNOME 3.0. [3]

Anyway, as Linus Torvalds once said (at a time where the Linux kernel 2.6 was thought to become 3.0): "Hey, it's just a name"...

The Statistics of the Week

Our Spanish contributor Francisco Vila has been busy lately, and he came up with some statistics (and some comments) about the LilyPond project. This week, we'll present the first part of his observations.

I was curious about the health of the LilyPond documentation
and I have made this graph.

Though most people browse it online or as a PDF file, LilyPond's documentation is also downloadable as a `tarball' archive on our website. This graph shows the evolution of this archive's size, over the years.

As you may know, "There are three kinds of lies: lies, damned lies and statistics" (Mark Twain). Nevertheless, what can we notice? In two years, the doc size has been multiplied by 4; this may have something to do with translations (that were basically not there in version 2.8). But the main cause is obviously the Grand Documentation Project that has been led by Graham Percival over the whole last year. On this subject, Graham commented the way these statistics were made:

if you *really* want to see a spike for GDP, let's look at
the size of patches in git. Or even better, look at the
percentage of patches which are documentation-related (i.e. modify
Documentation/ or input/ ) as opposed to code-related. ;)

However, I think the overall shape looks good... and promising for our future. Francisco, on his side, has gathered a different comment:

My two daughters said immediately that the drawing was "a
little path of stars". Isn't it lovely?

It certainly is :)

The Idea of the Week

To me, the "We Can" phrase is much less of a president's motto than a geek-defining mantra. You may know this geek-oriented US TV show called The Big Bang Theory; if not you can have a look at the first two minutes of this clip and pay attention to this line (slightly rephrased below), which I believe defines the very essence of geekness:

— Why do you send a signal from this laptop, through your local ISP, racing down fiber optic cabel at the speed of light to San Francisco, bouncing off a satellite in GS synchronous orbit to Lisbon, Portugal where the data packet will be handed off to submerge transatlantic cabels terminating in Halifax, Nova Scotia and transferred across the continent via microwave released back to your ISP and X10... to turn on a lamp???

— Because I can!

Anyway, from the just because we can collection...

This week's idea was posted by our Ukrainian contributor Dmytro Redchuk on the mailing list:

I thought of \repeat unfold in a context of some experiments
with "concatenateable" fragments of music, lyrics etc —
can not say now with what exactly;
so i can not tell whether i need this or not :-)

Of course, you must already be familiar with the widely-used \repeat unfold n {...} command, that prints the music expression n times. But... have you ever thought of using it not to print music?

\repeat "unfold" 0 { d4 d d d } % "zero" here
To be honest, i would expect this bar not to be engraved.

This "feature" is currently not available in LilyPond. Maybe someone will implement it some day... "What for", you might ask? Well, because we can! :)

LilyPond's companion projects

This week's "project" is more of an experiment, that could remind you of the special issue we had previously published about algorithmic composition.

On January the 5th, one of our French LilyPonders, Martial, inadvertently posted a mail on the -user list, announcing (in French) a new version of POV-ray (another Free Software project he seems to be involved in).

The Persistence of Vision Raytracer (aka POV-Ray) is a major project of the Free World (it has been developped for more than twenty years), that allows to render high-quality computed-generated 3D images. Similarly to LilyPond, its input format is plain code (it can, however, be interfaced with graphical modeling programs such as Blender). Its development seemed to have been slowing down for the past five years, but Martial's announcement of a new version proved that it's still well alive and promising.

This little mishap led to an interesting discussion, where Cameron Horsburgh believed that LilyPond had precisely been (remotely) inspired by the POV-ray language... But the coolest part is probably when Martial modestly took this as an opportunity to show us his work with computer-generated animations and LilyPond scores

Here are some explanations, translated from his website:

Some Python, some Pov and a lot of LilyPond:

the score (notes and rhythm) is randomly generated by some macros, randomly selected out of a prepared "pot", and then saved as a LilyPond file.

The image is generated from the same "pot": a macro somehow rotates a globul-blobtuberence field.

A random factor is seeded by a Python script involvind the system clock; then the score is engraved by LilyPond from the output file. Here's the source POV file that produced this LilyPond example.

Through the Python Imaging Library, the image is converted into EPS to be handled by LilyPond; the MIDI output is converted into mp3 through TiMidiTy.

[...] Instruments are arbitrarily chosen at the beginning of the POV source file.
Example:
#declare Instrument_1= "\"trumpet\"" or
#declare Instrument_1= "\"harpsichord\""

How cool's that? If you're geeky enough and want to experiment making 3D LilyPond scores "just because you can", then you'll definitely want to have a look at Martial's source code. Feel free to send us your creations, as the Report will happily feature them in its next issues!

The LilyPonder of the Week

Some day in March, your editor had the pleasant surprise to meet someone special on our -user list. I am, of course, referring to

Miklos Vajna, a Hungarian young developer whom we owe one of my favorite GNU/Linux distributions [4]: Frugalware.

What better way to introduce this distribution than by quoting this Distrowatch paper from 2006, which I believe is still dead on:

Frugalware Linux is one of those distributions that does not feature often in the news headlines. But those users who take the time give it more than just a passing glance are often surprised to find in Frugalware a clean, fast distribution with a great package manager and a few convenient system administration tools. Loosely modelled on Slackware and incorporating Arch's 'pacman' for managing installed applications, Frugalware Linux is not only a great operating system, it is also an active community project based on open source ideals.

As I said back then, Miklos is to Frugalware what our own "benevolent dictators" Han-Wen and Jan are to the LilyPond project; that should give you an idea of the amount of respect he deserves. So, what brings Miklos in the LilyPond community? Here's what he told the Report:

I play guitar for about 13 years, not that I'm that good in it, but with
time I tend to learn a bit. In the past five years most of the time I
played religious songs and that has nothing to do with LilyPond, it's
just a textbook with guitar chords. But from time to time I occasionally
play other old songs like Steve Wonder, The Beatles and such. I buy or
download the music sheet, print it out, then usually change it a bit,
since the original chords are for multiple guitars / instruments, etc.
When I'm happy with the shape and I'm tired of programming, occasionally
I try to let LilyPond create the music sheet for my version of the song.

What I like in LilyPond is of course its programming language-like
interface, which is quite comfortable for a programmer like me. I used
Finale once about 12 years ago and it was boring, I hate to use my mouse
too much. :) Also the community is wonderful, every time I ask something
on the list I get helpful answers. Not to mention the documentation,
IIRC the notation reference is 500+ pages. As a result usually my
question is not "Is it possible?" but "How to do it?". ;)

BTW, I first noticed LilyPond because I knew Han-Wen Nienhuys because of
his darcs2git and Jan Nieuwenhuizen because of ooo-build (I contibuted a
few patches to both projects), so it was something I had to try out as I
was sure it'll be quality software. :)

If you want to learn more about Miklos, you can have a look at this recent interview in the Frugalware Report — sorry, Newsletter.

The quote of the week

Upon special request from Graham Percival, this week's quote was found in a long discussion involving (yet another) Grahamized newbie named Tim Rowe:

The lilypond documentation really isn't the kind of documentation that you can go to when you want to know how to do something. It's designed to teach you how to use the software.

Comments are opened below.
[5]

... Aaand this concludes the fifteenth issue of The LilyPond Report.

Cheers,
Valentin Villenave

[1] I remember this was explained in a mail on -user (by Mats?), but I couldn't find it. Anyway, in a long (annoying) conversation in July 2006, Erik Sandberg used the 3.0 numbering, then a few days later the 2.10 numbering. My guess is that this is when the change took place.

[2] By the way, the third Open Movie project (codenamed Durian has just been announced, and it looks really exciting.

[3] ... And there I have managed to put both KDE and GNOME logos. Hopefully no flamewar today.

[4] Was my top #1 actually. And then KDE4 came out.

[5] In fact, I am not sure that I understand it completely, since I have been learning LilyPond the other way around: at first I only wrote very simple scores, and while reading the documentation my scores became more and more complex since everytime I found a new feature I just had to find a way to use it: "Hey, let's put a Voice-Following line somewhere! Hey, where could I use feathered beams in this piece? OK, let's add another measure for that..."
... And before you ask: yes, the "because I can" approach is the wrong way of composing music. But what do you know, I'm just a geek.

The LilyPond Report #14

Valentin Villenave — 2010-03-01T11:38:41Z

Welcome to this fourteenth issue of the LilyPond Report!

[...]
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

This Week's Desultory Editorial

Greetings,
[...]

News from the Free World

[OLPC situation]

What's up with LilyPond?

[Lily for childrens: testimony]

The LilyPond Feature of the Week

[...]

LilyPond's Companion projects

[DIY Keyboard project, article from Andrew Wagner]

The Idea of the Week

[...]

The Snippet of the Week

[Gilles' last snippet?]

The Bug of the Week

[sub-beaming broken]

The Interview of the Week

The LilyPond Report — Hello Reinhold, thanks for answering our questions today!

Reinhold Kainhofer — Likewise, nice to have a chat with you!

L.R. — Introducing you to our readers is not particularly easy, since you've been actively working on many various things within the LilyPond community... For instance, last week we've seen that you've just implemented native support for complex tempo indications (mixing text and metronome marks).

Reinhold — You are right, I am quite active, and not only for LilyPond. However, the things I have done in the last few days were not so much work, but simply got a lot of attention. Hmm, let me think; MusicXML import (musicxml2ly) was actually the first stuff that I worked on, in August 2007...

L.R. — Yes, the LilyPond Report has introduced this feature a couple weeks ago (has you once noticed, your name gets frequently mentioned in this column)...

Reinhold — I have also been contributing to the LilyPond Snippet Repository, of course, where I submitted 21 snippets so far. With John Mandereau, I've been investigating possible design improvements for the LilyPond website and online documentation; and I'm also a contributor or the Choral Public Domain Library.

L.R. — Indeed; by looking at the list of your scores, I understand you're involved in a choir?

Reinhold — Yes, I initially simply started using LilyPond to write scores for our choir (after having used abc2ps)... But then, some questions on the user mailing list intrigued me, in particular that strange incomprehensible language called Scheme... LilyPond is addictive, but it's also very complex; to be honest, I still have no real clue about the inner workings of LilyPond: when I try to implement something, I mainly take some existing code and try to adjust it to my situation (mainly by trial-and-error).

L.R. — This meets what I wrote last week: there's no real border between working with LilyPond and working on LilyPond...

Reinhold — Yes, invariably, you'll run into something that you need and that LilyPond does not do yet.

L.R. — ... Or you end up constantly pushing Lily, looking for anything that it cannot do yet!

Reinhold — Yes and no. As I said, I started by writing scores for our choir, then we had a performance of Schubert's Stabat Mater, for which no orchestra material was available for sale (only for rent...). Thus, I decided to get an old (out-of-copyright) edition and write the orchestral parts with LilyPond. That's also how I got into the territory of orchestral scores; and that's one area where you are constantly pushing the edges of what LilyPond can do and what it can't.

L.R. — ... Which led you to conceive your OrchestralLily package.

Reinhold — Yes, that mainly evolved from my orchestral scores, where I realized that for each instrument / group and for each movement I had to do the same definitions over and over again. And then I ran into other problems, so I put the solutions into that package too. Oh, this makes me think I should finalize version 0.03, which adds lots of new stuff over version 0.02. Docs are at http://kainhofer.com/orchestrallily/, and the latest version can be downloaded at http://repo.or.cz/w/orchestrallily.git.

L.R. — Er, this should have been my line actually... So, you're a singer?

Reinhold — Yes (mainly bass, but I can also sing baritone). I'm actually singing in 2-3 choirs: the choir of the St. Augustin church in Vienna (which is the church of the former emperor's palace in Vienna and where Sissy got married), where we have a choir mass (with a large orchestra) every Sunday; also the Chorvereinigung "Jung-Wien", which is mainly specialized on classical Viennese Waltzes, Polkas, etc; but we also do other concerts like the Stabat Mater, or a musical concert this fall.

L.R. — These choirs must be huge, if they can afford such large performances...

Reinhold — The first one is mainly funded by the parish, but the second one needs to finance itself: by getting subsidies from the city (which is quite hard this year, since all the money goes to the soccer championship), but also charging for the concerts — being in Vienna is especially hard for musicians, because there is so much competition here.

L.R. — Judging by the (impressive) résumé on your website, you are specialized in many different areas: mathematics, physics, economics... Is it because it "just happened", or are you genuinely interested in tons of things?

Reinhold — Yes, that's one of my problems: I take on so many things that I hardly have enough time. I'm interested in everything, so it happened automatically. For example, I studied Mathematics and physics, now I also took up some law studies, I'm taking solo singing lessons, try to improve LilyPond, and — because I might get bored during the weekends ;-) —

I'm also a referee for American Football.

L.R. — Oh, not to forget that you have a "normal" day job at the university of Vienna.

Reinhold — ... Which is apparently not too stressy, since I find the time to talk with you during my work hours ;-)

L.R. — And what about computers? Did you start programming because of your studies... or are you just another computer geek?

Reinhold — I'm simply a computer geek for no particular reason. I never studied programming or took courses, I simply learned it myself. I started programming Windows applications using Borland Pascal and Visual Basic, when I was in high school (I was in boarding school, so I was allowed to use the school's computer root!). Then at university I got in touch with Mathematica and in 1999 I worked for Wolfam Reseach (first as a summer intern and then as a contractor), the creators of Mathematica.

L.R. — Interesting! Is Mathematica still one of your favorite languages? I remember that a few months ago Trevor Bača told us he had been using it to process music; have you ever tried to do so?

Reinhold — To be honest: I know that Mathematica can process music and I suspect that it can be quite efficient and easy, but I never looked at it. My favorite programming language depends on the problem. Either C++ (for writing efficient numerical applications), Mathematica (if I need to calculate something symbolically), Python/Perl for simple scripts, etc. Scheme (and the C++ that LilyPond is written in, which is basically just Scheme written in C++ Syntax) is definitely not one of my favorites ;-)

L.R. — So, long story short, you ended up becoming a Free Software user and developer...

Reinhold — During high school I got in touch with computers and started programming, but that was only for Windows. My first contacts with Linux were at the time I did the summer internship at Wolfram Research, as one of my roommates there was very fascinated by it. The irony: when I met him a few years ago, I was the Linux developer, while he had meanwhile switched to a Mac... I'm using Linux on my office machine (running Debian sid) as well as on my laptop (running kubuntu hardy) and my wireless router (running OpenWRT). I've been quite involved in the KDE project for several years, where I also was the maintainer / lead developer of KOrganizer. I simply lack the time and also the motivation to work on it any more; now I'm "only" a happy KDE user (and seeing what Wilbert did with lilykde is really a great please, since it shows how great that desktop can be).

L.R. — You seem to care about software freedom; has it something to do with the fact that you're a scientist, and specifically a scientist involved in _public_ research?

Reinhold — Coming from science, for me it is absolutely natural that the things I do can be used and are useful to others. That's basically the idea of open source. I'm not after the money (otherwise I wouldn't be at university, but probably in some investment bank), but I want to do something that I like and that helps others, too. As for public vs private research, I think that's basically a matter of philosophy. If you are in a private company, things tend to be closed, results are not shared with "competitors", etc.; it's basically only about money and competitive advantages, but not about the people involved or the society as a whole. I don't know if people who like working there are not so interested in sharing their (sparetime) work with others in principle or if they get so used to these ideas as time passes.

L.R. — Yes; I couldn't help noticing that most, if not all, scientists I've met through LilyPond (Trevor Daniels, Torsten Anders, Peter Chubb and many others) were involved in public, government-funded facilities and/or universities. Now I won't jump to conclusions, since sometimes you don't really get to choose and life chooses for you... Let's talk music again. You do not seem to be into contemporary music at all, are you?

Reinhold — No, I'm mainly into classical choir music. I like contemporary music and listen to it, of course, but mostly I simply find it too shallow.

L.R. — Yet, given your background as a mathematician, you might be interested in Algorithmic Composition such as what Torsten Anders or Trevor Bača do...

Reinhold — Hehe, that's definitely an interesting area. To be honest, I'm currently thinking about switching my areas from financial and actuarial mathematics to computer science, in particular music information retrieval.

L.R. — Wow! Can you really do that? Wouldn't you lose your job or something?

Reinhold — Of course, I would have to give up my position as an assistant professor, but I already talked to one of the CS profs and I might get a position in a EU-funded research project (if that gets granted). The exact field, of course, depends on which project will be granted (in the next few weeks), but there are so many interesting issues in information retrieval...

L.R. — You seem to really have given it a thought indeed.

Reinhold — I would probably do this, because I realize more and more that financial mathematics is not what really interests me (economics was never so interesting, not the least because it's all about money, which has very low priority to me). Also, Mathematics is interesting, but then it always tries to be as abstract as possible, so I'm missing the real applications.

L.R. — Well, good luck for your plans, and for your future work with LilyPond; can you imagine yourself becoming a regular contributor/developer in a distant future?

Reinhold — Yes, absolutely. I'm quite motivated about LilyPond, although I'm seeing that it takes way too much of my time. Sometimes I'd rather write scores in LilyPond than fixing bugs that prevent me from writing the scores.

L.R. — Well, it's an addiction, as you noted... Thanks Reinhold; see you soon on the lists!

Reinhold — Thank you for the talk, too.

Thanks to Reinhold Kainhofer for this interview.

The Quote of the Week

[...]

And this concludes the fourteenth issue of The LilyPond Report.

Cheers,
Valentin Villenave

The LilyPond Report #13

Valentin Villenave — 2008-06-24T01:05:09Z

Welcome to this thirteenth issue of the LilyPond Report!

This week we'll talk a bit more about how Free Software, and particularly LilyPond, is developed, evolves, and ends up having a life of its own. We'll also talk 3D gaming, spectacular visualizations, but also visual impairments and music robots.
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

This Week's Desultory Editorial

Greetings,
I never get tired of saying that every week: Free Software is an amazing thing. For instance, using LilyPond doesn't makes us just "users", but actors as well; the limit between working with LilyPond and working on LilyPond is extremely thin, anyone of us can experience it. Therefore I never refer to LilyPond as product but a project.

You could think that this is just politics; well, to me it is, but in a good way. I remember an article by Richard Stallman where he referred to "computer-using citizens"; well, I do believe that being part of Free software, Free culture, Free knowledge projects defines us as citizens.
A new kind of citizens: citizens of the Free World.

This week I discovered something amazing, that you really need to see. A few weeks ago, José Padovani told us about the Processing visualization programming language
[1]; Michael Ogawa, a student at the University of California-Davis, has found an incredible way to use it.

Based on the history log of several famous Free projects, he's been able to build an organic visualization of their development by the community.

I've been studying software projects for a while now. Not the programming, but the people — the way they interact with each other through collaboration and communication. [...]

This visualization, called code_swarm, shows the history of commits in a software project. When a developer commits a file, it lights up and flies towards that developer. Files are colored according to their purpose, such as whether they are source code or a document. If files or developers have not been active for a while, they will fade away.

Here is, for instance, how the Python scripting language has been developped for the past fifteen years:

It's alive!!!

News from the Free World

Last week's Firefox 3 Download Day has been an overwhelming success; more than 8 millions downloads in 24 hours, about 18 millions in a week... Our special correspondent (namely myself) has been following the event very closely: staring at the browser window for 24 hours, hysterically hitting the F5 key, nearly having a heart attack at the beginning of the official countdown, when the Mozilla Foundation's servers went down for more than an hour, well I think we've all been there.

Strangely, on FF3 Download Day the LilyPond Report's statistics went through the roof as well — I can't really understand why, maybe it was because everyone was eager to test his newly-installed web browser... :-)

This special day made another great event stay almost unnoticed: Wine 1.0 is out, after 15 years of development. What's Wine? It's an amazing program that allows you to natively run MS Windows software on non-Windows operating systems: 32-bits GNU/Linux distributions, Solaris, BSD,... Photoshop, MS Office, you name it; it even offers (quite impressive) support for a huge number of Windows 3D games!

Wine will undoubtedly increase the GNU/Linux market share among people who want games... without necessarily wanting Windows. But is this event likely to affect the LilyPond world? For several years now, Wine has already had the ability to run Finale or Sibelius on GNU/Linux. Does it make these softwares really more appealing? I don't think so. People who "just want Finale", most of the time, also "want Windows".

Actually, the improvement of LilyPond's support for MusicXML, which we discussed last week, might play a much more significant role in making the switch easier between these applications and Lily, no matter the operating system used.

[UPDATE: Just after having written about people who "want Finale" and Windows, I've found this hilarious blog post by Matt Asay, saying "If you like WordPerfect, then Windows is what you want. [...] But if you do want to run WordPerfect you have problems that no operating system can fix for you".]

What's up with LilyPond?

This week an user named Claus Tirel had a short discussion with Mats and myself on the bug mailing list.

As he had reported a bug for LilyPond 2.8.1 (that was released in early 2006), he wondered why we would not process bug reports for this version (nor even for our latest stable version 2.10) anymore.

This discussion that happened afterwards made me understand that Claus wasn't familiar with the LilyPond releases and debugging cycles, and led me to search for a way to explain it to unexperienced LilyPonders.

Unlike big projects with more resources, that can afford maintaining "old" stable branches for years [2], LilyPond has two branches: the "stable" version (currently 2.10.33, released on September 20th 2007) and the "development" version (currently 2.11.49, released last week).

Software development, if I understand correctly, is made of two things: fixing issues, and implementing new features.
Sometimes, the two kinds are actually mixed: for instance, we'll solve this bug by rewriting this piece of code, adding this new property, etc.

Whenever any part of the code is modified, there's a risk that something will get broken in the process, or that something will change in the LilyPond syntax.
Therefore, the development version is where all the fun happens — but also the risks.

The stable version is released when every major bug has been dealt with, as we've seen in last week's issue. Once it is released, we will keep it up to date as long as we can, but only for modifications that do not break anything nor change anything major: no new feature gets implemented, for instance.

When a bug is reported with a stable release, developers fix it on the development branch as soon as they can; then, given the amount of changes that the fix implies, they have to decide whether this fix can be back-ported to the stable branch or not. In other words, not all reported bugs will get fixed on the current stable branch — in case our users want to benefit from all the bugfixes, we release a development version every couple of weeks.

The LilyPond Feature of the Week

This week's feature is untested, undocumented and unavailable until the next development release; I was originally planning to mention it as "Idea of the Week" but Reinhold Kainhofer has just announced he had officially "committed" it; so here it is!

You may remember that a few weeks ago Neil Puttock, Nicolas Sceaux and myself had made a snippet that allowed you to add text indications to metronome marks, using a separate function:

Well, thanks to Reinhold and Neil this feature has now been included in LilyPond, and is even integrated with the default \tempo function:

A tempo indication in general is either a text markup, a note=count or both. So far, lilypond only supported the note=count type of tempo indications in its \tempo command.

This patch extends the \tempo function to include an optional text string too, which is displayed in the tempo mark. It allows any of the following types of tempo settings:
 \tempo 4=120 \tempo "Allegro" 4=120 \tempo "Allegro" \tempo \markup{\italic \medium "Allegro"}
etc.

Kudos! I haven't yet taken the time to recompile LilyPond and test it, but Reinhold examples look quite promising:

All these lines were made using only one single function!

Next week, Reinhold Kainhofer will be the LilyPond Report's guest.

LilyPond's Companion projects

A few weeks ago, we've seen that LilyPond can be perfectly integrated in different specialized music projects; this week Dr. Peter Chubb, from the NICTA Australian Research Centre, showed us another example of such integration.

Hi folks,
Thanks to Lilypond and a lot of work from the team here, the
UNSW/NICTA robot clarinet won first prize in the Artemis Orchestra
competition.

What is this competition about?

The Artemis Music Orchestra challenges participants to create devices that play real musical instruments. [It] is an automated-music contest for technical students to demonstrate to the general public what embedded systems can do.

Among others, the contest rules specify LilyPond as the input format, and the instrument has to be unmodified.

The NICTA team chosed to build a Clarinet robot player. Peter gave us a few informations on that robot:

– it has a gumstix running Linux, that does Lilypond and some control loops, which talk to ...
– a robostix that controls fifteen solenoids, an air pressure modulator, a fake tongue and some fake lips to operate the reed.

According to Dr John judge, the project leader, "the Clarinet is a difficult instrument to play: [...] in the design of the mouth piece, we had to control the air pressure going into the clarinet, and the force on the reed".

Here's the result:

Quite impressive, isn't it? I also strongly invite you to have a look at the TeamDARE (Netherlands), who won the second prize with a much impressive guitar robot.

Peter Chubb was in charge of the music interpretation algorithms. In the process, this allowed him to write an "articulation" script that could greatly improve MIDI support in LilyPond.

I've been using Lilypond for many many
years --- you can see my work in Mutopia; I got involved to support Lilypond and for general clarinetty things (as I play clarinet myself).

As I told him that during my Algorithm-Composition series of interview, I had only met people who where quite frustrated with MIDI, he answered:

You're right, MIDI is very frustrating, but it's adequate for
instrument control ... and after all, that's what we need for a
robot. We chose MIDI as an intermediate language because:
– we could plug in a MIDI keyboard and control the instrument directly
– Lily already has rudimentary support for MIDI output.

Well, congratulations to both NICTA and DARE teams! And of course, many thanks to Peter; we're all looking forward to seeing some of your work getting implemented in LilyPond...

The Idea of the Week

This week's idea was suggested on the mailing list by a new user named David Griffel:

I'm transcribing some early classical prints. They use flags which are
bolder and clearer than modern flags - at least for 8ths and 16ths.

Would there be any interest in having this as another style of "ancient
flag"?

To which Luis Jure added that this was not only useful for ancient music engraving:

Actually I've been wanting to ask this for a long time, but not
related to old notation styles but rather to more "contemporary"
practices: it has been quite common for the last 50+ years to use
straight flags (see Stockhausen's Zeitmasze, 1956).

A few people started to investigate this, mainly Reinhold Kainhofer (again) and Werner Lemberg, who's our specialist when it comes to font and glyphs issues.

The good news is that LilyPond already makes it quite easy to define and use different flag styles, as Reinhold pointed out:

Actually, adding different flags in lilypond is quite simple: Simply use
\override Stem #'flag-style = #'flagstylename
and lilypond will use the characters "flags.flagstylename[ud][3456]" as flags.

Now, to implement straight flags, all one has to do is to create glyphs named flags.straightu3, flags.straightd3, etc.!

Well, I hope this will get implemented; these flags look really nice :-)

The Quote of the Week

This week's quote is from Hu Haipeng, our "blind musician from China", as he calls himself. A couple weeks ago, we already mentioned his contribution to paper sizes support; this week he told us a bit more about the place LilyPond takes in his life:

I'll thank you all for working hard on this software. I'm deciding whether I must go to New England Comservatory in USA to study composition, because the deans of my school refused me from learning this when I entered it as their only blind in 1999. Lilypond is the only tool for me to write music. It really helped me a lot!

Such testimonies make me feel happy, and somehow even proud to be (remotely) part of the LilyPond project. This reminds me of what Han-Wen and Jan once wrote in the Documentation's Preface: "Probably the most important motivation is that our program actually does something useful for people."

As we've already seen two weeks ago, LilyPond is particularly convenient for blind musicians. Perhaps it could become even better? For quite some time now, contributors have been investigating possible ways to implement some support for Braille music output.

Ralph Little is the one who's gone the furthest at the moment; he once told me that one interesting fallout of this feature would be to provide access to the Mutopia project to Braille users, as well as to allow blind composers to compose in Braille.

And this concludes the thirteenth issue of The LilyPond Report.

Cheers,
Valentin Villenave

[1] Speaking of Processing, if you're interested in it and/or want to test your whole new Firefox 3 you might want to have a look at John Resig's website, where he has entirely ported it to JavaScript/Canvas. It's very impressive too.

[2] Some Linux distributions offer an impressive support; not to mention RedHat, that is a company, the community-based Slackware distribution is probably the most impressive: security updates are still released for its seven-years-old version 8!

The LilyPond Report #12

Valentin Villenave — 2008-06-16T23:22:21Z

Welcome to this twelfth issue of the LilyPond Report!

This week, we'll talk command line on Windows, LilyPond development releases, MusicXML import, and friendliness vs usefulness; we'll also present (yet) another PostScript snippet, and a mysterious disease... the LilyPnod syndrome!
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

This Week's Desultory Editorial

Greetings,
This week, an old mail from Heikki Johannes Junes, and a patch I proposed made me want to write a short tutorial about something many Windows-users are not familiar with: the command line.

Click here to read the full tutorial.

For a couple of decades, the use of graphical interfaces have made computing a pleasant experience, and we all are familiar with such concepts as mouse pointer, windows, menus and icons. However, text-based interfaces are far from obsolete, and even the best software in the world is actually a command-line program.

If you're running a GNU/Linux, a Mac OS or a BSD operating system, you should easily find a terminal emulator application; this is because all these systems are based on some common concepts.

If you're running an MS-Windows system, it is more difficult. You can either browse the Start menu > Accessories folder and look for some "command" application, or you can simply go Start menu > Run and type "cmd" (a shortcut for "command").

(Yes, everything is in French on my computer; but the commands are the same.)

Here it is: the big, scary, black window with white letters.

As you can see, it looks much better on GNU/Linux, by the way:

Under GNU/Linux, the Terminal emulator allows you to launch any program you want (actually, it even allows you to launch many more programs than you could ever launch using only menus). On Windows, it only works with very specific programs, such as ftp, or... or lilypond, of course.

But first, we have to learn how to browse our folders and files. You basicalle need to know only two commands:

cd (meaning "Change Directory") allows you to move into your folders and subfolders;

dir (on GNU/Linux, it's ls, meaning "list") allows you to see what's in your folder.

Let's assume I have a LilyPond file somewhere. Is it in "My Documents"?

cd "My Documents"
dir

(Since there is a space in the folder's name, you have to enclose it with double quotes.)

Obviously, this folder is empty. Is it in "My Music"?

cd "My Music"
dir

We have two folders here: "lilypond_scores" and "best_scores". Let's move to lilypond_scores, but since the name is a bit long, we'll just type the first letters ("ly", for instance), and then hit the "TAB" key. The full name should appear: this is a very convenient feature called autocompletion, that can avoid you many typing mistakes.

cd lilypond_scores
dir

Okay, it's not the good one. To move back to the parent directory, we'll type two dots. Then, instead of cding into the "best_scores" folder then typing dir, we'll directly type dir, followed by the folder's name (we give the folder's name as an argument to the dir command).

cd ..
dir best_scores

Hum, nothing there. Oh, I remember! A few days ago, I created a movie_themes folder in "My videos"; the LilyPond file I'm looking for has to be there. Let's use one single command to go up to "My documents", open the "My videos" folder, and finally open the "movie_themes" subfolder:

cd "..\My Videos\movie_themes\"
dir

Here we are
[1]. Let's launch lilypond, wit a bunch of options. I do not want to get a .ps file, and I want the output to be in png format.

lilypond -ddelete-intermediate-files -fpng "the Great Theme.ly"

Voilà! Granted, it's much easier to just double-click on a file, but that would prevent us from using all these nice options. Besides, as we'll see below, some functions such as convert-ly or musicxml2ly are only available from the command-line.

To learn more about everything one can do with the command-line on *nix systems, you may want to read this excellent tutorial by Ladislav Bodnar.

Have fun!

News from the Free World

If you happen to have a SourceForge account, and haven't yet taken part in the 2008 Community Choice Awards, you only have a few days left to vote for LilyPond as "Best project" (you can use the button at the bottom of this page).

In the news, you may have already heard about the Firefox Download Day which is about to start within a few hours as I'm writing these lines. Already more than 1 million persons have pledged now; all you have to do is give a mail address and download Firefox 3 the day it's released, making it the most downloaded software in 24 hours.

As I've already written, I kind of dislike this third version of Firefox. However, when it comes to such events, it's all about the let's-show-them instinct, and personal tastes don't matter so much. Let's show them! Let's show the whole world how much we like Free Software, and let's demonstrate our freedom of choice, our power as citizens of the Free World!

Speaking of power, one last piece of news for this week: World's fastest supercomputer runs Linux. The IBM Roadrunner can run at 1.7 petaflops; it's built with the same AMD Opteron processors you could find in your computer, and the same IBM cores you can find in Playstations 3. It perform nuclear-weapons-stuff calculations... and it runs Red Hat Linux.

What's up with LilyPond?

Another week, another Development Release (this is how Linus Torvalds announces every kernel release on the Linux Kernel Mailing List). Last week we've seen that the LilyPond developers are getting ready for a stable release; the latest 2.11.49 version is clearly a new step in this direction.

As I told you some weeks ago, I lack convenient statistics to make my point, but there's been an obvious change in the development rhythm for the last couple of months. More bugs have been fixed in this timeline than in the previous months, and the software itself seems more and more stable.

I could notice two "waves" of bugfixes in the last two months:

– sixteen issues were labeled as "fixed" for 2.11.46 (whereas only one was fixed for 2.11.44, and five for 2.11.45); this is mainly due to Han-Wen's bug-fixing-marathon on May the 1st, and to the fact that .46 was released nearly a month after .45;

– six issues were labeled as "fixed" for the latest 2.11.49, including some severe ones such as Issue 535 and Issue 60 (whereas only one was fixed for 2.11.47, and three for 2.11.48).

The LilyPond Feature of the Week

A mail from Jonathan Kulp on our mailing list gave me the idea of this week's feature:

I decided to try musicxml2ly, and I have to say WOW!!

[...] I took two Finale files, one simple and one fairly complicated and long, and exported them as musicXML files, then ran the musicxml2ly command on them and finally ran them in lilypond, and I can't believe how well it worked. Major props to the developers for this!!

[...] This is an amazing improvement over what I tried before. THANKS!!! This is going to save me countless hours of work if I want to convert my many, many Finale files to lilypond. Awesome!

What Jonathan is referring to is the musicxml2ly Python script that comes bundled with LilyPond. As far as I can see, this script was originally written in 2004 by Han-Wen, and greatly improved in the last several months by Reinhold Kainhofer.

MusicXML is a XML-based interchange format for music notation. Though its licence is open and its specification freely available, it is mainly produced through commercial plugins, such as Dolet for Finale and Sibelius.

Recordare, the company behind MusicXML, provides an example of MusiXML file. I happily downloaded it to test the musicxml2ly script.

Here is a simple how-to for Windows users...

The file has been saved on the desktop ("Bureau", in French).

Launching the Command line (Start menu > Run > type "cmd" > OK).

Let's move to the Desktop...

Let's call the musicxml2ly script, and tell it which file to process.

Here is another way to do it, more complex (you'll probably prefer the above one):
– call the Python interpreter (python),
– tell it which script it has to launch (musicxml2ly),
– and which xml file it has to process (elite.xml).

OK! Let's check the Desktop: a new LilyPond file has appeared...

Oh, while we're at it, let's run LilyPond too:

Now we can compare the original output, produced with Finale 2005:

And the LilyPond output, much much nicer (though the tempo indication is printed twice, which is easily corrected):

Native MusicXML import that works great? We have it. Any questions?

The Snippet of the Week

You may have noticed that I love to embed PostScript drawings in LilyPond files; just a few hours ago, Stefan Thomas gave me a new opportunity to play with it:

Dear Lilypond-users,
is there a possibility to integrate symbols for percussion-beaters and sticks in a lilypond-file?

Well, here's the result of my (ugly and hairy) PostScript code (looks pretty cool though):
Percussion beaters

I think I'll never get tired of this feature... :)

The Quote of the Week

LilyPond is famous for it's extremely active mailing lists, especially the lilypond-user mailing list. Are these lists overcrowded? If you want to keep up with all the discussions, read all the mails, this can indeed take you quite a lot of time.

This week's quote is a dialog between Trevor Daniels and Graham Percival, the two guys in charge of the Grand Documentation Project.

Graham — The whole point of GDP is to reduce the burden of doc maintenance. I expect that maintaing the [documentation sources] will be about 2 hours per month (not counting reading email), and the snippet maintenance to be 1 hour per month (again, not counting email).

Trevor — [Reading emails:] This actually takes up most of my available time at the moment. I've seriously thought of unsubscribing
from -user, but then I'd quickly lose touch, so it's not really an option.

Graham — Speaking from personal experience, I vigourously remind myself that answering questions helps one person a lot, whereas working on the docs helps a lot of people a moderate amount. _ [...] Yes, it's not a friendly attitude to take, but I'd rather be helpful instead of friendly.

I'd like to have an opinion on that, but I don't really know what to say. I have mixed feelings: on one hand I do understand, from a rational point of view, what Graham and Trevor are talking about; on the other hand helping people, meeting people, knowing people is what I like in the LilyPond community.

That being said, when you've been around for as much time as Graham has, you understandably tend to get tired.

I guess I've never really been the "big picture" kind of guy — and therefore, we need both kinds of thinking: people who actually see the big picture, build a towering structure (rock-solid code, reliable documentation, etc.) and people who don't know anything about anything but just feel enthusiastic :-)

The Syndrome of the Week

How tired is Graham? Actually, he might ver well be more tired than any of us: in an attempt to study his condition based on scientific observations, I searched the -user and the -devel list and discovered...

The LilyPnod syndrome!

What's the LilyPnod syndrome? It's when you're getting so tired of writing the word "LilyPond", that youcan't spell it right anymore:

– April 22. 2007: When you run lilypnod-book you should see something...

– November 09. 2007: The LSR->lilypnod docs link has the potential to simplify the Doc
Editor's job...

– January 06. 2008: Fortunately, lilypnod
is extremely flexible...

– March 3. 2008: otherwise lilypnod-book will never find foo.tex...

– March 4. 2008: since John's familiar with lilypnod-book, I figured it was worth mentioning...

– April 19. 2008: Just process everything with lilypnod-book...

– April 24. 2008: the automatic paper formatting stuff that
lilypnod-book does for our manuals...

– May 23. 2008: all lilypnod commands that accepts pieces of text inside "" also accept \markup...

Graham Percival seems to be the only person affected so far (I didn't found any single mail that wasn't from him. Plus, the syndrome seems to be progressing: though he's been around for six or seven years, the first symptoms appeared in 2007, and it's been getting worse lately.

If you ever find yourself writing "lilypnod" in a mail... please get some rest!

And this concludes the twelfth issue of The LilyPond Report.

Cheers,
Valentin Villenave

[1] Windows is the only operating system where the backslash is used instead of the slash character. However, the slash is accepted by the command prompt as well — but it does not allow for auto-completion.

The LilyPond Report #11

Valentin Villenave — 2008-06-09T13:03:11Z

Welcome to this eleventh issue of the LilyPond Report!

This week we'll see what is going on on the development mailing list; we will also talk about Free Culture, silly contests, paper sizes and cross-compiling, and have a look at the upcoming next stable LilyPond version!
As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

This Week's Desultory Editorial

Well, getting back to normal was not easy after last week's huge and pandemoniac special LilyPond Report, that took me nearly a month to make.

After having read it, Trevor Bača asked me to keep an eye on my website's statistics:

probably you've just created an important starting point in the research of algorithmic music and compouter-aided composition ... and the traffic sources to the page will probably bear that out some weeks from now.

There was indeed some kind of an early bump in the stats (which, as I already told you, is always flattering); however I was struck by the lack of comments left below the "special" issue (except for Graham Percival, who raised an interesting point). Maybe many readers, expecting to find just another issue of the usual column, didn't have the courage or the time to read it, or felt too unfamiliar with the subject.

So did I myself, by the way; my point was precisely to imagine some kind of a bridge that could allow "normal" people like me to discover some esoteric stuff... I could have published this as a feuilleton through many installments of the Report; well, perhaps this will be possible for future investigations on other subjects...

During the weeks I spent being away (on the planet of computer-generated musics), many interesting things kept happening in the LilyPond community.

I couldn't keep track of everything, but I do remember a few things, so in this issue I'll try to give you some kind of a "previously on LilyPond" for those who'd have missed some episodes on the lilypond-devel list.

News from the Free World

Speaking of bridges, this week an interesting and promising one was created at the Creative Commons organization.

Une ébauche de logo pour le prochain timbre SEAM ?

(Je plaisante bien sûr. Malheureusement.)

There's a good chance that you already know the Creative Commons licenses, that provide "normal" people (again) to easily choose which freedoms they're ready to allow for their works; this user-friendliness has led CC licenses to be extensively used in all domains, from computers to art. Well, meet this new label: Approved for Free Cultural Works.

We've just added the seal you see at right to Creative Commons licenses that qualify as Free Culture Licenses according to the Definition of Free Cultural Works — Attribution and Attribution-ShareAlike. Public domain is not a license, but is an acceptable copyright status for free cultural works according to the Definition.

[...] Of course you don't have to agree with the definition of freedom used by the free software movement and the Definition of Free Cultural Works, and even if you do agree, there may be reasons for using a more restrictive license in some cases. However, this seal and approval signals an important delineation between less and more restrictive licenses, one that creators and users of content should be aware of.

This label relies on a definition of "free cultural works" proposed by the website freedomdefined.org, and based on the Free Software Definition from Richard Stallman's GNU project. It's a spectacular move, since until now Lawrence Lessig's Creative Commons and Stallman's Free Software Foundation had quite some differences.

However, this move was not really welcomed by some adepts of the CC licenses: among their main concerns, they fear this label might divide CC-licenses users between "good" and "evil", depending on whether the license they choose has the label or not. Quoting some mails I've read:

I used to see the different CC licenses as delimiting different _spaces_ of freedom. Now I feel like a _scale_ of freedom has been created, with this label at the top, and all non-approved licenses at the bottom.

Look at this website, freedomdefined.org: "Freedom defined"! Is it me or is this even worse than everything Orwell has ever written?

Hm, I would have been surprised if the Free community had suddenly ceased to be teared apart by debates and conflicts. This label, I think, is merely a reference, and a quite useful one for small projects or organizations in need of a legal frame: look, for instance, at the Mutopia project and its licenses; wouldn't it be much simpler (and legitimate) to simply use this label, instead of always having to specify "this license is allowed, but this one isn't, oh, wait, actually maybe it should", etc.?

Another piece of news: what makes a project a "good project"? Documentation, Usability, Support? Is LilyPond what you would call a "good project"?

This week, Oscar van Eijk has noticed that SourceForge.net was launching an event called "2008 Community Choice Award".

I'm involved in several projects (not Lilypond; here I'm just a
user :-) ), and IMHO LilyPond is an example project in usability,
documentation and support, so I nominated LilyPond in the category "Best
Project".

That looks really nice — I don't like SourceForge at all, but I do like such events for the community. Therefore I have put a link at the bottom of this page; feel free to click on it if you happen to have a SourceForge account, and if you approve Oscar's choice. You only have a few days left!

What's up with LilyPond? (1)

On May 30., Han-Wen Nienhuys, leader of the Lilypond project, stated:

The status is that we have 97 defects left, of which one regression,
and one marked high. After fixing these, I'll go through the list once
more to pick the low-hanging fruit. Then, from a bugs perspective, we
are ready to release 2.12.

A new stable LilyPond release? That would be a big event. Of course, if you are a bit familiar with the way LilyPond evolves, you know that the latest "development versions" (numbered 2.11.xx) already contain most of the features that are to be expected in 2.12. No Big Suprise, so.

But what are these features more precisely, and how will they make LilyPond 2.12 even better than 2.10? Here's a non-exhaustive list:

– New (mostly vertical) spacing code, that solves many more collisions

– Better support for microtonal music

– Better regression test system

– New glyphs

– New text markup commands

– New lines and spanners code

– Better page breaking algorithms

– Much, much better MusicXML conversion

– Better code for ties

– New dynamics engraving code

However, the most impressive change is to be found, of course, in the Documentation itself, that is being completely rewritten thanks to Graham Percival's "Grand Documentation Project". Note that LilyPond 2.12 will also make much easier to translate both the program and its documentation in other languages.

Speaking of documentation, Graham immediately answered that he'd rather avoid to make an official release before the GDP is more advanced:

In summary, you'd have to be crazy to be seriously contemplating a
release in the near future.

He added that this could be an opportunity to have a "beta" release state, as requested by some LilyPonders.

The Bug of the Week

When can a program be regarded as "stable" enough? You never know for sure that no annoying bug remains to be unveiled. Here's a short story I'd like to tell you, as the current BugMeister of LilyPond.

On May the 2nd, Daniel Tonda cross-posted a long mail to the bug and the -devel mailing lists. The subject of the mail was a bit verbose, the example he provided was not a minimal example; long story short, his mail didn't meet our bug report guidelines and I didn't have the time to deal with it immediately: therefore I bookmarked it and somehow forgot it afterwards.

On May the 3rd, another bug was reported by Stan Sanderson, and added to the tracker by Graham. Whereas Stan had noticed that all tablature-related regression tests
were broken, Graham's report only mentioned the banjo tablature test. Therefore, I confess I didn't pay much attention to it.

Oh, and if you wonder what a "regression test" is, click here...

Regression tests are used to verify that new versions are fully compatible with previous versions. Stan Sanderson is in charge of these tests and therefore plays an important role in preserving the quality of the LilyPond project.

2.11.47

2.11.48

On this page, for instance, you can see the differences between the last two versions of LilyPond, as shown on the left and on the right. You may not notice these differences; therefore the systems automatically highlights these (even the slightest ones) in blue.

While reviewing my bookmarked bug reports, yesterday I re-discovered Daniel's report, compiled his example, discovered that the stems were indeed missing. I stripped down his example to a minimal report and finally added it to the tracker.

Then Stan made me realize that it was the same bug that the one already in the tracker.

This was a major regression, and Han-Wen noticed it and fixed it immediately. It would have been really bad to release a "stable" version without supporting stems in tablatures, indeed.

The bottom line is
1) individuals fail. We all miss things, forget things, hide things.
2) the community is a wonderful way to compensate these faults, and make software live happily ever after. In this story, everyone, from Daniel to Han-Wen, has played a complementary role and, in the end, provided a balance to (mostly) my mistakes.

And this just feels great.

What's up with LilyPond? (2)

On a different tone, on May 26. Han-Wen also raised a debate about whether more non-programmer people should be allowed to take part in LilyPond development or not:

I have the feeling that the
number of people that need commit access is getting larger than it
needs be - we keep adding people that only do relatively small tasks.

[...] If we have lots of people
pushing to the master branch, a bad push by a single person will
affect all contributors.

I am in no place to judge this, being myself a non-programmer contributor with write permission to the source code (certainly the clumsiest and the most potentially harmful of all contributors, by the way). Therefore, I felt too personally concerned by what was at stake in this discussion (and not in a pleasant way) to take part in it.

However, when I try to have a neutral point of view on the subject, it seems to me that we deeply need fresh blood — and that means new people.

If these people could be skilled programmers with an extensive knowledge of Lily's secrets, that would be ideal; but I have this hope (this utopia, you might say) that even non-programmers can, in time, become experienced enough to do minor but sensible programming tasks: fix bugs, implement small features, etc.; I'm thinking about valuable persons such as John Mandereau, or more recently Neil Puttock.

Of course, most of these contributors mostly deal with documentation stuff, that could very well be developed on a separate area (a dedicated branch, or a fork, or whatever). John offered to keep track of such a system, to regularly merge the updated documentation into the main development branch. The discussion hasn't gone much further yet.

The Contribution of the Week

As I'm about to publish this Report, Michael Pozhidaev, a new user, has just introduced himself on the list:

I am blind programmer from Russia
and music takes a very important part in my life. Lilypond is just brilliant
software for me. Now I can make my own music publications by myself. Great thanks
to everybody, who made it possible!

Welcome Michael!

As a matter of fact, LilyPond is great for people with specific needs, and its text-based interface is particularly convenient for blind people. On the very last days of 2007, another blind person from China, Hu Haipeng, discovered LilyPond and has kept using it ever since.

Lilypond is written using ASCII, so it's the best way for blind to compose or read music without sighted people's help. I don't know whether there are many other blind people in European and American countries using Lilypond, but for me, this is a good thing.

This led to an interesting (though unsuccessful) discussion on the mailing list, since some of the documentation resources (particularly ornaments) were mostly available as undescribed pictures.

Anyway. This week, Haipeng took a really cool initiative: since some people had been discussing about various paper sizes for scores, he decided to contribute to LilyPond's development by implementing every paper sizes you can think of as pre-defined options in LilyPond:

I've also suffered from the paper size, and don't know how to handle them when writing oversized large orchestral scores like Mahler and Schoenberg did. Thanks to someone, who pointed out the Wikipedia [1]. I'm attaching my paper.scm, with most of the sizes copied from the page. Some may perhaps never be used in music output, but I added them as a preparation for accidental use. Hope it will be useful!

Now, thanks to Haipeng, you can feel free to use any paper size from various countries, such as

#(set-default-paper-size "government-letter")
or
#(set-default-paper-size "monarch")

:-)

What's up with LilyPond? (3)

On the lilypond-devel list again, John noticed an annoyance when trying to build LilyPond with the latest version GCC compiler. An impressive technical discussion followed, that proved that this bug wasn't because of LilyPond, not even because of GCC, but... because of the Intel x86 processor in your computer! According to Joe Neeman,

The summary is that you cannot expect sane behaviour with the double
data type using gcc (and apparently most other C compilers) on x86. In
particular, doubles may be truncated from 80 bits to 64 bits at any
time. Therefore, you can compare two (80-bit) doubles and find that they
are not equal only for them to become equal on the very next line.

This allowed me to have a look at the GCC bug tracker... And understand how happy I was to be in charge of LilyPond's tracker instead of this one :-)

The Snippet of the Week

A couple months ago, we briefly mentioned a nice feature that allows a voice to quote another voice.
However, some limitations remain in this function.

According to Gilles Thibault, a French user whom you may have met also on the English-speaking list:

the \addQuote function has three limitations:
– it isn't transpose-able ( \transpose c d \quoteDuring #"clarinet" s2. has no effect, for instance )
– it filters events: expressive marks and dynamics are not taken into account ( see \set Staff.quotedEventTypes )
– You cannot specify from where you want to extract the music.

Actually, it was not written to extract music from an existing expression.

So, here's a solution provided by Gilles:
Extracting unmodified fragments of a music expression

Nice work!

What's up with LilyPond? (4)

When you want to download and install LilyPond, you can choose about any version you want depending on your computer and your operating system. How is it possible? Do you think LilyPond authors have a bunch of different computers, running different operating systems?

Er, indeed they probably do, but all LilyPond versions are actually built on a single computer, running a single operating system (GNU/Linux), thanks to an unified build process developed specially for LilyPond, called GUB.

Grand Unified Builder (GUB) is the mini packaging system that we developed for building LilyPond binaries. It cross-compiles several packages and assembles them into a single package.

A month ago, Mark Hanlon, contributor of an interesting Music Notation project, asked on the list if it was possible to build binaries for Windows and Macs in Linux.

John answered: Of course, this is exactly how released binaries are built: binaries are cross-compiled on a GNU/Linux system for all platforms (although it's a
bit more problematic for MacOS 10.5).

21 days later... Mark: I am using the latest gub, started from scratch on Sunday
night, first ever gub build!

Han-Wen: You realize that you are probably the first or second person beyond me
& Jan to build a binary in GUB. Congrats!

Indeed; building LilyPond for linux is already an involved task, but building the whole set of GUB versions can be really challenging. For instance, have a look at the new technical discussion about compiling flags that it raised...

The Quote of the Week

On its very first issues, the LilyPond Report taught you how to curse, Graham's style.

Well, recently Francisco showed us how to beg, genuine LilyPond style:

From my humble position, I beg you please, please,
\repeat unfold 10 { please, } to make possible that translated docs are updateable after release.

Lily-talk: a new trend of geekness is born :-)

And this concludes the eleventh issue of The LilyPond Report.

Cheers,
Valentin Villenave

[1] By the way, I'm happy to see that Wikipedia can somehow be accessed in China...

The LilyPond Report #10

Valentin Villenave — 2008-06-01T18:38:23Z

Welcome to this special tenth issue of the LilyPond Report!

This issue is entirely dedicated to Algorithmic Composition. It regroups some pieces of information found in various papers, on various websites, and among the LilyPond community. I'd like to particularly thank Torsten Anders, Trevor Bača, Jaime Oliver, José Henrique Padovani and Andrea Valle for their help and their knowledge of the subject. As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.

[sommaire]

The (Special) Editorial of the (Special) Week

Greetings,
from its very beginning, one of the Report's purposes has been to introduce many LilyPond-related projects I used to call LilyPond's "companions". This week, we'll try to adopt a less Lily-centric point of of view, and see how integrated LilyPond can be among other music applications.

Therefore, this issue will be entirely dedicated to algorithmic music languages. It is largely based on contributions from Torsten, José, Jaime, Andrea and Trevor, and many papers, websites etc. Where I originally intended to write just another article, the subject proved to be so complex that it progressively turned into this huge entire column.

To make reading easier, it will be divided into (hopefully) digestable pieces, than you can read one by one or just browse. Please remember that this is just a (somewhat messy) introduction, written by a non-specialist. All I intend to do is to to share some of my discoveries with you, in a very non-scientific way; if you're really interested in this stuff, please follow some of the links I've gathered here.

Here begins the story of my journey through the world of computer-generated music...

Day #1: When computers write music...

— Day one.— Algorithmic music in the LilyPond community.

I recently realized that many people were involved or simply interested in interfacing LilyPond with diverse experimental music representation languages:

– Bernardo Barros is trying to translate SuperCollider output into LilyPond scores,
– Karim Haddad is pursuing a similar goal
with OpenMusic,
– Jaime Oliver is working on a Pure Data export,
– José Padovani translates music from Common Music to LilyPond,
– David Psenicka's Fomus has a built-in LilyPond export, which allows non-free languages such as PWGL to be converted into LilyPond scores,
– Graham Percival uses the Marsyas sound analysis software to produce native LilyPond code,
– Torsten Anders has developed his own constraint system, Strasheela,
– Andrea Valle has been working on a complex algorithmic system that integrates (using a Python wrapper?) LilyPond, SuperCollider and TeX,
– Víctor Adán wrote a set of Python modules, named Cuepatlahto, for creating Lilypond scores in a conceptual approach,
– Trevor Bača has a similar project named Lascaux; these two projects have been merged into the Abjad project, that is still in active development.

Quite an impressive list — and yet I may very well be missing somebody or something. Anyway, I have to investigate this promising stuff.

All I want is something to write in next week's Lilypond Report...

Day #2: This strange thing called composing...

— Day two.— Automated composition vs Computer-aided composition

What is this strange thing called algorithmic composition?

As far as I can see, it is a way to create music using a series of mathematical operations.

Today, Trevor Bača told me something I had never realized: "Algorithmic" composition doesn't necessarily imply composition with computers: composition could be plenty algorithmic centuries before computers showed up: it's customary, in fact, to see references to Mozart's dice games, the canonic transformations of the twelve-tone row, and even French isorhythm à la Machaut at the beginning of a great many articles, books and dissertations on algorithmic composition".

When I try to think about it, I realize that, de facto, mathematics are a vital part of any music language (and particularly Western music from the past five centuries, where this relationship has been getting stronger and stronger — as the music was more and more precisely written? The art of polyphony, from gregorian counterpoint to baroque fugue and contemporary serialism, relies on mathematical principles. It's even more visible when it comes to rhythmic structures: even the crappiest techno music is basically made of $4^$ n measures.

I have also read a 2003 essay by Torsten Anders; this made me understand the distinction between automated composition and computer-aided composition:

– Automated composition "is usually expected to compose music as fully as possible [...] to allow scientific verification by comparing the original and the model-composed music.".

– On the contrary, computer-aided composition is still an artistic act. I remember Trevor also tried to explain the same idea: When graphic designers sit around and use Photoshop, are they engaged in "algorithmic design" or "algorithmic photo manipulation"? Really stop and think about this question for a moment. This helps us imagine how the compositional can be somehow computer-assisted without being fully algorithmic.

OK. Somehow, this makes sense.

Day #3: Teaching music to computers?...

— Day three.— Basic historical approach.

Miller Puckette

Today, thanks to José, I've learned about someone important named Miller Puckette. I have to show you his official picture. Apparently, this man has written one of the most important piece of software for contemporary music.

I have begun to read his book The theory and Technique of Electronic Music. At the beginning, he says "electronic techniques to
– record,
– synthesize,
– process, and
– analyze musical sounds [...] came into [their] modern form in the years 1948-1952, but [their] technological means and artistic uses have undergone several revolutions since then".

It's interesting to note that according to him, sound recording, synthesis, processing and analysis share a common historical ground (that merely corresponds to the beginning of modern computer science).

A 1956 advertisement

Indeed, the first approach to algorithmic composition seems to have been based on statistical analysis of existing music. I have just found the following glorious pamphlet, proudly published in 1956 and entitled A Machine to Compose Music:

Since the functioning of the human mind may seem to be beyond analysis, the thought that a machine could compose music might seem absurd at first. However, by the use of statistical analysis and the application of information theory, such a machine could be built.

This first orientation was clearly what Torsten referred to as automated composition. This approach, as far as I can tell, originated alongside with the Artifical Intelligence utopia [1]; it was this era where humans wanted computers to compete with them, either by writing fake Bach scores or by playing chess against them.

The 1956 advertisement mentions the use of statistics applied to music. This is particularly interesting, since this kind of analysis seems to be the common ground of most automated composition experiments since then. In a 1996 essay on Grammar-Based Music Composition, John Mc Cormack reports that "A diverse variety of stochastic processes have been applied to music composition. Most involve two stages:
– First, some existing musical expression or quantity is analysed.
– Following the analysis, re–synthesis is applied using the selected stochastic technique".

Statistics and randomness: that's about all a computer can do to pretend becoming an "intelligent" composer [2]. Quoting McCormack again, for instance, "one popular approach to computer composition is the use of Markov chains [...] Most attempts using this form of modelling first require some existing composition to be analysed".

Interestingly, none of the people I've interviewed so far ever referred to the concept of "intelligence" any more. Visions seem to have changed during the past 15 years; a paper by Bruce L. Jacob, written in the early 90s, expresses this progressive slide, by referring to the computer as both an intelligence and a tool:

The question what is algorithmic composition is analogous to the question what is artificial intelligence. Both are hotly debated. Neither are answered easily. Both ask fundamental questions about the authorship of ideas.

[...] Algorithmic composition is as old as music composition. It is often considered a cheat, a way out when the composer needs material and/or inspiration. It can also be thought of as a compositional tool that simply makes the composer's work go faster.

A tool.

In other words, let's not try any longer to teach the computer how to write music on its own, but instead teach it how to write what we want it to write. No more, no less.

Day #4: Chaos und Ordnung...

— Day four.— Randomness and creativity. Generating notes in LilyPond.

Honestly, I'm confused.

Randomness. Is that the real name of "creativity"?

I remember Mozart's dices. Computers are good at generating (apparently) random numbers. Does it make them good at composing too?

Today I've discovered this german guy named Karlheinz Essl. I tried to read his thesis in German, and was struck by a formula: "die labyrinthische Dialektik von Chaos und Ordnung" (the labyrinthine dialectic of chaos and order).

As far as I understand it, "pure" chaos (if there's any) couldn't be qualified as art: the composer has to put some order in it. The artistic process is precisely this delicate balance between randomness and rules.

No matter how, a computer has to be taught what it has to do, and how to do it. The purpose of a composition program is exactly to allow the user (the composer) to easily define such rules.

This implies, at some point, that the way the program is designed can influence the way you compose; according to Miller Puckette, "The design of the software cannot help but affect what computer music will sound like, but we software writers must try not to project our own musical ideas through the software".

For instance, LilyPond itself can compose music; have a look at the snippet entitled "Generating random notes".

\score { #(let ((random-state (seed->random-state 128))) (ly:export (make-music 'SequentialMusic 'elements (map (lambda x (let ((idx (random 12 random-state))) (make-music 'EventChord 'elements (list (make-music 'NoteEvent 'duration (ly:make-duration 2 0 1 1) 'pitch (ly:make-pitch (quotient idx 7) (remainder idx 7) 0)))))) (make-list 24))))) }

The code behind it tells Lily
– to produce exclusively single notes (make-music 'NoteEvent)
– with a fixed duration (ly:make-duration 2 0 1 1, that is quarter notes)
– not above the upper g ((quotient idx 7))
– choosing among all notes in the scale ((remainder idx 7))
– without any accidental (0)
– and to stop after having produced 24 notes.

GNU Solfege

Another similar example is the GNU Solfege application, created in 1999 by Tom Cato Amundsen, a former Swedish LilyPonder. Solfege is written in Python, and includes about four hundred pre-defined rules (and one hundred ready-to-use modules) to randomly generate scales, chords, harmonic progressions etc. Nothing LilyPond couldn't do... but in a much simpler way.

A twelve-notes serie in GNU Solfege

However, both LilyPond and Solfege are only suitable for relatively simple composing rules. As Jaime noted, "I suppose one of the things LilyPond is not suited for is to make algorithmic operations on musical material except for the usual transposition, doubling durations, etc..

Most programming environments are perfect for this because they use number representations. Therefore the need to be able to go from each representation to the other. I imagine that LilyPond users that are interested in algorithmic procedures would benefit of being able to create numbers and converting them into LilyPond code."

Day #5: Wind it up and let it go!...

— Day five.— Aesthetic approach. Meet Trevor Bača.

I had to ask Trevor.

Trevor Bača

There's one thing I noticed about Trevor Bača (besides the fact that he is a distinguished LilyPond contributor and sponsor): he often has a lot of things to say.

I remembered having read a long and interesting mail of his on the list in 2007; therefore a few days ago I asked him his opinion about algorithmic composition. Here's his first answer: "Hi Valentin, excellent topic!
I have a huge amount to say about this ... probably too much ..." — see what I'm talking about? :-)

OK, so, Trevor, what do you think?

— What matters to me is the model of the score — the "formalization", if we want to borrow Xenakis's term.
Lilypond has an incredible model... but one that isn't iterable or traversable or subject to the other composerly operations one expects of an environment for computer-assisted composition. LilyPond files are for input, not manipulation.

— That meets what José told me yesterday. Ok, so are you saying that you need specific tools?

— Yes. Five or four years ago, Víctor Adán and myself independently developed two different systems in Python, to give ourselves formal(ized) control over musical score. His was named Cuepatlahto, mine was named Lascaux.

— Interesting! That reminds me of the beginning of LilyPond...

— I flew to New York so Víctor and I could compare our codebases directly and we decided to unify out projects. The result is Abjad, a new system which takes the best of both Lascaux and Cuepatlahto and then cleans up the model and adds more.

— OK. I understand this system is still in development...

— Yes. Basically, it can be seen as an extension of the interactive Python interpreter, designed to let you build up larger and larger instances of notational elements like tuplets and staves, all of which know how to format themselves as Lilypond input.

It can be compared to some existing systems: for example, Chris Ariza's athenaCL models events in time and focuses on the creation of event-list stye list-of-floats output; Abjad models music score and focuses on the creation of hierarchically nested Lilypond output..

— Why creating such a system? What does it allow you to do?

— What's really interesting to me are not what computational *tools* composers use to compose but what composers are actually *doing* when we use computational tools. In my music I'm interested in building up massively composite textures and working out gestures with so many points of local change as to become difficult to manage by hand. I also (re)analyze my own musical material to extract fully or partially formed patterns to seed, grow, expand or conflict with new material in the same piece (or even in newer, later pieces). So I use computers.

I think what matters most is that some stuff is waaay to difficult to do by hand and that computers help a damn lot. It's much easier to manage, for example, a palette of 18 different types of attack in the flute, all floating around and interacting with each other in patterned ways if you've got a symbolic model hanging around somewhere. And it's also much easier to go back and ask questions of your own material such as "how many occurrences are there of consecutive quintuplet thirty-second notes between measures 31 and 90 that feature interval adjacencies not greater than a minor third?" when you have an instance of a data structure of your own material that you can iterate and interrogate.

— Hm. Let me rephrase: what does it allow you to do, concretely?

— In 1993 I started work on a piece that I would later call "Poème récursif". The original was a short 13-page score for 16 percussionists running 64 + 1 measures. Here's a page from that original version of the score.

1993 version

Twelve years later, in 2005, I wrote a second 'verse' of "Poème récursif". The dimensions were expanded to 64 parts deep and 256 measures long.

2005 version

You can immediately tell what role LilyPond played in the compositional process. The sketch from 1993 is completely hand-written, it took me weeks to write it out, even after the structure of the piece was determined almost completely. I remember wanting to go much deeper at that time, but doing so by hand was completely prohibitive: it meant working with many thousands of notes per page.

Enter LilyPond. Working with code on the front end and LilyPond on the back end, I was able to work through dozens and dozens of different configurations of all sorts of parameters, trying and trying and trying again until I was able to map, frame, fracture and recombine exactly the right parts of exactly the right structures to get what I wanted.

— I think I'm starting to understand... (am I?)

— The truth is that I've always been deeply uncomfortable with this process. What exactly do we convey when we mine algorithms or patterns? Searching for variations — whether by hand or machine — is one thing. But it seems to me that there is some sort of line we can cross over in passing to certain algorithms that are so ... massive, perhaps ... that they can generate entire pieces and also produce monolithic, even closed results. There's nothing wrong with these results, of course. But there's a certain type of sorcery needed to communicate effectively within the "wind it up and let it go" way of working. And I think that magic may yet be too black for me.

— "Wind it up and let it go"?

— I've stolen this bit from Andrea's way of talking about his own process. And I believe he's even referred to himself as "radically of the wind it up and let it go" mentality. Which goes to show that there are those among us that can cast that particular type of black magic ... :-)

After this Report was first published, I received a mail from Victor Adan, adding some comments to what Trevor had already said:

I would just like to add a comment about algorithmic composition in general.
The definitions I know always revolve around the *method* of composing, and so
debates arise as to which methods are valid as algorithmic and which are not.
Does using random processes make one an algorithmic composer? Does mapping cellular
automata to pitches? Does using 12 tone rows? Constraint programming?

It seems to me that algorithmic composition is as much an aesthetic paradigm, a way
of thinking about composition, as it is a methodology. In algorithmic
composition it's not so much the specific sequence of sounds or notes
that matters as it is the algorithm that produces the sequence. i.e. it is
as much (or more) about how a thing is generated as it is about what
is generated. Music is a byproduct—a measure, even—of the algorithm,
which becomes the artifact of main interest.

Thanks to both Trevor and Victor; good luck on your common project!

After having discussed with Trevor, there was one little thing I had to investigate. He had mentioned Christopher Ariza, that some of the other guys had already mentioned.

I opened his website, thinking I might found some other cool composition software I wasn't aware of...

And then I saw it.

This list of composition systems.

Er...

Let me count...

118 items!!!

Gasp...

Day #6: Lost in Implementation...

— Day six.— Common Lisp. Testing Fomus.

So many programs; so many systems; so many people...

One of the first things Torsten told me, a few days ago, is that "there exists an amazingly large number of algorithmic composition systems". I could never have imagined how much this statement was true...

However, Torsten did also provide me with a good angle to start: "Various programming languages are used, but Common Lisp is kind of the lingua franca of algorithmic composition (I assume for historic and cultural reasons)".

Indeed. A few weeks ago the Report mentioned Igor Engraver software, that is written in Common Lisp; there's also quite a few music-dedicated languages that are based on Common Lisp, such as:

– Fomus, a translator of algorithmic composition output into different notation formats (Lilypond, MusicXML...)

– Common Lisp Music, a sound synthesis system

– Common Music Notation, a notation system, like Lilypond though less advanced in default notation quality

– Common Music (and its father, Schottstaedt's Pla), a composition system

– Compo, a composition system

– Patchwork, a visual programming composition system (user programs graphically by connecting boxes with patch chords) — and its two derivatives:

– OpenMusic,
or the
– non-free [3] PWGL.

Hard not to get lost, isn't it?! [4]

What makes Common Lisp so attractive? According to Torsten, "Common Lisp is a compromise which brings together the various features of several now obsolete Lisp dialects, it is a *huge* language" [5].

There's more: Common Lisp is a high-level [6], multi-paradigm [7] language.

For instance, José Padovani told me what Common Lisp (and particularly Common Lisp Music) allows him to do:

José Padovani

I generate some codes that write down a lilypond code. Then, I run LilyPond to see my "experiments". These codes are not organized in a library like Fomus, as I am not a programmer, but a composer.

In a composition of mine, I'm using Lisp code to quantize some onsets from a spoken text. The quantizer is very particular, since I want the measures to begin according with the accented pulses, etc.

In another project, I have written a lisp code that creates some "textures" with genetic algorithms and writes down the result for 18 string instruments, with some quantizing rules. What is nice is that, firstly, I had initially used these functions to generate senoid grains; then since I liked the result, I decided to do something similar for string orchestra.

Well, that made me want to get my hands dirty. I decided to try the first program in the above list: Fomus.

— Testing Fomus —

Fomus was developed by the former LilyPonder David Psenicka; as I said, it's written in Common Lisp (though José told me its author is recoding it in C++). It's name stands for FOrmat MUSic, and according to its description "The purpose of it is to facilitate the conversion of "raw" algorithmic output data into readable music notation, a process that can be frustrating; [...] FOMUS attempts to remove this frustration by making a reasonably intelligent attempt at spelling notes, quantizing offsets and durations into readable rhythms, laying out the information into voices and staves". It supports the following output formats:
– LilyPond
– MusicXML
– Common Music Notation
– MIDI

So, many export formats. This choice, as far as I understand, is what Torsten refers to as an "output-format-neutral music representation" [8]

Fomus is released as a compressed archive; after uncompression, one can either load Fomus inside a Common Lisp interpreter, using
(load "load.lisp") and then (use-package :fm), or install it as a standalone application through an ./install.sh script (unlike LilyPond, it does not contain its own interpreter, so one might want to install, for instance, the Gnu Common Lisp implementation).

As a matter of fact, I couldn't make it work (there seemed to be a problem with the AutoAccidentals code). However, I did find some interesting snippets in the documentation:

Most of Fomus remains yet to be documented. However, it looks very interesting, and its syntax looks so familiar that I wish it could be entirely implemented in Scheme, and therefore provide a nice superset to LilyPond. This way, one could use it as a more user-friendly standalone and cross-platform application. (I was running Windows when I tried it, and it was a bit painful — not to mention it kept assuming I should have Adobe Acrobat installed).

Moreover, it seems to support only LilyPond 2.4 export (when we were still using the TeX backend). A nice project, that needs interesting ideas less than resources to implement these... [9].

Day #7: Of sounds and notes

— Day seven.— Meet Andrea Valle. Electroacoustics and algorithmics. "Fluid" systems.

Today I received a mail from Andrea Valle.

I'm actually using a very eclectic approach, sometimes using Python, sometimes SuperCollider for "gluing" stuff together, sometimes my output is LilyPond, sometimes graphical notation.

So, I'm a bit skeptical about a general system, as I change my solutions on a piece basis.

Andrea Valle

Browsing Andrea's MySpace and Youtube page, I understood that he's interested in very different music languages, including acousmatic systems.

I had almost forgotten that many (if not most) of the contemporary music created in the past half century is actually electroacoustic music, made of sounds rather than notes [10]. "Many algorithmic composition systems are also genuinely sound synthesis systems", as Torsten made me realize.

In an article that is to be published soon at Nime2008, Andrea makes some interesting assumptions about the relationship between elctroacoustic and algorithmic music:

Algorithmic musical composition has been proposed and practiced widely
starting from the '50s. [...] An interesting shift in perspective has occurred roughly from the '60s up to present day.

The first approaches to algorithmic composition were driven by instrumental scoring. But [...] the idea of a purely algorithmic approach, in which a strict formalization rules the whole composition process, is no more pursued in its integrity and has migrated from the instrumental domain to the electroacoustic one.

And why is that?

In fact, considering the final output of the composition process, while in the electroacoustic domain the synthesis of the audio signal is a trivial task per se, in the instrumental domain the generation of musical notation still remains a very difficult task.

This notational issue has prevented the diffusion of real algorithmic practice for instrumental composition. Such an approach, in which the composition process is turned into a completely algorithmic workflow –from the first idea to the final score–, can be defined as Integrated Algorithmic Composition.

You would have guessed that this "Integrated Composition" thing is what Andrea Valle (the "wind it up and let it go" kind of guy, remember?) is most interested in.

How does he achieve that?

Instead of starting from a solid framework where to insert
modules...

(That would be the Common Lisp approach we've already mentioned.)

[... it is possible to start from an indefinite variety of
available modules to be plunged –when necessary– into an
open environment. Such an environment
is fluid because it is intended as a glue capable of attaching
different modules together.

In other words, it's no more about implementing modules inside a single program, but about writing a "meta-program" (a so-called "wrapper") around several completely different programs.

A Python wrapper for graphical notation

This method can be used for dealing with randomly-generated notes (stochastic process), or for reacting in real time to some events, for instance an audio input:

A composition project involving parameter extraction from audio signals.
[...] was to use as starting material an excerpt from Sophocles' Antigon, which was read by a philologist so to respect as possible the reconstructed Greek classic pronunciation. Three voices sing melodies generated from data resulting from the analysis of the original audio file, in particular from the fundamental frequency and the first two formants.

The Praat software has been chosen for the analysis task. The SuperCollider application has been chosen both as system glue and as an audio module: as a language, SuperCollider is rich in data structure, highly expressive, provides an interface to the OS environment, allows for string manipulation; as an audio server, it provides state-of-the-art sound processing.

And for the music score output? Let me give you a clue — its name is Pond...

Click here to listen to a sample.

I'm not sure if electroacoustic music is really more "purely algorithmic" than music involving notes and scores. Though I haven't studied the question as Andrea has, I have the feeling that most electroacoustic musics, at a some level, rely on composer's arbitrary choices as much as any other music (the structure, the gestures for instance).

What is true, however, is that most facilities dedicated to contemporary music around the world, deal with sound processing and synthesis as well as they deal with algorithms and programming. Some of the most famous of these are:

– the IRCAM in Paris, France (where Miller Puckette used to work, and where the Lilyponder Karim Haddad works too, as far as I can see),

– the CCRMA at Stanford's University, USA and

– the ICCMR in Plymouth, England, where Torsten Anders is involved.

Day #8: The Max paradigm

— Day eight.— Miller Puckette, Max Mathews. Testing PureData.

Remember this guy named Miller Puckette? A few days ago, José told me to read a paper of his, entitled "Max at seventeen". Who's Max? It's actually a software developed by Puckette at the IRCAM in the 80s. It was named after one of the most important programmers in the past 50 years: Max Mathews.

Back to the 50s — you know, when some people wanted machines to write music like Bach? Well, some other people actually began to regard the computer, not as a brain, not as a tool, but as a... music instrument.

As soon as 1949, the CSIRAC (Australia's very first computer) was able to play digital sound. However, it's in the USA that Max Mathews developed the MUSIC programming language, that is, according to José Padovani, the "father' of all modern audio programs (for instance, the popular Free Software CSound, created by Barry Vercoe, one of Mathews' students).

In the 80s, Miller Puckette (one of Vercoe's students, again), at the IRCAM, began to imagine a new style of programs, designed for "real-time computer music performance".

How do you define a music instrument? Do you see a piano as a 88-keys keyboard and three pedals? Well, to Puckette, "in a multi-tasking environment we could describe a piano as 91 tasks, one for each key and pedal. The performer—the piano user—chooses in which order to trigger the 91, and how many times they will be triggered".

Do you see where this is going? Max is mainly about defining some events, and then triggering them while performing. It's a real-time oriented application, that can deal with pre-recorded sound samples, but also with notes, as you will see.

Many music software are oriented towards real-time use. These includes SuperCollider, which the LilyPond Report already mentioned in a previous issue, and ChucK. As far as I can see, however, both are more designed to deal with sound processing and synthesis than notes.

— Testing PureData —

Therefore, I decided to install Pure Data (pd), which is the Miller's open-source implementation of Max.

Since I was running Windows (again), it allowed me to notice how cross-platform is pd (whereas the original Max was only developed for Mac) — as far as I can see, it relies on the Tk library. When you launch it, and start a new project, you see nothing but a blank window. According to Puckette, this is a deliberate choice:

On starting Max, the user sees nothing but a blank page—no staves, time or key signatures, not even a notion of note," and certainly none of instrumentalvoice" or sequence." Even this blank page carries stylistic and cultural freight in at least one interesting respect: the whole idea of incorporating paper in the music-making endeavor was an innovation of Western Art Music, for which many other musics have no use at all. So say we all... OK, let's try to put something on this page: As you can see, some various objects are available but it is mainly about {boxes}. A box can contain a musical event, or trigger another event, or both. In fact, {you build your own interface}, the way you want. I couldn't go any further, but let's ask YouTube to do it for me:
Please have also a look at [this video->http://www.youtube.com/watch?v=2aXxgICaZwU] to see how the same principle can be extended. The computer as an instrument. That could be... Well, that could be actually {fun}, to quote Miller Puckette: Computer science has never found a metric for determining whether or not a computer program is fun to use. [...] We speak ofplaying" a violin, not working" it. If using a computer program feels like working in a bank or a hamburger chain restaurant, musicians won't (and shouldn't be asked to) do it. [...] The computer should ideally feel in the musician's hands like a musical instrument, needing only to be tuned up and then played. {{{Day #9: The composer as a software user}}} --{Day nine.}-- Lexikon Sonata. Real time vs "Offlineness". GUI vs CLI. Yesterday's examples demonstrated sound sampling from real recordings. However PureData/Max can also deal with MIDI format, and therefore generate its own notes -- which brings us back to algorithmic composition. Quoting Miller Puckette, who seems quite frustrated with MIDI (but who isn't?): because of the limitations of 1987 (you could have a GUI or a programmable DSP engine but not both in the same address space) Max became aMIDI program.'' There was nothing fundamental in Max that was derived from MIDI, however; as far as Max was concerned, MIDI was an I/O interface and nothing more.

— The Lexikon Sonata—

The other day I told you about Karlheinz Essl. I had to show you a composition of his: the Lexicon sonata, that has been composed (and performed) using Max.

I am impressed by the overall expressiveness of this work. You can distinctly hear (maybe feel) dramatic progressions, musical gestures, tension etc.

The output, as far as I understand, is entirely in MIDI, though the sound is actually not so bad. There's nothing pre-recorded here, everything is generated on the fly, and it clearly demonstrates what real-time musical gestures can be made of. Please have a look at the web page where he explains it all: the way music events are named, for instance, is particularly interesting. Esprit, Joyce, Clouds, ...

— Real time—

This ability to create events while performing, like you would do when improvising with an actual music instrument, is what is commonly referred to as a "real time" process.

What does real-time change, in terms of user experience?

Jaime Oliver

Jaime — Well, changing parameters while listening makes a great difference for me.

The fact that it is a real-time application doesn't contradict that it is a quite complete programming environment able to save information in different representations. So being real-time doesn't imply that it can't produce information that could then be used 'offline' or use 'offline' information in real time.

As a composer/computer musician, my personal interest is to be able to switch representations, and of course one of them is the musical score as a mean to communicate with other musicians, especially performers with traditional occidental instruments. This means to use algorithmic techniques to generate melodic, harmonic or rhythmic materials and output them into LilyPond.

José — Although it is very powerful for designing real-time systems, PD is not suited to non real-time symbolic processing of scores, or symbolic music structures of the western tradition.. What I mean is: a "quarter" or a "Staff"doesn't mean much in PD.

I use PD very much, but not for non-real-time music purposes: it's convenient for real-time sound experiments and for interactive music projects (like compositions for instrument and live electronics).

— Can you really create music scores using a real-time program?

Jaime — Pd allows for both sound processing and music scores creation: athough it doesn't have a score format as such, you could represent a 'musical score' in many ways, the most simple ones being lists, arrays and text files. I've seen Miller's attempts to make music scores and they are not even close to LilyPond's functionalities; his attempts are more a temporary display.

Therefore I am currently developing an external that takes pitch-duration pairs and converts them into a LilyPond score.

Torsten — As José mentioned, PD was not intended originally for creating notated scores, but many other composition systems are, and most of these output into Lilypond.

By the way, creating Lilypond scores this way is much more easy than, say, creating a Finale file — the textual format of Lilypond is well documented and relatively easy to create automatically.

One other aspect I like about creating Lilypond files automatically for algorithmically created music is the ability to annotate the score with analytical information which was part of the algorithmically created result.

— Is there a strong, fundamental design difference between "real time" and non-real-time music software?

Torsten — Is there a strong difference between improvisation and composition?

There are systems which focus on sound synthesis, systems which focus on interactivity with various means, systems which focus on notated score output, etc. Some are addressing multiple concerns, but nevertheless these concerns differ greatly in their needs.

José — These past years, the two trends are getting closer. PWGL, for example, is capable to process and synthesize sound in real-time and has created a particular own notation app.

Marcos Alessi Bittencourt managed to make a score box inside PureData, using a C translation of Common Music Notation.

It would also be very nice to visually represent Common Lisp functions in PureData (possibly using Open Sound Control), like PatchWork and its derivative do. There's a Lisp interface object for Max, and now a Java implementation as well...

— Interfaces—

— José, you were talking about visual representations; there's indeed a fundamental difference between text-based programs, and graphical programs such as Pure Data or OpenMusic... Don't you guys like to have a visual output?

José — I am interested in visualizations of all kinds. For instance, I have used some of Common Music Markov functions to create Computer Assisted Poetry, based on poems of Carlos Drummond de Andrade (a well-known Brazilian poet), and have generated some visualizations of spoken texts in Processing, a Java based language for visual arts (though it is more visual oriented, it has some librarys to synthesize sound and can be linked with PD, SuperCollider or Max/MSP with the Open Sound Control protocol).

Trevor — I'm extremely dependent on actual notation when I work, and I know for a fact that I would never have been able to write my most recent pieces without some sort of computational front end (the code) and some sort of music notation back end — and that back end is LilyPond.

Torsten — The overall goal is similar in various systems: to provide a rich toolbox for defining computational music theory models.

But whereas some put particular emphasis on making these tools accessible for a non-programmer, code-based programs address a different audience: users which have either a bit programming experience or are willing to learn that. (Though they still don't need to be professional programmers: I am a musician too.)

I fully understand that visual programming is beneficial for many users which have often little programming background. However, for my own purposes a visual programming language does not "scale up" — complex music pieces simply end up in spaghetti :)

Day #10: Musics from the Emerald City

— Day ten.— Torsten Anders. Rules, constraints. Testing Strasheela.

My journey was coming to its end, but I couldn't leave without paying Torsten Anders a visit.

As I already told you, Torsten works at the ICCMR in Plymouth, England. On the ICCMR's website, one can read:

"We are chiefly interested in computational modelling of Music. We are developing new intelligent musical systems that will be able to evolve their own rules for musical composition and ability to interact with musicians and listeners in much more sophisticated ways than the present ones can do. We predict the emergence of new kinds of music content, most of which will be generated on the fly, requiring new modes of representation, access and interaction."

Looks promising, doesn't it?

Torsten Anders

Please have a look at the list of publications by Torsten, that are all freely available [11] I've already quoted his 2003 essay:
Composing Music by Composing Rules: Computer Aided Composition employing Constraint Logic Programming;
the following quotes will be taken from there, as well as from his thesis
Composing Music by Composing Rules: Design and Usage of a Generic Music Constraint System.

As you can see through these titles, Torsten aims to deal with music "rules", and program "constraints".

Basically, the latter is nothing but the implementation of the former. Music is based on rules? Let's teach a computer how to apply these rules!

For centuries, compositional rules were an important device for expressing compositional knowledge in a music theory.

The attitude of constraint based computer aided composition is particularly close to the way many musicians and music textbooks are thinking or talking: musicians often describe, for instance, a certain compositional style by a set of compositional rules. The concept of constraint based computer aided composition is hence intuitively understood by many musicians.

Many musicians thus feel comfortable with a computational model based on the notion of rules. For example, rule-based approaches attracted much attention among composers, because by defining rules composers can formalise virtually any explicitly available compositional knowledge as a task which the computer can solve automatically.

What is a compositional rule, precisely?

For a composer, it is particularly intuitive to describe a score or a musical style by modular compositional rules.

A rule restricts score objects (e.g. sets of notes) and their parameters (e.g. durations or pitches). Frequently, a rule controls mutual dependencies between multiple score objects and parameters.

Compositional rules are particularly well suited to describe music, because rules describe the multi-dimensional nature of music in a modular way. When listening to music, we perceive various aspects such as rhythm, harmony, voice leading or instrumentation
simultaneously.

Similarly, the formalisation of a task as complex as composition is greatly simplified when the task is stated in a modular way. When the task description is broken down into rhythmic rules, melodic rules, rules on the harmony etc. then the various musical dimensions are formalised one by one.

The programmer's goal, therefore, is to allow the composer to define his own compositional rules, as "constraint satisfaction problems" (CSP).

Basically, we could see this program as a filter or a sieve for randomly generated events. Some of these events will be left random (as unknown variables), other will be constrained.

Since the program does not want to affect the composer's style, this randomness has to be the most generic possible: in other words, if you do not feed the system with any rule, it should be able to "invent" any music possible — his search space would be infinite.

A most generic music constraint system would allow its user to implement any music theory model conceivable. From the perspective of the present research, such a system is an ideal system.

The set of solutions for this extreme CSP (which is only theoretical) contains any conceivable score. The user can freely constrain any unknown information in order to reduce the set of solutions as desired.

So, it's not about starting from scratch and making music, but on the contrary, starting from an infinity of possible scores, and narrowing down these possible solutions to fit what you have in mind.

These rules can be applied in different ways:

Composers often prefer a way of working which is situated somewhere between composing ‘by hand' and formalising the composition process such that it can be delegated to the computer.

For example, the composer can determine some aspects of the music (e.g. certain pitches) by hand and constrain other aspects by rules. Alternatively, the composer may specify the high-level structure (e.g. the formal structure) manually and let the computer fill in the details.

[Deterministic constraints] can be used together with other compositional rules. [...] A traditional musical example which uses a similar technique can be found at the beginning of the first movement in the fourth symphony of Johannes Brahms. All intervals between the first eight pitches of the starting theme are falling thirds (or rising sixth) – thus forming a simple pattern of the pitch classes, which is further controlled by the underlying harmony.

This approach was already implemented in PatchWork and OpenMusic (through OMClouds, PWConstraints and Situation). Though I do not know anything (nor could I find any easily accessible piece of information) about these, it's time for me to introduce you to the result of Torsten's research: time to meet.. Strasheela!

Strasheela by Leonid Vladimirsky

— Testing Strasheela—

Strasheela is the name of the scarecrow in the Russian story The Wizard of the Emerald City. This software is actually written in Oz, so you might get the allusion.

Strasheela is one of the best documented projects I have seen and read about during this investigation. Not only are Torsten papers fully accessible, as we've already seen, but Strasheela's documentation itself is nice and looks very promising — LilyPond's Documentation Editor himself, Graham Percival, is credited on the site; could that be a coincidence?

Launching Strasheela through Emacs

Strasheela needs an implementation of Oz to work; the only one available is Mozart, easily usable through the GNU Emacs editor. This solution seems to be more or less cross-platform, but I only tested it under GNU/Linux.

I earlier referred to Common Lisp as a multi-paradigm language; so is Oz, though it was originally conceived following a very specific paradigm: "concurrent constraint programming, which subsumes concurrent, logic, and functional programming", which is the reason why Torsten chose it in the first place: it makes Oz/Strasheela the first "special constraint programming platform
for music composition".

There would be many things to say, for instance, on the way random notes are generated and filtered (what Torsten calls the "search process"). This requires such a huge amount of computing that Strasheela gives the ability to define a so-called "search-strategy" that can make the process quicker and more efficient.

Another thing I'd like to briefly mention is the internal representation of music in Strasheela. From what I understand, its nature is actually double: precise textual representation vs abstract data typology (ADT). Here are some examples.

In the following snippet, you can see a simple melody and its textual representation.

On a more abstract level, here's a motif declaration from Beethoven's 5th Symphony:

motif (durations : [2, 2, 2, 8] pitchContour : [=,=,-])

Now for some Constraint Satisfaction Problems.

In the following example, we want to obtain a symmetric twelve-tone series:

Torsten kindly provided me with an example demonstrating a microtonal chord progression in 31 equal temperament. "Besides demonstrating a formal idea on creating such chord progressions, it shows how analytic information can be added automatically to the LilyPond score".

Harmonic progression - 31ET

You can also see the code behind it, and read this discussion about it.

Additionally, I just have to show you, in bulk, some cool functions I've noticed in Strasheela examples:

hasDiatonicPitch hasUniqueMelodicPeak isConsonant_lessStrict isConsonant_strict palindrome markovChain

Like Fomus, Strasheela is output format neutral. It can export in MIDI, CSound, LilyPond; it can also be interfaced with Fomus, to obtain MusicXML files.

I can't, obviously, sum up all of its functions here (that would imply that I've fully understand it, which I have not). But I have to say I have been seduced by its promises.

Having installed it, I unfortunately couldn't make it work at all. I kept getting a "broken pipe" Operating System Error, which was very frustrating.

However, please have a look at these examples on the website, and maybe you'll be seduced as I've been:

– This example implements the classic music theory, as defined by Fux in 1725;

– A very impressive Counterpoint example involving rhythm parameters as well;

– An example of Strasheela's Real-Time capabilities: when interfaced with SuperCollider, Strasheela can compose a counterpoint or an accompaniment to a melody on the fly.

This latest feature is really impressive. As far as I can see, Torsten wrote about it with the scientist/composer Eduardo Miranda, who works at Plymouth too.

Making Strasheela capable for real-time performances makes it an extremely valuable and plastic tool; as we've seen, many composers like to instantly hear or see the result of their work, even if they're not performing per se.

And what about the user-experience? After having seen how convinced Torsten was about the utility of text-based interfaces, and moreover in a very specific language, I didn't expect him to consider other possible implementations... But here's what he has to say about Strasheela's future:

Porting Strasheela to a language which is more widely-used in the computer-aided composition community or which offers special support for unexperienced programmers makes the system more easily accessible.

An example of a widely-used language is Common Lisp, because there exist a considerable number of Lisp-based composition systems Visual languages enjoy much popularity among musicians (especially data-flow language such as Max and its relatives).

It would be interesting to investigate how visual programming can facilitate the use of a generic music constraint system like Strasheela in a way that scales well even for complex constraint problems.

If this were to happen, Strasheela could play a major role in tomorrow's computer-aided music creation!

Conclusion

Well, this very long and very special issue of the LilyPond Report is now over.

I'd like to finish it by quoting... Miller Puckette, who else:

Certain forward-looking music educators are investing many hours trying to encourage students to build their own computers. I hope someday to see an international culture of home-built computers and homemade computer music software.

[...] Communities are necessary for knowledge to grow, especially with non-commercial operating systems. But I can imagine a future in which computer music expertise (including how to assemble a machine, install an OS, and run software) is at least within a village or two of most people of the world.

[...] People almost anywhere can or soon will be able to get hold of a computer and involve it in their music-making; [...] this will engender an increasingly audible change in the music of the world.

This meets my own personal conviction: more than ever, we citizens of the Free World have the power to decide what the future will be made of.

Well, I hope this Report allowed you to make as many astonishing discoveries as I've made myself while preparing it; nevertheless I look forward to getting back to the weekly column you've been reading so far.

On an anecdote level, this series of discussions led José, Torsten, Trevor and Andrea to decide to create an open platform dedicated to algorithmic composition. Will it be a Wiki, a forum, a mailing list? None of them knows yet; but I'm looking forward to see what they will come up with — and be assured that the LilyPond Report will give this event a proper announcement when it happens!

And this concludes the tenth issue of The LilyPond Report.

Cheers,
Valentin Villenave

[1] Interestingly, David Cope refers to it as "artificial creativity" or "musical intelligence", both expressions being derived from the concept of "Artificial Intelligence". According to Torsten, "Concerning music an Artificial Intelligence, particularly impressive results were obtained by him".

[2] An interesting quote by Brian Ferneyhough:

All decision making (I believe) has random elements. These can be constrained in various ways. It is the constraint system which transmits a sense of order. There are computer programs which can rapidly write you a symphony in the style of Mozart: what they are patently unable to do is come up with the flashes of perverse insight which makes a piece REALLY Mozartian. The Imp of the Perverse is our true spirit guide.

[3] This point raised a comment from Torsten:

"PWGL is freely available, but not open source (its creator Mikael Laurson obviously had some bad experience in the past when IRCAM took his PatchWork sources and turned it into Open Music — without even acknowledging him). I would *not* call it non-free, though it is not open source".

Well, that unfortunately happen to be the definition of non-free (as in freedom). I regret what happened to Laurson; I don't know what was the license used in PatchWork (I couldn't find any documentation available about it), but if it was a strong license such as the GPL, the IRCAM could hardly get away with it.

[4] I wouldn't dare to make a silly joke here... Oh, yes I would: What can you say when you meet a deaf algorithmic composer? — Answer: you say "Read my Lisp" :-)

[5] Interestingly, the old Lisp was precisely used for Artificial Intelligence...

[6] Quoting Torsten, The programmer works on a very high level in this language. By contrast, programming in a lower-level language like C or C++ requires that the programmer has to care for many details himself and cannot define many things so concise (so the code gets much longer). BTW: Even more lower level is assembler, which is only used today when maximum speed is required. Something in the middle between Lisp and C are languages like Java or C#. And there is a clear tendency towards higher-level languages, more recent languages which inherited many features from Lisp are Perl, Python, Ruby, and Groovy.

[7] Unlike, say, the Scheme or the C++ language (both used in LilyPond), that are respectively used for functional and object-oriented programming, Common Lisp is truly multi-paradigm: in other words, you can use it to program the way you want. Quoting Torsten's 2003 essay, "The expressiveness of multi-paradigm programming is probably also one of the reasons why Common Lisp is often employed as the foundation for computer-aided composition environments".

[8] This led to an interesting discussion between Torsten and Trevor:

Torsten — The advantage of targeting a single format (Lilypond) allows you to support the full output format more easily. The advantage of a more general approach is that outputting alternative formats is more easy.

Trevor — You're absolutely correct that one of the major design decisions any system that functions as an 'automatic notation generator' must make is whether to generate format-neutral or format-specific output. And you're also right that the trade-offs are clear — wider potential applicability with format-neutral output and more granular control with format-specific output.
Early versions of my first software output MIDI and Csound scores before Lilypond output, but work on my actual pieces for the following couple of years minimized my need for those two formats and maximized my need to notation visualization (as Lilypond scores). So I guess my last project represents an organic move away from multiple-format output towards Lilypond only for practical reasons.

[9] As we've seen, Fomus supports many output formats. However, it's interesting to note that both José and Torsten confessed being mainly interested at Fomus as "a score translator that can write LISP generated music events in Lilypond.".

IMNSHO, it could be a sensible choice for Fomus' future to focus on LilyPond export.

But then again, I may not be completely neutral here :-)

[10] Admittedly, a note is also a sound, of a special kind. But still, you get my point.

[11] Though no license is specified, which makes these not redistribuable... and makes me guilty of multiple copyrighht violations :)

Right after having published this LilyPond Report issue, I received a number of comments and additions from some of the composers who contributed to it. Most of their Errata have been merged into the paper, but I'd like to quote here an interesting comment from Trevor:

It's been quite clear to me that there's a fascinating (if loosely-knit) ecosystem of composers working with LilyPond in different ways. But what's been absent up until now has been any public notification of this fact.

If — for a counterexample — you're interested in the OpenMusic tools available from IRCAM, then you'll quickly discover a very widely publicized community of composers and their projects. But this has not been the case with LilyPond.

I think your article probably marks the beginning of a small changing point, where it will now be clear to a public audience that there is in fact a dedicated collection of composers who have made LilyPond an integral part of their working process.

The LilyPond Report #9

Valentin Villenave — 2008-05-12T18:53:46Z

This week's issue discusses open-source hardware, accordion support, intermediate files, LSR downtimes and bug statistics. I'd also like to announce that next week's issue will be a very special "LilyPond Report", entirely dedicated to... oh, you'll see. As always, you can post your comments at the bottom of the page, or even register and contribute to the LilyPond Report's next issues.[sommaire]

This Week's Desultory Editorial

Greetings,
well well, this is already the tenth issue of our LilyPond Report (since the first issue was numbered as #0), and so far I think the experiment has been pretty successful... and the success is actually yours much more than mine: without all the interesting conversations, ideas, contributions that are constantly to be found among the LilyPond community, none of this would be possible.

Of course, keeping up for ten weeks or for fifty-two weeks is a completely different story, and I have no idea whether the LilyPond Report will still be alive in 2009 or not; honestly, this is the kind of questions I'm trying to avoid at all cost right now.

So-called "Newsletters" mainly exist in large projects such as GNU/Linux distributions, and most of the time these are published on a monthly basis, or are kept sparse (sometimes these are nothing but a collection of links). This is why, although Graham Percival once referred to the LilyPond Report as a "newsletter", I never intended to actually run a "proper" newsletter, and I hope this way it will be more fun, more flexible, and more likely to last.

To see what kind of things I have in mind, please come back next week and have a look at the official #10 issue: thanks to some distinguished LilyPonders, a very special and interesting Report is indeed being prepared. Stay tuned!

News from the Free World

Software can be free, we already know it: using a GNU/Linux distribution you can have a whole Operating System entirely free (as in "free speech"). Drivers are a bit more problematic: for instance, your 3D accelerator chip or your WiFi card may require you to use non-free drivers or firmwares. However, can the hardware itself become free?

For the past decade, countless initiatives have been aiming to make it happen. The OpenCollector project is trying to list them, though their diversity makes it unbelievably difficult. Interestingly, this website refers to the Hippie movement more than once, and mentions that the tradition of Homebrew Computers dates back to the 70s.

Open-source hardware includes amateur and professional projects, from plain do it yourself to huge communities or corporations. Exactly like free software, these projects have in common an ability to exist without any guaranteed income, a cooperative development that relies entirely on the Internet and a constant commitment to making their specifications available to the public.

Open-source MP3 players are generally proposed as DIY kits: let's mention, for instance, Frank's, Butterfly and other kits.

Open-source game consoles are a bit more involved. You may know about the Hydra kit, that is absolutely not free; I'd like to mention the French DOGS project, that looks quite interesting, and, on a much more professional-looking level, the GP2X. We can also expect more projects to appear in the future.

Open-source graphic cards are very promising. The OpenGraphic project, started by Timothy Miller, is showing us a promising path; their first chip is now ready and available for pre-order. However, even major companies seem to make their specifications available, so things are moving really fast here.

Open-source processing cores, similarly, are now fully functional. The OpenCores project has been started to achieve "the ideal of freely available, freely usable and re-usable open source hardware". As noticed in a Free Software Magazine interesting paper, "Open hardware goes mainstream": even the Sun Microsystems company has made their Sparc processor publicly available, under the name of OpenSPARC.

The UltraSPARC T1 chip

OK, I could stop here. BUT... this is precisely where things are getting more interesting. What if... anything could be free (as in "freedom")?

We already talked about Free Beer. There are also
– Free Robots,
– Free Prosthetics,
– Free Particle Accelerators,
– Free Bikes and Bicycles,

The C,mm,n prototype

– no less than three different Free Car projects: the Open Source Green Vehicle, the C,mm,n project, and the OScar — all three have impressive designs!

– And tomorrow... Free Space exploration?

While the world we know is often about short-term benefits, there are people working everywhere to invent something else, something new, something Free.

In two weeks, we'll have a look at a project Andrew Wagner told us about some days ago: the Do-It-Yourself keyboard.

The LilyPond Feature of the Week

Last week we had a look at LilyPond's support for Bagpipe; this week I'd like to mention support for Accordion.

As Graham noted in a recent mail, both "the bagpipe and accordion stuff [...] have no special code-features; they're all done with "extra" .ly files". (You can have a look at these files in your LilyPond installation directory; however most of the time, you'll just want to \include them in your score.)

However, this is only partially true, since the LilyPond "Feta" font itself includes some special glyphs for the Accordion.
Accordion support is not well documented yet; however you can have a look at some LSR snippets to see some possibilities:

– Diatonic accordion (this snippet includes a macro for jEdit)

– Discant symbols

– Hopefully more to come...

A few months ago, David Kastrup explained in a long mail how support for accordion could be improved. His request has been added to our tracker, and is now waiting for someone to handle it... If any skilled programmer/accordionist ever reads this, well, you know what to do!

The Idea of the week

This week's idea has been around for quite a while.

As you may remember, LilyPond originally produces Level 3 PostScript files, with a .ps extension. These files are afterwards turned into PDF files, that most people are much more familiar with.

So, here's a simple idea Laura Conrad once mentioned, before Graham summed it up:

I agree with Laura: we should treat the .ps files as temporary and delete them.

[...] I think that deleting the .ps files is a good default option. Most users don't want ps, and many users who investigate the ps files won't know how to deal with them properly. Anybody who really wants a ps file can invoke with —ps.

Indeed: seeing "exotic" files appear on your computer is always confusing for new users, particularly for Windows users who not only get an unwanted .ps file, but also a .log file as well.

Granted, there's already an option that does the trick. This require to launch LilyPond from the command-line:

Launching LilyPond from the command line

notice the —ddelete-intermediate-files option

Mmm... Not very handy, is it?

The LSR-downtime of the Week

Some of you have noticed that the LilyPond Snippet Repository was going through a rough time.

As you may know, the LSR is developed by Sebastiano Vigna and gracefully hosted at the University of Milano, Italy.

– On May the 1st, a broken router affected the LSR server. It was corrected in less than 12 hours.

– On May 6th an update in the Java code behind the LSR made it unavailable again; it was fixed a couple hours later.

– On May 11th, the LSR database got flooded by a too important amount of connections. It was fixed within 6 hours after reported.

This tells us two things: the first one is that we need the LSR more than ever: every time anything goes wrong, LilyPonders notice and report it. Secondly, this proves how dedicated and reliable Sebastiano is: well done Seba, and keep up with the good work! :-)

The Bug(s) of the (80 past) Week(s)

Several days ago, Graham Percival has stated that "we have a fair number of new people fixing some bugs". I don't know exactly what (or who) he meant, but I have to say that these past few weeks, I have indeed felt that we were achieving some kind of balance between the bugs appearing bugs being fixed — was this feeling justified? I had to investigate this.

Now for some statistics. Our Google bug tracker was opened on August 20th 2006, but for some time it only hosted fake (but amusing) test reports. It wasn't really used until 18 months ago, where it was interfaced with our bug mailing list.

Here is a histogram showing:
– in red, the total number of mails sent to this mailing list
– in blue, the number of mails that were automatically sent from the tracker
– in yellow, the ratio between these two curves.

Statistics of the bug-lilypond list

From October 2006 to May 2008

Hmm. Does it prove anything? I'm not sure. Mails are sent by the tracker every time an Issue is created, fixed, or simply commented, so these types of data lack reliability.
However, we can see three distinct bumps in January 2007, in September 2007 and in January 2008, that affect the tracker activity as much as the list.

There's something else. If you look closer at the histogram, you can see that, while the blue and red curve have been slowing down for the last four months, the yellow curve is starting to go up, as of April 2008. In other words, for the very first time, there is more activity on the tracker (i.e. more bugs discussed and/or fixed) than on the mailing list!

For example: since May the 1st, 4 new issues have been opened, while almost 30 issues have been dealt with on the tracker (most of them have been fixed; on May 1st only, Han-Wen handled about 20 issues on his own! — mostly ties and slurs-related, if I remember correctly).

So, is LilyPond in good shape? I do think so — even if we lack convenient tools to tell exactly how good.

And this concludes the ninth issue of The LilyPond Report.

Cheers,
Valentin Villenave