Digested

I once heard a radio show describing an ant that lives in the South American rain forests whose bite is so painful that all the creatures of the forest steer clear of it's nests. If a man is bitten, a multple day coma ensues where one is writhing in pain. The idea is that large animals in their carelessness can damage a nest, and the only defense against either purposeful or inadvertent damage is a bite that definitely gets attention.

Free software is paradoxically fragile like an ant that can be crushed, but if roused to bite can inflict serious damage. Think SCO vs. Groklaw. Anyone who ignores the license requirements of the GPL and gets a friendly visit from Eben Moglen learns that it is much easier to steer clear of license violations.

To a lesser degree, the Apple khtml saga illustrates the sharp edge of free software, and used effectively can keep others from inadvertently damaging a project. A little prick here and there can get needed attention.

Then there is the annoying prickishness of someone like Larry McVoy who seems to purposely stir up the nest for no reason.

I can't complain. I enjoy the slow relaxed pace of life in the Kootenays, so how can I complain when a thunderstorm takes out the ISP at 19:24:43 and service isn't restored till 02:46:56? Yes that was last night, Friday the 20th. Maybe nobody noticed until they came home from the bar? If there was a serious issue requiring parts replacements, the fixes wouldn't happen till monday sometime.

And of course when I saw that everything was down, I did the responsable thing; went to bed.

Does money changing hands affect free software?

I like money. The more the better. But then again, I'm not willing to pay the price necessary to get enormous amounts of money. I could probably double my income within a month or two, but that would require that I give up many other things that I enjoy, such as a quiet country lifestyle, my family, my faith, my hobbies.

In other words, the commercialization of free software isn't a problem. I'm sure that most KDE developers would love being able to work full time on what they enjoy doing. There are some developers who are paid to work on KDE, and a comment from Andrew Morton recently noted that most of the kernel developers are paid to do so.

There are certain aspects of commercialization, or rather certain characteristics of high tech business that can cause a problem.

Ownership: Obviously tight intellectual property control is anathema to free software. But what about when a commercial interest releases their software as gpl or similar? Who owns, as in controls, sets future direction, drums up developer support, maintains communication systems, etc. for the development effort? We have many examples of this situation that we can learn from.

Trolltech owns Qt. There is no question. When a KDE developer finds a bug in the Qt classes and sends a patch, Trolltech rewrites the fix. They want to maintain ownership of the codebase, all the while releasing it under the GPL. This is one of the more successful commercial open source stories. They get income by selling the libraries to those who want to develop proprietary software. They nurture the relationship with the KDE community by substantial contributions of developer time, and in return they get testing and a rather admirable case study for their product. Does Trolltech own KDE? They definitely influence KDE greatly. When Trolltech released Qt bindings for D-Bus, it probably sealed the decision for KDE to use D-Bus in future versions. This wasn't a decision by fiat however; many KDE developers were leaning that way already. There seems to be a healthy symbiosis between the two interests, and a common view of what constitutes good taste. It is in Trolltech's interest to maintain an excellent relationship with KDE, so we see good results. If that ever changes, KDE developers will need to take over maintenance of the Qt classes.

Note that I have no difficulty with Trolltech's licensing schemes. If someone wants to develop a proprietary application, let them pay the going rate for whatever libraries they need. Indirectly I benefit from that arrangement, since the revenue is put back into the Qt libraries or KDE developers are hired.

Easy Software Products owns CUPS. I am not as familiar with the business end, but the software is available under the LGPL and GPL. Many printer manufacturers provide drivers that work with CUPS, and they sell either software or consulting along with other printing and document products. They own the software, but it is made useful by the extensions of others.

OpenOffice. SUN owns OpenOffice and are constantly trying to get others to contribute. They will probably get as many contributions as Trolltech does to Qt. Not many. Why would I hire someone or contribute myself to something someone else owns? Realistically future OpenOffice development will have to come from SUN itself. How does the free distribution of OpenOffice contribute to SUN's bottom line? I understand from a business point of view why Trolltech releases Qt under a free license. Does SUN get enough revenue out of StarOffice to maintain development? This is a side effect of commercialization, one that again favors the monopoly.

Safari. Hah. Who owns khtml/WebCore? The answer is KDE/Apple. The twain don't meet. This is an interesting case study, and contains a salutary lesson for the next time someone shows up with a gift horse. The rather ham handed ways that Apple proceeded could have killed a less resilient project. As Zack so eloquently stated, the development of khtml changed from a rather interesting challenge to a frustrating parsing of questionable source code while fending off impatient unknowledgable users. I give full credit and thanks to those who have continued khtml development. Very strange pronouncements have emanated from various quarters, such as eskewing code quality for feature implementation. I have a hunch how far that would get in the khtml developer circles. How dare khtml reject the generous code offerings from Apple! There is a very interesting precedent to do just that. Remember the LVM/EVMS competition in the kernel? How the kernel developers rejected a working but inelegant solution from IBM?

When a commercial interest tries to take ownership of something or tries to get others to contribute to what they own, problems develop. Why should I work for someone else if there is no benefit that comes to me? IBM contributes to the kernel even if HP, SUN and other competitors may benefit, because they don't own the kernel. Trolltech, Novell, DataKonsult, Open Office Polska Company and others contribute to KDE because none of them own KDE.

Another aspect of commercialization is branding. I am interested primarily in one brand, Derek Kite. Whatever benefits that brand, I am interested in contributing to. I know that I alone cannot do everything, and recognize the value of community, so I am interested in the KDE brand. I can be made interested in other brands however. Contact me for my rates. In other words, no, I'm not going to contribute to a software project where a commercial interest takes all the credit. Unless I'm paid to do so.

The Linux desktop is maturing and becoming a viable choice for ever greater numbers of users. Commercialization will happen as people step in to fulfil demand for Linux desktop services and products. As a result many more people will be making a living from KDE. This will contribute to an increasingly rich software offering. But danger lurks. Something about software, particularly the desktop makes people stupid. Someone in the near future will make a play to dominate the free desktop market. They will do it by the old standbys of exclusion and differentiation. KDE will once again be attacked viciously for having the temerity to exist. It will turn out badly for many. Comparisons to the Unix wars of old will be made.

It's not money that is the issue. It is control.

I've got the basic statistics generating code for svn done. Some testing is in order, but it seems to work.

A few minor frustrations/learning opportunities. If you svn log -r [rev] [path] you get a nice formatted log. If you --xml, it spits it out in xml format, for which I've written a nice parser in php (bleh). So I figured in the name of code reuse and efficiency (laziness) I could use the php class to generate the statistics. Well, no. the --xml arg does everything as expected except include the lines changed count, which I need. So back to perl and lines like

if ($i =~ /^r\d*\s\S\s(\S*)\s\S\s(\d*\D\d*\D\d*\D\d*\D\d*).*\|\s(\d*)/) {

So as to keep my good name with the admins, and not be blacklisted while I run loops through a week's worth of commits, I set up the local svn mirror. rsync.kde.org::svnmirror has what I needed, so I did the rsync, went to work, came home, went to bed, went to work came home and it was done. So I tested with svn log file:///home/extra/kde_svn and got the same error that anonsvn.kde.org users got last week. The rsync seems to fetch the repos/db/current file near the end, which indicates the last commit. It was out of sync with the fetched revisions, hence the error message. A quick rsync fixed the problem.

How big is the KDE svn repository in fsfs format?

@:/home/extra$ du -hc kde_svn
12K     kde_svn/dav
1.6G    kde_svn/db/revprops
13G     kde_svn/db/revs
64M     kde_svn/db/transactions/411053-1.txn
64M     kde_svn/db/transactions
14G     kde_svn/db
8.0K    kde_svn/html
4.0K    kde_svn/locks
14G     kde_svn
14G     total

@:/var/www$ du -hc kde_repository
...
19G     kde_repository
19G     total

The second set of numbers is the cvs repository. Kudos again to the svn migration team for preserving the important stuff, but mucking out the unnecessary cruft.

ade commented on the history available within the KDE repository. I thank coolo and ossi for their efforts to maintain an accurate record of development. You cannot know where you are going unless you know where you come from. KDE is as much a social phenomenon as a collection of software. Other projects have at their center a mailing list, but KDE seems to revolve around the source repository. Hence the importance of preserving the history, and my decision to document what goes in.

I am in the process of (actually have a blank open document in Kate) rewriting the statistics generating code for the Digest. A blank slate is daunting yet liberating. What information should I gather, and how should it be presented? I ask for comments and suggestions.

Currently a perl script goes through the cvs repository, file by file, selecting commits between the dates passed as arguments. From the cvs logs, it gathers everything and outputs an xml file (may62005.stat).

The commit summary data is as follows:

totalcommits

totallines

newfiles

modfiles

numdevs

For each module the following data is gathered:

modulename

moduleatomiccount

modulefilecount (which doesn't make sense)

and each author has this data:

authorname

authorfilecount

authoratomiccount

authorlines

Which with a bunch of lines and html is used to create the statistics page in the weekly Digest. svn log creates a line of data like this:

r410589 | dkite | 2005-05-07 16:56:21 -0700 (Sat, 07 May 2005) | 1 line

Throw in the list of files per commit, a rather long running loop through all the touched files, spit out an xml file, et voila.

So after all that blather, let's cut to the chase. What do you want? Some people mentioned graphs. Graphs of what?

So much about free software challenges conventional wisdom.

I don't know for sure if my experience is representative in any way, but I suspect it may be.

I started my Linux experience with Redhat 5.2, then 6.idon'tremember. The rpm madness drove me to Debian, where I enjoyed solid engineering that characterizes the distribution. As advancements in the desktop, compiler and libraries left Debian behind I installed Gentoo. Oddly enough the long builds were very similar in length to the dial-up apt-get upgrades, especially using the KDE packages regularly put together by Ivan E. Moore II. To digress a little; Watching the effort Ivan put into the project moved me to find a way to contribute.

I remember wondering at the time how the free desktop would survive the demands of a less technically adept user base. Attempting to freeze a release in software that was under rapid development seemed to be a colossal wasted effort. It seemed an affront to my profoundly lazy nature. And to call something 'stable' that was frozen even a few months previous didn't make sense. Real bugs, real functionality, real usefulness had been incrementally added over that time.

So I did the only reasonable thing. I ran right off of cvs HEAD. Gentoo had easy ebuilds that made simple work of keeping up to date. I noticed two strikingly important things: the maintenance of the packaging was almost trivial since it pulled directly from the KDE repository, and, the experience was very stable. Oh there were the odd issue, mostly due to rapidly changing software. Most everything worked well, and the important improvements were experienced immediately.

This doesn't make any sense. I should have had failures, data loss, or at least a bad experience. I'm a realist. I knew I was living on the edge. But oddly enough, the most unstable and treacherous times were when KDE was preparing for a release. It should be the opposite. This in spite of the valiant efforts by Stephan Kulow and Dirk Mueller before him as release coordinators. It seemed that the release process broke something.

I don't think this is restricted to KDE. The Kernel developers recognized that fast moving development and attempts at bi-annual or annual releases was fundamentally broken. The distributions maintained (and paid for) a huge duplicated effort. And the kernels were still unstable. Now the releases happen regularly, new stuff is put in and deployed quickly. In general the stability has been reasonably good, again surprisingly.

What brought this whole thing to mind is my recent experiences. My gentoo installation, similar to my Debian and Redhat installations, got too big and ugly, and needed to be mucked out. The easiest way is to reinstall. Lacking time, I installed Kubuntu. It was quick and painless, and I got what I thought was a stable installation. Finally I was no longer living on the edge.

Or so I thought. I'm seeing behavior that I've never seen before. OOM killing, having to delete KDE configuration files. In fact my box isn't as stable as it was before. And I'm running into the frustrations of living with the builds done by others for their purposes. The audiocd ioslave doesn't do mp3's. Mplayer would need a forced install with all the attendant maintenance issues.

Sure I could wait for the bugfixes, or updates to the packages. A slight problem though. I already see fixes and features since KDE 3.4 that I need, that won't be in 3.4.1. No minor stuff either. For example the kpilot work by Jason vanRijn Kasper fixes some issues that I need.

Is it possible that the problems that I've described are from trying to apply conventional wisdom solutions to free software? Almost anything can be done by brute force, by applying unlimited resources to the problem. Free software doesn't have unlimited resources. The duplication of effort to try to package something unpackageable, to the satisfaction of no one seems crazy. To try to announce, organize, test and finish a release when everyone knows the results will be unsatisfactory again seems crazy. Sure, these things could be fixed with tighter discipline. Oh yeah, that doesn't work unless releases every three years is deemed acceptable. Maybe our headaches will stop when we stop banging our head on the wall. Trying to do something that doesn't work to solve a problem that doesn't exist to satisfy conventional wisdom that is wrong.

I have become a firm believer in the importance of source code distribution. The resulting installation represents what the designers of each project had in mind. The feedback mechanism is shortened, duplication of effort is avoided. The only problem is the computer resources and complexity. Gentoo is in some ways purposely complex, not desiring to coddle the users. That isn't necessary. And Moore's Law again comes to the rescue. How long does it take to build KDE on a well endowed AMD 64? That and the compiling speed improvements in GCC 4.x. Beineri's timings show a dramatic improvement in build times, with more speed improvements to come. Instead of trying to control the uncontrollable, organize the unorganizable by packaging everything neatly and instantly obsolete, maybe the solution is to build on the strengths of free software

In the interests of building community spirit, I propose that the KDE community adopt a song. I could be sung at the yearly conferences to set the tone, and I'm sure some enterprising programmer will figure out a way to have us all sing together over the internet.

The song would reflect the harmonious relations that exist within the community. The helpful suggestions when someone messes up. The consensus seeking when a decision needs making. The warm feelings we have towards one another.

I nominate the following, and will enjoy the nominations of others:

Sam Hall by Johnny Cash

I grew up with my parents playing Johnny Cash and Frank Sinatra. The music of my youth was Cat Stevens and Neil Young. Even when I knew everything and my parents didn't, I still listened to Johnny Cash. There was an honesty and depth to his singing that couldn't be ignored, even if country wasn't my taste.

Since my daughter left home I've enjoyed the opportunity to start listening to music again. I picked up Cash's last album, Cash American IV: The Man Comes Around. It comes with something everyone will dislike until they listen to it.

The title track is a gospel song that Cash wrote. The second track, Hurt, written by Trent Reznor from Nine Inch Nails begins to show the depth of this work. The original was raw and rough and noisy. Cash is if possible more raw while being quieter. The feelings and hurt from drug addiction portrayed in this song is quietly and powerfully rendered. Cash covers some other very familiar tunes; Bridge Over Troubled Water by Paul Simon, Personal Jesus from Depeche Mode, In My Life, a Beatles tune that Cash redefines. First Time I Ever Saw Your Face was sung by Roberta Flack. Nancie who has perfect pitch commented that his voice seems to go off somewhere from time to time, but you can't stop listening. In Desperado, an Eagles tune, Cash seems to be beseeching a close friend, with Don Hanley doing backup vocals. All these tunes start playing in my mind when I hear the title, but Cash makes them fresh and alive.

The other songs are older, some written by Cash himself, such as Give My Love to Rose, and Tear Stained Letter, Streets of Laredo which are old tear jerkers. He does a beautiful rendition of the Hank Williams classic I'm So Lonesome I Could Die with Nick Cave. Sam Hall is another old Cash tune makes me think that my attitude isn't so bad after all. Quite fun. And one of the most surprising is We'll Meet Again, sung originally by Vera Lynn during WWII, again redefined by Cash.

There is a raw feel to this album. Cash's voice has always been special, never technically perfect but full of feeling. This comes across in the sparse arrangements. Yea yea, I hate that stuff, it's so old, blech. As I said, there is something to hate for everyone on this album, until you listen. Corny tearjerkers that, oops, start you crying. Out of tune harmonies that demand another listen. How could you describe a song where fiddle, dubro and clarinet play incongruously in the background. Huh? I've got to listen to that one again.

Someone had some good sense to just let the man sing. He does.