The Dude abides.

For Sale: Kabinet Malaysia (Malaysian Cabinet) 2009

We were having dinner at Gandhi’s, an Indian vegetarian restaurant in Brickfields, where we noticed that the Malaysian Ministerial Cabinet is apparently for sale at Gandhi’s. We took some snaps:

On the Microsoft Office ODF support fiasco

The ODF-support-in-Microsoft-Office-2007-SP2-fiasco-and-its-fallout (phew, try saying that fast five times over!) is, from the perspective of this participant of the Great Document Format Wars, pretty silly.

Yes, its true that ODF (for version 1.1 and below) has had under-developed formula specification. Calling it undeveloped is being charitable. For those of us who toil by the craft of the code, the specification was pretty much non-existent.

Having said that, most officeware that support ODF worked just fine because they emulated the behaviour of OpenOffice, the dominant ODF supporting office suite. This state of being, while not being particularly ideal, wasn’t that big a deal because:

1. Reverse engineering ODF file format is really just an unzip followed by an xmllint (to format the XML) piped into less. I know this because I spent a few weeks writing a parser of the file format that would become ODF. It wasn’t particularly difficult then (I had ZERO reference documentation), it shouldn’t be difficult now (given tons of reference documents available today) .
2. First versions of standards tend to not be perfect. Despite the fear-mongering I’ve heard from various parties, non-perfect standards are actually perfectly OK. The idea is to get a reasonably decent standard out and improve it incrementally over time. If we were to wait for each and every standard to be perfected before being released, we’d get nowhere.
3. Formula specification support is slated for ODF 1.2. ODF 1.0 was the original major release, followed by a minor 1.1 release. With ODF 1.2, formula specification should be properly formalized. It must be noted that formula specifications require time to perfect (a lesson that Microsoft learned the hard way when it was revealed that formula specification in OOXML was less then perfect). Hence, the OpenFormula team spent a significant amount of time developing the extensive OpenFormula specification which ODF 1.2 will support.

Microsoft Office 2007 SP2 shipped with ODF support. The raging debate across the Interwebs is whether Microsoft intentionally or unintentionally broke ODF support as basic formulas have fail to compute across different applications. Doug Mahugh argues the Microsoft perspective that they had followed the standard to the letter. Others have pointed out that Microsoft’s offering is not compatible with other ODF supporting officeware.

To this observer, following the letter and not the spirit of an open standard seems to serve no useful purpose to users of the application other then to lock them into an open standard that is, for all intents and purposes, proprietary. Sure, ODF <= 1.1 has had substandard formula specification. However, most applications interop just fine, so why can’t Microsoft Office?

From an engineering perspective, attempting to replicate OpenOffice’s behaviour would have been the correct approach. Everybody knows about ODF’s <=1.1 substandard formula specification. This was due, in part, because Microsoft publicized the flaw wide and deep when ODF was being adopted as national standards (oh, the debates in national TC’s then were just so much fun </sarcasm> :p).

My point is that if you know something is broken, the correct engineering approach is to see how others have fixed it and to follow their footsteps if the fix is reasonable. As it so happens, others have replicated OpenOffice with decent results. If Microsoft had done the same, their users would be able to share documents between other ODF supporting office suites just fine. Now, that would truly be serving the spirit of an open standard!

On a related note, Gray Knowlton (who leads the product managment team for Office Developers) has a blog post on removing Rob Weir as the Chair of the ODF TC. He says:

“I’m not saying Microsoft (or anyone) should be the chair instead, but I am saying that Rob is unfit as a leader given his inability to separate his personal venom from his role as a leader in driving the standard forward.”

For those of us who have been following  the Great Document Format Wars, Gray is being disingenous. To a large extent, ODF has come as far as it has because of Rob’s (and other ODF TC members) fantastic work on it.

As for personal venom, it takes two to tango. Microsoft is certainly no blooming lily where personal attacks are concerned. After all, it takes two parties to build a healthy working relationship. On this, Gray cannot place the blame solely on Rob’s shoulders.

Given how ODF has flourished under Rob’s leadership, there doesn’t seem to be any meritorious reason for him to vacate the Chair.

On using fuse-encfs to keep stuff private

fuse-encfs is a user level virtual filesystem which, when mounted, stores encrypted data in one directory and simultaneously exposes the decrypted files in another directory. The advantages of a user level virtual filesystem are obvious:

1. Transparent to applications (ie, applications write to the filesystem and the filesystem handles all encryption/decryption)
2. No need to load kernel modules (and recompile them for newer kernel versions)
3. Available to all local users on your *nix system

fuce-encfs’s mechanism is simple enough. There is a decrypted directory and an encrypted directory. The decrypted directory is explicitly mounted. Once mounted, when files in the decrypted directory are written to, the changes are encrypted on the fly and appear in the encrypted directory. When unmounted, the decrypted directory is empty and the actual data is kept safe in the encrypted directory.

Installing fuse-encfs should be easy on modern *nix systems as it should come bundled. For example, in Fedora, I’d do:

yum -y install fuse-encfs

First up, we create the unencrypted directory (the mountpoint) and the encrypted directory

mkdir -p ~/encfs ~/.encfs

Next, we create the encrypted filesystem:

encfs ~/.encfs/ ~/encfs/

Choose standard mode and enter a password to protect the filesystem. That’s pretty much it.

Writing to ~/encfs/ will automatically encrypt data and store it in ~/.encfs. Once done saving data to the encrypted filesystem, simply unmount it:

fusermount -u ~/encfs/

EncFS provides a number of useful options (such as idle auto-unmount, multi-user filesystems and on-demand mounting). With simple scripting, it is therefore possible to protect private data easily and without hassle.

On the Malaysian community of “rockstars”

The Rails rockstar fiasco has had fallout all over the blogosphere. Perhaps one of the most insightful opinion pieces was written by J Aaron Farr, in his aptly titled “A Community of Rockstars“. Farr writes that:

The problem with the Ruby on Rails community isn’t simply sexism, unprofessionalism, or ghetto-ism, but that Rails is, at its core, a culture which values individual ego more than community. David and his posse have purposefully encouraged a tradition of edginess, individual expression, one-upmanship, and disregard for such fluffy, fake egalitarian notions as consensus, inclusion and good manners.

He goes on to compare the Rails community with the Apache Software Foundation community:

But beyond that, I have spent the last six years as a contributor to projects in the Apache Software Foundation and the differences between the Apache culture and the Rails culture astound me. Apache, like Rails, is a strict meritocracy, but I believe that may be where the similarities end. There’s nothing wrong with giving credit where credit is due, but the Apache philosophy asserts that collaborative peer development trumps rockstars. Rockstars are an anathema to the ASF culture. In the long run, a large, diverse community of contributors provides a stability and quality that cannot be matched by a benevolent dictator who may face conflicting interests, may be unable to scale with the community, or who might murder his wife.

…

If it’s unsurprising that a community of rockstars would gravitate to a hip tool like git, which favors individual forks, compared to a staid tool like subversion, which forces developers to work together in a centralized repository, then it should be equally unsurprising that the rockstars don’t care that they offend or drive others away. Their culture doesn’t value building a community. If it happens, great. But they aren’t doing it for you or anyone else. This is a community of ones.

Farr shows clearly that individual egos simply don’t work in building a community. Long lasting community building exercises are founded upon consensus, inclusion or good manners. Community building requires humility and strength of character, to be able to accept shortcoming of the self and improving on them as well as accepting the shortcoming of others and guiding them to contribute constructively. It is a demanding, selfless and herculean task and certainly not a responsibility to be shouldered lightly.

Having said that, community building is an area where the Malaysian free software champions can certainly improve upon.

Part of community building is being able to participate in qualitative discussions openly and discussing issues by putting individual egos aside. Community growth in Malaysia has been retarded by a phenomenon which shares many characteristics with that of the Rails community: an edginess to the character, misinterpreting feedback as personal attacks or negative criticism, name calling and lashing out, and lack of inclusiveness. These attributes make community building and honest conversations impossible. These are certainly not signs of a mature and thriving community.

Farr points to a solution:

Cultures which value diversity and collaboration over individual ego are required to cultivate an enticing environment. It’s difficult to attract a wide range of contributors if the environment is constantly abrasive. This is why you’ll find Apache members spouting off about the importance of consensus, inclusion, and equality amongst peers. We’ll even go so far as to suggest community over code. Perhaps we must do so simply because such virtues are not necessarily in our nature. Open source developers tend to be an ego-driven, strong minded, smart-ass bunch.

One is tempted to peer into the crystal ball and see how things go from here.

Perhaps a new breed of young ones will shun the current culture and build their own progressive culture. Perhaps the current generation will evolve and adapt to a more inclusive and egalitarian culture. Perhaps we will see a shift from disparate community to a bigger united movement with more stakeholders from the industry, universities and government.

All in all, a change in our culture would do the free software movement good.

Porting over generatedata.com to PostgreSQL + CSV bugfix

generatedata.com is a GPL PHP based web tool to generate test datasets. I needed it to run on PostgreSQL so I hacked it to support PostgreSQL. Get the file here. And here. There is nothing MySQL specific in the SQL, so  ideally the code should be restructured to support common databases. A data abstraction layer would be useful

In addition, the CSV output mechanism looked fubared for cases where the CSV enclosure and delimiter was not escaped, so I fixed it as well. Get the file here.

If I’ve time, perhaps I’ll find out why the frontend is slow. Clearly, something inefficient in the JavaScript.

Enjoys!

Mandatory disclaimer: It’s not my code. I have made only made minor fixes.