Beauty and the beast (on elegance versus performance)

When you are programming you have to choose between code elegance, clean code, versus code performance.

It starts with the programming language itself which you choose. Pick a low abstraction programming language, like C, and you'll have a lot of performance. You could go even further and write in assembler, for even more performance. But consequently the code will be large and hard to read. The program will tend to focus on the specifics of the machine or language, how to do it, instead on the specifics of the problem to be resolved, what to do. In other words, it will be low in abstractions. Otherwise, if you pick a higher abstraction programming language, an OOP or FP language, your program will be more succinct and readable, but you may get performance penalties. This time your program will be rich in abstraction, but it may also suffer on performance.

Even in one programming language you still have options for elegance versus performance. If you choose to use the simplest and cleanest solution you come up with, you may have poorer performance than if you would have used a more complex algorithm, but also harder to understand. Also if you choose to generalize the solution to include a family of problems, in the eventually of program growth, you would get into performance problems.

I faced this issue recently. I tried to write a program as simple and readable as possible. I used Groovy. I used a lot of reflection; the language itself is also heavily based on reflection. I tried to generalize each aspect to include as many cases I could think at the time. I used a lot of dynamic attributes and collections instead of static ones. The program was very domain centric, easy to read and understand (I think).
Note: I don't want to seem to be bragging; there were also plenty of hacks, shortcuts, to resolve a problem in minutes, instead of spending hours to do the wright way.

It all went fine until I tried to use large input data. It took forever to process the data. All the reflection, dynamics and generalities came back on me, and had dramatic effects on performance. I tried to improve some critical points of the program, hot-spots, but the overall performance was still poor.

So in the end you always have to pick: code elegance versus performance. Sometimes the domain will force you to choose one, because of its restrictions. But you will always have to balance these two aspects. If you favor performance you will have hard time maintaining the code on the long-run, If you favor code elegance you will have performance issues, like I had. It's a fine balance, you have to get it right.

What's your opinion: Code elegance or performance? Beauty or the beast?

Programmer profitability

How profitable are you as a programmer? Have you ever asked yourself that question? This is a tricky question, so let me give you a hint: it would be more profitable for you as a programmer to be better at what you do? I don't say if it would be good for you to be better programmer, but it would be more profitable? OK, enough with the questions.

It is accepted by everyone that programmer productivity varies by a large margin (by 10 times or more, but the numbers are debatable). So let's say you are twice as productive as the average programmer. Are you paid twice as much? Probably not. So a programmer's pay is not directly proportional with the value he is providing. Probably if you're twice as productive as the average programmer you'll be paid with let's say x% more, where surely x is smaller than 100. This sounds wrong, but it's true. And it's true because the work a programmer is doing is extremely hard to quantify. A programmers work cannot be quantified like a manufacturing task for example, where you could be paid by the number of items produced. But a programmer doesn't produce items by the number, even if there is an old practice to measure programmer’s productivity by the line of code they've written. But this is the worst measure of a programmer. A programmer could solve a problem in half the code and twice as elegant than another one, and that doesn't mean he is less productive, quite the opposite. But this is another story.
Not to mention another tricky aspect, a good programmer can come up with solutions to problems that the let's say the average programmer couldn't crack at all. And that's invaluable.
So from an economical point of view you're less profitable if you are a better programmer. So how can a programmer be motivated to be better at his job? We could blame this aspect for the ever decreasing level of competency in our industry, yet another story.

If being a better programmer isn't profitable for you, this means it is more profitable for the employer. And it is, because you get twice the value at less than twice the price. So it's a bargain for the employer to get the best programmers out there. But unfortunately usually most of the companies hire on number of programmers, not on value. You usually hear 'we need x more programmers', not 'we need y programmers with that level of skills or z with another level of skills'.

In the end I have a small advice for you: be a better programmer for yourself, because it feels good to be one of the best, and not because it gets you more money, because it doesn't scale up.

On Java EE complexity

Reading an article I remembered the words of one of my friends (Doru Girba) about the increasing complexity of Java, the language and technologies. And I must agree with him: EJB it's a nightmare (at least version 2), Java enterprise application start to add more and more complicated technologies, which add more complexity rather than to simplify things. I start to feel all this complexity myself, being involved in a massive enterprise project.
But there is also hope for better. Spring (a JEE application framework) simplified a lot the JEE landscape; it heavily influenced EJB version 3 with it's innovative ideas.
Another good sign is the development of more and more dynamic languages that run on JVM. The most promising (from my point of view) is Groovy: a dynamic version of Java (dynamic typing, closures, MOP, and others). Groovy also provides and web framework, inspired by the famous RoR, called Grails. Groovy's advantage over other dynamic languages is that it is compiled into Java bytecode, so it looks live ordinary Java classes.
Coming back to the article I mentioned in the beginning, it's about the recent acquisition of the company that develops Groovy and Grails by SpringSource (the company that develops Spring framework) - "More Weapons for the War on Complexity: SpringSource Acquires Groovy/Grails Leader". A combination of Spring and Groovy and Grails, running on a light-weight application server, based on OSGI, looks like a very promising idea. And the most interesting is the new SpringSource motto: "Weapons for the War on Java Complexity."

Conclusion: Keep it simple! Do not complicated things more than it has to be.
"Everything should be made as simple as possible, but not simpler." - Albert Einstein
"Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away." - Antoine de Saint-Exupery

The perfect definition for "Meta"

Defining the word "meta" is always hard, this includes programming.
Following is the Wikipedia definitions for "meta":

Meta is a prefix used in English in order to indicate a concept which is an abstraction from another concept, used to complete or add to the latter.

This way too abstract and hard to understand. A better definition for the word "meta" is given by Guy Steele in his OOPSLA'98 talk "Growing a Language" (video and paper), for the meaning used in programming but also for a more general meaning.

Meta means that you step back from your own place. What you used to do is now what you see. What you were is now what you act on. Verbs turn to nouns. What you used to think of as a pattern is now treated as a thing to put in the slot of an other pattern. A meta foo is a foo in whose slots you can put foos.

If you carefully analyze this definition you'll find a very deep meaning and you'll probably realize what "meta" means. And also if you truly understand this definition than you probably understand the whole meta-programming stuff. So just read that definition very carefully.

Software Visualization

Software visualization is defined as “the use of the crafts of typography, graphic design, animation, and cinematography with modern human-computer interaction and computer graphics technology to facilitate both the human understanding and effective use of computer software.”(John T. Stasko et. al.). It is a specialization of information visualization, whose goal is to visualize any kind of abstract data, while in software visualization the sole focus lies on visualizing software.


Software visualization and reverse engineering: Software visualization has been widely used by the reverse engineering research community during the past two decades. Many of the approaches provide ways to uncover and navigate information about software systems. The graphical representations of software used in the field of software visualization, a sub-area of information visualization, have long been accepted as comprehension aids to support reverse engineering. Software visualization has become one of the major approaches in reverse engineering, Koschke reporting that 80% of interviewed researchers consider visualizations as being important or absolutely necessary in software reverse engineering.

The enormous interest in visualization as an aid for reverse engineering and problem detection can also be inferred from the large number of tools that have been developed for this purpose: Rigi, SHriMP, CodeCrawler, Mondrian, etc.

My interest in this domain appeared during my visit to SCG, during my diploma thesis, when I created a set of new visualization for Lisp systems, developed to underline the differences of the language and to help understand and browse complex Lisp systems. For more information see my diploma thesis.