What makes a programming language attractive and competitive? How can we evaluate its quality? There are many different aspects of this problem.

Ease of reading and writing, or: directness of the mapping between the problem in your head and the model you are creating on the computer. This can be highly domain dependent, for instance languages such as LaTeX, Matlab and R are designed with specific problems in mind and cater to users from that domain. Their limits show quickly when you try to stretch them beyond their envisioned purpose. Speaking of general programming languages, I think Python deserves to be mentioned as a language that is extremely readable and writable. It has other shortcomings though – see below. Prolog is also highly read- and writable if it suits your problem.

Runtime performance. Arguably this is one of the few reasons to bother with using C++. For the majority of programming projects though, performance is much less of a problem than one might think, especially if one considers how close the performance of many JVM languages get to C++. When programmers think about their overall productivity and effectiveness in developing and maintaining a system, C++ is often not the best choice, obviously.

Scalability to large teams. The key property here is: does the language do anything to help me, as a developer, work with code that other people wrote? Ease of maintenance may be strongly correlated with usability in large teams. An anti-pattern here is languages that allow for solving the same problem in a huge amount of ways with very variable syntax. For instance, Perl and C++ can lead to notoriously unmaintainable code if used carelessly. Some say that Scala also suffers from this problem. Basically, the language helps here if it prevents me from doing things that other developers might not expect, and that I might forget to document or communicate. This is why Gosling famously called Java a blue collar language; it restricts you enough to make teamwork quite practical. It even restricts the layout of your source file hierarchy. (Now we begin to see that some goals are in conflict with each other).

Scalability to large systems. This is related to the preceding property, but whereas team scalability seems to be mainly about avoiding the creation of code fragments that surprise people other than their creators, system size scalability seems to be about avoiding the creation of code fragments that surprise other code fragments. Here one needs invariants, good type checking, static constraints of all kinds. Scripting languages like Perl and Python, lacking static typing completely, are some of the worst in this regard, since we cannot even be sure at startup time that methods we try to invoke on objects exist at all (Python).

Scalability over time (maintainability). If there is both system size scalability and team scalability, then the system is also likely to be able to live for a long time without great troubles.

Developer efficiency and rapid prototyping. Depending on the nature of the system being developed, this may depend on several different properties listed above.

Availability of quality tools. Mature runtime environments, such as the JVM, have many more high quality tools and IDEs available than a language than Ruby. Mature languages also have more compilers for more different architectures available.

These points begin to give us an idea of how we can evaluate programming languages. However, I also believe that making a good language and making people use it is largely about luck and factors outside the design itself. Just like there’s a big step between imagining and specifying an utopian society and making that social order an actuality, there’s a big step between designing an ideal programming language and achieving widespread adoption for it. We have seen a way forward though: with generalised runtime environments such as the JVM and the CLR, we may develop and deploy languages that take advantage of a lot of existing infrastructure much more easily than before. And what I hope for is in fact that it becomes even easier to deploy new languages, and that new languages are as interoperable as possible (insofar as it doesn’t constrain their design), so that we could see more competition, more evolution and more risk taking in the PL space.

But they also complicate reasoning about programs enormously. Both human reasoning and automated reasoning. Pointers allow any part of the program to have side effects in any other part of the program (if we have a reference to an object that originated there), and they make it very hard to reason about the properties that an object might have at a given point in time (since we generally have no idea who might hold a reference to it – it is amazing that programmers are forced to track this in their heads, more or less). In my effort to design my own language, multiple pointers to the same objects – aliases – have come back from time to time to bite me and block elegant, attractive designs. I believe that this is a very hard problem to design around. Aliased pointers set up communication channels between arbitrary parts of a program.

Nevertheless attempts have been made, in academia and in research labs, to solve this problem. Fraction-based permissions track how many aliases exist and endow each alias with specific permissions to access the object that is referred to. Ownership analysis forces access to certain objects to go through special, “owning” objects. Unique or “unshared” pointers in some language extensions restrict whether aliases may be created or not. But so far no solution has been extremely attractive and convenient, and none has made it into mainstream languages. (I know that someone Philipp Haller made a uniqueness plugin for the Scala compiler, but it is not in wide use, I believe.)

If we are to attempt further incremental evolution of the C-family languages, aliased pointers are one of the most important issues we can attack in my opinion.

Hence the trouble with obtaining funding for research. In order to obtain funding, you have to communicate something that seems incommunicable without the full commitment of the reader. Grant dispensers want to know a number on a scale: “what’s the quality of this paper between 0 and 1?”, but this quality number cannot be communicated separately from the full substance of the paper and its environs. And thus we end up with keywords, catchphrases that become associated with quality for short periods of time, as a way of bypassing this complexity, an approximate way of indicating that you are doing research on something worthwhile.

This reflects a broader problem in society of evaluating authorities. I cannot evaluate my doctor’s, or my dentist’s, or my lawyer’s work, since I don’t have the necessary competence. Accordingly, I base my trust on the person and some of their superficial attributes, instead of judging the work by itself. It seems that the same kind of thing becomes necessary sometimes in choosing what researchers to fund.

It also points to a faculty that must have evolved in human being since millennia: the capacity for evaluating important properties of things we do not understand well very quickly, for danger, nutrition, etc. Only that this faculty does not translate well to research…

This is hopefully the start of a short series of posts in which I attempt to relate the concepts of value and value creation, in particular as they were understood by Friedrich Nietzsche, to the modern world, in some kind of way. Comments of all kinds are encouraged!

In the beginning (understood as ancient Greece), there was philosophy. That is to say, most systematic inquiry into matters worth thinking about was collected under this umbrella term. Ethics, politics, epistemology and metaphysics went side by side with physics, biology and astronomy. As millennia passed, the collective human knowledge and scholarly labour grew, and some philosophical disciplines got their own name, cut the umbilical cord, and got to stand on their own feet.

There are many definitions as to what a philosopher is; one definition would be those who study the academic subject of philosophy in academic institutions. The German philosopher and philologist Friedrich Nietzsche wrote at length about what a philosopher really is; in his definition a philosopher is someone who creates values. Nietzsche rejected morals and universal truth as laid down by a God or higher authority; instead they are created by subjective human beings, and by philosophers in particular.

Perhaps [the genuine philosopher] himself must have  been critic and sceptic and dogmatist and historian and also poet and collector and traveller and solver of riddles and moralist and seer and “free spirit” and almost everything in order to pass through the whole range of human values and value feelings [...] But all these are merely preconditions of his task: this task itself demands something different – it demands that he create values.

(Friedrich Nietzsche, Beyond Good and Evil, s. 211, Walter Kaufmann transl.)

We may understand a scholar to be a person who processes knowledge. Good scholarship entails marshalling what has been written and studied previously, perhaps with a view to settling a question or supporting a perspective. Scientists and philosophers can make use of scholars in their work. To the extent that the scholar does more than merely process knowledge, he or she is something more than a scholar.

In contrast, a scientist, as we understand him or her today, is someone who combines scholarship and primary investigation (in the form of calculation, experimentation, measurement and so on) in order to create models of nature and the world, in order to gain the power to explain. The classical scientific process involves repeated refinement of hypotheses until one that cannot be proven wrong has been found.

Today, science, which formerly was known as natural philosophy, has grown enormously large, and to most people probably appears to have much greater value than philosophy. The scientific mindset is widely appreciated and respected throughout the world — perhaps too respected. Scientists learn as one of their highest virtues to be skeptical and to reject assertions that are made without a basis in measurement or theory. Paralysis by skepticism is very much a possibility. To see the danger in this, we have to recognise that a great deal of valuable things in human history have been created without such a basis – by people who have been something like the ones Nietzsche describes.

The dangers for a philosopher’s development are indeed so manifold today that one may doubt whether this fruit can still ripen at all. The scope and the tower-building of the sciences has grown to be enormous, and with this also the probability that the philosopher grows weary while still learning or allows himself to be detained somewhere to become a “specialist” – so he never attains his proper level, the height for a comprehensive look, for looking around, for looking down. [...]

Indeed, the crowd has for a long time misjudged and mistaken the philosopher, whether for a scientific man and ideal scholar or for a religiously elevated, desensualized, “desecularized” enthusiast and sot of god. And if a man is praised today for living “wisely” or “as a philosopher”, it hardly means more than “prudently and apart”.

(Friedrich Nietzsche, Beyond Good and Evil, s. 205, Walter Kaufmann transl.)

In fact, scientists today do not, in my experience, work like the ideal scientist described above. Scientists often use their own judgment and their own values in order to influence how their science is to be used. Einstein and Oppenheimer had opinions about the use and misuse of the nuclear bomb. Creators of vaccine may have opinions on how it is to be distributed and may be able to influence this. Sometimes these value statements made by scientists are pure judgments, applications of an ethic that the scientists already believe in. However, sometimes the situation is so new that the scientists effectively have to create values. To the extent that they do this, these scientists dabble in ethics, morality and philosophy, but this is often overlooked, as is the fact that scientific method itself was created by philosophy.

Nietzsche calls for philosophers to make use of scientists and artists, and create values in the service of mankind. He calls for a new recognition of the true role and dignity of philosophy, which does not at all need to mean a reduction of the value of science, but rather an expansion of the whole system. Philosophy stands naturally above science and scholarship and uses them as its tools. The activity of creating values based on philosophical insight by necessity goes on constantly and should not be confined to little nooks in the margins of society. The full extent of and need for this activity needs to be acknowledged.

Has the situation changed since Nietzsche wrote Beyond Good and Evil in 1886?

Type theory was invented by Bertrand Russell as a solution to his own well-known paradox, which cast doubt on the foundations of mathematics upon its discovery.  The problem posed by the paradox is essentially: given a set of all sets that do not contain themselves, does that set contain itself? Another interesting problem arising from self-referentialism. Type theory resolves this issue by classifying the members of sets according to some type, and in such a system a set definition like this one is not permissible.

(The drama surrounding Russell’s paradox and Principia Mathematica can now also be read in the form of a “logicomix” graphic novel by Doxiadis et al!)

Type theory did not end its life as a metamathematical hack that solved a set theoretical problem. It has now come of age. Type systems are, of course, at the heart of the development of many modern programming languages. Java, C#, Lisp, ML and Haskell are but a few of the ones that depend completely on a nontrivial type system. (Even though C and C++ have type systems, they are kept in a crippled state by the fact that the programmer is allowed to ignore their commandments at will.)

What is the benefit of a type system in a programming language? In the words of Benjamin Pierce,

A type system is a syntactic method for automatically checking the absence of certain erroneous behaviors by classifying program phrases according to the kinds of values they compute.

So, generally speaking, a type system helps distinguish correct programs from incorrect programs. The parser also does part of this job, of course, since many programs that cannot compile will not parse. But there are programs that are syntactically correct but at the same time semantically unsound according to the evaluation rules of the language. Consider the following Java statement:

T x = new U();

This is clearly syntactically acceptable, but is it semantically valid? In order for it to be, either U needs to be the same type as T, or a subtype of T. But Java programmers are free to define new subtypes of existing types in source code. In order for the parser to check the correctness of the assignment, it seems that either the grammar would become enormously large, or it would place awkward restrictions on Java syntax. So we leave more refined aspects of correctness to the type checker, which is applied later, post-parsing.

What does a type system look like? As used in programming languages today, a type system consists of a set of typing judgments. The judgments can be stacked together to form the typing derivation of an expression in the language. Every expression normally has a unique typing derivation, and when a type can be derived, we call the expression well typed. As Benjamin Pierce’s formulation suggests, each typing rule makes some assumptions about the kinds of values that constitute the various parts of an expression, and then says something about the kind of value that will be computed by the expression as a whole.

Type systems can look prohibitive at first sight, since they use quite a few algebraic symbols. But in general, the structure and interrelationships between the symbols is the most important thing to discern from them, and focussing on this aspect makes reading them much easier.

In the following mini-type system, typing judgments will have the form , where is a typing context, is a term, and is a type. Terms are essentially fragments of the source code, and from smaller terms we can construct larger terms using syntactic forms. So for example, if a, b and c are terms, then we might use the if-syntax to construct the larger term

if (a) {b} else {c}

in a java-like language. The typing context is essentially just a map, mapping already typed terms to the types assigned to them. As we type increasingly large terms, we gradually add more information to the typing context. The symbol is just notational convention and has no particular meaning. The judgment is read as “term t has type T in typing context “.

A fragment of a minimal system might look like the following.

Three typing rules have been introduced here, T-ASGN for assignments, T-IF for if-statements, and T-BINOP for binary operations on integers, like + and -.
Every rule has the same structure: above the line, assumptions that must hold true for the rule to be applied; below the line, the conclusion we may draw if the assumptions are true.

Above, the following example was given.

T x = new U();

Using T-ASGN, we can type an assignment statement like this one. The assumption T <: U says that T is a subtype of U. We have also assumed a lookup function vartype which gives the declared type of the variable we assign to. In English, we can read the rule T-ASGN as saying: “Assuming that the type of the right hand side is a subtype of, or the same type as, the declared type of the variable, the assignment will evaluate to a value that has the declared type of the variable”. Of course not all programming languages implement assignment in this way.

In the T-BINOP rule, we must of course substitute an actual operation on integers for the special word intBinop for it to be valid. For addition, the rule can be read as “Assuming that the left hand side and the right hand side are both integers, then the result is also an integer”.

Finally, the T-IF rule says “assuming that the truth condition is a boolean, and that the two alternative paths have the same type, then the if-statement evaluates to the same type as that of the two conditional paths”.

The shape of the type system follows the evaluation rules for the language quite closely, so that a well-typed term is also always possible to evaluate. We can thus avoid accepting programs that might get stuck without any valid evaluation rule at some point. (Consider what happens in Python if you try to invoke a method on an object, and the object doesn’t have a method with that name.) We are not limited to tracking just the kinds of values being computed in a type system. We can track various kinds of safety properties, such as memory regions being violated, as well, or we may track multiple properties at once. Type and effect systems is one way of tracking both values being computed and resource violations.

A good source of more information on type systems applied to programming languages would be Benjamin Pierce’s Types and Programming Languages.

It’s been just over a year since I started studying for my Ph.D. — formally, I entered the program in April 2009. With at least two years to go, how do things look with some hindsight? What do I think it means to obtain the Ph.D. degree, or, more specifically and usefully, to be a researcher in computer science?

Much of what I’m noticing are things that sound obvious and natural, like everyday truisms, when expressed with words, but the idea I have of it goes a little bit deeper than that. For instance, we all get told over and over throughout our lives, starting in high school, that we have to become good communicators. So it’s not going to be a surprise to anyone when I say that I think the process entails becoming a much better communicator than I’ve ever been before. Maybe what’s different is that I am trying to communicate things that haven’t been communicated before, things that I invented — or things that have hitherto been communicated only by a very small number of people. (Most of the communication I did prior to becoming a Ph.D. student may not have been terribly original.) Basically, reading and understanding a large amount of scientific papers, and understanding them with a particular use in mind, either having or getting a sense of how they fit into your own work. Then, writing your own papers, and communicating, somehow, what you thought, and what you were the first person to think, so that somebody else might read it like you read the works of others, and use it similarly. Then, presenting research, discussing it, and understanding what is being presented and discussed by others — similar challenges in speech instead of in writing.

I can’t speak for other fields, but in computer science ( I work with programming languages and software engineering), I find that a lot of this, for me, has been about building up a certain mental dexterity with formalisms. Understanding the implications of formalisms as you read about them and seek to apply them. Communicating formalisms to others. Some of this is still difficult, in particular the “communicating to others” part, but I think I am achieving things in this regard.

Communication, then, where does it take us? One of my mental images of academic knowledge is a big directed acyclic graph (a tree) where papers reference other papers. A surprisingly big part of writing a paper is ensuring that your work can get assimilated into this graph easily — placing it well, referencing the right things, making sure that you can be referenced easily. Also: defining the boundaries of your work extremely well — here’s where it begins, here’s where it ends. We assume precisely this and arrive at precisely that. It really seems that these things can never be made clear enough.

Which leads to another mental image of research: the paper/unit of work as a building block. The more solid it is, and the harder and sharper its surfaces and edges are, the better structures can be built from it (though I think there are other kinds of valuable works too). That’s one direction I think I need to be aiming for as an aspiring researcher.

There is now an opportunity to do a paid internship for 2-6 months at Japan’s National Institute of Informatics in this project. If you are a masters or Ph.D. student who is interested in programming languages and software engineering, and you think the project sounds interesting, please do consider applying. Unfortunately the internship is only open to students of institutions who have signed MOU agreements with NII. The list of such universities, which includes many institutions from around the world, as well as more formal information, is available here. If you know or want to learn Scala, all the better!

A common misconception is that these techniques only are useful with large projects and large teams. I find, though, that they can help reduce much of the growth pains even in small projects, helping them reach maturity much faster.

Use version control. Classical, but invalid, counter arguments include “it’s a hassle and too much work to set up”, or “there’s only one person working on this project anyway”. Even if it’s only you, you will benefit massively from being able to undo your changes far back in time. It will let you experiment safely. Plus, setup is no longer an issue with free and easy-to-use services like github and bitbucket. My tool of choice is now Mercurial, and I used to use SVN. And there are many other good choices.

Use a debugger. If there is a debugger available for your language, and there most certainly is, then you should use it to find nontrivial errors, rather than extensive printf style testing.

Don’t optimise prematurely, but when you need to, use a profiler. Profilers tell you where a program’s performance bottlenecks are. You can profile things like heap usage (what classes use most space in Java, for instance) and CPU usage (which functions use the most CPU time). For Java, I’ve discovered that the NetBeans IDE has a very good built in profiler. Eclipse also has one, but it didn’t work on Mac last time I checked. For C/C++, GProf used to be good and probably still is.

Use unit testing wisely. All of the above apply even to very small projects, but I think some projects are too small to need unit tests, at least initially. You be the judge. I find that unit tests can have a lot of benefit when applied to the fragile, complicated parts of a system, where many different things interlock. If you are ambitious you can also write tests first and code later — test driven development.

Use a good IDE if you can. For a language like Java, where you have to type a lot of code to get something done and spread out your code across lots of files, a good IDE that can generate boilerplate code and navigate quickly can really speed up your work. It’s beneficial for other languages too. But I have no problem with people who use pure vim or emacs, after all these are practically IDEs.

I believe that honing your software development skills as an academic can pay off. Also see: Daniel Lemire on why you should open source your projects. (I will get around to doing this eventually, I promise )

Recently I watched a friend give a presentation on a research topic he’s been working on for years. I found the presentation to be fascinating, and the clearest explanation of his work that I have seen to date. But I felt compelled to criticise him on one point.

In order to lighten up the speech a bit, he had chosen to include characters from a popular science fiction movie on every other slide, using them to explain the results he had attained in theoretical computer science. The link between the characters and the results was nearly non-existent; the pictures were clearly only there to lighten the presentation up a bit. I had been irritated by people’s tendency to do these things for some time, so I decided to point it out. One extreme example of this tendency gone too far occurred recently in a presentation about the database CouchDB – readers can Google for the slides to see the full controversy, though they are somewhat NSFW. (I don’t want to make moral judgments in this context, but I think the academic/professional domain can be kept free of these controversies. Save those battles for where they belong!)

So there’s a tendency for people to sugarcoat their presentation topic sometimes. The arguments in favor of doing this are that it can lighten an intrinsically heavy subject a lot, and save people from nearly falling asleep from compounded boredom (such as a conference where 30+ presentations about results in theoretical computer science are given). Essentially it mixes in some sugar with the sour stuff, yielding what might be called a sweet and sour talk. The medicine becomes easier to swallow.

But isn’t there something essentially contradictory about mixing contemporary pop culture so freely with results that, in this case, were about essentially pure mathematical theory? For one thing it takes the essentially perennial and debases it, linking it up with images that are hopelessly stuck in a short timeframe. For another, it can be seen a vote of non-confidence in your own ideas. It can be seen as saying “I know this is boring and useless to you, so please bear with me, and look at these amusing pictures until it’s over.” I’m not a good presenter, but in order to become one, I think I need to have sufficient confidence in my ideas to present them unsweetened unless the circumstances are extreme. I need to make my audience see the value in my ideas. Also, it’s quite different if the sugar coating is of the kind that helps people get into your idea, or if it’s the kind that just distracts (this case).

My view is therefore that one should use one’s lumps of sugar with restraint. Maybe a situation where this is called for is when the audience necessarily contains some people who are on the level that you need to be talking to, and many other people who are not on that level, and cannot possibly be brought up to it. In this situation, the sugar might be used to keep the second group somewhat alive and alert. And this is in fact the kind of situation my friend wrote the presentation for originally. So, no scorn on him, just a word of warning to the general public!

Examples cited by Reuters UK include surf science, golf management and winemaking. I agree that these are probably vocational qualifications rather than fields meriting university study. But for blue skies projects in the natural sciences or the humanities, the payoff and effects on society are very hard to judge in advance. After all, we very often conduct the research precisely to evaluate these benefits.

The incentive situation with basic research is different today from what it was during the cold war era. When basic research was a national affair, not to be shared freely in the scientific community, it was probably possible to gain a national advantage by investing more in basic research. Today it’s all too easy to make the argument that other countries will reap the benefits, so why pay for the investment? Essentially a reverse prisoner’s dilemma: out of selfishness, you are tempted not to invest, but everybody benefits more if everybody invests. But surely this is too simple a view of the situation.

Where will countries that cut down on basic research be in the league tables of the future?