It’s been just over a year since I started studying for my Ph.D. — formally, I entered the program in April 2009. With at least two years to go, how do things look with some hindsight? What do I think it means to obtain the Ph.D. degree, or, more specifically and usefully, to be a researcher in computer science?

Much of what I’m noticing are things that sound obvious and natural, like everyday truisms, when expressed with words, but the idea I have of it goes a little bit deeper than that. For instance, we all get told over and over throughout our lives, starting in high school, that we have to become good communicators. So it’s not going to be a surprise to anyone when I say that I think the process entails becoming a much better communicator than I’ve ever been before. Maybe what’s different is that I am trying to communicate things that haven’t been communicated before, things that I invented — or things that have hitherto been communicated only by a very small number of people. (Most of the communication I did prior to becoming a Ph.D. student may not have been terribly original.) Basically, reading and understanding a large amount of scientific papers, and understanding them with a particular use in mind, either having or getting a sense of how they fit into your own work. Then, writing your own papers, and communicating, somehow, what you thought, and what you were the first person to think, so that somebody else might read it like you read the works of others, and use it similarly. Then, presenting research, discussing it, and understanding what is being presented and discussed by others — similar challenges in speech instead of in writing.

I can’t speak for other fields, but in computer science ( I work with programming languages and software engineering), I find that a lot of this, for me, has been about building up a certain mental dexterity with formalisms. Understanding the implications of formalisms as you read about them and seek to apply them. Communicating formalisms to others. Some of this is still difficult, in particular the “communicating to others” part, but I think I am achieving things in this regard.

Communication, then, where does it take us? One of my mental images of academic knowledge is a big directed acyclic graph (a tree) where papers reference other papers. A surprisingly big part of writing a paper is ensuring that your work can get assimilated into this graph easily — placing it well, referencing the right things, making sure that you can be referenced easily. Also: defining the boundaries of your work extremely well — here’s where it begins, here’s where it ends. We assume precisely this and arrive at precisely that. It really seems that these things can never be made clear enough.

Which leads to another mental image of research: the paper/unit of work as a building block. The more solid it is, and the harder and sharper its surfaces and edges are, the better structures can be built from it (though I think there are other kinds of valuable works too). That’s one direction I think I need to be aiming for as an aspiring researcher.

There is now an opportunity to do a paid internship for 2-6 months at Japan’s National Institute of Informatics in this project. If you are a masters or Ph.D. student who is interested in programming languages and software engineering, and you think the project sounds interesting, please do consider applying. Unfortunately the internship is only open to students of institutions who have signed MOU agreements with NII. The list of such universities, which includes many institutions from around the world, as well as more formal information, is available here. If you know or want to learn Scala, all the better!

A common misconception is that these techniques only are useful with large projects and large teams. I find, though, that they can help reduce much of the growth pains even in small projects, helping them reach maturity much faster.

Use version control. Classical, but invalid, counter arguments include “it’s a hassle and too much work to set up”, or “there’s only one person working on this project anyway”. Even if it’s only you, you will benefit massively from being able to undo your changes far back in time. It will let you experiment safely. Plus, setup is no longer an issue with free and easy-to-use services like github and bitbucket. My tool of choice is now Mercurial, and I used to use SVN. And there are many other good choices.

Use a debugger. If there is a debugger available for your language, and there most certainly is, then you should use it to find nontrivial errors, rather than extensive printf style testing.

Don’t optimise prematurely, but when you need to, use a profiler. Profilers tell you where a program’s performance bottlenecks are. You can profile things like heap usage (what classes use most space in Java, for instance) and CPU usage (which functions use the most CPU time). For Java, I’ve discovered that the NetBeans IDE has a very good built in profiler. Eclipse also has one, but it didn’t work on Mac last time I checked. For C/C++, GProf used to be good and probably still is.

Use unit testing wisely. All of the above apply even to very small projects, but I think some projects are too small to need unit tests, at least initially. You be the judge. I find that unit tests can have a lot of benefit when applied to the fragile, complicated parts of a system, where many different things interlock. If you are ambitious you can also write tests first and code later — test driven development.

Use a good IDE if you can. For a language like Java, where you have to type a lot of code to get something done and spread out your code across lots of files, a good IDE that can generate boilerplate code and navigate quickly can really speed up your work. It’s beneficial for other languages too. But I have no problem with people who use pure vim or emacs, after all these are practically IDEs.

I believe that honing your software development skills as an academic can pay off. Also see: Daniel Lemire on why you should open source your projects. (I will get around to doing this eventually, I promise )

Recently I watched a friend give a presentation on a research topic he’s been working on for years. I found the presentation to be fascinating, and the clearest explanation of his work that I have seen to date. But I felt compelled to criticise him on one point.

In order to lighten up the speech a bit, he had chosen to include characters from a popular science fiction movie on every other slide, using them to explain the results he had attained in theoretical computer science. The link between the characters and the results was nearly non-existent; the pictures were clearly only there to lighten the presentation up a bit. I had been irritated by people’s tendency to do these things for some time, so I decided to point it out. One extreme example of this tendency gone too far occurred recently in a presentation about the database CouchDB – readers can Google for the slides to see the full controversy, though they are somewhat NSFW. (I don’t want to make moral judgments in this context, but I think the academic/professional domain can be kept free of these controversies. Save those battles for where they belong!)

So there’s a tendency for people to sugarcoat their presentation topic sometimes. The arguments in favor of doing this are that it can lighten an intrinsically heavy subject a lot, and save people from nearly falling asleep from compounded boredom (such as a conference where 30+ presentations about results in theoretical computer science are given). Essentially it mixes in some sugar with the sour stuff, yielding what might be called a sweet and sour talk. The medicine becomes easier to swallow.

But isn’t there something essentially contradictory about mixing contemporary pop culture so freely with results that, in this case, were about essentially pure mathematical theory? For one thing it takes the essentially perennial and debases it, linking it up with images that are hopelessly stuck in a short timeframe. For another, it can be seen a vote of non-confidence in your own ideas. It can be seen as saying “I know this is boring and useless to you, so please bear with me, and look at these amusing pictures until it’s over.” I’m not a good presenter, but in order to become one, I think I need to have sufficient confidence in my ideas to present them unsweetened unless the circumstances are extreme. I need to make my audience see the value in my ideas. Also, it’s quite different if the sugar coating is of the kind that helps people get into your idea, or if it’s the kind that just distracts (this case).

My view is therefore that one should use one’s lumps of sugar with restraint. Maybe a situation where this is called for is when the audience necessarily contains some people who are on the level that you need to be talking to, and many other people who are not on that level, and cannot possibly be brought up to it. In this situation, the sugar might be used to keep the second group somewhat alive and alert. And this is in fact the kind of situation my friend wrote the presentation for originally. So, no scorn on him, just a word of warning to the general public!

Examples cited by Reuters UK include surf science, golf management and winemaking. I agree that these are probably vocational qualifications rather than fields meriting university study. But for blue skies projects in the natural sciences or the humanities, the payoff and effects on society are very hard to judge in advance. After all, we very often conduct the research precisely to evaluate these benefits.

The incentive situation with basic research is different today from what it was during the cold war era. When basic research was a national affair, not to be shared freely in the scientific community, it was probably possible to gain a national advantage by investing more in basic research. Today it’s all too easy to make the argument that other countries will reap the benefits, so why pay for the investment? Essentially a reverse prisoner’s dilemma: out of selfishness, you are tempted not to invest, but everybody benefits more if everybody invests. But surely this is too simple a view of the situation.

Where will countries that cut down on basic research be in the league tables of the future?

In order to evaluate Mendeley, let’s ask ourselves what we want from a bibliography management system in the modern research environment. At a bare minimum, we want an easy way to catalogue and search PDF documents, and of course compile the all-important reference list at the end of the laborious writing process. Mendeley does this, as well as bring a social networking aspect into the picture. It tries to recommend papers that are relevant to your work, as well as give you an easy way of sharing interesting papers with colleagues.

In contrast to Aigaion, which I wrote about previously, Mendeley is not a web based system but a desktop application. This definitely has benefits as the interface is quite slick. I can set the application to watch my “papers folder”, and any PDFs I save to that folder, or its subfolders, will automatically be scanned and entered into Mendeley. Metadata, such as author, title and references, is automatically extracted from the document in most cases, though I found I had to manually revise it sometimes. There’s a built in command that searches for the metadata by paper title on Google Scholar, which comes in very handy in such cases.

Mendeley is built around an internal PDF viewer where the user can highlight text, add little stickies with notes, and so on. This works quite smoothly, but on the Mac platform, it’s definitely not as polished as the Mac’s built in Preview PDF viewer. Mendeley is using its own PDF rendering layer, and it shows in the slower loading times when you scroll the documents. Some additional work could be done here. This is my only major complaint so far, though.

Much like the Evernote application, Mendeley has the option of storing all the papers on a central server, so that I can easily access them (and any annotations I might have made) from a different computer by signing in with my user name and password and then syncing the files. This means I don’t have to give up the benefits I get from a centralized server. It might be nice, however, to have the option of running my own Mendeley server, so I’m not dependent on the Mendeley company’s server somewhere – but then I would forgo the social networking benefits of course.

This application has similarities to how last.fm is used for music, in that people build a profile based on what they consume. Indeed, Mendeley is describing itself as a last.fm for research (video presentation). Let’s compare research and music as forms of media.

• Most music listeners probably don’t make their own music – most people who read research papers probably write their own papers.
• Songs sample other songs (the remix culture), but it’s relatively recent – researchers have always done this in order to establish basic credibility.
• The atomic unit of music is the song. The atomic unit of research is the research paper (the PDF in today’s internet based world, at least in my discipline) – but could this change in the future? Do we have to constrain ourselves to the article format?

In summary, Mendeley is probably the most useful, workflow friendly bibliography system I’ve tried so far. If you’re in research, I’d recommend you give it a try. If I get time, I plan to also investigate a more Mac-centric tool called Sente in the future.

The Savage Minds blog recommends that you don’t use Mendeley as your main tool yet due to its relative immaturity, but I have seen no showstopper bugs so far.

As an academic, I would like to publish prestigious articles. It would be nice to publish 10 papers at second or third rate conferences, but they might all be made irrelevant by a single article at a first rate conference (or even an article in Nature or Science, say). So quality is a better measure than quantity.

I would also like to come up with new and influential ideas, but I suspect I would probably be happier if I managed to influence 10 very highly regarded people than if I managed to influence 10 000 laymen. (These exact numbers were computed using the “wild guess” algorithm and further evaluation may be needed.)

In professional life, I’ve found it dangerously easy to fall into a mode of thinking where you evaluate yourself by your income. This is true up to a point, but I’ve found that there’s a point beyond which additional income has diminishing returns in terms of how much it adds to my overall rewards from life. So beyond this point, quality is a better measure than quantity. What are my tasks, how do they force me to learn and evolve, what kind of satisfaction do I feel and why? So quality is a better measure than quantity.

User satisfaction with computer software can, to some extent, be measured using response time and latency. A snappy, responsive user interface usually produces more satisfaction than a sluggish one. But this can often be compensated for to a surprising extent by having appropriate progress indicators, animations and design features that placate the user in some way, assuring them something is being done. This is in a sense the opposite of the money situation: up to a certain point, quality makes up for quantity, after that point (when the slowness becomes impossible to mask), quantity becomes increasingly important.

What’s most interesting is perhaps the convertibility between quality and quantity. In engineering a device or a software system, quantitative metrics can be crucial tools in the construction process, but the final user experience must be qualitatively right. So quantity is a tool to construct quality. And in the real life situations where quantity is actually the best measure — bargaining, comparing, communicating, constructing, … — I think of it as a way to mask qualities. The numbers are simply easier to consider than the vast number of qualities that lie underneath.

I started using Aigaion for mine. Then I found out that there’s an open system called bibsonomy, which is potentially much better since it lets you tag and share bibliographies socially, and it seems to already know about all the major computer science papers.

Again (see: The problem with standards), I’m frustrated by the fact that I can’t move my data around between applications as I like without lots of manual effort. A worthy research problem would be making data truly application independent once and for all.