Monomorphic

The Gregorian calendar has been in use since 1582. Among its features is a moderately complicated rule for leap years: if n mod 4 is 0, then n is a leap year. However, if n mod 100 is 0, then n is not a leap year, unless n is a multiple of 400.

In addition, we live in a world with timezones and regional differences in when countries go on and off daylight savings time, if they have such a system. As yet another example of Japanese rationality, Japan does not have a DST system.

Implementing date and time computations correctly can be very hard for computer programmers and is invariably a source of many hidden bugs that may take a long time to discover. Yesterday, a large amount of Sony’s Playstation 3 game consoles stopped working normally.  This was later fixed. There was speculation the error was due to incorrect leap year handling. It wouldn’t be the first time this occurred if this was indeed the reason.

In a software company where I used to work, there would usually be massive troubles every time some country went on or off daylight savings time, or any other time calculation hit a sensitive spot. I’m fairly sure that the world’s software systems, including government, finance, insurance, health care, suffer untold billions of damage every year due to the complexity of the system. Maybe we should simplify it.

I suggest having “years” with 365 x 4 + 1 = 1461 days instead of the usual year for starters. This would move the leap year problem ahead until year 2100, when the next special rule comes in. By that time, software engineering technology should have improved enough that this should no longer be an issue, I hope. If not, we can invent another system by then. Let’s also scrap all daylight savings time everywhere. It’s easy to do and the savings would be huge.

Academics and practitioners, having rather different goals in life, tend to approach software development in quite different ways. No doubt there are many things each side of the fence can learn from the other, but I think academics in particular could often benefit quite a lot by adopting some of the practices used in industrial development. And not just computer science academics!

A common misconception is that these techniques only are useful with large projects and large teams. I find, though, that they can help reduce much of the growth pains even in small projects, helping them reach maturity much faster.

Use version control. Classical, but invalid, counter arguments include “it’s a hassle and too much work to set up”, or “there’s only one person working on this project anyway”. Even if it’s only you, you will benefit massively from being able to undo your changes far back in time. It will let you experiment safely. Plus, setup is no longer an issue with free and easy-to-use services like github and bitbucket. My tool of choice is now Mercurial, and I used to use SVN. And there are many other good choices.

Use a debugger. If there is a debugger available for your language, and there most certainly is, then you should use it to find nontrivial errors, rather than extensive printf style testing.

Don’t optimise prematurely, but when you need to, use a profiler. Profilers tell you where a program’s performance bottlenecks are. You can profile things like heap usage (what classes use most space in Java, for instance) and CPU usage (which functions use the most CPU time). For Java, I’ve discovered that the NetBeans IDE has a very good built in profiler. Eclipse also has one, but it didn’t work on Mac last time I checked. For C/C++, GProf used to be good and probably still is.

Use unit testing wisely. All of the above apply even to very small projects, but I think some projects are too small to need unit tests, at least initially. You be the judge. I find that unit tests can have a lot of benefit when applied to the fragile, complicated parts of a system, where many different things interlock. If you are ambitious you can also write tests first and code later — test driven development.

Use a good IDE if you can. For a language like Java, where you have to type a lot of code to get something done and spread out your code across lots of files, a good IDE that can generate boilerplate code and navigate quickly can really speed up your work. It’s beneficial for other languages too. But I have no problem with people who use pure vim or emacs, after all these are practically IDEs.

I believe that honing your software development skills as an academic can pay off. Also see: Daniel Lemire on why you should open source your projects. (I will get around to doing this eventually, I promise )

Scala is a little wonderland sprinkled with useful things you can mix and match as you like to improve your coding experience while staying on the Java platform. The Option classes, the structural case matching, the compact declarations, lazy evaluation… the list goes on. But at the heart of it is the decision to mix freely the functional and imperative programming styles.

How does this work in practice?

• Statements can have side effects, like in Java
• The final statement evaluated in a function is its return value by default
• Every statement evaluates to a value, even control flow statements like if… else, unlike in Java

The bottom line is that some problems call for a functional programming style, and others for an imperative one. Scala doesn’t force you into a mold, it just gives you what you need to express what you’d like to express. This can lead to very compact code. Here’s a function that recursively finds all files ending in .java starting in a given directory. The File class here is the standard Java java.io.File!

Remember, the last expression evaluated is the return value.

 def findJavaFiles(dir: File): List[File] = {
    val files = dir.listFiles()
    val javaFiles = files.filter({_.getName.endsWith(".java")})
    val dirs = files.filter({_.isDirectory})
    javaFiles.toList ++ dirs.flatMap{findJavaFiles(_)}
  }

But we can write it even more compactly at the expense of some clarity:

 def findJavaFiles(dir: File) = {
    val files = dir.listFiles()
    files.filter(_.getName.endsWith(".java")).toList ++
 files.filter(_.isDirectory).flatMap{findJavaFiles(_)}
  }

Now write this function in Java and see how many lines you end up with.

I just got a new laptop, courtesy of the lab. Naturally, it’s of the fruity kind. One of the first steps: install essential software.

I thought I’d make a list of software I consider absolutely essential on any new computer, and it became longer than I thought.

General use:

NetNewsWire for news reading

DropBox for file syncing

OmniFocus as a task organizer (the GTD methodology actually works — it has liberated me from reciting a long list of things to do in my head all day long)

CircusPonies Notebook for note taking

iStat Pro for system monitoring

If I want to develop software:

Eclipse

Fink and MacPorts so I can get various unix tools (I can’t settle for one or the other, since some tools are in one of them only, but normally Fink is nicer since the packages are precompiled)

Apple’s developer tools

If I want to read and write papers:

TeXShop

Mendeley Desktop

So these are the “absolute essentials”. Of course web apps like gmail count too, but they require no installation. Anything I’ve missed?

One thing I do not install, but perhaps should, is Apple’s MobileMe. Considering how fruity my environment is, there ought to be some benefit. But between Dropbox, my own DAV server for calendars, and built-in syncing of apps like OmniFocus, I can make things stay in sync anyway, so MobileMe is probably not worth the cost… I think.

Flood-it, a color filling game. This version was made by Lab Pixies for the iPhone, but many others exist.

There’s a veritable torrent of little games constantly being released for the iPhone. One of the more likable ones is Flood-It, which I’ve been playing recently. The premise is extremely simple: you start off with a grid divided into squares of different, randomized colors. You are given a tool that works a bit like the bucket fill in a picture editor. At each turn, the player chooses a color to fill the grid with, starting from the upper left corner. The monochromatic area slowly grows, and the aim is to fill the entire grid with a single color within a limited number of turns.

A recent analysis showed that finding an optimal solution to games like Flood-It is a NP-hard problem. In addition, deciding whether the game can be solved in n steps for some n is NP-complete. The analysis relies on a reduction of Flood-It to an instance of the SCS problem (shortest common superstring). (It’s important to note that what is NP-complete is deciding whether a particular board can be solved in a certain number of steps, not solving the game with a bounded number of steps. This can be done in polynomial time.) For those who need a summary, ACM Communications had an excellent review of the state of the P/NP problem in September last year.

For a NP-hard problem H, there exists a polynomial time reduction of any problem in NP to H, meaning that if we can solve H in P-time, we can solve any problem in NP in P-time. Many optimization problems in society rely on approximate solutions to difficult problems: routing traffic, assembling DNA sequences from partial subsequences, mathematical theorem proving… On the hypothesis that evolution has turned people into efficient solvers of hard problems (i.e. we have good heuristics in our brains from birth and from experience), we ought to pay people to play these games on their phones, but map real problems into game instances, so that people effectively work while they’re playing. We ought to design games that act as front-ends for real combinatorial problems.

A computer game, as we understand it, can be defined as a very smooth learning curve, and if we only “play” very tricky instances of combinatorial problems, the game would probably present too much of a barrier to new players. So maybe the best way of executing this kind of scheme would be that a majority of all game instances do not represent real problems, but mere training or verification of already solved problems — but every once in a while, a real problem pops up. The player should still get paid though.

A double benefit would be blurring the line between work time and  play time, what is useful and what is useless — I think this line is often artificially constructed. Has technology ever before given us the possibility to literally turn work into play?

Acknowledgements. I am indebted to Christian Sommer for showing me the complexity analysis of Flood-it.

The Flood-It game, easy difficulty setting, with the player having made some progress.

Nomura's Jellyfish. Picture by Kenpei at the Osaka aquarium. GFDL license.

Nomura’s jellyfish, a species frequently encountered in Japan and China, is one of the largest in the world. The body can reach a diameter of 2 m. Since they create big problems for the fishing industry, Japan has now sought China’s help on the issue. It is thought that a recent proliferation of the species, huge swarms appearing every year since 2000, originates at the mouth of the Yangtze river.

Evolution can do fascinating things sometimes. Upon reading about this, a doubtlessly romantic and delusional notion entered my mind. What if the sea ecosystem, or a subset of it, say 10-100 species, perceive the human fishing industry as a threat that needs to be defended against, and in response create an evolutionary niche where a new kind of species can thrive, a species whose only purpose is to obstruct fishing? A romantic notion since it plays off the mythical idea that human beings are at war with nature, or that nature is good and man is evil, something I don’t really believe in. But an interesting one nonetheless. Is such a development possible?

A cold, bright morning in Tokyo’s somewhat fashionable Azabu-Juuban district. I’m looking for a clinic, but I can’t find it. I’ve only visited it once before, more than a year earlier. I look for landmarks that I might remember, bring out the map on my phone, pay attention to every detail in the hope that I will recognize something.

The morning has turned into a game. It’s me against the city layout, me against my memory, me against entropy and the temporal degradation of my cognitive faculties. The ludic dimension has entered my life again. And soon enough, I find the place I was looking for.

When we have a sense of competition, that a victory against something or someone is possible, our awareness of life is heightened in every way. We pay more attention, we notice more, we become more here and now. The endless simmering chatter in our heads, nearly meaningless thoughts that usually refuse to yield anything meaningful, gives way to absolute focus.

It occurs to me that a society where everyday tasks can be carried out like they are games, victories to be won, might be a more moral society, with greater happiness and life awareness for everyone. In such a society, even if you lose a particular game, you win something else.

In his first amendment to Human, All Too Human (1886), entitled Miscellaneous Maxims and Opinions, Friedrich Nietzsche states that

300. HOW FAR EVEN IN THE GOOD THE HALF MAY BE MORE THAN THE WHOLE. — In all things that are constructed to last and demand the service of many hands, much that is less good must be made the rule, although the organiser knows what is better and harder very well.He will calculate that there will never be a lack of persons  who can correspond to the rule, and he knows that the middling good is the rule. — The youth seldom sees this point, and as an innovator thinks how marvelously he is in the right and how strange is the blindness of others. (Helen Zimmern transl.)

Friedrich Nietzsche did not describe software making – I can only assume that he was describing authors and ideologists – but this seems to capture the difficulties of software development only too well. And it seems to give a recipe for how to overcome the communication difficulties (abandon exotic, over-refined solutions and focus on an easily understood middle ground, so that everybody can get together and comprehend the architecture). This was originally published in 1886.

With that, merry christmas!

In the chilly yet sunny winter afternoon, I took a walk past the imperial palace in the centre of Tokyo. I find sunny winter days refreshing.

The palace is interesting to behold. It is fronted by lots of that most precious of Tokyo commodities, open space. Supposedly, during the height of the land bubble, the land on which the palace is built was worth more than the state of California. This is in turn surrounded by some of Tokyo’s most prestigious office buildings in the Marunouchi and Hibiya districts. Tokyo station is just a few minutes away on foot.

The scene is one of juxtapositions. Open space meets tightly packed high rise buildings. Traditional Japanese architecture counters sleek office buildings. Yet this  never feels contradictory, because there is an underlying theme of restraint and control.

As you might expect from a royal residence, the public courtyard is immaculate. The grass is so well cut and even as to resemble a golf course. The trees on the lawn are of uniform height, lushness and distance from each other. The gravel is supremely even.

The office buildings are similarly controlled: shades of grey and brown, a certain minimalism and homogeneity in design that is easier found here than in Europe, the sense that unnecessary detail has been removed.

There is a sense of power in all this; a will and a shared set of ideas that have been realized to a high degree. The homogenous, flat skyscraper with a grid of windows is the triumph of human, platonic ideas over the organic and the irregular. The palace garden is man’s will taming the uncontrolled vegetation we find in nature. Yet such control is always a question of scale. We can cut and prune the trees, but we cannot control the color of their leaves or the exact angle of every branch. And we can cut and prune the buildings, but generally, we cannot control the shape of the overall skyline in detail. Something organic manifests itself in the multitude, even as some parts are controlled.

Power and rebellion, in constant struggle and symbiosis.

The Scala collections library (in version 2.7.7) has a MultiMap trait for mutable collections, but none for immutable ones. I hacked something up to use while waiting for an official version. I’m finding this to work well, but I don’t have much experience with collections design, so it’s likely to have some flaws. Also, this is a class and not a trait, so you can’t use it with any map you like. And from a concurrency perspective, maybe it’s sometimes better to use backing collections other than the HashSet and the HashMap.

 
import scala.collection.immutable._
 
/**
A multimap for immutable member sets (the Scala libraries 
only have one for mutable sets). 
*/
class MultiMap[A, B](val myMap: Map[A, Set[B]]) {
 
	def this() = this(new HashMap[A, Set[B]])
 
	def +(kv: Tuple2[A, B]): MultiMap[A, B] = {
	  val set = if (myMap.contains(kv._1)) {
		  myMap(kv._1) + kv._2
	  } else {
		  new HashSet() + kv._2	     	   
	  }
 
	  new MultiMap[A, B](myMap + ((kv._1, set)))
	}
 
	def -(kv: Tuple2[A, B]): MultiMap[A, B] = {
	  if (!myMap.contains(kv._1)) {
	    throw new Exception("No such key")
	  }
	  val set = myMap(kv._1) - kv._2
	  if (set.isEmpty) {
	    new MultiMap[A, B](myMap - kv._1)
	  } else {
		  new MultiMap[A, B](myMap + ((kv._1, set)))
	  }
	}
 
	def entryExists(kv: Tuple2[A, B]): Boolean = {
	  if (!myMap.contains(kv._1)) {
	    false
	  } else {
	    myMap(kv._1).contains(kv._2)
	  }
	}
 
    def keys = myMap.keys
 
     def values: Iterator[Set[B]] = myMap.values
 
    def getOrElse(key: A, elval: Collection[B]): Collection[B] = {      
      myMap.getOrElse(key, elval)
    }
 
    def apply(key: A) = myMap(key)
 
 
 
}

Usage:

 
   var theMultiMap = new MultiMap[String, Int]()
 
   theMultiMap += (("george", 1))
   theMultiMap += (("george", 3))
   theMultiMap += (("bob", 2))
   theMultiMap -= (("george", 1))