Programming, simplicity and art

Programming, the writing of computer code in order to solve a specific problem, is a new intellectual discipline. It has a history going back to logic and mathematics, but it is relatively new as a human endeavour. It is constrained by hardware, by mathematics, by programming languages, and what we might call technical culture: APIs, programming interfaces, and concepts already invented by others. It is a vast canvas for thought: in the machine, imagination becomes physical reality.

For mysterious reasons I became fascinated with computers immediately when I saw one as a child in the 1990s (a friend’s Commodore 64). They seemed to be a window into a different world. As a teenager I tried to teach myself how to program. Sometimes I was self taught, sometimes formally trained, and over the course of my life it feels like I’ve touched (with varying aptitude) everything from hacky PHP and Basic to more rarefied Prolog and Haskell (today I almost exclusively use Scala). Having studied computer science academically and worked in a few fields, today I work in bioinformatics, where we study DNA and genomes – perhaps the “code” of biology and cells.

In my own estimation, my skill and creativity in programming is still increasing as I approach middle age. But over time I have come to value different things and think differently about this craft. It struck me that some of the perspective shifts I have made might apply to human creativity in general.

The biggest single such shift was a new focus on simplicity that I found only recently. In my twenties, I worked on many projects that were maybe too complex for the sake of being complex. I was probably (I’m a little ashamed to say) impressed by the difficulty of the things I myself was doing. This style of work was not pointless, as I could still produce meaningful results, and I was constantly learning. It was the free exploration of new frontiers. But it was powered by youthful arrogance: I would not easily have listened to older people telling me at the time to keep things simpler. I wouldn’t have known what they were trying to tell me. I would have insisted on covering all this ground on my own.

At some point, my designs got too big to sustain, and I was held back by my refusal to simplify things I was making. This was part a reluctance to part with work that I had put a lot of time into, and in part an indecisiveness and a desire to let my project be all things to all people. Slowly I realised that this had to change. I took the plunge and gave one of my projects a much sharper focus. The quality improved dramatically and I could keep up the work for much longer. I was wondering why I had not arrived at such a simple truth earlier.

What does such a sharp focus look like? I will try to generalise. In order to find simplicity we need a source of truth, a meaningful signal to measure against. In programming this could be the fundamental limits of algorithms (how quickly can we sort strings, how much memory do we need to represent them?) or of the domain problem (how quickly can we align DNA against a subject genome?). It could be product requirements (how fun can we make a video game that we are developing?). Once we have a source of truth, we can measure the value of any changes or additions we are making. Generally, adding code increases cognitive load, as the project becomes harder to understand and communicate. When we can measure value, we can investigate the ratio of the value gained to the cognitive price paid for it. It may not be a good return on investment. If not, we should refuse to add that code. Perhaps we make a note of the decision made and the reasons why we made it.

Simplicity often correlates with removing things, though size is not the only measure. What we should care about the most is cognitive load, the ease of understanding for a reader (and of course, ease of use for users). Code that is easy to understand is honest, and easy to return to and build on in the future. It is easy to communicate to others, so it can be a good basis for teamwork. But because this often does mean that we must mercilessly remove things that we put many hours into, the surviving code might often be just the tip of an evolutionary iceberg. Maybe in order for five branches to survive in the repository, fifty branches had to be attempted (hopefully leaving some records of what was learned). In order to find the best algorithm for a research problem, maybe ten algorithms had to be tried, and the nine unfortunate ones ripped out for the sake of simplicity. Keeping code of unclear value around (opportunistically) leads to confusion, although it can certainly be kept in some private branch in case it is needed in the future.

In general, a lot of experimental, creative processes that are trying to home in on a goal will have a lot of hidden history, in arts and crafts as well as technology. Certainly in technical and scientific research. A perfect gem can be the result of laborious experimentation that perhaps nobody remembers anymore. How much work went into discovering even the basic designs and appliances that we use in our homes every day? How many melodies did Bach never write down?

Although that may seem tragic, it brings vast benefits and is maybe the only way to reliably have a sustainable creative process in certain fields.The world might always remain complex, but we can mostly demand that productions of the mind should be simple. (I would make an exception for fields like philosophy and some of the arts, where the objective may sometimes be to undo ingrained simplicity for the sake of increased contact with reality.)

Finally as I write this first blog post in a couple of years: let’s keep blogging alive! Blogs were once part of a better internet that we could still preserve to some degree, and, I think, a format that will outlast the insane social media frenzy of today. He who has eyes, let him read.

Some books I read in 2021

Another year of the pandemic. I thought I’d wrap up by briefly reviewing some books I’ve read during the year.

Friedrich Nietzsche – The Birth of Tragedy. Despite this being probably Nietzsche’s earliest famous book, I somehow never managed to get a foothold in it previously. Here he formulates his early philosophy in terms of the Dionysiac and Apolline.

Friedrich Nietzsche – Human, All-too-human. (Re-read) Nietzsche’s turn to what he considered a more “scientific” viewpoint after his crisis and break with Wagner.

Seneca: Letters From a Stoic. Seneca’s letters to his protégé about life, stoic philosophy and politics are incisive and almost humorously eloquent. Every time I read something from Roman times I’m struck by how similar their society and political lives seem to have been to ours. Stoicism as a philosophy I don’t feel ready to comment on.

Leo Tolstoy – The Cossacks and Other Stories. The painterly style here is every bit as good as in War and Peace. The titular story impresses, as do the Sevastopol Sketches and the later Hadji Murat about an aging warrior.

Dino Buzzati – The Tartar Steppe. Magical realism. A meditation on organizations, ambition and time. Giovanni Drogo spends his entire life waiting for an event that might never happen. People posture, follow rules to a fault, die.

Neal Stephenson – Fall, or Dodge in Hell. This sci-fi novel explores the ultimate conclusions of the present “post-truth” algorithm-driven political landscape that we live in very well. It also explores, very elaborately, the notion of a simulated digital afterlife in ways that I found believable (assuming that we accept the notion of a simulated consciousness for the sake of the story). However, after a certain point the book becomes tedious and gets lost in its own plot. By that time readers should feel free to skip to the end.

Ernst Jünger – Storm of Steel. Jünger’s shocking, aesthetically concerned, unflinching account of WW1 is truly surreal and allows us to see the the world through the ambitious young eyes of an exposed frontline soldier at the time.

Immanuel Kant – Critique of Pure Reason. As of the end of 2021 I’ve begun on this key work in earnest, making some progress with the help of Jay Bernstein’s lecture course. If I make it through the entire book I will report back.

I will end with an admonition. I don’t have much to say about the ongoing Covid pandemic, except that if you are hesitant to get vaccinated, you may be putting other people at risk. We can and should debate the politics of how the pandemic is being managed – a topic for a different time – but getting vaccinated is common sense (it certainly is not a political stance), so please do it.

Happy 2022.

Covid-19 and time

I can now conclusively answer the question raised at the end of my blog post from December 2019: the 2020s are not a decade of orderly peace. What a strange year. But weren’t years always strange?

Time passes not only quantitatively but also qualitatively. A year spent with Covid-19 seems to have passed differently from a year without it. Maybe boredom has increased (possibly for the better), maybe focus has increased. For many there has been, and continues to be, untold suffering. The mRNA vaccine, based on a technique that was overlooked by academia for a long time, at this moment looks to become one of the great success stories of science. Possibly and hopefully. Many countries have aggressively and successfully vaccinated large swathes of their populations; Japan is unfortunately still in the early stages of this process. The pandemic year almost seems to have removed us physically from the world before, opening up a widening chasm that we may or may not be able to cross again. Looking across that chasm from 2021, both the present time and that time before the pandemic seem alien and strange.

Around this time last year, when lockdowns were starting to happen in some countries, the prospect of a two week lockdown seemed to me an unbearable burden. (To date Japan has not had a hard lockdown.) Today, of course, that seems like it would have been a small price to pay, given what some other countries, like Taiwan, have achieved. I must not be as good at deferring present benefits for future rewards as I thought.

On a different note, for the technically-minded: During the past year I was able to channel a lot of energy into research on k-mer counting, a genomic data problem. A medium post (in the early stages of this work) and eventually a paper were published. The main achievement was to find a novel way of grouping k-mers — short genomic subsequences — into evenly sized bins, which greatly speeds up many kinds of processing. This would not have been possible without previous work on FastKmer (Ferraro Petrillo et al) and the new randomized algorithm PASHA for generating compact universal hitting sets (Ekim et al). This work may also be an interesting case study on the low hanging fruit available when aggressive engineering, specifically for the purpose of improving parallelism, is applied to existing algorithms.


It seems clear by now that Covid-19 is one of the great crises of our time, more so than the financial crisis of 2008, judging by the size of the measures announced and already taken by various countries and by the impact on human lives. In Japan I have been following live data from Toyo Keizai, the Tokyo government, and an independent source tracking hospital bed capacity.

While this will be a tragedy for many, it will also be an opportunity for societies to reset themselves and to make changes that otherwise could not have been made. New perspectives urgently come to the fore. Perhaps we discover how much of the everyday frenzied activity we were caught up in was superfluous. Certain viewpoints and positions may have to be abandoned as people come together around trust, resilience, survival, innovation.

Thus the present crisis is also a time for reflection and renewal.

The year and decade in review. 2020s: orderly peace?

2019 comes to a close, and with it the 2010s. Below are a few thoughts on these periods of time.

The most significant book I’ve read in 2019 is probably Hannah Arendt’s The Origins of Totalitarianism. The German title, literally “Elements and Origins of Totalitarian Rule” more closely reflects the contents of this monograph. Arendt explores antisemitism, imperialism and totalitarianism to form a grand analysis of totalitarian forms of government, which she considers to be genuinely new and unprecedented. Those who make it through the somewhat slow early chapters will be richly rewarded. It’s a very timely book – although written in the 1950’s, most of the ideas feel like they could be from last week. Elements of totalitarian rule are absolutely something we should worry about.

Another notable book from this year has been Edward Snowden’s Permanent Record. Aside from the obvious political dynamite, I found myself relating to a lot of the experiences he had growing up. Perhaps this is a generational story. In the late 90s, the Internet suddenly became relatively mainstream and for a short while, it was a very special place, seemingly full of utopian promise and all kinds of possibilities and exploration. For many born in the mid-80s this coincided with our teenage years.

I’ve lived in Japan throughout the 2010s, the final part of the Heisei (平成) era. In 2019 this era came to a close and we are now officially in Reiwa (令和). I can’t easily summarise the 2010s. Both my personal life and Japan seem to have undergone great change during this time, and sometimes it’s hard to separate one from the other. The Fukushima incident in 2011 was perhaps a watershed moment that Japan is still grappling with. Although the future of nuclear power has not yet been resolved, the country’s response to such a tense incident has in many ways been admirable, and the famous Japanese virtue (sometimes a double-edge sword) of stability certainly came through. The surrounding world is also changing, and Japan, though still a relatively separate culture, is becoming considerably more open and mixed as a society, perhaps out of necessity. Tourism and labour imports have both increased significantly. This raises interesting questions about what kind of society Japan might be in 10 – 20 years.

During the decade I have had diverse personal and professional experiences. I lived in Tokyo, Osaka, then Tokyo again. I was able to complete a PhD thesis. I visited many countries for the first time, and became interested in bioinformatics (mainly as a field in which to apply fundamental computer science and software engineering). I took up several new hobbies, obtained permanent residency in Japan, and was able to improve my Japanese to the point of reading novels, although I’m still not quite where I’d like to be with the language. I’ve been reading a lot of philosophy and general literature and tried to systematically develop a worldview (fragments of which sometimes appear on this blog). Not everything I tried to do worked out the way I expected, but the learning has felt very valuable, and I do feel much wiser and more capable about my approach to many things. I expect to be sincerely expressing the same sentiment in the year 2029, though.

One technical focus this year was improving my Spark (and Scala) skills and developing an algorithm for De Bruijn graph compaction (similar to what Bcalm does). I was pleased with the efficient research process I was able to achieve, probably my best ever on this kind of project. In terms of my professional path, the overall trend for me seems to be towards smaller firms and greater independence. (Although I remain with Lifematics, I will now also be available for consulting and contracting opportunities in bioinformatics as well as general software development. If you are reading this and think you would like to work with me, do get in touch.)

Thus ends a politically very strange decade, from a global perspective, and we enter the brave new world of the 2020s. Will it be a time of “orderly peace”, as the name 令和 suggests?