AI and the politics of perception

Elon Musk, entrepreneur of some renown, believes that the sudden eruption of a very powerful artificial intelligence is one of the greatest threats facing mankind. “Control of a super powerful AI by a small number of humans is the most proximate concern”, he tweets. He’s not alone among silicon valley personalities to have this concern. To reduce the risks, he has funded the OpenAI initiative, which aims to develop AI technologies in such a way that they can be distributed more evenly in society. Musk is very capable, but is he right in this case?

The idea is closely related to the notion of a technological singularity, as promoted by for example Kurzweil. In some forms, the idea of a singularity resembles a God complex. In C G Jung’s view, as soon the idea of God is expelled (for example by saying that God is dead), God appears as a projection somewhere. This because the archetype or idea of God is a basic feature of the (western, at least) psyche that is not so easily dispensed with. Jung directs this criticism at Nietzsche in his Zarathustra seminar. (Musk’s fear is somewhat more realistic and, yes, proximate, than Kurzweil’s idea, since what is feared is a constellation of humans and technology, something we already have.)

But if Kurzweil’s singularity is a God complex, then the idea of the imminent dominance of uncontrollable AI, about to creep up on us out of some dark corner, more closely resembles a demon myth.

Such a demon myth may not be useful in itself for understanding and solving social problems, but its existence may point to a real problem. Perhaps what it points to is the gradual embedding of algorithms deeply into our culture, down to our basic forms of perception and interaction. We have in effect already merged with machines. Google and Facebook are becoming standard tools for information finding, socialising, getting answers to questions, communicating, navigating. The super-AI is already here, and it has taken the form of human cognition filtered and modulated by algorithms.

It seems fair to be somewhat suspicious — as many are — of fiat currency, on the grounds that a small number of people control the money supply, and thus, control the value of everybody’s savings. On similar grounds, we do need to debate the hidden algorithms, controlled by a small number of people (generally not available for perusal, even on request, since they would be trade secrets), and pre-digested information that we now use to interface with the world around us almost daily. Has it ever been so easy to change so many people’s perception at once?

Here again, as often is the case, nothing is truly new. Maybe we are simply seeing a tendency that started with the printing press and the monotheistic church, taken to its ultimate conclusion. In any case I would paraphrase Musk’s worry as follows: control of collective perception by a small number of humans is the most proximate concern. How we should address this concern is not immediately obvious.

The minimal genome of Craig Venter’s Syn3.0

The J Craig Venter Institute has published a paper detailing the genome of their new Syn3.0 synthetic organism. The major accomplishment was to construct a viable cell with a synthetic, extremely small genome: only 473 genes and about 500 kbp.

Even though it is considered to be fully “synthetic”, this genome is not built from scratch. Instead, the starting point is the Mycoplasma genitalium bacterium, from which genes and regions are deleted to produce something that is much smaller, but still viable. This means that even this fully synthetic genome still contains regions and functionalities that are not fully understood. M. genitalium was also the basis for JCVI’s Syn1.0, which was produced in 2008, but the genome of Syn3.0 is the smallest so far – “smaller than that of any autonomously replicating cell found in nature”. Syn3.0 should be a very valuable starting point for developing an explicit understanding of the basic gene frameworks needed by any cell for its survival – the “operating system of the cell” in the words of the authors.

Since so many genes are still basically not understood, the authors could not rely entirely on logic and common sense when choosing what genes to remove. They used an approach that introduced random mutations into the starting organism, and then checked which mutations where viable and which were not. This allowed them to classify genes as essential, inessential or quasi-essential (!). The deletion of essential genes would cause the cell to simply die. The deletion of quasi-essential genes would not kill it, but would dramatically slow its replication rate, severely crippling it. The final Syn3.0 organism has a doubling time of about 3 hours.

Some of the points I took away from this readable and interesting paper were:

Synthetic biology methods are starting to resemble software development methods. The authors describe a design-build-test (DBT) cycle that involve several nontrivial methods, such as in silico design, oligonucleotide synthesis, yeast cloning, insertion into the bacteria, testing, and then (perhaps) sequencing to go back to computers and figure out what went wrong or what went well. Thus, a feedback loop between the cells and the in silico design space is set up.

A very small genome needs a very tightly controlled environment to survive. The medium (nutrient solution) that Syn3.0 lives in apparently contains almost all the nutrients and raw materials it could possibly need from its environment. This means that many genes that would normally be useful for overcoming adverse conditions, perhaps for synthesising nutrients that are not available from the environment, are now redundant and can be removed. So when thinking about genome design, it seems we really have to think about how everything relates to a specific environment.

The mechanics of getting a synthetic genome into a living cell are still complex. A huge amount of wet-lab (and, presumably, dry-lab) processes are still needed to get the genome from the computer into something viable in a cell culture. However, things are going much faster than in 2008, and it’s interesting to think about where this field might be in 2021.


Method and object. Horizons for technological biology

(This post is an attempt at elaborating the ideas I outlined in my talk at Bio-pitch in February.)

The academic and investigative relationship to biology – our discourse about biology – is becoming increasingly technological. In fields such as bioinformatics and computational biology, the technological/instrumental relationship to nature is always at work, constructing deterministic models of phenomena. By using these models, we may repeatedly extract predictable results from nature. An example would be a cause-effect relationship like: exposing a cell to heat causes “heat shock proteins” to be transcribed and translated.

The implicit understanding in all of these cases is that nature can be turned into engineering. Total success, in this understanding, would amount to one or both of the following:

  1. Replacement/imitation as success. If we can replace the phenomena under study by its model (concretely, a machine or a simulation), we have achieved success.
  2. Control as success. If we can consistently place the phenomena under study in verifiable, fully defined states, we have achieved success. (Note that this ideal implies that we also possess perfect powers of observation, down to a hypothetical “lowest level”).

These implicitly held ideals are not problematic as long as we acknowledge that they are mere ideals. They are very well suited as horizons for these fields to work under, since they stimulate the further development of scientific results. But if we forget that they are ideals and begin to think that they really can become realities, or if we prematurely think that biology really must be like engineering, we might be in trouble. Such a belief conflates the object of study with our relatedness to that object. It misunderstands the role of the equipment-based relationship. The model – and associated machines, software, formulae. et cetera – is equipment that constitutes our relatedness to the phenomena. It cannot be the phenomena themselves.

Closely related to the ideals of replacement and control is the widespread application of abstraction and equality in engineering-like fields (and their application to new fields that are presently being clad in the trappings of engineering, such as biology). Abstraction and equality – – the notion that two entities, instances, moments, etc., are in some way the same – allow us to introduce an algebra, to reason in the general and not in specifics. And this is of course what computers do. It also means that two sequences of actions (laboratory protocols for example), although they are different sequences, or the same sequence but at different instances in time, can lead to the same result. Just as 3+1 and 2+2 both “equal” 4. In other words, history becomes irrelevant, the specific path taken no longer means very much. But it is not clear that this can ever truly be the case outside of an algebra, and that is what risks being forgotten.

We might call all this the emergence of technological biology, or technological nature, the conquest of biology by λόγος, et cetera. The principal danger seems to be the conflation of method with object, of abstraction with the specific. And here we see clearly how something apparently simple – studying RNA expression levels in the software package R, for example – opens up the deepest metaphysical abysses. One of the most important tasks right now, then, would be the development of a scientific and technological culture that keeps the benefits of the technological attitude without losing sight of a more basic non-technological relatedness. The path lies open…

Is bioinformatics possible?

I recently gave a talk at the Bio-Pitch event at the French-Japanese institute. I was fortunate to be able to speak about some of the ideas I’ve been developing here among so many interesting projects (MetaPhorest, HTGAA, Yoko Shimizu, Tupac Bio, Bento Lab etc).

The topic of my talk was “Is bioinformatics possible”? A deliberate provocation, since of course many people including myself work with this every day. I simply mean to suggest that there are intrinsic problems in the field that are not usually discussed or thought about, and that it might be valuable to confront those problems.

The slides are available, if anyone is interested.

The bigger topic that is hinted at, but not discussed, might be the instrumental relationship of humans to nature. I hope to return to this problem soon.

Reactive software and the outer world

At Scala Matsuri a few weeks ago (incidentally, an excellent conference), I was fortunate to be able to attend Jonas Bonér’s impassioned talk about resilience and reactive software. His theme: “without resilience, nothing else matters”.

At the core of it is a certain way of thinking about the ways that complex systems fail. Importantly, complex systems are not the same as complicated systems, although in everyday speech we tend to confuse the two. Perhaps a related or even identical question is: how do composite systems fail?

Using a terminology that originates with the Erlang language, Bonér talked about the “error kernel”, which is the part of a software system that must never fail, no matter what. As long as this innermost part stays alive, other parts are allowed to fail. There are mechanisms to replace, restart or route around failures in the outer parts.

This style of design leads to a well-structured failure and supervision hierarchy. Maybe this style of thinking itself is the most important contribution. In most software systems being designed today, the possibility of errors or failures is often a second class citizen, swept under the carpet, and certainly not part of a carefully considered structure of possibilities of failure. What if this structure becomes a primary concern?

Once errors are well structured and organised in a hierarchy, it also becomes easy to decide what to do when errors occur. The hierarchy structure clearly indicates which parts of a system have become defunct and need to be replaced or bypassed. Recoverability – being able to crash safely – at every level takes the software system a little bit closer, it seems, to biological systems.

Biological systems, Bonér pointed out, usually operate with some degree of inherent failure, be it disease, weakness, mutations or environmental stress. Perfect functioning is not typical, and it seems to me that for most organisms such a state may not even exist.

Recoverability at every level, resilience, and error hierarchies – “let it fail” – is truly a significant and very humble way of thinking about software. It means that as the developer, I acknowledge that the software I am writing does not control the universe (although as a developer I often fall prey to that illusion). The active principle, the “prime mover”, is somewhere outside the scope that I control. When it produces some unforeseen circumstance, we must respond properly. Reactive software to me seems to quietly acknowledge this order of things.

I have only had a very brief opportunity to try out Akka, Typesafe’s actor framework, in my projects so far, but I felt inspired by Boner’s talk and hope to use it more extensively in the future.