Lintsbakh Machine

By Yuri Tsivian, U of C

I am not sure how well what I am going to say and show may fit into the context of our discussion. The conference is mostly about the present and the future, and my paper is a sally into the past. The machine I want to discuss was not intended to be a game, though there is enough fun in playing with it. Its relation to the world of computers is a theoretical one; as a student who helped me to make it work suggested to me, the model I am going to present looks like a hypothetical multimedia device, by analogy with Turing machines – a 1936 theoretical prototype of modern computers. I know too little about Turing to insist on this analogy, particularly since Lintsbakh’s hypothesis was proposed twenty years earlier and has remained unknown to people involved in computer science. But, on the other hand, it is exactly this early date and the lack of wide recognition that makes me think it may be of interest to people working in the area of human/computer interface design – and to those who, like us here, seek to put this area into a cultural perspective.

The following account will hardly be effective without a few audio-visual illustrations which Lintsbakh – the hero of my talk of whom more later – was only able to present in the form of symbols, drawings and notations in music. I will use the time given to me at my panel to illustrate the points I introduce below using modern tool to give life to Lintsbakh’s theoretical ideas, but all the images you will see on the screen are facsimiles from his book, as well as the melodies I will ask you to listen to. This made me think it unnecessary to supply the following brief account with copies of drawings, though you will find references to them inserted in the text.

Jacob Lintsbakh (1874-1953) – an Estonian linguist and a mathematician living in Petersburg -- was one of those people that believed in the possibility of an artificial universal language of the future. In 1916 Lintsbakh publishes a book – in Russian – in which he criticized the existing artificial languages like Volapuc and Esperanto and offered his own, based on a completely different principle. The mistake of Esperanto and others, Lintsbach claimed, was that they emulated the phonetic principle of natural languages. The universal language of the future will be based on writing rather than speech. What he meant by writing was more than using a pen: it was close to what we now call multimedia – a semiotic ensemble that includes pictures, body language, the language of music, mathematical formulae, glossalalia, geometrical graphs, and stylized animated cinema which Lintsbakh variously called (1916 was the twentieth year since Lumieres’ invention of cinema) "living schemes," "the cinematograph of icons" and the "manual language of the mind."

The amazing thing about Lintsbakh was that, dreamer as he was, his book would not stop at the stage of philosophical prophecy or vague theoretical suggestions, but actually creates an elaborate network of rigid semiotic, linguistic and mathematical correspondences as a result of which any utterance that we are able to formulate in a natural language can be translated into another medium. I am going to try to address Lintsbakh’s project in as much detail as I have time for, but this will be far from giving him his due. Although I think I was able to grasp the mathematical principle of its construction, I will probably leap some important stages of Lintsbakh’s line of argument, but I must assure you that for all its strangeness, there are no lapses or glitches in the mechanism of the language machine he constructs.

He begins with a picture (in this, Lintsbakh falls in with Francis Bacon’s tradition of putting visual images before words) representing a sample situation which the book stick to throughout. The picture looks like this (fig. 28; I’ll keep Lintsbakh’s numeration of his examples). Its verbal equivalent is: On a road that lays under a mountain a horse is running from right to left. It drives a sledge on which a man is sitting. Picture 29 disjoints the situational coherence of the previous one, leaving each of its five elements context-free. Let us now (Lintsbakh says) give back to the picture its former spatial coherence, but now rendered, not simultaneously, but in sequence (the sequence on fig. 31, for instance, corresponds to the above verbal description); let us call (he continues) this sequential type of coherence a "cinematograph."

The problem with this specific sequence is that it is arbitrary, for (in Russian) it is very much up to the speaker which word in the verbal utterance comes first: whether the road is mentioned before or after the mountain, for instance. But we can decide to arrange our sequences so as to give it a sense of spatial orientation. If we agree that the objects which are farther away from us be always shown first, it will make it clear that the road runs this, not that, side of the mountain (fig. 32).

We can also inform such kind of sequencing with a temporal dimension, and this without losing the three dimensions of space. For the sake of convenience, let of rotate the sequence on fig. 32 vertically, to be read from top down, like a Chinese inscription. Then our reading of it along the time axis will run left to right if we put several such "cinematographs" side by side.

The sequence of pictures on figure 45 tells what had taken place before the man embarked on the sledge ride. If you read this as columns, the information you obtain will be about space: column one, for instance begins with a line which in this case is not a road but the horizon. It says that in a house under a mountain there lived a man. Early in the morning (note the sun under the line) he got up, left the house, harnessed a horse, etc. To give a sense of purpose to the ride, says Lintsbakh, add what he calls "an inserted mental scene" (fig. 46) – he was heading to the city.

This may look clear on a book page, but what would we do if we wanted to run it real time, as the real cinematograph does? One way to do it is to run it as an alternating space-time series, as we one were cutting a loaf of time into slices of space (medical scanning might be a more modern analogy): four frames for space t1, another four frames for space t2… etc. Figure 52 shows a longer version of the sample story, and that is how it would look if Lintsbakh had convinced someone to put his theoretical "cinematograph" on a real film (which in 1916 was technically as feasible as now, but would involve a serious investment). I asked Gunars Civjans who designed this interface for me to run it at the speed of seven frames per second, for on the strength of one or another psychological theory current at his time Lintsbakh believed this was the speed with which mental images flashed in human mind. (RUN FLASH)

So far there has been little or no mathematical reasoning involved, for all that Lintsbakh has been doing so far is to lay geometrical (we would nowadays say ‘analogical’) foundation of what would become his language machine; in other words, what he need to go on is a visual utterance with four identifiable space-time dimensions. Having set those, he goes on from there to the analytical (and also what we might call ‘digital’) stage. Once we have this four-dimensional model of an utterance, we will be able to encode it mathematically by using the grid of Cartesian co-ordinates and a calibrated time axis added to it. One problem here is that the calibration of time has to be the same as the calibration of the three spatial dimensions, and Lintsbakh opts for the simplest one: the binary scale of notation which allows to define any point within our space-time ‘cinematograph’ in terms of a string of ones and zeros. (As we will see in a minutes, speaking of music and gestures, he finds binaries boring, and for the sake of expressivity occasionally switches to the octuple scale).

As it follows from this, our sample utterance about the man and the horse can be expressed algebraically, which chapter I will mercifully omit, although, in the wake of his seventeenth-century predecessors, Lintsbakh believed that mathematical symbols are more perfect than words. What makes Lintsbakh’s machine interesting for us (that is, in the perspective of its anticipated future, not of its philosophical past) is not so much its mathematical backstage, but the stage, the display, the ideas of interface design before the date. There is nothing, he writes, that prevents us from using our organs of speech to re-encode any kind of binary encoding by simply substituting, say, the vowel "o" for all "0"s and the vowel "a" for all "1"s (table on page 155). The only thing is, he adds, that this two-sound language will be monotonous, and sound like the croaking of frogs; but again, nothing prevents us to switch to a less uniform, though no less rigorous scale of notation, using consonants along with vowels; then the picture of the house in which the man lives would render as: (SOUND RECREATION THE OF EXAMPLE ON PAGE 176).

A similar series of operation applied to the language of music gives us, first, the monotonous binary music (figs. 116-117), for instance if we denote zeros by "C" (do) and ones by "G" (sol); but we may as well deploy the entire octave if we assign certain tones to this or that combination of digits. Likewise, the language of gestures (another important avatar of Lintsbakh’s machine) should not consist of mere hand-up and hand-down signs (figs. 123-124), but use the rich vocabulary of body language that people acquired in the whole course of the history of dance (fig. 133).

It would turn my talk into a tedious (indeed, monotonous) experience if I went on discussing Lintsbakh’s alphabet of colors that serves in a similar capacity, or some other media his machine is capable of using (fig. 142, perhaps with color codes added?). The only I want to show before I conclude is the way Lintsbakh uses some of them to render his sample utterance: On a road that lays under a mountain a horse is running from right to left. It drives a sledge on which a man is sitting. (Fig. 121). The underlying shape is the picture showing this in the analog format; the grid serves to digitize (analyze) it by scanning it from left to right and assigning to each of its points a numerical and algebraical values; in terms of music, the summit of the mountain, for instance, corresponds to the highest tone; he also suggests that louder sound may be used for closer objects and less loud for those farther away; or mellower timber for animate objects (man, horse) and harsher for the sledge and the mountain. Thanks to modern technology (and to young people who helped me to make it work) this can be now easily displayed in the medium of sound; as well as it can be shown in the language of gestures which the signs at the bottom are referring to: (PLAY GESTURES + MUSIC]

To conclude, let me try to put Lintsbakh’s idea into a broader cultural perspective: how does it look in the light of a conference on games? In his famous book Homo Ludens (Playing Person) Johann Huizinga defines the game as an activity which has no purpose other than itself, saying that in this sense all human culture is one large game. As a language Lintsbakh’s machine is immensely impractical – unless we imagine the speaker as a man-band equipped with a carry-on blackboard, portable piano and a baby film projector (although, of course, the piano is not strictly necessary – you can whistle the phrase you want to say.) But at one point in his book Lintsbakh lifts his eyes from the narrow mathematical path to ask exactly this question. He says that the future of Universal language is in the hands of artists, not linguists or scientists. My task is modest, he says: I am laying a logical/mathematical foundation for the forms I am not able to predict.

Let me now quote yet another theory: a definition of art that stems from Russian Formalist School of the twenties later refined by Roman Jacobson in Prague and still later in the United States. I know some think what they say is too limited, but as definitions go, this is the best we have. According to this theory, art is the use of practical means for non-practical ends. The whole practical point about walking, for instance, is to take a person from one place to another; but what about dancing? Dance equals walking minus its practical goal. By the same token, the practical function of using language is to communicate, that is, to exchange messages; but when we read poetry what we respond to is not a message, but to the language as such.

I know this may sound simplistic and it probably is, but let us assume for a moment that the practical goal of computers is to compute, and do it fast so that we can save time on computing. Interface is another practical device that saves us time learning how to handle computers. But if someone, instead of saving time chooses to lose it playing with interface for its own sake, there may be a chance computer games have a future as an art form. What I wanted to say by showing Lintsbakh’s model (apart, of course, from the main, perfectly unpractical purpose of playing with it) is that there existed a person, an actual 1916 species of Huizinga’s Homo Ludens, who decided to play with what we now call multimedia even before they became an available reality, and play it by mathematical rules that the modern computer, as you have seen, is able to understand.