the rhythm images hit the eye like the rhythm of sounds reach the ear   by Luca Tanzini (Dipartimento scienze della comunicazione university of Siena: tanzini@media.unisi.it)     Plato believed that the world was made according to musical principles and harmony and rhythm ruled man's inner self. Aristotle wrote: "Given that some re-create the world through figures and colors, and others through sound, in the arts as well, all of us re-create by means of rhythm, dialogue, and harmony, and these or those separately or mingled."     De Natura Animalium, Aristotele.     Since its origins, Occidental Europe has been teeming with theories that link aural sensation to visual sensation, music to painting. Music theorists were the first to approach the idea. They tried to create a "fusion" of music and color by creating an instrument that could produce different colors for different musical notes.   The first attempt at "painted music" was in 1725 and 1735, when the Jesuit Louis-Bertrand Castel introduced the clavecin oculaire (ocular clavichord). The instrument was meant to paint sounds with corresponding colors in such a way, claimed Castel, that a deaf person could enjoy and judge the beauty of a musical piece through the colors it created, and a blind person could judge colors through the sound. The instrument functioned like a traditional clavichord, excepting that each note was associated, in accordance with Castel's own exhaustive studies, with a particular color that would be displayed upon the playing of each note.     Cover of "L'optique des couleurs" by Louis-Bertrand Castel     On the 16th of January 1877 Bainbridge Bishop patented a coloring organ that simultaneously played music and projected colored lights through illuminated windows.     Bainbridge Bishop's "Color Organ"     In 1893 Bishop published "A Souvenir of the Color Organ, with Some Suggestions in Regard to the Soul of the Rainbow and the Harmony of Light," a short pamphlet in which he describes his experiments and ideas on the relationship of notes and the primary colors of a rainbow. In 1895 the Englishman Wallace Rimington conceived of a small music box that contained many apertures with colored glass and an electric wire. The apertures could open and close ­ projecting colors on a white screen ­ by playing a soundless keyboard.     Wallace Rimington and his "Color Organ"     The construction of such instruments continued throughout the 19th Century in the attempt to discover the "scientific" link between sound and color, but the period that saw the greatest experimentation was the first three decades of the 20th Century. In that period, everything was tried: organs that produced music or color, or keyboards that created colors without making a sound. Nevertheless, the marriage between music and color could also be made by endowing the picture with a temporal dimension like that of music. This concept saw a flowering of experimentation and theoretical hypotheses in Europe in the 10 years preceding the Great War. Influenced by the experiments and research of Bishop and Remington, in 1909 the Russian composer Aleksandr Skrjabin wrote the symphonic poem "Prometheus," in part of which the notes are meant to correspond to certain colored lights.     Aleksandr Skrjabin     Skrjabin wanted to create a keyboard of lights; colors would correspond to traditional keys according to his own visionary idea of a cosmic synthesis of sound and light. Skrjabin commissioned Alexander Mozer to build the device. Mozer, a photographer and electro-mechanics teacher at the Technical Institute in Moscow, completed the device in a few months time to be ready for the first demonstration of Prometheus (15 March 1911). The device had a fundamental component all Mozer's own: 12 colored lamps placed in a circle on a wood base were lit up by pulses. It is currently on display at the Museum House of Skrjabin in Moscow. Arnold Schonberg must have had Skrjabin in mind when he began composing Die Gluckliche Hand (The Happy Hand) in 1909. The score specifically outlines plans to project colors on a screen that move with the music: "The game of light and colors is not based only on intensity, but on values that can only be compared to the heights of sound. Sound and color mingle freely only when their relationship is, at root, reciprocal. In a letter to the Viennese publishing house "Universal Editions," Schonberg declared "What I'm looking to do is the exact opposite of what cinema normally hopes to achieve. I demand the greatest unreality! The general effect doesn't have to be dream, but something similar to music, to harmony. "     Arnold Schönberg     In both the almanac Der blaue Reiter and the book The Spiritual in Art, Wassily Kandinsky attempts to construct a theory of pictorial harmony by analyzing the effect color has on the viewer. "If color is a means of influencing the soul, you can than say that pictorial art in the future will use colors as sounds, and canvases will be divided into two categories: simple compositions (or melodies) and complex (or symphonies)."(1)     Cover of the Almanac Der blaue Reiter     Kandinsky believed in an art in which the comprehensive dynamic created by different means (color, sound, movement) would hit on a deep, internal level capable of resounding, in various ways, on the spirit of the viewer.   With the Futurist brothers Ginanni-Corradini, better known as Arnaldo Gina and Bruno Corra, conceived of chromatic music while they were studying Byzantine mosaics in Ravenna. They declared their idea in the manifesto Arte in 1910, claiming that colors create both a harmonious music and a sonorous one. They could, they exclaimed, express feeling and states of being with notes and equally compose harmonies, motifs and symphonies. "You can create a new and more rudimentary form of pictorial art by using a mass of color harmoniously mingled, one on top of the other, in such a way as to please the eye without representing a figure. This would correspond to what in music is called harmony; we can therefore call it chromatic harmony. Like music (a series of notes over time), color can give shape to a temporal art that is an assortment of chromatic tones successively hitting the eye, a movement of color, a chromatic thread." (2)     Arnaldo Ginna     Corra sought to put the idea of music to color into practice; he built a piano with 28 keys that correspond to 8 differently colored electric lamps. By pushing one key, a color would be projected over a background. By pushing many keys, the colors would form a harmonious light. This method soon revealed its simplicity: the effects were pretty, but lacked an emotional core, the fusions were arbitrary, little intensity and nothing of true "orchestral effect."     Bruno Corra     Dissatisfied with his first music-color experiment, Corradini decided to venture into new territory: abstract cinema. This time, colors were painted directly onto film in the hopes of creating a chromatic symphony capable of visually reproducing feelings and emotions with music that inspired the compositions.   In 1914, the painter Leopold Survage, doubtless inspired by the first design animations, came up with the moving painting, or movement of color forms, that worked as a succession of paintings taken in one by one: "The colored rhythm is neither illustrative or interpretive of a musical pieceSound is the primordial element of music. Combinations of musical sounds make music. [...] The key element of my dynamic art is the visual colored form that functions like the sound of music. [...] from a psychological point of view, neither color nor sound ­ absolute, isolated ­ touch us and influence us; it is the alternate sequences of colors and sound that touch us."(3) Survage's ambition was not to create a meeting point for diverse languages, but to invent a new, autonomous, artistic medium: colored rhythm was, like film, thought to be another language altogether with music as its model and the movement of colors its preoccupation. Survage's argument, compared to Corra's, comes closer to forming abstract shapes, not mere colors.     Still from Leopold Survage's film, Le Rytme coloré, 1913     Viking Eggeling began working on stretching the limits of painting and painting's static framed structure. To Eggeling, moving beyond the traditional picture meant not only a projection in space but the kinetic DILATAZIONE of the temporal form, the achievement not only of a new dimension, but a completely new structure: a new structure of movement that would develop over time and engender a linguistic model capable of working with it. As the ties to music were so central, the fusion of form and movement in an organic moment had to be structurally perfect piece by piece yet permanently mobile. Eggeling defined the language of movement by making an analogy: "The process consists of dramatic developments and evolutions in the sphere of pure art (abstract forms): it's analogous to the movement a listener experiences when he listens to music. As in music, here too the storyline is made with pure material and this pure material adds to the tension-and-release." (4)     Viking Eggeling     Eggeling's philosophy was greatly influenced by the personality, and theories, of Ferruccio Busoni, a composer, musician and Bach scholar. From Busoni came the idea of contrapuntal music, an extremely formal, precise model that posits the idea of an elastic contrapuntal score. But Eggeling's vision didn't reach so far to an equivalent elastic form of contrapuntal music; he was more interested in finding a universal language that synchronized formal principles and spatial opposites, constructing a rhythmic evolution he felt to be terribly necessary.     Still from Viking Eggeling's film, Diagonale Symphonie, 1921-1924     The idea of an expression that could be to the eye what sound was to the ear was taken up again roughly 10 years later by the French director Germain Dulac: "The film we all dream of making is a visual symphony of rhythmic images, one that only the sensibility of an artist can piece together and put up on the screen."(5)     Germaine Dulac     For Dulac, the relationship between music and image meant a chain of free association that continuously renewed itself and avoided strict formality, aiming at a flow that stirred emotions, a horizon of suggestivity and movement more than a rigidly defined linguistic pattern.   Léon Moussinac, in his own words, echoes this thought: "The cinematic poem as I see it is so nearly related to the symphonic poem, given that the images of the former are to the eye what the sounds of the latter are to the ear [...] the subject is no longer the essence of the work, but the pretext or, better yet, the visual theme."(6)   Abel Gance corroborates: "There are two kinds of music: that of sound and that of light. Wouldn't we say that it can play upon our senses with the same energy and the same refinement?"(7)     Abel Gance     Eugen Deslaw maintained that "the rhythm of images should eliminate the documentarian and instructive aspects of the scenes. They don't need to be understood, simply felt. Silent films aren't made anymore, so I decided to make a sound film as surreal as possible. "The musical accompaniment will be made with the Rumharmonium of the futurist composer Russolo. The music will be styled to the same rhythm of the images. Russolo called this rhythmic accompaniment."(8)     Luigi Russolo's INTONARUMORI     Fernand Léger said, "The future of cinema depends on purely fantastic and imaginative inventions [...] cinema errs in its subject [...]freed from this negative weight, cinema can become the gigantic microscope of things never before seen or heard."(9)     Fernand Léger     The final outcome of early 20th Century thought was to forget subject, make cinema a visual music and express oneself through a rhythm that stood for nothing but itself. They also erred. The search for pure art or pure rhythm was channeled only for the benefit of animated images, the discoveries and freed spirit with which the experimentation of music to colors had begun. In the end the visual rhythm as it stands for itself means nothing: there is no pure rhythm in cinema, just as there is none in literature, music and theatre. As Antonin Artaud wrote in 1927, "The idea of a pure cinema is wrong, as it is wrong in any art form to enforce a principle upon it." (10)     Antonin Artaud     One of the few to see things clearly was the French director Henri Fescourt. "Visual music," he said, "is a possibility and manner for the cinema of tomorrow. To what should it apply?" (11)         Music to Colors: Life and Art     The avant-garde sought to form, and develop to its fullest, experimentation. Music to Colors takes it a step furtherwe are concentrated on life energy, the consciousness of living, not objective art. Rhythm is only important as it involves a natural process of creation and destruction of life energy. Making music, like any action, means to make, act, be. It results in an action's end and in its gesture, without objective purpose. Music to Colors is only one "instrument" that allows us to realize the creation and destruction cycle.   It involves a system of mixed technologies (analogical ­ digital) that allow us to realize and adapt the ideas of the first experimenters in a live music performance.     Model of Music to Colors     The first part of the system is an interface Pitch to MIDI converter that transforms an audio analogue sign that can come from any sonar source into MIDI digital messages.     The Transformation of the Audio Analogue Sign into MIDI Messages   The Pitch to Midi interface converter is connected to a simple MIDI interface that receives and transmits signals to and from a computer's serial or USB port.     Reception/transmission of MIDI messages   It should be made clear that the MIDI protocol does not transmit sound, but information relative to the process. This information is played by one or more instruments hooked up to the system and transmitted to a computer. The MIDI messages are digital signals made up of numerical sequences in binary form that then travel in serial form. The amount of information the MIDI program can carry depends on the instruments being used and the manner in which they are being used. The speed of the transmission is high: 31, 25 Kband, in which 31250 bits a second allows an input-output correspondence.     MIDI System     The final part of the system consists of a computer with a serial or USB port attached to a video projector. The computer receives MIDI messages that have been generated during the performance. The software, Music to Colors, programmed in LINGO (program language of the Micro-media director application) enables the system to respond to sounds, automatic montages of video sequences to the MIDI messages that in real time are received by the computer.     MIDI messages received, audio-video projection montage   Music to Colors creates a literal communication line between the musician's sound and the computer's image. There is nothing hyper high tech or avant-garde about the system because it has been created with computers and instruments by now in disuse, with a completely different approach from the technology of VJs that has been in fashion for some years now. In fact, with Music to Colors, you can use any kind of video material. This means that the usual feedback will not be the result of automatic computerized processes. Instead, the "automatism" factor will prime the visual rhythm of the images recorded during the acts (photographs, video takes). The flow and rhythm of the visual will avoid narration and instead look to dismantle the "record of the experience" and thus constructing a natural platform of Music to Colors. Luca Tanzini September 2004   (1) W. Kandinsky - F. Marc, Der blaue Reiter, Monaco, maggio1912.   (2) B. Corra, Musica cromatica, in "Il pastore, il gregge e la zampogna", 1912.   (3) L. Survage, Le Rytme coloré, "Les soirées des Paris", 26/27 luglio-agosto 1914.   (4) V. Eggeling, Elvi Fejtegetesek a Mozgomüvesztröl, "MA", n. 8 1921.   (5) G. Dulac, In Schèmas, n. 1 dicembre 1925.   (6) L. Moussinac, In Naissance du cinéma, Povolotzky 1925.   (7) A. Gance, Le temps de l’image est venu, conferenza tenuta nel 1925. In "L’art cinématographique", Alcan, 1927.   (8) E. Deslaw, Cinema and Robots, in "Close Up", n. 3, marzo 1929.   (9) F. Léger, A propos du cinéma, in "Cahiers d’Art", 1931.   (10) A. Artaud, Le cinéma et l’Abstraction, in "Le Monde illustré", 29 ottobre 1927.   (11) H. Fescourt, Sensations ou sentiments?, in "Cinéa-Ciné pour tous", agosto 1926.