Epilogue for research, Fall 2007

My research takes two forms: I can read the available material others have written relating to my topic and I can learn from experiments I do myself.

The topics that will be most directly helpful to my research include electroencephalography, scientific studies on entrainment, and technical knowledge of the hardware used for image capture and playback.

As for experiments I can do on my own, I want to begin my building a strobe light device that can receive flash-rate information from a computer (probably implemented via Arduino). Running a binaural-beat program in synch with the light rate is also an option.

The second step to doing some primary research on my own would be to head into DIY neurofeedback (http://openeeg.sourceforge.net/doc/, http://www.dcc.uchile.cl/~peortega/ae/). This would not be a light undertaking, and would start in July at the very earliest. With such a complete setup, I could monitor and quantify my own response to a flashing light with some amount of scientific rigor.

While the artistic exploration of frame rate is my chosen primary independent research area, I will continue the simultaneous pursuit of knowledge in other fields. Alternative pixel grids for image sampling is one of my most recent interests (hexagonal and triangular tessellations of the field). The human voice is another interest (particularly with regard to the concepts of fundamental frequency and register). I mention these to reinforce that my experiments with cinema framerate will be a long-term (and not all-consuming) project.

At this point, I believe that the conscious mind may perceive a range of frame rates in correspondence with a range of semantic association. As I’ve mentioned before, I imagine the range as: slower=passive/dreamy, faster=active/fearful/epiphany. Actual entrainment is unnecessary for frame variability to color the viewing experience.

The several areas I am interested in exploring with frame rate are distinct enough that it is productive for me to define them as separate here:

1)Entrainment-based cinema.
2)Frame rate matching to a semantic range.
3) Subjective interaction (interference patterns) between capture device rate and rate of filmed subject (e.g. wagon-wheel effect).

Discussing these as separate categories is helpful to my project advancement, as it offers a better clarity of purpose.

The ends of my interest in entrainment is not mass mind-control, it is transcendental self-awareness and communal experience. Trance has a very bad connotation and I doubt that I could erase suspicion of it with several pages of writing. For the most part this suspicion is a healthy impulse anyway, so trying to persuade those concerned otherwise wouldn’t be the best use of my time.

If anyone has been keeping up with the entries of this blog, please let me know. It’s admittedly quite dense, but I hope that its density has been informative for any reader intrepid enough to slog through it. Moreover, if you’re that interested in my project enough that you’ve read through this whole sketchspace, maybe I’ll have somebody to collaborate with next quarter.

In a Nutshell

My idea is to manipulate frame rate in a way that substantially contributes to
the meaning of video. It is essentially the development of a new cinematic
technique I am developing, not a specific video piece; as such this is a
long term project that will involve a large amount of research and filmmaking,
likely to result in a body of multiple films that will together illustrate the
dimensions of the technique in practice.

Specifically, I am interested in correlating frame rate with the range of human
brain waves observable from EEG recordings, as they are classified in the field
of neuroscience. It is my hypothesis that such a variation of frame rate will
not effect the audience only on a conscious level, but also has the potential to
directly influence the rate of a viewer's brain waves, via the process of photic
driving, a subset of a phenomenon called entrainment.

I hope this summary is a convenient introduction for anyone who might be interested in my idea but would otherwise get lost in all the text.

Project Dimensions

The following is an update on the progress and interaction between my ideas. I plan to update this post and reform in into a more cohesive entry soon. As it is however, it fleshes out the dimensions of my project fairly well.


A dream machine is “viewed” with closed eyes. The human eye's advanced
capacity for perceiving high resolution images is set aside while input is
reduced to a primitive binary pulsation of extreme luminance values.

The a priori “content” of the viewing experience exists only in the temporal
domain. The perception of this stimuli may be somewhat synesthetic in nature,
triggering an audio-like sensation in the perceptually fuzzy range of Hz that is
too fast to be rhythm and too slow to be perceived as pitch. With no spatial
differentiation (to use a digital analogy, no intra-frame information), photic
driving might be better described as a sub-visual stimulus.

I must continue with substantial research into the biological reality underlying
the phenomenon, but at this point it seems that visual entrainment functions by
either A) exploiting only certain specialized retinal cells (possibly related to
ganglion cell transmissions via the retinohypothalamic tract?) by bombarding
them with an unusual stimulus, or B) engaging at some kind of meta-level with
all the eye's receptors, using the optic nerve as a route for “hacking” the
brain.

It's easier to conceive of this as a design error rather than imagine that
nature intended us to respond to photic driving and other forms of entrainment,
but I'm not yet willing to rule out that our hardwired potential for deep level
synchronization and conscious submission may in fact prove to be evolutionary
advantages (to briefly touch on this without getting lost on a tangent, just
consider the utility of rhythm and trance in religious experience and work
songs).

Photosensitive epilepsy, albeit a condition afflicting only about 5% of chronic
epileptics (though exaggerated in the media and public imagination), is
convincing evidence of how powerful, and potentially dangerous the stimulus of a
flickering light may be, even if only to a specific subset of the general
population. A historical perspective reveals that, particularly in biology, the
study of abnormal conditions is often the most potent frontier in the advance of
scientific knowledge, ultimately with broad implications and pervasive
relevance.

Is there any more controversial and underdeveloped field of scientific study
than neuroscience? The answer is no – no I can't, won't and don't want to make
any pretense of being a scientifically rigorous torch bearer. With that
disclaimer, I've recently become aware of Karl Pribram and David Bohm's
collaborative Holonomic brain theory as a fascinating new research area that
might benefit my project development.

I feel like a reasonable knowledge of non-invasive (EEG-based) neuroscience is
necessary for me as I develop my project to keep the results of my experiments
from being a random collection of cheap tricks that I can't reduplicate or
explain. I plan to craft video with intelligently varied frame rate semantically
integrated with the emotional and narrative content of the images. I expect the
frame rate manipulation to range from subtle and consciously imperceptible to
blatant strobing that would probably turn-off a close-minded audience. However,
the avantgarde art crowd has enough kids playing hot potato with
big ideas and I
wouldn't find any consolation if my work had to hide behind a massive artist's
statement or in Post-rationalization-land. If my technique gets
implemented and felt successfully, I wouldn't expect anyone watching such video
to be able to explain why it was affecting. To be successful though, it surely must be felt.

Back to the dream machine. Though it has no inherent atemporal visual detail, a
dream machine purportedly assists the user in generating self-emergent image
content by guiding the brain to a lucid dreaming state. It would be amazing if
photic driving could heighten the degree to which viewers imaginations were
provoked to dig into their associated memories to supplement the on-screen
content of a film. I don't think this would happen. At least not dramatically,
nor for the majority of people. I doubt most people would be able to fully
process complex images before their eyes while in a state of theta wave
dominance. Maybe young children and viewers who got bored and nodded off at the
start would have the best experience at these movies.

If 24p film were projected at 24 Hz, there would be noticeable flicker and
resultant eyestrain. Each frame is typically shown two or three consecutive
times, interrupted several times by a mechanical shutter, for a true refresh
rate of 48 or 72 Hz. Over 60, the flicker fusion threshold, is enough for most
people to consciously ignore the strobing.

Digital video cameras have no physical shutter per se. Shutter speed is the time a charge is allowed to build up on the CCD(s). I have no independent source to confirm this assertion yet, but it is my current understanding that when shutter speed is lower than frame rate on a digital camera, multiple frames are allowed to bleed together. This results in ghosting and image blur. The interaction of shutter speed/ exposure time and frame rate is something I need to explore further.

I have many questions. How far apart do the peaks and valleys of a wave
representing a stimulus have to be for this wave to be capable of entrainment?
Is there a maximum amplitude of such a wave beyond which it is no longer
effective? Is periodic change in luminous intensity (candelas) the only variable
for defining the Hz of the entraining stimulus or could relatively minor changes
in the detail of a frame (even with zero overall luminance change) be effective?
Are the valley and the trough most effective when they occur for equal
durations?

More to come.

Cultural Web

The core of my idea is the experimental use of photic driving as a new realm in which to explore and devlop cinematic language. As such, cinematic technology, including historical and obscure technical forms is one half of my research plan. Research into Neural Correlates of Consciousness and the application of the physical concepts of entrainment and coherence to the measurable parameters of human states of perception is a field that demands highly specialized knowledge and training. I cannot pretend that I am going to move very far beyond amateur status in this field of study in the short term, yet I still intend to learn a great deal about the topic so that my understanding of it neither generates false leads for my project, nor taints it with the stigma of pseudoscience. As audio is often presented in conjunction with video in an integrated relationship of mutual reinforcement (and surely audio is given prime importance in many cases), I think it natural to have a strong simultaneous interest in the field of sound correlated with the predominantly visual frame-rate research I am undertaking. My focus in this related area is on very low frequency tones (including infrasound), the gradual border ranging between perceptions of pitch and rhythm in the 0-60 Hz range, acoustical beating and other interference patterns. I hope to actively explore the connections between photic and auditory driving, as I have already generated some interests I am enthusiastic to explore from mapping sound theory concepts to the visual domain, such as harmonics and resonance.

Dream machine, Brion Gysin
entrainment, coherence (physics)
resonance
harmonics
trance
auditory driving
photic driving
epilepsy
chronobiology
over/undercrank
refresh rate
flicker fusion threshold
shutter speed
frate rate selection, “ramping”
vfr, media containers
cinematic techniques
interference patterns
acoustical beating
Alvin Lucier
quantization error
neural correlates of consciousness
binocular rivalry
suprachiasmatic nucleus
stroboscope
light painting
electroencephalography

The bridge

NOTE: Please read the prior post in order for this one to make sense.

Attention to our involuntary (and pseudo-voluntary) biological signals continues to strongly influence my art, however, it's unlikely that I will continue to use biofeedback. And whereas Follow Your Heart! is an immersive, interactive installation experience designed for one individual at a time, I do not expect the research and artwork development I'm now undertaking to result in any of these attributes.

I am uncertain at this stage whether I would need to develop specialized hardware for the image capture and digital projection of variable frame rate video. But even if this does become the case, then at least the content I create in this specialized medium could be viewed by many people at once. This future potential for shared experience and greater audience exposure is something I'm enthusiastic about.

As in Follow Your Heart!, the research going into my new work is based on my fascination with the relationship one has to their unconscious. Encouraging disorientation of a person's normal sense of self and facilitating transcendental experience is still my ultimate goal. The concept from Buddhist philosophy of anatta, possibly translated as non-selfhood, is something I want my work to explore.

In Follow Your Heart!, the unconscious self plays the leader for the conscious will to follow. If some degree of tension or resentment is momentarily provoked as the participant struggles to keep up, all the better. Resonance or dissonance between the video's frame rate and the viewer's dominant brain waves may create a meta-drama on top of the film's narrative content. If the pull of the film on the viewer's conscious mind is out of step with the direction and rate of frame rate entrainment, it is unlikely that the range of feelings Follow Your Heart! aimed to elicit would emerge in this gap. Rather, a simple awareness of the disparity might emerge and the conscious mind would self-correct.

Music is often used to reinforce the emotion of a scene. Combining visual entrainment and cinema would likely be used in the same mutually reinforcing way. For example, very high frame rate could be employed when a character is afraid, trying to think quickly, avert disaster and has an epiphany. Very low frame rate could be used when a character is basking in a state of happy relief, dreaming or drunk.

Much less often, but occasionally to intensely powerful effect, music can be very carefully chosen in deliberate emotional dissonance with a scene (e.g. triumphant music set to a massacre, sad music bringing tears of happiness). Low frame rate accompanying a sudden, sorrowful revelation and lots of on-screen action might work in a similar way, to underscore the hopelessness of a character's efforts, like in a dream when running but being held back by the air.

Then there is the special case of interference patterns that may be generated between frame capture rate and periodic on-screen events cycling against still reference objects: This would be a wholly unsubtle (but, I am inclined to imagine, beautiful) way of illustrating how the quality of perception is reflected on what is being perceived. A statement that sounds obvious, but is conveniently forgotten so often.

Working toward implementation and experimentation is certainly preferable to merely writing about a permanently hypothetical cinematic vocabulary for the proposed technique of frame rate variance. But I suggest the possibilities I have imagined here to give the reader a sense of the potential significance I hope may eventually emerge from the latent idea.

Take you to the bridge

I will elaborate on the bridge between my projects; how my hopes and plans for new artwork draw upon my the substance of my previous work. But not in this post. In the next one. Before I create that post, I must introduce and outline the direction of my new research and what artistic potentialities I have been imagining.

Last March, while on Spring break in Tokyo with my girlfriend and another friend, I woke up in the middle of a night bus to Kanazawa. Electric snakes of streetlight had got into the cabin around the edges of closed curtains and were scanning the dark stock of sleeping passengers in a violent, bright barcode. The face of my girlfriend asleep in the seat beside me had become an astoundingly incoherent, fascinating and disturbingly inhuman playground of cubist shards. I stared for a long time, in a sustained strain to see her beneath the flickering projections of the huge highway zoetrope we were shuttling along in, as though on some cosmic voyage.

The circumstance of my inspiration is vaguely reminiscent of an experience that contributed to Brion Gysin's plans for the Dream Machine, a stroboscopic entrainment device.

"Had a transcendental storm of colour visions today in the bus going to Marseilles.
We ran though a long avenue of trees and I close my eyes against the setting sun.
An overwhelming flood of intensely bright colours exploded behind my eyelids: a multi-dimensional kaleidoscope whirling out through space. I was swept out of time.
I was out in a world of infinite number. The vision stopped abruptly as we left the trees.
Was that a vision? What happened to me?"
-Extract from the diary of Brion Gysin 21/12/1958

I'd made my own dream machine several summers before. However, it failed to lead me into a hypnagogic state in my first several sessions with it, so I gave up on it, somewhat disappointed. Had I made an error in the calculations of it's construction? Was I not being receptive to the machine? Was brainwave synchronization pseudo-scientific bullshit? Not knowing what was to blame, I suspended judgment of the whole affair.

Back on the night bus, I closed my eyes and considered my impulse to film what I'd witnessed at some point, perhaps setting it as the opening to a feature film. I thought of the unsupported frame rates on the Panasonic AG HVX200 that could be reached with a hack and I wondered how fast the bus was moving, how far the streetlights were apart and what angle the beam was allowed to sweep through the cracks in the curtains.

Why is there no capability to alter frame rate on the fly with digital cameras? Was hand-crank technology my recourse? I wanted to know what the scene might look like plunged down to 1 fps, then ramped back up past 24 Hz. I wondered what interference patterns would appear when the frame rate was dissonant with the rate of the lights flickering over my girlfriend's face and I wondered if I could discern her as whole again if the two rates were harmonic. It struck me that varying the speed of the bus and the frame capture rate of the camera would be a song for two voices written against the still tapestry of life.

This was a leap of inspiration in a solution precipitated from a long period of mental fog. Earlier, I had discussed with a friend the idea of different characters/objects in a scene being displayed at different frame rates. This was the case in a 2D space video game he'd been working on and we imagined the potential of the idea as a subtler form of leitmotif (such as heralds all appearances of Darth Vader). Full scenes could be shot this way as well, contributing to mise en scène in a similar manner as a film's color palette.

But, as with color, what semantic and emotional range might different frame rates correspond to? An explicit answer to this question was something mysterious to me for a while, though I had some rough intuitions born from visions and scenes in my dreams. While educating myself with a Wikipedia article on binaural beats, I came across a table that paired the frequency of brain waves with usual associations for each dominant state:

> 40 Hz
Gamma waves
Higher mental activity, including perception, problem solving, fear, and consciousness
13–40 Hz
Beta waves
Active, busy or anxious thinking and active concentration
7–13 Hz
Alpha waves
Relaxation (while awake)
4–7 Hz
Theta waves
Dreams, deep meditation
< 4 Hz
Delta waves
Deep dreamless sleep

Age is proposed as another correlated variable. I imagined a film of children playing in theta range fps with fast energy and the erratic quantum movement of little electrons. However, the true effect of this proposed contribution to cinematic language is something that needs to be born out in experimentation and practice, not mere hypothesis.

Can a model of sampling rate be applied to human perception itself, and if so, would brain wave frequency be an accurate way to quantify the range of different modes we perceive in? This is a subject I plan to read much more about. Unfortunately, there appears to be a paucity of literature on this topic that is of a rigorously scientific nature. However, this impression is only from my initial survey of a small pool of resources so far, so I'm hardly discouraged.

One of my hypotheses is that the cinema standard of a fixed 24 fps (up from 16 and 18 in the silent era), although a product of myriad historical determinants, including economics and projection mechanics, happens to be a rate near the brain wave frequency of an average person engaged in viewing film.

This observation begs the question of whether cinema can successfully be injected with elements of photic/visual driving (This is one name for a process of entrainment whereby a stimulus can induce trance by locking onto and leading a viewer's brain waves in one direction or the other). Thus my ambition is not only to explore whether I can subtly strengthen the presentation of images and narrative by strategic variation of frame rate-- I believe that this contribution to cinematic technique may be powerful enough to transport viewers to specific states of perception in tune with given scenes.

Motivated manipulation of frame rate is the direction in my research and experimentation I'm most excited to explore. However, it is neither the first nor most recent field of my attention in research-art. Frame rate is the second of three.

The first: It was about two years ago that I saw an experimental film which became the paragon for my endeavors. The film was made by my girlfriend (though that's a detail subordinate to the fact that it was all a dream I had, so not really). It basically depicted the birth and death of a snowflake; I'll suffice to say that the innovative beauty of the film was in the complex way it treated depth and time.

Film compositors treat effects and video as a “stack” of 2D layers. Though this is typically toward the end of merging multiple layers into a one seamless channel, it is a design principle that can also be exploited in a way that ruptures the illusive spatial depth of live-action video.

Imagine a particular channel being used as a character itself in a larger work: The silhouette of a human could define the boundaries of a layer, the content of which could convey more information as a window into the character's psychological state than as a solid human body. The shadow of a building, a flock of birds and a puddle of water could all be screens. To make a dense collage from many video-layer objects strikes me as a much more elegant and potent tool for multi-channel display than either split screen or multi-panel installations.

I suspect that the majority of research necessary for advancement in this field has to do with developing sophisticated algorithms for boundary detection, both within a frame and over time (so as to minimize the disadvantages of blue/green screen). Yet, the basic architecture of video compositors is built for this technique, so I'm optimistic about experiments in this direction.

Third: The application toward film of the high dynamic range imaging techniques employed in specialist photography. This interest stems purely from my frustration with the small dynamic range and consequent lighting problems of cameras, digital in particular.

I'll paraphrase Stan Brakhage to say that there is no such thing as a wrong exposure value. Right now I have an interest setting up high contrast scenes and changing camera exposure on the fly to reveal the bias of the lens. But ultimately, some knowledge of the human eye and a comparison of it to pixel arrays leaves me disappointed in the man made one.

Where I'm coming from

This is a brief artist statement for my most recent project.

Follow Your Heart! is an externalization of a participant's heart signal as a spot of light pulsing across a dark floor. The instructive title of the installation compels the participant to follow this other self around the room like an inverted shadow and the emergent feedback loop of rising heart rate sets a natural narrative arc for the experience.

The installation is an attempt to summon up acute self-awareness, even to the point where a participant might have competitive, playful, or frustrated feelings at their external heart. The holy grail of emotion that I hope this physically exerting experience could evoke is the intimate feeling of happy surrender, as toward a loved one or an idea of God. To that end, the heart of light would return to orbit the exhausted participant and lead them back out of the dark.


My full artist statement exists in the form of the original Proposal entry for April 10, 2007 of my DXARTS 202 project development blog:

http://kjellhansenin202.blogspot.com/2007/04/proposal.html

Follow Your Heart! couldn't debut as scheduled at the end of May because of a problem implementing the wireless transmission of the heart signal to the control computer. Because I want the independent research and work I do this year to advantageously tie into the sound and video sequences I'm in, further effort toward Follow Your Heart! is therefore on hiatus. I will elaborate more on this decision and the continuity across my transition to new work in my next post.