Klarinet Archive - Posting 000530.txt from 2005/03

From: Tony Pay <tony.p@-----.org>
Subj: Re: [kl] Conducting without a baton
Date: Mon, 21 Mar 2005 16:58:47 -0500

It might be worth posting here something I wrote a few years ago, about t=
he
possibility of constructing a 'conducting machine'.

I hoped at the time that someone -- perhaps a Sibelius programmmer -- mig=
ht
take up the challenge of programming it, and I contacted them, but didn't=

push very hard. If anyone is interested in collaborating with me on the
project, I would be interested. (Only talented programmers need apply:-)=

The relevance to this discussion is of course that in order to construct =
a
*useful* conducting machine, you need to specify what 'good conducting' i=
s
rather precisely. That's interesting in itself.

David McLune has presented here his own taxonomy of gesture, which seems
useful. Another part of what may be useful is presented here below under=
the
heading, "Physical Intuitiveness".

See what you think.

-------------------------------------------------------------------------=
---

Blueprint for a Conducting Machine

This paper is a first attempt at a design for a machine to conduct human
players. Such a machine would be a sort of super-metronome. If it were
successfully built, it would expand the possibilities of performing a
certain type of complicated music, involving independent groups of
players, and could also be a pedagogical device for both performers and
conductors. The implementation is imagined at the moment to be a
program running on a microcomputer, which would output to a standard
screen, possibly projected at a larger size to enhance visibility.

Possibilities and inherent limitations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The primary purpose of a conducting machine would be to assist players
in the task of playing together, by representing for them visually the
metre against which they have to play. However I think it is important
to realise clearly from the start both what such a machine might be able
to do, and what it would not be able to do. =20

A human conductor enters into a relationship with his players and the
music that a machine is unlikely to be able to emulate in the forseeable
future. Even from a technical point of view, he is able to beat ahead
or behind the players in order to speed them up or slow them down; and
he is able to make other gestures to change or modulate what he
perceives them to be doing. The machine, lacking a representation of
what the players are doing, would be unable to respond to them in this
way. Though a more interactive relationship might be achievable in a
simple situation (if only one player were involved, for example) in such
a simple situation the machine wouldn't really be necessary. It might
of course turn out to be interesting to develop such a limited
relationship. But in any case, the possibility will not be considered
further here.

A human conductor is also involved in real time with the emotional and
structural features of the music =99 again, beyond the abilities of the
machine. However, this does not mean that the machine would be unable
to represent expressive features if these were programmed in to start
with. Even though it would 'have no idea' whether or not it was
communicating successfully with the players, it should be able to beat
in both a legato and staccato style, communicate sostenuto, espressivo
and parlando, and initiate and control accelerandi, ritardandi and
pauses; as well as beat a sequence of bars of varying time-signatures
and tempo.

Two-dimensional representation of the point of the baton
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to do these things, the machine needs to be quite subtle. At
first sight, the most obvious way of modelling a conductor would be to
represent the point of the baton on screen in some way. The point of
the baton seems to be the focus of attention for a player: an expert
conductor, indeed, seems to use it to create expressive shapes that
transmit much of the character of the music. However, I shall argue
that though this naive representation would be well in advance of
anything available at the moment, it falls short of what we require.

Quite a large amount of a conductor's usefulness is that he tells you
not only what you should do now, but also what you should do at a time
'just later than now'. Players need this information, not only in order
to prepare themselves to change what they do, but in order to prepare to
be together with others who also have to make the change.

This is why upbeats are important. A change in tempo is typically=20
signalled by a different length upbeat. Now, a change of upbeat is
possible in a two-dimensional representation of the point of the baton.=20
But the problem of showing a speeding up and slowing down is of a
different order. You want to indicate that you are about to speed up or
slow down in the next moment, and when this moment arrives, you still
want your players not to change anything! You want them to begin,
together, to speed up or slow down. (What changes is the second time
derivative, so to speak.) Moreover, this is a more common case than the
one of making an abrupt change of tempo, so it is something that our
machine should routinely be able to do.

The grammar of arm movements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The best human conductors do this job rather well. But they are not
constrained to express themselves merely with the point of the baton.=20
Consider a famous book on conducting: The Grammar of Conducting, by Max
Rudolf. In the taxonomy of various different sorts of beat that Rudolf
presents in his book, the diagrams of the arcs to be traced by the point
of the baton corresponding to each beat are accompanied by detailed text
describing the types of arm movement to be used while tracing them. The
details of the movement of different parts of the arm are a very
important expressive vehicle for a conductor, as Rudolf is at pains to
point out. And the fact that every able conductor without exception
uses them to some degree or another suggests that the control of a
repertoire of arm movements is an indispensable part of the human
conducting vocabulary. Our machine too must use them, or something like
them, not only to ensure that the players stay together, but to achieve
flexibility of expression.

We have to throw away the naive two-dimensional representation of the
baton point, and consider anew how we may show the metre of the music no
less effectively than a human conductor, yet in the simplest possible
way. It would be unfortunate if the simplest way to do this were to be
the presentation of a likeness of a human conductor in detail on the
screen. Fortunately, it turns out that this is not the case, partly
thanks to a remarkable property of our visual systems that I shall
shortly describe.

Before describing it, however, I want to put forward the fundamental
idea that the precise and elegant diagrams of baton movement in Rudolf's
book are more usefully regarded as arising naturally from the
composition of much simpler movements of parts of the arm relative to
each other, and that if these simpler movements were modelled, it would
be possible to generate not only the beats recommended by Rudolf, but
also many others.

Rudolf describes a type of beat that is executed solely from the wrist,
and proceeds to consider beats that involve the forearm, keeping the
elbow still. He then moves on to develop his taxonomy, which includes
light-staccato and full-staccato, neutral-legato and expressive-legato,
marcato, tenuto, combined legato-staccato and accented beats. But he
fails to note the further implication that any movement of the arm is
composed of simpler movements of its parts, preferring instead to draw
the baton-point diagrams and supplement them with textual comment =99
albeit highly informative, inspiring and musically relevant comment.=20
This observation is not meant to be a criticism of Rudolf's strategy: he
is after all concerned with teaching conducting as well as analysing it. =

Decomposing a complex activity into its most basic constituents is not
always the best way for a human being to go about learning how to
execute it. The problem of acquiring a new language springs to mind as
an obvious example.

However, if we approach the situation from the opposite direction, and
regard the relative movements of the various joints of the arm as
modules, we can see that any arm movement is composed of a superposition
of movement of the elbow relative to the shoulder, movement of the wrist
relative to the elbow, and movement of the hand relative to the wrist.=20
Theoretically, these movements are all independent. Such a 'natural'
decomposition makes it much easier to analyse and then to model any one
of Rudolf's different styles of beats, because consideration of these
different styles shows that they can indeed be easily described as the
superposition of different proportions of simple movements of the joints
relative to each other. In fact, I think that the precise curves that
Rudolf draws are relatively unimportant when compared to his
characterisation of what amounts to the grammar of how the modular
movements should combine. The arcs described by the point of the baton
are a sort of epiphenomenon, and are probably mostly not attended to
directly by players, though of course there are isolated moments when
attention directed toward the point of the baton is of great importance.

If we can represent the composition of these modular movements in
various proportions so that they can easily be seen on screen, we will
have a infinite library of beats from which we can derive the ones we
want by the application of further constraints: Rudolf's grammar, above,
being the result of the application of one possible set of constraints.=20
We could go on to model the style of one particular conductor. It may
even be possible to find additional combinations of the modules and
create new ways of beating that usefully extend the normal range of a
human conductor. (As a somewhat silly example, you could construct a
beat that did three with the elbow, four with the wrist and five with
the hand.)

But how can we represent this composition of modular movements? Does
this not amount to drawing all the possible positions and movements of
an arm on screen?

How to represent it
^^^^^^^^^^^^^^^^^^^
A remarkable series of experiments by Johansson has shown that our
visual system attempts to interpret two moving spots of light on a
screen as though they were the ends of an invisible rigid connecting
rod. For example, if two spots at the opposite ends of the diameter of
an imaginary ellipse move around the ellipse together, staying opposite
each other, the distance between them obviously changes continuously.=20
(At one point, this distance is the major axis of the ellipse, at
another the minor axis.) But if we see these moving points of light,
and nothing else (no ellipse, no diameter) we instantly and
involuntarily interpret the motion as of a rigid yet invisible rod,
rotating about its midpoint in a plane inclined to the screen. The
inclination of this plane is such that we can interpret the ellipse
actually delivered on the screen as a foreshortened circle. This circle
is the path considered to be 'really' described by the moving lights, in
their imagined role as the ends of the rigid rod.

Johansson's subjects proved to be unable to not do this. In other
words, even when they knew that they were seeing just luminous dots
moving in an ellipse on a flat screen, they were unable to eradicate the
three-dimensional interpretation.=20

Amazingly, the illusion occurs even when the ellipse is changed to a
rectangle. The points move round the rectangle, starting together at
opposite corners, and always remaining equidistant from the centre of
the rectangle. (Remember, the only things visible are the moving
luminous points.) But now the rod seems to twist and turn in a
complicated way in space, to preserve our perception that the moving
lights are rigidly connected.

Johansson and his team decided to investigate what occurred when the
situation was more complicated. They attached a flashbulb to each of
the joints of one of their colleagues: two for wrists, two for elbows,
two for shoulders, two for hips, two for knees and two for ankles =99
twelve in all. Then they filmed him, first seated in a chair, and then
getting up and walking about a darkened room. All that was visible on
the film was a collection of moving spots of light. Yet experimental
subjects viewing the film instantly understood the lights as a moving
figure. They could even tell when the figure was simulating a limp!

See:

http://www.atm.damtp.cam.ac.uk/people/mem/papers/LHCE/lucidity-preface.ht=
ml

This fact about our visual system is fascinating in its own right, but
it is clearly a godsend for the problem of representing a moving arm on
screen. What it means is that we need do no more than solve the
mathematical problem of how simple motion of a limb translates into the
motion of a central perspective projection of its endpoints on screen,
represent those endpoints in an elegant manner (say, as different
brightly coloured small circles), and we have our representation. From
the programming point of view, movement of the wrist relative to the
elbow (one module) is the same as movement of the fingertips (or
baton-point) relative to the wrist, or the elbow relative to the
shoulder. Musicians (or rather, their visual systems), presented with
such a display, will automatically reconstruct the requisite arm
motions, with which of course they are already familiar. It may even be
that such a system is even easier to decode in peripheral vision than
the 'real thing'!

Physical intuitiveness=20
^^^^^^^^^^^^^^^^^^^^^^
Clearly it is possible to model any combination of different amplitudes
and phases of each modular movement, subject only to the constraint of
representing rigid connection between the spots of light representing
each joint. Not all of these combinations will be needed, or indeed
appropriate, though it may be possible usefully to interpolate between
the combinations classified in Rudolf's taxonomy (and it may be that my
'silly' example turns out not to be so silly after all). The most
economical way to start modelling is probably to imitate the movements
of a human conductor to establish the normal motions of each module.=20
But I should like to suggest that when we consider a time-sequence of
such combinations, there is a further abstract constraint that may be
useful, and that may constitute a grammar with a validity transcending
the personal.

We need the notion of a system of signals being in some sense physically
intuitive to explain how it is that a really excellent conductor
(excellent, that is, in the limited sense of conducting that we are
considering) manages to be clear immediately to an orchestra that he has
never worked with, without needing an initial period during which they
learn his personal system of signals.

Physical intuitions (what have been called 'folk-theories of physics')
form a very large part of our successful interactions with the world.=20
By this I mean simply that in our normal daily lives, we know what
objects are likely to do, more or less. We don't try to move a grand
piano with one finger, and we leap up from the table if someone spills a
carafe of wine, expecting it to run off the table into our laps. These
intuitions can be very fast-acting, and are either built into our
systems or are a product of early learning. Sometimes our intuitions
fail, as when we encounter a siphon, or a gyroscope, but mostly they
serve us well. They exist for survival purposes, but are available to
be used for whatever purposes we fancy.

In particular, they are available as common property between conductor
and orchestra (and, incidentally, will make life difficult for an unwary
conductor who tries to go against them). Of course, different
conductors have different ways of showing their intentions. But our
machine must be understandable by the widest possible audience. The
requirement that any system of communication satisfy the demands of
physical intuitiveness should guarantee the greatest possible
accessibility.

As an example of what I mean by a time-sequence of combinations that is
physically intuitive, I would like to put forward a brief sketch of one
possible solution to the problem of initiating and sustaining ritardandi
and accelerandi. It may be that there are better or at least
alternative solutions to these problems. (Interestingly enough, Rudolf
has little to say about the problem of showing an accelerando.)=20
Experiment is necessary here.

If a moving system begins to be coupled to a stationary one, our
physical intuition is that the moving system will be slowed by starting
to bring the stationary one into motion. Thus a movement confined to
the wrist, if subsequently coupled to the forearm and upper arm, would
be thought intuitively likely to slow down. An alternative viewpoint
would be that the larger moving object of hand plus lower and upper arm
needs greater energy to keep it moving, and so is again likely to slow
down. This would be particularly true if the impression were given that
the motion was encountering some resistance, as if the arm were moving
through treacle. All of this is physically intuitive, and would
communicate directly to a player. I recall the leader of an orchestra
telling me once that I 'looked as though I wanted to slow down' at a
particular point. "No," I said. "I'm not slowing down." "Yes, you
are," he said. "It's something to do with how you're moving your arm."

On the other hand, if the movement of the elbow were energetic, and
ahead in phase of the movement of the hand relative to the wrist, it
would be intuitive that the hand would speed up. This device, of
encouraging accelerando with the elbows, can be observed in several
human conductors.=20

These physically intuitive strategies may be contrasted with the
superficially similar but possibly counterintuitive strategy to use 'the
size of the beat' to signal accelerando and ritardando. I have heard it
suggested that the intention to slow down is simply signalled by a
larger beat, and the intention to speed up by a smaller beat. However,
there is then the difficulty that a larger beat has to go initially
faster in order to continue at the same speed as the previous beat
before slowing down (and vice versa for the smaller beat). Of course,
it can work; any system of signs that is consistent can be obeyed by
musicians (who after all are sometimes required to do their jobs with
little or no help at all); but it runs counter to our physical
intuition. Very commonly, in music, slowing down is accompanied by a
loss of energy; to have a signal for a loss of energy that requires an
increase of energy in order to execute it clearly risks confusion.=20
Perhaps the change of tempo may work, but other aspects of the music may
be interfered with.

What is to be modelled
^^^^^^^^^^^^^^^^^^^^^^
Tempo, size, phase and character of the three beat 'modules' (elbow,
wrist and hand/baton) for the standard bars (1 to 8, with the different
groupings for 5, 7, 8, and subdivisions as an option). This is the
fundamental problem. I am imagining that these parameters may be set on
writable icons and slider bars, rather like the RGB sliders on the
colour definitions for a palette. Particular presettings will produce
particularly useful characters of beat, much in the same way that
particular presettings of palette are more or less normal, but other
options will be available. The hand/baton has configurable length,
whilst the other arm-part dimensions are fixed at sensible lengths for
the screensize.

The 'character' parameter represents a spectrum between slow, continuous
motion and fast sweep. I suspect that 'bounce' will be an emergent
property of how the modules interact (at least, that's what it looks
like in the mirror!)

'Stops', and normal, delayed and anticipated upbeats

Small preliminary 'drift' movements to signal 'about to give upbeat'.

-------------------------------------------------------------------------=
--

Tony
--=20
_________ Tony Pay =20
|ony:-) 79 Southmoor Rd tony.p@-----.org
| |ay Oxford OX2 6RE http://classicalplus.gmn.com/artist=
s
tel/fax 01865 553339
=20
... How do I set a laser printer to stun?

-------------------------------------------------------------------
Klarinet is a service of Woodwind.Org, Inc. http://www.woodwind.org