Reference: Church, R. Breckinridge, Martha W. Alibali, and Spencer D. Kelly, eds., Why Gesture?: How the Hands Function in Speaking, Thinking and Communicating, Gesture Studies (Amsterdam: John Benjamins Publishing Company, 2017), vii https://doi.org/10.1075/gs.7

Gesture Study

3

• 제스처 스터디는 20세기쯤부터 고성장 (bourgeoning) 특히나 말을 할때의 역할로서 연구되었다.
• “window on the mind.” ::Hearing Gesture, Susan Goldin-Meadow (2007):: frequently in a way that reflects an imagistic version of what is being spoken.

4

• 아리스토 텔레스는 행위를 일으키는 것과 (what causes a — efficient cause), 행위를 하는 이유 (what behaviour for — final cause)로 나누었다.
• 산책을 한다고 할때, working metabolic 과 근육 시스템이 efficient cause, 건강이 final cause이다.
• 이 글에선 what gesture is for가 될것이다.

397

• 이 책에서 첫 테마는 제스처에 대해서는 생물학적, 정신학적 그리고 사회적으로 해석해 본다.
• 두번째 테마로는 제스처의 기능을 모든 시간대의 프레임으로 해석해본다. – moment-to-moment, ontogenetic, and evolutionary
• 세번째 테마는 the methodology for studying gesture is necessarily varied.

398

• Finding that gesture occurs across different contexts and under different task requirements tells us that its functions are multi-faceted and flexible.
• 네번째 테마는, 제스처의 기능은 producer를 위해서만 기능하지 않고, 보는 사람을 위해서도 작동한다. The gestyoure supports speech to enhance internal activities of the speaker, such as thinking and language production, while simultaneously supporting speech to enhance communication to listeners, influencing the listener’s thinking and language comprehension.

398-401 (Neurological evidence Analysis)

401- (Psychological evidence Analysis)

• Gesture appears to be linked with language to support the way spatial information in speech ::(Alibali et al., Chapter 2; Ozyurek, Chapter 3)::
• As another example, gesture appears to reflect action in a simulated form in problem-solving contexts (Hostetter & Boncoddo, Chapter 7; Nathan, Chapter 8)
• 특히나 챕터5 에서 McNeill and Lopez-Ozieblo’s
  • Growth Point Theory (GPT):
  • (1) gesture and speech are synchronized;
  • (2) gesture’s format, which is gestalt, 3D, and imagistic, is distinctly different from speech’s format, which is analytic, 2D, and linear;
  • and (3) because these two formats are different, the combination of gesture and speech modalities reflects a more complete version of an idea than either modality alone.

402-404

• De Ruiter의 경우 스피치가 제스처에 영향을 끼친다고 하며 제스처가 스피치의 supplement information으로 (that is redundant with speech)라고 말해지지만 이 책에서의 연구를 통해서 사실 서로가 영향을 받으며,
• GPT 를 통한 연구는 gestyoure content mirrors speech content, because gesture’s format is 3-D and nonlinear, it is never fully redundant with speech.
• Gesture provides visuo-spatial information that reflects 3-di- mensional, dynamic, as well as perceptual features (Hostetter et al., Chapter 7).
• 제스처는 embodied cognition와 연결되어있다.
  • our understanding of concepts may be grounded in the way we physically interact with the world, which is reflected in the way we gesture about the world (Cook & Fenn, Chapter 6; Hostetter et al., Chapter 7; Nathan, Chapter 8; Novack & Goldin- Meadow, Chapter 17; Hostetter & Alibali, 2008; Alibali & Nathan, 2007; McNeill, 2005; Núñez & Lakoff, 2005)
• 챕터 17에서 제스처는 특이한 유니크한 폼의 액션이다.
  • 즉 제스처는 어떻게 직접적으로 세상에 영향을 끼치는가에 대한 정보를 담은 직접적으로 세상에 영향을 끼치지 않는 액션이다. it represents information about a direct effect on the world without having a direct affect on the world (e.g., twisting a jar lid results in an open jar in a way that producing a twisting gesture does not; see also Goldin-Meadow, 2015, and Novack & Goldin-Meadow, 2016, for discussion).
    • This in betweenplace may serve a particularly important purpose for cognition.
  • However, gesturing about acting on objectsis more likely to lead to generalization and retention than actually acting on objects(see Novack & Goldin-Meadow, Chapter 17 and also Novack, Congdon, Hemani-Lopez & Goldin-Meadow, 2014; and Congdon, Novack, Brooks, Hemani-Lopez, O’Keefe & Goldin-Meadow, under review; Wakefield, Hall, James & Goldin-Meadow, 2017).

404-405 Social Evidence (Chapter11-16)

405- (Gesture Functions in all time frames)

Moment-to-moment

Developmental time frame

Evolutionary time frame

제스처 메카니즘

6

• A number of evolutionary perspectives suggest that gesture evolved either as a precursor to spoken language (Bates & Dick, 2002; Corballis, 2002; Rizzolatti & Arbib, 1998; Tomasello, 2008) or simultaneously along with it (McNeill, 2012).
• In addition, there are powerful mechanisms of gesture on the much shorter timeframe of moment-to-moment processing, which spans from seconds to minutes. For example, when people are faced with challenging spatial and motoric tasks, they produce more representational gestures when they speak, than when they face simpler tasks (Alibali, Yeo, Hostetter & Kita, Chapter 2).

제스처의 기능

7

• Tinbergen (1963) points out, a behaviour can be functional without being the direct product of some specific evolutionary mechanism.
  • human hands evolved to interact with real objects in the environment, but they were co-opted over time to also serve the communicative function of gesturing about imaginary objects not present in the here and now.
• The gestures may enhance or disrupt common ground (Nathan, Alibali, & Church, Chapter 13), clarify or confuse an important concept (Singer, Chapter 14)

챕터 2 Representational gesture help speakers package information for speaking p. 15-

15

• What role do gestures play in speaking? Current theories of gesture production have three primary foci:
  1. the role of gestures in communicating information,
  2. the role of gestures in producing speech, and
  3. the cognitive processes that give rise to gestures.
• These distinct perspectives emphasise different aspects of the complex behaviour that we recognize as gesture.

16

• We focus in particular on representational gestures, which are movements that represent semantic information via form (handshape), trajectory, or location.
• 이러한 제스처는 beat 제스처와 다르다. which are motorically simple gestures that manifest aspects of the structure and prosody of speech but do not convey semantic content (McNeill, 1992), and 또한 인터렉티브 제스처와도 다르다, from interactive gestures, which are used to regulate turn-taking and other aspects of interaction among participants in a communicative situation (Bavelas, Chovil, Lawrie, & Wade, 1992).
• 스피치와 제스처는 다른 기호적 특성을 가진다
  • Gesture: Distinct meanings converge into a single, synthetic gesture.
  • In contrast, speech is analytic and combinatorial, in the sense that the meaning of the whole depends on the meanings of the individual elements.
• 글로벌과 통합적인 특징으로, gestures are adept at expressing spatial, motoric, and relational information (Alibali, 2005)
• 이 글은 Information Packaging Hypothesis Kita(2000)에 의해 개발된 메또돌로지를 이용해 분석해본다.

Information Packaging Hypothesis란?

- *Information Packaging Hypothesis* “helps speakers organize rich spatio-motoric information into packages suitable for speaking” (Kita, 2000, p. 163) 
- 키타에 따르면 공간-동적(spatio-motoric) 생각은 “alternative informational organization that is not readily accessible to analytic thinking” (p. 163) 

17

• 키타의와 몰(2012)의 의견에 경우
  • In later work, Kita and Özyürek (2003) further specified this process in their Interface Model, which holds that gesture production and speech production processes are linked bidirectionally. 스피치와 제스처 사이엔 익스체인지가 일어남.
  • Through this process, gesture and speech converge in content; more specifically, gestures encode information equivalent to the infomation speech encodes within a processing unit for utterance formation (roughly a clause for adult speakers) (Mol & Kita, 2012).
• 이 생각은 다른 두개의 컨템 스피치 속 제스처에 대한 의견과 반대된다.
  1. First, the Lexical Retrieval(회복) Hypothesis (Krauss, Chen, & Gottesman, 2000): – Briefly, gestures activate spatial-dynamic features of concepts, which in turn feed activation to lexical items, facilitating speakers’ retrieval of those lexical items.
  2. Second, the Image Activation Hypothesis(de Ruiter, 1998; Wesp, Hesse, Keutmann, & Wheaton, 2001). gestures serve to maintain activation on mental images while they are encoded in speech.

20

- IPH에서 키타가 한 실험.
    - [image:5DDDBD6B-5494-438C-A9C3-0C293094973A-1343-0001218ACC14AD90/Screenshot 2020-04-03 at 22.32.02.png]
- As predicted from the IPH, participants produced more representational gestures (but not more beat gestures) in the hard condition than in the easy condition, while using comparable content in speech. 
- 결과적으로 더 어려운 그림을 설명해야할 때 더 많은 제스처를 사용함

제스처의 제한

22

- 아이들의 제스처를 제한 시킨뒤 설명을 시켰을때, 
    - 비교문이라던가  (about information that was not perceptually present, such as information about the initial equality of the object), 
    - 아니면 변화 transformation that the experimenter had previously performed.
    - 또는 hypothetical states or transformation (e.g., “if you put these two together, then this would be longer that this”)
- 제스처를 허용했을 경우
    - tended to focus on information that was perceptually present (e.g., “this one’s taller”) 
    - often combination with deictic information with perceptual gesture information.
        - (i.e., a gesture toward one of the task objects that also depicted or highlighted a perceptual feature of the object, such as a flat palm held at the top edge of one of the glasses, to depict the height of the glass) 
- **Thus, prohibiting gestures reduced children’s focus on perceptually available spatio-motoric information in their explanations.** 

24

- participants who were allowed to gesture expressed a greater percentage of key events with semantically rich verbs than did speakers who could not gesture. This finding is compatible with the view that speakers package information differently when they produce gesture and when they do not. 
- In addition, speakers who could not gesture were more likely to begin units with a filler (i.e., “um,” “uh,” “and,” or “then”) than were speakers who were al- lowed to gesture. 

30

- (말하기 전 집중용으로) Alibali et al. (2014) argued that the boy’s gesture served to focus his attention on the width of the dish, and highlighted information about width for verbalization. 
- (할 말이 정해진 상태에서 어떻게 말할 것인지 표현할 것인지 possible option를 explore하는 용으로 제스처를 사용) In other cases, speakers’ choices about what information to express are relatively constrained or specified in advance. In such cases, speakers may use gesture to explore possible options for how to express that information. 

챕터 7 (155-)

155

- As movements of the body, gestures are actions, albeit representational ones that do not actually manipulate the physical environment.  
- gestures provide perceptual experiences 
- Gesture as Simulated Action (GSA) 프레임워크는 
    - which claims that gestures emerge from **perceptual representations** and **links with action** that are formed in the minds of speakers. We then consider how gestures’ relationship to perceptual-motor representations might play a functional role in strengthening those representations in the minds of speakers. 

156

- Researchers who argue for the **embodiment of cognition** claim 
    - (1) that psychological processes are grounded in the sensorimotor(감각운동) system 
    - and (2) that **action is an integral part of perception** (see Glenberg, Witt, & Metcalfe, 2013 for a review). 
    - Thus, perceptual judgments are not the result of abstract calculations, but involve activation of our own previous, current, and expected sensorimotor experiences.  (간단한 거리 감각에도 얼마나 그 거리에 도달하기까지 에너지가 필요한가에 따라서 더 멀리 느껴지며, 누가 총을 들고 있는가 아닌가는 우리 스스로가 총을 들고 있느냐 아닌가로 판단된다)

157 (GSA의 연구들)

- **The GSA framework** follows the claims of embodied cognition more **generally to assert that when speakers talk about perceptual and motor experiences**, they activate perceptual-motor representations of those experiences, and **these representations reactivate the same neural areas that were involved in actually having or observing those experiences**. 지각적 운동적 경험의 제스처를 할때 실제로 두뇌의 그 부분을 액티베이트시킨다.

158

- For example, speakers gesture more when they are describing spatial or motoric information than when they are describing abstract information (Alibali, 2005). 

159

- Hostetter (2014) found that speakers gestured at high rates both when describing highly motoric objects (e.g., tools) and when speaking to a listener who could see their gestures. 
- Although the framework is termed Gesture as Simulated *Action*, this should not be taken to imply that the framework excludes gestures that occur with per- ceptual representations (such as thinking about the size or shape of an object). 

160

- judgments of size and distance of objects are affected by our own past and anticipated experiences with those objects (e.g., Witt & Proffitt, 2005). 
- Moreover, increasing evidence suggests that **viewing an object with a particular shape activates a motor plan for how to grasp or use the object (Bub, Masson, & Cree, 2008)** as well as a motor plan for **how to trace the shape of the object** (Bach, Griffiths, Weigelt, & Tipper, 2010). 
- It is also not unusual for speakers to gesture as **they describe something that they have only read about** and have not directly experienced. 
    - embodied cognition is that psychological processes are grounded in the sensorimotor system (e.g., Glenberg et al., 2013); such psychological processes include reading. 

168

- This means that gestures can be thought of as highlighting perceptual-motor representations in two ways. 
    1. Gestures’ perceptual-motor representations’ production signals that a particular kind of representation is formed in the speakers’ mind, mainly one that reactivates the neural experience of the perceptual and motor event the speaker is describing.
    2. The motor plan involved in gesture can strengthen the speakers’ representation, which can affect how easy it is for the speaker to attend to, remember, or describe gestured elements of that representation 

챕터 17(381-

381

- When we say here that **gestures are representational actions**, we mean that they are meaningful substitutions and analogical stand-ins for ideas, objects, actions, relations, etc. 
- 여기서 **representational** 의 사용은 representational gesture (a category of gestures that look like the ideas and items to which they refer (i.e., iconic and metaphoric gestures) 와 혼동되면 안된다. ➡︎ **apply to all types of nonconventional gestures**, including representational gestures (iconics, metaphorics), deictic gestures (points), and even beat gestures (rhythmic movements closely coordinated with speech). 

382

- Gesture is *action*in that it involves movements of the body. 
- 하지만 제스처는 다른 방식의 액션이다.
- one that *represents* the world rather than directly impacting the world.

Properties of a Movement to be Identified as a Gesture

382-3 — Processing movement as gesture

- 재현적인 제스처는 비재현적인 제스처와의 구별이 필요하다. 
- 이 부분은 왜 사람들이 움직임을 제스처로 보게 만드는가를 설명한다.
- 첫번째 조건으로는 **빈 손**이다 ➡︎ 하지만 모든 빈 손(춤이나 운동)이 제스처로 인식되진 않는다. ➡︎ However, unlike dance or exercise, the movement itself is *not* the goal of a gesture. 

384 — The unique functions of gesture in communication, problem solving, and learning

- 제스처와 다른 움직임은 인식적으로 분리된다는 benefits가 있다.