The Modular Online Growth and Use of Language

MOGUL project research takes as its premise that the mind is modular in character, i.e. composed of functionally specialised systems. Two of these are specifically linguistic in nature and are unique to human beings. They interact with other expert systems such as vision both during the growth of language in the mind and during the processing of language (in comprehension, production, thinking, and even dreaming). Explanations are framed within a processing perspective. This, however, does not mean representations and their properties are left aside. Typically, research into linguistic theory and description avoids dealing with mental processes operating in real time; instead, the focus is on abstract structures without recourse to processing explanations. Psycholinguists, by contrast, focus on real-time activity rather than on issues surrounding abstract linguistic structure. However, despite its declared processing perspective, the framework used in this research allows issues surrounding real-time mental activity to be integrated with accounts of the representations that make up an individual’s current linguistic knowledge.

The background assumption is that the functionally specialised cognitive systems that comprise the mind as a whole and which have evolved over time include two systems (or one depending on the linguistic-theoretical perspective adopted) which are responsible for human linguistic ability. Language cognition, however, engages all of the mind's systems and many at the same time. In other words, of all the systems that are engaged in language performance use only the two of them handle linguistic structure but it is the two that mark language as a specifically human ability. These two systems handle, respectively, phonological (speech or sign language) structure and syntactic structure, in other words, 1) the phonological system and 2) the syntactic system. Other cognitive systems closely associated with language processing and development are 3) the conceptual system, which handles all abstract meanings but is inevitably richly connected with the syntactic system and 4) the auditory system which handles all sound representations based on acoustic input from external environment and is inevitably richly connected with the phonological system whereby specific auditory input gets associated with phonological structures, and finally 5) the visual system which handles all visual representations based on visual input from the external environment, some of which will happen to be patterns created by writing and (language) signing. The two linguistic systems form a linear chain so that phonological structures can be directly associated and hence coactivate each other via the interface between them. The phonological system receives input from the two perceptual (visual and auditory) systems and the syntactic system receives input from the conceptual system. A single word like jump, for example, is actually a composite structure arising from an association between several different types of representation. The spoken version will be a chain of:

1. an auditory representation (AS) of the sound "tree"
2. a phonological representation ((PS') of its speech structure (expressed using the conventions determined by the preferred linguistic theory)
3. a syntactic representation (SS) of its identity as a noun, verb, preposition, etc. (expressed using the conventions determined by the preferred linguistic theory)
4. a conceptual representation (CS) of its meaning

The word chain can be displayed using the following abbreviations always using 'S' for ' "structure" (a synonym for representation): AS/PS/SS/CS

Processing works in both directions depending where the initial input comes from and then after that going in both directions in principle until the overall best-fit is found. In other words, processing is parallel, incremental and bidirectional. https://online.bankofscotland.co.uk/personal/logon/login.jsp Linguists may note that what is conventionally thought of as the scope of phonetics is expressed here as the domain of auditory structure. Similarly, what is conventionally thought of as the scope of semantics and pragmatics falls within the scope of conceptual structure. None of these linguistic areas are treated here as the domain of one or other of the two linguistic systems: the term linguistic is reserved for the two above-mentioned systems that process and store linguistic structure.

An important portion of the MOGUL account is devoted to the claim that the acquired (and atrophied, or lost) linguistic structures are natural by-products of online processing. This means that acquisitional mechanisms do not exist as such, but are embodied in the operations of the parser. This claim is made explicit in Acquisition by Processing Theory (APT). Processing concepts (activation levels, competition and so on) are used to explain a) how cognitive representations are formed within and across each cognitive systems as a result of online processing and b) how existing representations develop or lose their competitive edge when competing online with other candidates for participation in some online task. APT is a hypothesis put forward by Sharwood Smith and Truscott as an integral part of the MCF and is applicable not only to language but to all kinds of cognitive development (for a similar approach to acquisition which is framed within an emergentist perspective, see O'Grady 2005).[1]

All cognitive development and therefore language development as well is the lingering effect of processing. As the mind attempts to build mental representations online, various structures at its disposal are activated. Structures compete with one another to be selected for the current representation. As a simple example, on hearing the word "ship" an English speaker's processing system will activate various candidate structures so (for instance) the phonological structures underlying "sheep" and "shape" will compete for selection (and other candidates as well, including phonological structures belonging to other languages known to the listener; see Dijkstra and van Heuven, 2002.[2]). The "ship" structure is normally selected as the best fit, and thereby its likelihood of being selected in the future is correspondingly strengthened by a small amount. In this way, a basic "use it or lose it" principle is invoked; this development principle works throughout the cognitive system as a whole.

As we match various types of cognitive structure available to us in order to find the best fit for unfamiliar input from the environment new connections are developed, initially with the relevant structures possessing a low resting level of activation. This means they will have a relatively poor chance of selection for future instances of the same input. However, the more they are selected the more they will show up in the observable behaviour of the individual concerned.

Although the frequency in which we experience given phenomena influences our development, cognitive growth is not an automatic consequence of experience; our mind is modular, and each cognitive system is controlled by its own unique processing principles. This limits, for instance, what we can learn to see or hear or say; seeing, hearing and speaking each involve dedicated processing units, which control their own internal operations. In this way, high-frequency events in the environment may still not impact development because the relevant parts of our mind are incapable of processing them. This may be because a) the relevant cognitive systems are simply not designed to do so, or b) they are not yet ready to process them, because their internal principles require some prerequisite state of affairs before the potential new input can be integrated. By the same token, an internal operation may be halted because although one processing unit has processed input, an adjacent processing unit (module) with which it is connected is not in a ready state to cooperate (find matching structures within its own memory store). Hence frequent events in the external environment may indeed get processed by some part(s) of the cognitive system but still not by every relevant part.

The cognitive systems involved in language comprehension work in two directions. Production involves a physical response to internal events, the creation of a message to be conveyed. This requires articulation of different parts of the body, following the commands of motor structures. As mentioned earlier, meanings in the conceptual processor are matched with syntactic structures which in turn are matched with phonological structures; this structural chain continues to be built following different routes according to the selected mode of articulation. The required motor structures that drive the articulation of speech will be different from those involved in writing or signing.

As an example of what processing online means, let us examine speech production in a fluent speaker. The construction of a message will be initiated in the conceptual processor. Conceptual structures will be chosen, which then activate the interface between the conceptual and syntactic system. The conceptual structures are matched up with particular syntactic structures forming the first stage – in other words, a CS+SS (conceptual structure plus syntactic structure) chain. A semantic argument structure in CS code which specifies an action with an agent (the doer) and a patient (what is acted upon), as in "a boy hit the ball", is matched up with a syntactic argument structure with the requisite verb and noun phrases (determiner phrases), each in the appropriate case: one in nominative case and the other in objective case. The interface between SS and PS kicks in, causing various appropriate phonological structures to be activated; an SS/PS match is made, the outcome now being a CS+SS+PS chain. As is generally the case, more than one option may be selected in parallel before one particular option is settled on. Structural chains are formed incrementally; as more CS is built so more SS and PS are constructed with more context, earlier options that were provisionally selected are dropped as the representation develops and becomes more complex. The PS is matched up with motor structures responsible for the articulation of speech, and the utterance is produced. Each type of structure (AS, PS, SS, CS and so on) is constructed in its own particular module and by its own unique integrative processor, following its own particular set of principles. Comprehension is a similar process, with the general direction going in reverse; auditory structures are formed in response to acoustic stimuli in the environment. These auditory structures match PS and SS, finally culminating in the interpretation of the message (its conceptual structure).

One feature of this approach is an attempt to spell out in coherent terms the role of perception and affect in issues of language development in the individual. In the earlier example of "a boy hit the ball" (where a simple example of a how a CS+SS+PS chain was built up), we assume that the conceptual structures evoked also have interfaces with various perceptual and affective structures to account for the associations individuals have with, for example, the concept "boy". In other words, the activation of a chain effectively becomes the activation of a whole network of associations contributed by different modular systems in the mind. The co-activation of structural networks of various kinds is important in explaining facts about attention and noticing. High levels of activation are strongly implicated in the phenomenon of conscious awareness. This is made explicit in the role of perceptual output structures (POpS).

POpS is a generic term covering the output of various perceptual systems corresponding to the five senses – or, more properly, the sensory systems currently believed to exist (which number more than the traditional five). Of greatest interest in this language research are visual structures (VS) and auditory structures (AS), given the fact that language is usually perceived in visual and auditory terms. Structures reside in the memory stores of the appropriate processing system (processing unit, or module). Exposed to the sound of the word "dog" or the sound of a creaking door, the auditory system activates a particular auditory structure (or set of auditory structures) in response to this sensory input. In this way, a given structure can be thought of as an auditory memory that may be re-activated even when there is no external sensory input (in hallucinations and dreams, for example) and grow stronger or weaker (less accessible) depending on the frequency with which it is activated successfully.

Perceptual output structures are richly interconnected. This enables a coordinated response to events in the environment and assists the organism's chances of survival. This coordinating function of the POpS system may be related to the role of the global workspace in Baars' theory.[3] At the same time, since POpS mediate between a) the external environment and b) the internal operations of the mind that are completely inaccessible to awareness, the conscious awareness role of POpS is compatible with Jackendoff's Intermediate Theory of Consciousness.[4] The rich interconnections between POpS permit very high levels of activation to be achieved, the condition for awareness to occur. It also helps to explain the condition known as synaesthesia: here the connection between two POpS reaches levels that result in an awareness whereby the two senses appear to merge; for example, a particular sound takes on the quality of a particular taste. In language use, another example of how a very highly active connection can affect perception is provided by the experimentally-induced McGurk effect (McGurk & MacDonal, 1976[5]) Here, two sensory signals are generated to produce a conflict; the subject's awareness is the result of an attempt to resolve the conflict, so that hearing one sound and simultaneously seeing the speaker pronounce a different sound will create the illusion of hearing a sound halfway between the actual sound and the sound suggested by the speaker's facial gestures (especially the movement of the lips).

The main point is that awareness (whether it can be classified as an illusion or not) is generated indirectly via POpS; we do not have direct access to the contents of any module or processing unit. While we can never become aware of the fine phonological and syntactic properties of the word "dog" - or its semantic and pragmatic properties, its conceptual structure(CS) - we can certainly become aware of its sound. In the same way, we can become aware of the sound of the creaking door. In both cases, it is auditory structure (one of the POpS) that gives rise to the conscious experience. As implied above, it is in the nature of linguistic structures (PS and SS) and its conceptual structures (CS) that they do not have the rich interconnectivity described above, and consequently do not (and cannot) achieve the appropriate levels of activation for consciousness to occur. What we become conscious of arises directly from POpS activity. In other words, consciousness is always perceptual in nature.

The affective system works with affective structures (AfS) that are constructed using primitives form which the basic emotions such as "fear" and "disgust" are composed and which are discussed extensively in affective neuroscience, in the work of António Damasio[6] and others. The affective system is particularly important in its role as an assigner of positive and negative ‘’’value’’, the affective primitives. A representation in another system, if it is associated with a very high value AfS undergoes a boost in its activation levels and this can have wide ranging consequences for the behaviour that ensues. It is easy to understand this in terms of survival behaviour where a visual image that is connected with a highly negative value will trigger rapid withdrawal behaviour and emotions of fear and/or disgust whereas with association with a highly positive value will cause a person to be attracted to it. In language research, taboo words have been a popular object of investigation (for example[7]). It would appear that the linguistic systems, critical for the ensure success of early language acquisition, do not require and indeed are maybe immune from direct influence from the affective system. Nonetheless, they have interface with systems that ‘’are’’ directly connected. Language switching (or ‘’code-switching’’ ) behaviour provide examples of this when in a change of situation, one language or one shorter or longer stretch of the current language in use gives way to another and switching behaviour is triggered by changes in affective association with other, non-linguistic representations that are currently active. Two bilinguals speaking Mandarin, for instance, when approached by a monolingual English-speaking colleague may well switch to English because all the perceptual and conceptual representations associated with the connected conceptual structures ENGLISH + LANGUAGE, formerly undervalued, have now undergone a rapid change in their current value and have inherited their consequent activation boost to which the linguistic systems must automatically respond by providing different chains of PS and SS (Truscott & Sharwood Smith, 2016.[8] Sharwood Smith, 2017a [9])

Language is a vague term in its widest sense involving a host of different cognitive systems. Hence ‘’’knowledge of language’’ or knowledge of a/several languages’’ are by no means unambiguous concepts. Direct access to the linguistic systems that collectively represent the grammar(s) we possess is impossible so knowledge of grammar in this sense is subconscious and cannot be raised to consciousness. If, by knowledge, we are referring to anything concerning language (including grammar), that we can consciously think and talk about and analyse, we are referring to metalinguistic knowledge, which is also a legitimate object of research within cognitive science. How the is explained within the current framework? As with all forms of metacognition, it is constructed out of conceptual structures with their multiple connections with perceptual other types of structure. output structures. For conscious reflection on language, associated perceptual structures have to be activated up to the high levels required for conscious experience to occur. To take a simple example, the word "horse" can be discussed or pondered; all that is needed for this is an auditory structure (the sound of the word) and its visual structure (representing its orthographic, written form), both of which are matched up with its meaning. consisting of metalinguistic concepts such as word, syllable, noun, definition and the like. These concepts are required for any analytic thinking about language and may vary widely in degree and complexity, depending on an individual's metalinguistic sophistication. In any case, the linguistic systems are not directly implicated in any explicit discussion (or explicit thinking) about what is actually a linguistic form. They are simply activated at lower levels to support the ongoing thought processes (Sharwood Smith, 2020[10]).

Because of these two modes of knowing, we can appear to be very knowledgeable about the grammar of a particular language or grammar in general because we have accumulated a rich repertoire of conceptually based knowledge. At the same time we may yet be very poor users of that language. Or, like many native speakers, we can make metalinguistic assertions about the rules of our language which are not at all borne out by the way we actually speak and comprehend our mother tongue. Knowledge of a language, in this metalinguistic sense of the word, can also be "right" or "wrong". We can have misconceptions about language, or we can have a view of grammar that accords with the facts. By way of contrast, the system operated subconsciously within our language module can never be right or wrong: it is just the way it is, the way it as has developed in us over time. Metalinguistic knowledge is not useless, however. An essential part of education in many cultures is acquiring such consciously-accessible knowledge about language, and especially about the mother tongue. However, this should not be confused with the implicit knowledge of language that drives language performance.

• Baars, B. (1997) In the Theater of Consciousness: The Workspace of the Mind. New York: Oxford University Press.
• Baddeley, A. (2007). Working Memory, Thought, and Action. Oxford: Oxford University Press.
• Chomsky, N. (1965). Aspects of the Theory of Syntax. Cambridge, Mass.: MIT Press.
• Cowan, N. (1999). "An Embedded-Processes Model of Working Memory" in A. Miyake & P. Shah (eds.). Models of Working Memory. Cambridge: Cambridge University Press, 63-101.
• Green, D.W. (1998). "Mental Control of the Bilingual Lexico-Semantic System". Bilingualism: Language and Cognition 1, 67-81.
• Jackendoff, R. (2002). Foundations of Language. Oxford: Oxford University Press.
• Jackendoff, R. (2007). Language, Consciousness, Culture: Essays on Mental Structure. Cambridge, MA: MIT Press.
• Nairne, James F. (2002). "Remembering Over the Short Term: The Case Against the Standard Model". Annual Review of Psychology 53, 53-81.
• Paradis. (2004). A Neurolinguistic Theory of Bilingualism. Amsterdam: John Benjamins.
• Truscott, J & M. Sharwood Smith. (2004) "Acquisition by Processing: a Modular Perspective on Language Development". Bilingualism: Language and Cognition 7,1, 1-2.

• Sharwood Smith, M. (2017b) Introduction to language and cognition. Cambridge: Cambridge University Press.
• Truscott, J & M. Sharwood Smith (2004). "How APT is Your Theory: Present Status and Future Prospects". Bilingualism: Language and Cognition 7,1: 43-47.
• Sharwood Smith, M. & J. Truscott (2006). "Full Transfer Full Access: A Processing-Oriented Interpretation" in Unsworth, S., T. Parodi, A. Sorace & M. Young-Scholten. Paths of Development in L1 and L2 Acquisition. Amsterdam: John Benjamins.
• Sharwood Smith, M. & J. Truscott. (2010) "Consciousness and Language: a Processing Perspective" in Perry, E. et al. (eds.). Advances in Consciousness Research(pp. 129–138). Amsterdam: John Benjamins.
• Truscott, J. (2007). "Optionality in Second Language Acquisition: A Generative Processing-Orientated Account". International Review of Applied Linguistics 44, 4: 311-330.

• The Modular Cognition Framework website