Formal language

In the 17th century, Gottfried Leibniz imagined and described the characteristica universalis, a universal and formal language which utilised pictographs. During this period, Carl Friedrich Gauss also investigated the problem of Gauss codes.[2]

Gottlob Frege attempted to realize Leibniz's ideas, through a notational system first outlined in Begriffsschrift (1879) and more fully developed in his 2-volume Grundgesetze der Arithmetik (1893/1903).[3] This described a "formal language of pure language."[4]

In the first half of the 20th century, several developments were made with relevance to formal languages. Axel Thue published four papers relating to words and language between 1906 and 1914. The last of these introduced what Emil Post later termed 'Thue Systems', and gave an early example of an undecidable problem.[5] Post would later use this paper as the basis for a 1947 proof "that the word problem for semigroups was recursively insoluble",[6] and later devised the canonical system for the creation of formal languages.

In 1907, Leonardo Torres Quevedo introduced a formal language for the description of mechanical drawings (mechanical devices), in Vienna. He published "Sobre un sistema de notaciones y símbolos destinados a facilitar la descripción de las máquinas" ("On a system of notations and symbols intended to facilitate the description of machines").[7] Heinz Zemanek rated it as an equivalent to a programming language for the numerical control of machine tools.[8]

Noam Chomsky devised an abstract representation of formal and natural languages, known as the Chomsky hierarchy.[9] In 1959 John Backus developed the Backus-Naur form to describe the syntax of a high level programming language, following his work in the creation of FORTRAN.[10] Peter Naur was the secretary/editor for the ALGOL60 Report in which he used Backus–Naur form to describe the Formal part of ALGOL60.

An alphabet, in the context of formal languages, can be any set; its elements are called letters. An alphabet may contain an infinite number of elements;[note 1] however, most definitions in formal language theory specify alphabets with a finite number of elements, and many results apply only to them. It often makes sense to use an alphabet in the usual sense of the word, or more generally any finite character encoding such as ASCII or Unicode.

A word over an alphabet can be any finite sequence (i.e., string) of letters. The set of all words over an alphabet Σ is usually denoted by Σ^* (using the Kleene star). The length of a word is the number of letters it is composed of. For any alphabet, there is only one word of length 0, the empty word, which is often denoted by e, ε, λ or even Λ. By concatenation one can combine two words to form a new word, whose length is the sum of the lengths of the original words. The result of concatenating a word with the empty word is the original word.

In some applications, especially in logic, the alphabet is also known as the vocabulary and words are known as formulas or sentences; this breaks the letter/word metaphor and replaces it by a word/sentence metaphor.

A formal language L over an alphabet Σ is a subset of Σ^*, that is, a set of words over that alphabet. Sometimes the sets of words are grouped into expressions, whereas rules and constraints may be formulated for the creation of 'well-formed expressions'.

In computer science and mathematics, which do not usually deal with natural languages, the adjective "formal" is often omitted as redundant.

While formal language theory usually concerns itself with formal languages that are described by some syntactical rules, the actual definition of the concept "formal language" is only as above: a (possibly infinite) set of finite-length strings composed from a given alphabet, no more and no less. In practice, there are many languages that can be described by rules, such as regular languages or context-free languages. The notion of a formal grammar may be closer to the intuitive concept of a "language", one described by syntactic rules. By an abuse of the definition, a particular formal language is often thought of as being equipped with a formal grammar that describes it.

The following rules describe a formal language L over the alphabet Σ = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, +, =}:

• Every nonempty string that does not contain "+" or "=" and does not start with "0" is in L.
• The string "0" is in L.
• A string containing "=" is in L if and only if there is exactly one "=", and it separates two valid strings of L.
• A string containing "+" but not "=" is in L if and only if every "+" in the string separates two valid strings of L.
• No string is in L other than those implied by the previous rules.

Under these rules, the string "23+4=555" is in L, but the string "=234=+" is not. This formal language expresses natural numbers, well-formed additions, and well-formed addition equalities, but it expresses only what they look like (their syntax), not what they mean (semantics). For instance, nowhere in these rules is there any indication that "0" means the number zero, "+" means addition, "23+4=555" is false, etc.

Formal languages are used as tools in multiple disciplines. However, formal language theory rarely concerns itself with particular languages (except as examples), but is mainly concerned with the study of various types of formalisms to describe languages. For instance, a language can be given as

• those strings generated by some formal grammar;
• those strings described or matched by a particular regular expression;
• those strings accepted by some automaton, such as a Turing machine or finite-state automaton;
• those strings for which some decision procedure (an algorithm that asks a sequence of related YES/NO questions) produces the answer YES.

Typical questions asked about such formalisms include:

• What is their expressive power? (Can formalism X describe every language that formalism Y can describe? Can it describe other languages?)
• What is their recognizability? (How difficult is it to decide whether a given word belongs to a language described by formalism X?)
• What is their comparability? (How difficult is it to decide whether two languages, one described in formalism X and one in formalism Y, or in X again, are actually the same language?).

Surprisingly often, the answer to these decision problems is "it cannot be done at all", or "it is extremely expensive" (with a characterization of how expensive). Therefore, formal language theory is a major application area of computability theory and complexity theory. Formal languages may be classified in the Chomsky hierarchy based on the expressive power of their generative grammar as well as the complexity of their recognizing automaton. Context-free grammars and regular grammars provide a good compromise between expressivity and ease of parsing, and are widely used in practical applications.

Certain operations on languages are common. This includes the standard set operations, such as union, intersection, and complement. Another class of operation is the element-wise application of string operations.

Examples: suppose ${\displaystyle L_{1}}$ and ${\displaystyle L_{2}}$ are languages over some common alphabet ${\displaystyle \Sigma }$.

• The concatenation ${\displaystyle L_{1}\cdot L_{2}}$ consists of all strings of the form ${\displaystyle vw}$ where ${\displaystyle v}$ is a string from ${\displaystyle L_{1}}$ and ${\displaystyle w}$ is a string from ${\displaystyle L_{2}}$.
• The intersection ${\displaystyle L_{1}\cap L_{2}}$ of ${\displaystyle L_{1}}$ and ${\displaystyle L_{2}}$ consists of all strings that are contained in both languages
• The complement ${\displaystyle \neg L_{1}}$ of ${\displaystyle L_{1}}$ with respect to ${\displaystyle \Sigma }$ consists of all strings over ${\displaystyle \Sigma }$ that are not in ${\displaystyle L_{1}}$.
• The Kleene star: the language consisting of all words that are concatenations of zero or more words in the original language;
• Reversal:
  • Let ε be the empty word, then ${\displaystyle \varepsilon ^{R}=\varepsilon }$, and
  • for each non-empty word ${\displaystyle w=\sigma _{1}\cdots \sigma _{n}}$ (where ${\displaystyle \sigma _{1},\ldots ,\sigma _{n}}$are elements of some alphabet), let ${\displaystyle w^{R}=\sigma _{n}\cdots \sigma _{1}}$,
  • then for a formal language ${\displaystyle L}$, ${\displaystyle L^{R}=\{w^{R}\mid w\in L\}}$.
• String homomorphism

Such string operations are used to investigate closure properties of classes of languages. A class of languages is closed under a particular operation when the operation, applied to languages in the class, always produces a language in the same class again. For instance, the context-free languages are known to be closed under union, concatenation, and intersection with regular languages, but not closed under intersection or complement. The theory of trios and abstract families of languages studies the most common closure properties of language families in their own right.[11]

Closure properties of language families (${\displaystyle L_{1}}$ Op ${\displaystyle L_{2}}$ where both ${\displaystyle L_{1}}$ and ${\displaystyle L_{2}}$ are in the language family given by the column). After Hopcroft and Ullman.

Operation		Regular	DCFL	CFL	IND	CSL	recursive	RE
Union	${\displaystyle L_{1}\cup L_{2}=\{w\mid w\in L_{1}\lor w\in L_{2}\}}$	Yes	No	Yes	Yes	Yes	Yes	Yes
Intersection	${\displaystyle L_{1}\cap L_{2}=\{w\mid w\in L_{1}\land w\in L_{2}\}}$	Yes	No	No	No	Yes	Yes	Yes
Complement	${\displaystyle \neg L_{1}=\{w\mid w\not \in L_{1}\}}$	Yes	Yes	No	No	Yes	Yes	No
Concatenation	${\displaystyle L_{1}\cdot L_{2}=\{wz\mid w\in L_{1}\land z\in L_{2}\}}$	Yes	No	Yes	Yes	Yes	Yes	Yes
Kleene star	${\displaystyle L_{1}^{*}=\{\varepsilon \}\cup \{wz\mid w\in L_{1}\land z\in L_{1}^{*}\}}$	Yes	No	Yes	Yes	Yes	Yes	Yes
(String) homomorphism ${\displaystyle h}$	${\displaystyle h(L_{1})=\{h(w)\mid w\in L_{1}\}}$	Yes	No	Yes	Yes	No	No	Yes
ε-free (string) homomorphism ${\displaystyle h}$	${\displaystyle h(L_{1})=\{h(w)\mid w\in L_{1}\}}$	Yes	No	Yes	Yes	Yes	Yes	Yes
Substitution ${\displaystyle \varphi }$	${\displaystyle \varphi (L_{1})=\bigcup _{\sigma _{1}\cdots \sigma _{n}\in L_{1}}\varphi (\sigma _{1})\cdot \ldots \cdot \varphi (\sigma _{n})}$	Yes	No	Yes	Yes	Yes	No	Yes
Inverse homomorphism ${\displaystyle h^{-1}}$	${\displaystyle h^{-1}(L_{1})=\bigcup _{w\in L_{1}}h^{-1}(w)}$	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Reverse	${\displaystyle L^{R}=\{w^{R}\mid w\in L\}}$	Yes	No	Yes	Yes	Yes	Yes	Yes
Intersection with a regular language ${\displaystyle R}$	${\displaystyle L\cap R=\{w\mid w\in L\land w\in R\}}$	Yes	Yes	Yes	Yes	Yes	Yes	Yes

• Combinatorics on words
• Free monoid
• Formal method
• Grammar framework
• Mathematical notation
• Associative array
• String (computer science)

• "Formal language", Encyclopedia of Mathematics, EMS Press, 2001 [1994]
• University of Maryland, Formal Language Definitions
• James Power, "Notes on Formal Language Theory and Parsing" Archived 21 November 2007 at the Wayback Machine, 29 November 2002.
• Drafts of some chapters in the "Handbook of Formal Language Theory", Vol. 1–3, G. Rozenberg and A. Salomaa (eds.), Springer Verlag, (1997):
  • Alexandru Mateescu and Arto Salomaa, "Preface" in Vol.1, pp. v–viii, and "Formal Languages: An Introduction and a Synopsis", Chapter 1 in Vol. 1, pp.1–39
  • Sheng Yu, "Regular Languages", Chapter 2 in Vol. 1
  • Jean-Michel Autebert, Jean Berstel, Luc Boasson, "Context-Free Languages and Push-Down Automata", Chapter 3 in Vol. 1
  • Christian Choffrut and Juhani Karhumäki, "Combinatorics of Words", Chapter 6 in Vol. 1
  • Tero Harju and Juhani Karhumäki, "Morphisms", Chapter 7 in Vol. 1, pp. 439–510
  • Jean-Eric Pin, "Syntactic semigroups", Chapter 10 in Vol. 1, pp. 679–746
  • M. Crochemore and C. Hancart, "Automata for matching patterns", Chapter 9 in Vol. 2
  • Dora Giammarresi, Antonio Restivo, "Two-dimensional Languages", Chapter 4 in Vol. 3, pp. 215–267