Wild cards and character ranges

We can represent all simple regular expressions according to our formal definition, but in certain cases, doing so would be tedious. For instance, suppose I want to represent the set of all English phonemes \(\Sigma\). Using our formal definition, we would need to list out all of the phonemes joined by \(\cup\): \((\text{i} \cup (\text{ɝ} \cup (\text{a} \cup (\text{ɪ} \cup \ldots))))\).

To make this less tedious, Python (and many other languages) introduce additional special characters into the definition of \(R(\Sigma)\). The most basic is the wildcard ., which matches any single Unicode code point (alphanumeric or otherwise) except the newline \n. A regex-engine character is not always a phoneme: phones such as tʃ and aʊ contain multiple code points in our transcription. The examples below join phones into strings and therefore use . at the character level.

with open("cmudict-ipa", encoding="utf-8") as f:
    entry_rows: list[list[str]] = [
        line.strip().split(",", maxsplit=1) for line in f
    ]
    entries: dict[str, list[str]] = {
        word: ipa.split() for word, ipa in entry_rows
    }

import re

regex_dot_bstɹ_dot_kʃən = '.bstɹ.kʃən'

for w, ipa in entries.items():
    if re.fullmatch(regex_dot_bstɹ_dot_kʃən, "".join(ipa)):
        print("".join(ipa), f"({w})")

If you want to match the . itself (or any special character we introduce below), you need to escape it.

regex_period_bstɹækʃən = r'\.bstɹækʃən'
string_æbstɹækʃən = "".join(entries["abstraction"])

re.fullmatch(regex_period_bstɹækʃən, '.' + string_æbstɹækʃən[1:])

Besides ., we can also use character ranges to target more specific sets, like ASCII upper- and lower-case alphabetic characters ([A-Za-z]), lower-case alphabetic characters ([a-z]), or numerals ([0-9]). Do not use [A-z]: the code points between Z and a include punctuation such as [ and _.

regex_numeric_bstɹækʃən = '[0-9]bstɹækʃən'
string_4bstɹækʃən = '4bstɹækʃən'

re.fullmatch(regex_numeric_bstɹækʃən, string_4bstɹækʃən)

In addition to ranges, there are even more compact escape characters. For instance, \w matches Unicode alphanumeric characters plus _ under Python’s default Unicode rules. It is thus broader than the ASCII class [A-Za-z0-9_] and matches characters such as æ.

regex_alphanumeric_bstɹækʃən = r'\wbstɹækʃən'

re.fullmatch(regex_alphanumeric_bstɹækʃən, string_æbstɹækʃən)