Morphological well-formedness

In the section on phonotactic acceptability, we saw that judgments about possible words are gradient. The same is true of morphological combinations: re-doable is relatively natural, de-publicize is more marginal, and ness-happy is poor. So what should a model predict if well-formedness is not merely yes or no?

The bundled trials.csv contains ratings for morphologically complex English words and nonwords. We will use these ratings as the morphological acceptability target: a model should assign higher scores to forms that participants rated more highly.

The acceptability data

We begin with the judgment data.

import pandas as pd
import matplotlib.pyplot as plt

trials = pd.read_csv('data/trials.csv')

print(f"Total judgments: {len(trials)}")
print(f"Unique words: {trials.word.nunique()}")
print(f"Unique subjects: {trials.subject.nunique()}")
trials.head()

The judgment column contains raw ratings and judgment_z contains z-scored ratings, normalized per participant to account for individual differences in scale use.

fig, axes = plt.subplots(1, 2, figsize=(10, 4))

axes[0].hist(trials.judgment, bins=20, edgecolor='black', alpha=0.7)
axes[0].set_xlabel('Raw judgment')
axes[0].set_ylabel('Count')
axes[0].set_title('Raw judgments')

axes[1].hist(trials.judgment_z, bins=40, edgecolor='black', alpha=0.7)
axes[1].set_xlabel('Z-scored judgment')
axes[1].set_ylabel('Count')
axes[1].set_title('Z-scored judgments')

fig.tight_layout()
plt.show()

We can also look at the average z-scored judgment for each word:

word_means = trials.groupby('word').judgment_z.mean().sort_values()

fig, ax = plt.subplots(1, 1, figsize=(10, 4))
ax.plot(range(len(word_means)), word_means.values, linewidth=0.5)
ax.set_xlabel('Word (sorted by mean judgment)')
ax.set_ylabel('Mean z-scored judgment')
ax.axhline(0, color='gray', linestyle='--', linewidth=0.5)
fig.tight_layout()
plt.show()

print("Most acceptable:")
print(word_means.tail(5))
print("\nLeast acceptable:")
print(word_means.head(5))

The modeling target

The binary CFGs developed above distinguish licensed from unlicensed strings. The ratings require an ordering within and beyond that binary division. We will obtain such an ordering by assigning probabilities to rules and then summing the probabilities of the parses for each word.

Before we can estimate those probabilities, we need two kinds of annotation. Morpheme boundaries supply the terminal sequence, while morphological trees supply the rule counts. CELEX provides both for a large English lexicon. So we next ask how those analyses are represented and how to load them.