NC-FFN: A Feed-Forward Layer Where Every Neuron Can Introduce Itself in Logical Language

An Old Question

The feed-forward layer (FFN) is the most opaque component of the Transformer architecture. Attention layers at least have a name—"heads"—so you can loosely say "this head is looking at the preceding noun." The FFN? Two linear transformations sandwiching a GELU activation, with tens of thousands of neurons in between, each computing GELU(W2 @ GELU(W1 @ x)). Ask "what is neuron #4096 computing?" and nobody can answer.

Over the past two years, the mainstream interpretability approach has been post-hoc dissection: train a sparse autoencoder (SAE) to "pry" readable features out of an existing FFN. Anthropic did monosemanticity, OpenAI did SAE probes—the results are good but costly. Every model needs a retrained SAE, and the SAE itself is a black box.

In this June 2026 paper, Thomas Marshall takes a different approach: stop dissecting after the fact—build an FFN that is legible by construction.

NC-FFN: Replacing GELU with Fuzzy Set Operations

The core change is simple. Replace the FFN hidden units GELU(W1 @ x) with explicit fuzzy set operations:

• Intersection: \(A \cdot B\) — "A and B are both true"
• Set-difference: \(A \cdot (1-B)\) — "A is true but B is not"

Each unit's inputs are first squashed into \([0, 1]\) via sigmoid, then combined with these operations. This is the NC-FFN (Negation-Capable FFN), because set-difference gives a bounded, positively-encoded "negation"—"A but not B"—which GELU or gated units cannot express structurally.

Parameter neutrality is the key design: NC-FFN has exactly the same parameter count as a GELU FFN. It is not "trade more parameters for interpretability"—it is "same parameters, different structure."

The hybrid partition is an engineering detail: 75% GELU + 25% Boolean, because purely Boolean FFNs hit a trainability ceiling—fully Boolean networks suddenly spike and fail during training. The 75/25 mix is the empirical optimum.

Three Results

Result 1: Reasoning capability improves

Using a pure FFN stack on N-bit parity (a classic reasoning probe), the conclusion is:

> Bounded multiplicative units are the most parameter-efficient basis for reasoning at shallow depth.

Two axes:

• Bounding (sigmoid-bounded) buys "width-efficient, shallow-depth reasoning."
• Unbounding (unbounded multiplication) buys "depth composition."
• Additive baselines (GELU) plateau in between.

Set-difference (complement) additionally provides a shallow-depth margin—"A but not B" offers one more dimension of discriminative power than plain "A and B."

Result 2: Language modeling ties GELU

At 125M parameters on OpenWebText, NC-FFN and the GELU baseline tie in perplexity at equal parameter counts. No semantic gain, no loss.

But there is a "small and persistent syntactic deficit"—concentrated on grammatical licensing and quantifiers, e.g.:

• Comparative + "than": *She is taller than* him.
• Passive participle + "by": *The book was written by* him.
• Negative polarity item + "nor": *He didn't smile, nor* did he speak.

What these constructions share: several tokens separate the licensor from the licensee. "Taller" appears at position \(t\), "than" at \(t+3\). The FFN is within-token—it can only operate on a single token's representation and cannot carry information from \(t\) to \(t+3\).

This is NC-FFN's structural shortcoming: within-token set operations cannot express sequence-level dependencies.

Result 3: Self-forgetting quantifiers close the gap

The paper's "central result": adding a sequence-level fuzzy quantifier module solves both problems at once.

Module design:

• Soft existential: a soft "exists" operation over the sequence.
• Soft proportion: a soft "most" operation over the sequence.
• Each unit has an independently learnable forgetting rate, initialized at the "no forgetting" limit.

After training:

1. The syntactic deficit recovers within epoch 1—by epoch 2 the gap is halved. 2. Slight lead over the baseline on LAMBADA. 3. Logical structure is preserved and transfers to deeper layers rather than eroding into soft gating as before. 4. Forgetting rates self-organize: the median half-life per unit is about 1.5 tokens, with zero near-permanent units. Quantifiers are local, predictive operators, not latches.

The most striking is point 5:

> Quantifier units read as grammatical-licensing detectors without any dictionary learning.

Each unit fires on a licensor word, then uses its ~1.5-token memory to carry membership forward to the licensed function word:

• Comparative → predicts "than"
• Passive participle → predicts "by"
• Negative polarity item → predicts "nor"

The forgetting rates tune themselves to the "licensing window" length. This was not designed—the training objective rewarded remembering a licensor for exactly the licensor-to-licensee distance.

Three Deeper Findings

Logic is task-shaped

> A network's logical content is a dynamic readout of what its objective rewards: on tasks rewarding multiplicative reasoning, Boolean structure crystallizes precisely; where only prediction is rewarded, it erodes into soft gating.

This answers a fundamental question: legibility is not free. NC-FFN provides readable structure, but whether that structure gets exploited depends on whether the task rewards it. Most prediction in language modeling does not need explicit logic, so deep-layer Boolean erodes into soft gating. Reasoning tasks do need it, so shallow Boolean crystallizes.

Layer 0 is irreplaceable; deeper layers are redundant capacity

Causal ablations: ablating the Boolean blocks in layer 0 (the embedding-adjacent layer) collapses the model; ablating deep-layer Boolean blocks has almost no effect.

This is consistent with the finding above: genuine two-operand logic crystallizes only in layer 0; deeper layers are "redundant capacity"—allocated but not strongly exploited.

Legibility is local, not network-wide

Honestly, the paper admits: readable units are only a few percent of the network, concentrated in later layers. A fully Boolean FFN (no GELU stabilizer) does not train. Legibility is currently only free within the "partition"—the 75/25 mix is an engineering compromise.

What This Means

For interpretability

There are currently two paths: 1. Post-hoc probes (SAEs, probing): pry readable features out of existing models. 2. Architectural design: directly build legible-by-construction models.

NC-FFN belongs to the second path. Its core contribution is not "a bit better than GELU" (it actually ties), but that every unit is a named operation—intersection, set-difference, quantifier—readable without an SAE.

This drops the cost of interpretability from "train an SAE per model" to "read the model's built-in logical structure." If this path pans out, interpretability research infrastructure changes completely.

For the "logic vs. learning" debate

An old question: do neural networks "learn logic" or "learn statistical patterns"?

NC-FFN gives a precise answer: both, depending on what the task rewards. On reasoning-rewarding tasks the network uses explicit logic; on prediction-only tasks, logic erodes into statistical gating. Logic is not learned—it is pulled out by the task.

For self-forgetting

The self-forgetting quantifier is the paper's most elegant design. Traditional RNN forgetting gates are global (the LSTM forget gate); NC-FFN's forgetting is per-unit—each quantifier unit learns on its own how long to remember.

The result: different licensing constructions have different window lengths (comparatives vs. passives vs. NPIs), and each unit tunes to its own optimum. This is not design—it is emergence.

Honest Limitations

• Requires a GELU stabilizer: purely Boolean models do not train; 75/25 is an empirical ratio. Fully Boolean FFNs remain an open problem.
• Legibility is local: only a few percent of units are readable, concentrated in later layers.
• Scale unverified: tested only at 125M parameters. What would GPT-5-scale NC-FFNs look like? Unknown.
• No semantic gain: a slight LAMBADA lead, but overall perplexity ties. NC-FFN's advantage is legibility, not performance.

My Take

What excites me most is the finding that "logic is task-shaped."

We used to think of "interpretability" as a model property—this model is interpretable, that one is not. NC-FFN tells us interpretability is a task–model interaction property: the same architecture crystallizes logic on reasoning tasks and erodes into statistics on prediction tasks.

This means: if you want an interpretable model, changing the architecture is not enough—you must change the training objective. An NC-FFN trained only on next-token prediction will degenerate in its deeper layers into a GELU cousin. Only by adding reasoning-rewarding tasks does Boolean structure persist.

This is a directional hint for interpretability research: rather than chasing "inherently interpretable architectures," chase "training pipelines that reward interpretability." Architecture provides possibility; the task decides reality.

The second exciting point is the emergence of "self-forgetting quantifiers as grammatical-licensing detectors." Nobody told the model "comparatives are followed by than"—the model discovered the concept of a comparative on its own, learned on its own to "remember for 1.5 tokens," and tuned its memory window to exactly cover the licensor-to-licensee distance.

This is a clean case of structural emergence: the architecture provides the primitive of a "forgettable sequence operator," the training objective provides the pressure of "predict the next token," and grammatical-licensing detectors emerge from their combination—no human prior in between.

There is an isomorphism with "structure determines function" in biological evolution: DNA provides the amino acid sequence (architecture), the environment provides selection pressure (task), and protein function (the licensing detector) emerges. No designer—only a loop of constraints and selection.

---

Paper: Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors Author: Thomas Marshall arXiv: 2606.31845