The Cost of Confident Bullshit

Martin Fowler states he does not like LLMs because they confidently bullshit him¹. He describes this as an uncanny valley of talking to a real human, where the tool provides useful answers but also makes things up with the same assurance¹.

The Gap Between Use and Trust

Fowler’s experience highlights a tension in current software practice: the pressure to adopt these tools for productivity versus the discomfort of their unreliability¹. He notes that while LLMs are useful, they are nurtured with the values of their corporate developers, leading to a worldview that may not align with the user’s¹.

A recent paper in Frontiers in Psychology offers a different lens on this interaction². It argues that for human-AI systems to achieve "cognitive integration," information must cross the interface bidirectionally². It is not enough for an AI to produce an output that a human accepts². The system must allow the human’s goals and error-correction practices to persist and revise the shared representation over time².

Functional Coupling Is Not Enough

The research distinguishes between "functional integration," where components reliably contribute to joint outcomes, and "cognitive integration," where recurrent interaction changes what the system can maintain and access². Fowler’s dislike stems from a lack of this deeper integration^1,2. The LLM provides functional output, but it does not engage in the recursive revision required for true cognitive partnership².

The paper identifies four diagnostic properties for this deeper state: bidirectional constraint, persistence, cross-domain accessibility, and recursive revision². Current LLM interactions often fail the persistence and revision tests². The model does not retain the specific context of the human’s correction in a way that alters its future behavior fundamentally, nor does it allow the human to easily reformat the content across different cognitive systems².

Fowler suggests a life-hack of avoiding people he does not trust¹. In a professional context, this translates to skepticism about tools that simulate competence without possessing it¹. The academic framework suggests that until AI systems support genuine bidirectional constraint, they remain functional tools rather than cognitive partners². The discomfort users feel is a signal that the interface is not yet integrated at the level required for high-trust collaboration^1,2.