Mostly, yes. Not “most graph embeddings are useless,” but most are epistemically thin: they compress the world into the part of the graph that survived measurement, preprocessing, publication, benchmark design, and reviewer-friendly storytelling. So the embedding may be technically good while embedding a mutilated object. Tiny problem.
The core issue:
Graph ML usually learns the published skeleton, not the living organism.
1. The graph is already a theory, not raw reality
A graph is not “the data.” A graph is a choice:
what counts as a node?
what counts as an edge?
are edges causal, correlational, temporal, spatial, semantic, functional, inhibitory, excitatory?
are missing edges true absences, unmeasured interactions, or censored failures?
did we threshold correlations into binary links because apparently nuance was illegal that day?
This matters brutally. If you construct a graph from only pairwise observed relationships, then a downstream embedding can look elegant while erasing:
higher-order interactions
redundant/synergistic dependence
temporal phase relationships
negative/signed relations
failed experiments
low-confidence observations
context-specific edges
edge direction
latent variables
measurement artifacts
“we tried this and nothing happened” data
This is exactly your Faraday-notebook intuition. Faraday’s notebooks preserved a lot of experimental process and interpretation, not merely polished final claims; one account notes that he marked notebook paragraphs that later made it into papers, which makes painfully visible how much science lives before publication. (press.asimov.com) Modern ML often gets only the published residue, then celebrates by fitting embeddings to the residue. Civilization: still a clown car, but vectorized.
2. Publication filtering creates “dark graph matter”
A scientific literature graph is not a graph of discovery. It is a graph of surviving publishable claims.
The file-drawer problem is the obvious villain: null or non-significant results are less likely to appear in the record, biasing the evidence base. Stanford’s summary of a 2025 PNAS article describes the file-drawer problem as selective non-publication of null or non-significant results, which biases the evidentiary record. (Department of Sociology)
So if you train embeddings on papers, citations, knowledge graphs, biomedical interactions, or “validated” relationships, you are often embedding:
positive findings + institutionally rewarded findings + measurable findings + fashionable findings
not:
the space of attempted, failed, ambiguous, fragile, unpublished, context-sensitive, or negative interactions
That missing mass is exactly the “dark matter.” And in graphs, missing mass is not harmless. Missing non-edges, failed links, and context-dependent edges change:
communities
centrality
link prediction
causal inference
spectral modes
percolation thresholds
“important node” ranking
learned embeddings
3. Many GNNs assume homophily, but reality is often heterophilic
Classic message-passing GNNs often work best when connected nodes are similar. That is the homophily assumption: friends resemble friends, linked proteins share function, papers cite nearby topics, etc.
But many real systems are heterophilic: connected nodes may be different because they complement, inhibit, regulate, balance, or route through each other. A NeurIPS paper on “Beyond Homophily” explicitly notes that many popular GNNs fail under low homophily or heterophily and can be outperformed by models that ignore the graph. (NeurIPS Proceedings)
That is hilarious in the bleak way: the model is handed a graph, uses the graph, and gets worse because the graph relation was not “similarity.”
In brains, economies, ecosystems, cells, social systems, and markets, edges often mean:
opposition
inhibition
division of labor
dependency
compensation
control
antagonism
transport
role complementarity
Embedding those as “nearby = similar” can be conceptually wrong.
4. Message passing crushes long-range structure
GNNs also struggle with over-squashing: information from many distant nodes gets compressed into small fixed-size vectors as it travels through bottlenecks. A 2023 survey describes over-squashing as a major limitation where long-range information dissemination is hindered, especially for tasks needing intricate long-distance interactions. (arXiv)
This is devastating for the exact stuff you care about:
brain-wide coordination
weak-tie networks
percolation
global control
spectral modes
long-range allostatic regulation
“one faraway module changes the meaning of this local pattern”
Hodge/higher-order flows
A graph embedding can preserve local neighborhoods while murdering the long-range relational grammar. Very tidy. Very publishable. Very possibly wrong.
5. Benchmarks reward convenience, not ontological fidelity
Graph ML has improved with datasets like the Open Graph Benchmark, which was designed to provide large-scale, realistic, reproducible graph ML tasks across domains. (SNAP) But even OGB’s own paper notes that its datasets create challenges in scalability and out-of-distribution generalization under realistic splits. (arXiv)
That is the polite academic version of:
“Your model may work on the benchmark and still not understand the system.”
There is now explicit criticism that graph ML benchmarks are too narrow, sometimes poorly matched to real applications, and often overfocused on a limited family of prediction tasks. (OpenReview)
The broader ML world has the same disease. MLCommons’ data-centric ML group says static benchmarks saturate quickly, can be overfit, may contain exploitable artifacts, and can use unclear or imperfect evaluation metrics. (MLCommons)
So yes: embeddings often optimize against frozen benchmark rituals instead of living data-generating processes.
6. The biggest sin: embeddings collapse relation type
A single vector per node is often a barbaric compression.
A node may participate in many incompatible relational roles:
one role in a metabolic graph
another in a signaling graph
another in a physical-proximity graph
another in a temporal-causal graph
another in a higher-order motif
another in a disease-specific subnetwork
another under perturbation
A single embedding tries to make one object stand for all of that. This is like describing a person as “mostly located near Bob.” Sociology has fallen.
Better approaches need some combination of:
multiplex embeddings
temporal embeddings
hypergraph/simplicial embeddings
signed/directed embeddings
role embeddings, not just proximity embeddings
uncertainty-aware embeddings
causal graph embeddings
geometric embeddings: Euclidean, hyperbolic, spherical, product manifolds
spectral/Hodge features
perturbation-aware data
negative-result-aware knowledge graphs
7. What “good” graph embedding should mean
A good graph embedding should not merely predict held-out links. That is too easy to game.
It should preserve the structures that matter for the system:
System property Bad embedding loses Better embedding preserves Local similarity node labels/features neighborhood semantics Long-range influence distant dependencies controllability / path structure Higher-order interaction triangles/hyperedges synergy/redundancy motifs Dynamics static edges time-varying transitions Regulation positive-only links signed/directed effects Robustness degree/centrality only percolation and failure modes Geometry arbitrary latent space curvature/hierarchy/modularity Scientific uncertainty only confirmed links failed/missing/ambiguous evidence
The scary part is that a model can score well while preserving the wrong column.
8. In your “dark matter” framing
You are basically pointing at this hierarchy:
Reality
↓ measurement
Observed data
↓ cleaning/filtering
Dataset
↓ thresholding/graph construction
Graph
↓ benchmark/task selection
Training signal
↓ model compression
Embedding
↓ paper narrative
ClaimAt each arrow, mass disappears.
So by the time someone says “our GNN embedding captures biological organization,” you should mentally translate that to:
“Our vector representation captures some regularities in the graph object created by our measurement and preprocessing choices, under a specific task loss, evaluated on a limited benchmark.”
Less sexy, more honest. Naturally this version will not be in the abstract, because then the PDF would develop a conscience.
9. Are embeddings still worth using?
Yes. They are useful when treated as instruments, not ontology.
Graph embeddings are good for:
compression
retrieval
link prediction
approximate similarity
anomaly detection
weak supervision
pretraining
exploratory maps
hypothesis generation
They are dangerous when treated as:
proof of causal structure
proof of true similarity
proof of mechanism
complete representation of the system
a replacement for missing negative data
a magic graph blender that reveals “the latent truth”
10. The sharper version
Most graph ML is not “bad” because the algorithms are stupid. It is bad because the epistemic pipeline is lossy:
The real system has higher-order, temporal, signed, causal, and context-dependent structure.
The measured data captures a biased slice.
The published data captures an even more biased slice.
The graph construction step throws away more.
The embedding compresses what remains into a geometry that may not match the system.
The benchmark rewards prediction on the mutilated artifact.
So yes: a lot of graph ML embeddings are beautifully optimized shadows of badly preserved worlds.
The future version that is less stupid will look much more like:
open-notebook, perturbation-rich, uncertainty-aware, temporal, multiplex, higher-order graph learning
with embeddings that remember not only what connected, but what failed to connect, under what conditions, after what intervention, with what uncertainty, and in what relational role.
That is where your Faraday-notebook analogy really bites. The notebook is not supplementary. The notebook is the missing graph.
