SciAtlas: When 43 Million Papers Are Woven into a 'Cognitive Map' of Science
*Structured summary of the original Chinese forum post.*
Key points
- Motivation: Researchers struggle not to find papers but to find *connections*. Keyword matching misses referential relations (e.g., "protein structure prediction" ↔ AlphaFold); vector retrieval captures similarity but not citation-based influence. SciAtlas replaces fuzzy semantic similarity with structured knowledge topology and deterministic graph propagation instead of costly, hallucination-prone agentic LLM reasoning.
- Scale and schema: 43.3M papers (cleaned from OpenAlex's 480M), 109.7M authors, 3.76M keywords, 120K institutions, 280K sources, and a four-level topic hierarchy (4,520 topics → 4 domains). Twelve relation types span semantic (CITES: 214M, RELATED_TO), conceptual (HAS_KEYWORD, COOCCUR), directional, social (COAUTHOR: 2.06B — the densest), and publication layers.
- Four-layer cognitive architecture: semantic (what a paper says), conceptual (shared core ideas via keyword co-occurrence), directional (four-level discipline hierarchy), social (who and where). Different downstream tasks activate different subgraphs.
- English-only filtering, abstract-length filtering, and PDF-availability filtering (traceability to sources).
- Authors deliberately not deduplicated — a pragmatic engineering tradeoff given widespread name ambiguity.
- LLM keyword extraction: Qwen3-30B-A3B-Instruct-2507 extracts 3–8 reusable, general keywords per abstract (keeping "protein structure prediction", filtering bespoke framework names), with importance scores modulating HAS_KEYWORD edge weights; co-occurrence within a paper creates weighted COOCCUR edges.
- Embeddings precomputed with bge-large-en-v1.5; the graph is served from Neo4j (Cypher). Updates: daily OpenAlex API increments, bi-monthly bulk change files, GROBID for PDF metadata recovery.
- Seed merging: exact title hits gain +0.35, fuzzy +0.10; seeds are scaled by citation importance:
w_p^seed = s_p^pre × (1 + γ × imp(p)). - Graph propagation: 2-hop subgraph constraint (≤500 nodes per entity type per hop), Random Walk with Restart over edges weighted by type — CITES 1.00, RELATED_TO 0.90, AUTHORED 0.80, COAUTHOR/COOCCUR 0.60. Update rule:
r_v^(t+1) = α·s_v + (1-α)·Σ_u r_u^(t)·P(v|u); convergence at L1 difference < 10⁻⁶ or 50 iterations. - Final ranking:
s_p^final = min(1, 0.35·s̃_pre + 0.45·s̃_graph·g_p + 0.20·imp_final(p)), with a graph-support gating floor g_p ≥ 0.25 so graph scores don't fully override semantic relevance. Notably, graph topology outweighs initial semantic similarity by design. - Transparency & speed: every result ships with per-path score decomposition and explanations; full retrieval runs in "significantly under 2 minutes".
- Literature reviews with tunable venue/author/institution weights.
- Idea grounding: retrieve related papers → segment full texts → LLM-refined queries → passage-level retrieval → similarity/difference analysis to detect prior art, supporting evidence, or genuine novelty.
- Trend prediction, collaboration mining, academic trajectory exploration.
- Strengths: largest open academic KG of its kind; pragmatic quality/coverage tradeoffs; effective low-cost keyword extraction.
- Unresolved issues: the quantitative evaluation section (baselines, ablations, NDCG) was unavailable in the source; English-only filtering risks systematic bias in medicine and social sciences; undeduplicated authors cause ambiguous queries (especially for names like "Wang Wei"); the 50-iteration random walk remains opaque without manual graph traversal; keyword quality is bottlenecked by the LLM's domain understanding.
- SciAtlas technical report (arXiv:2605.22878)
- Project page: http://scigraph.openkg.cn/
- OpenAlex, Neo4j, bge-large-en-v1.5, Qwen3-30B-A3B-Instruct-2507
Data construction decisions
Neuro-symbolic three-path retrieval
1. Keyword matching: LLM-extracted query keywords; exact match (full score) plus vector match (threshold 0.7, top-3). 2. Semantic matching: query embedded and searched over title (top-60, weight 0.4) and abstract (top-60, weight 0.6) channels, reranked with bge-reranker-large, top-15 each. 3. Title matching: for reference-paper queries — GROBID title extraction, LLM confidence filtering (top-10), exact (1.0) or fuzzy match (0.65×LCS + 0.35×Jaccard, threshold 0.88).
Downstream applications
Strengths and open tensions
Strategic significance
SciAtlas marks a shift from metadata management to *cognitive topology construction*. Paired with parametric, opaque LLM assistants (e.g., Elicit, Consensus), it supplies explicit, traceable, structured grounding — LLMs handle high-level reasoning, SciAtlas handles fact retrieval and relation verification. It also enables cross-disciplinary connection discovery (e.g., GNN papers applied to molecular representation via COOCCUR and citation chains). It is not a path to fully automated science — hypothesis generation, experiment design, and inference still need humans — but it gives researchers a navigable map in the fog of information explosion.