YaCy.Uno Design Specification: A Decentralized P2P Search Engine Built on Uno Platform and .NET 9

Forum topic · ✨步子哥 · 2026-02-17

Summary

YaCy.Uno is a C#/.NET 9 reimplementation of the YaCy decentralized peer-to-peer search engine, built on Uno Platform 5.x for true cross-platform support across Windows, macOS, Linux, WebAssembly, iOS, and Android. The project is fully compatible with the YaCy P2P protocol (network name 'freeworld'), interoperating with Java and Go YaCy nodes, and uses SQLite FTS5 for full-text indexing, ASP.NET Core Kestrel for HTTP, and Material Design 3 for its UI. Key architecture components include a search engine with BM25-based ranking (BM25 relevance, title match, URL depth, inbound links, freshness, language), a polite multi-threaded web crawler with robots.txt compliance and incremental fetching, a DHT with 256 partitions and redundancy factor 3, and 30+ document parsers (HTML, PDF, Office, RSS, JSON) via AngleSharp, PdfPig, and Open XML SDK. Chinese full-text search is supported via jieba.NET segmentation. It offers multiple run modes (desktop app with embedded server, headless server, WASM, mobile, CLI via Spectre.Console, Docker), REST/SSE/SignalR APIs, peer types (virgin/junior/senior/principal), and ships under Apache License 2.0 with default ports 8900/8943.

YaCy.Uno is a design specification for a decentralized peer-to-peer search engine implemented in C# on .NET 9 using Uno Platform 5.x. It is fully compatible with the YaCy P2P protocol (network freeworld) and can interoperate with existing Java and Go YaCy nodes. It targets Windows, macOS, Linux, WebAssembly, iOS, and Android, with a Material Design 3 UI.

Key points

Identity and ports: Name YaCy.Uno, default ports 8900 (HTTP) / 8943 (HTTPS), network freeworld, YaCy P2P Protocol v1 compatible.
Comparison with other versions: Unlike Java YaCy (Solr + Kelondro, Jetty, port 8090) and YaCy-Go (SQLite FTS5, Fiber, port 8899), YaCy.Uno uses SQLite FTS5, ASP.NET Core Kestrel, and Uno Platform XAML, and uniquely supports mobile (iOS/Android), WASM, and a Spectre.Console TUI.

Architecture

Three frontends: Uno Platform XAML UI (Desktop/WASM/Mobile), ASP.NET Core Web API (REST/SSE/WebSocket/SignalR), and Spectre.Console TUI.
Application services: Search, Crawl, P2P/DHT, and Index services over a core engine.
Core engine: SQLite FTS5 full-text index, 30+ document parsers, and YaCy-protocol-compatible P2P layer; unified SQLite (EF Core + FTS5) storage.

Technology stack

| Component | Choice | |---|---| | Runtime | .NET 9 | | UI | Uno Platform 5.x + Uno.Material (MD3), CommunityToolkit.Mvvm | | Web | ASP.NET Core 9.0 (Kestrel), SignalR, SSE via IAsyncEnumerable | | Search | SQLite FTS5 (tokenize = 'unicode61 porter') | | Parsers | AngleSharp (HTML), UglyToad.PdfPig (PDF), Open XML SDK / NPOI (Office), System.ServiceModel.Syndication (RSS/Atom), MetadataExtractor (EXIF) | | Chinese tokenization | jieba.NET (ITextSegmenter / JiebaTextSegmenter; CJK text is segmented before writing to FTS) | | HTTP client | HttpClient + Polly (retry/circuit breaker) | | Logging / DI / Config | Serilog, Microsoft.Extensions.DI, Microsoft.Extensions.Configuration | | Testing | xUnit, NSubstitute, Mvc.Testing, BenchmarkDotNet |

Core modules

Search engine

FTS5 virtual table documents_fts indexes url_hash, url, title, description, content, host, language, content_type, alongside a documents metadata table. Query flow: QueryParser → FTS5 bm25 matching → local index plus parallel P2P remote search (DHT-selected peers) → result merging in RankingEngine → snippet generation with FTS5 highlight → SSE/SignalR streaming.

Ranking factors: BM25 relevance 40%, title match 20%, inbound links 15%, URL depth 10%, freshness 10%, language match 5%.

Crawler

Components: CrawlScheduler (priority queue + per-domain rate limiting), persistent UrlQueue, HttpLoader with Polly, RobotsParser, SitemapParser, ContentExtractor (AngleSharp), parser pipeline, and index submitter. Features: parallel workers via Channel<T>, polite crawling, configurable depth/type filters, incremental fetch (If-Modified-Since / ETag), real-time status push.

P2P network

Implements YaCy endpoints: /yacy/hello, /yacy/search, /yacy/seedlist, /yacy/transferRWI (JSON { word, docRefs }, pushed by RwiIndexTransferService), /yacy/transferURL, /yacy/crawlReceipt, /yacy/query, /yacy/profile, /yacy/idx.json, /yacy/message.

DHT: 256 partitions (2^8), redundancy factor 3 among senior peers, word-hash → peer-hash nearest-node selection, background index distribution. Peer types: virgin (new), junior (behind NAT), senior (public IP), principal (senior + seed-list service).

Document parsers

IDocumentParser interface exposes SupportedMimeTypes, Priority, and ParseAsync; parsers cover HTML/XHTML, PDF, DOCX/XLSX/PPTX, legacy DOC/XLS/PPT, RSS/Atom, XML/SVG, JSON/JSON-LD, plain text/CSV/Markdown, and image EXIF.

Storage

SQLite (EF Core + FTS5) holds documents, documents_fts, rwi_terms (RWI), seeds, crawl_queue, crawl_profiles, bookmarks, blacklist, and config tables; the filesystem stores the database file, logs, and exports under data/.

API design

REST endpoints mirror YaCy-Go: search (GET /api/search, /api/search/stream SSE, /api/suggest), crawl (POST /api/crawl, status/pause/resume/abort), peers, DHT stats, and admin (/api/stats, config read/update, index browse/export/import). Real-time communication uses SSE for streaming results and SignalR for crawl/P2P events, with WebSocket as a WASM-compatible fallback.

UI design

Material Design 3 with primary gradient #667eea → #764ba2, Segoe UI / Inter / Noto Sans CJK fonts, 12–16px card radii, elevation shadows, light/dark themes with system follow. Pages: Hero (centered logo + search box), Search results, Crawl management, Peers, DHT visualization, Stats dashboard, Config, Help. Desktop uses a sidebar navigation; mobile uses a bottom tab bar.

Deployment and operations

Run modes: Desktop (Uno window + embedded Kestrel), headless Server, WASM, Mobile, CLI (yacy search/crawl/status/peers/serve), Docker (multi-stage build on mcr.microsoft.com/dotnet/aspnet:9.0).
Requirements: .NET 9+, 256MB RAM minimum (1GB+ recommended), 100MB disk plus index growth, Windows 10+ / macOS 12+ / Linux (glibc 2.17+).
Security: Kestrel TLS, Basic Auth + JWT for admin, RSA 2048 node identity keys (YaCy-compatible), rate limiting, configurable CORS, input validation.
i18n: .resw resources, initial English (en) and Chinese (zh-CN).
License and process: Apache License 2.0, GitHub Flow with PR + code review, GitHub Actions CI/CD.

The full specification also contains the detailed project layout for YaCy.Uno.Core, YaCy.Uno.Server, YaCy.Uno.App, and YaCy.Uno.Cli, plus tests, docs, and Docker folders.