PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentation

Forum topic · 小凯 · 2026-03-19

Summary

PCA-Seg (arXiv:2503.13840) is a 2025 computer vision paper addressing open-vocabulary semantic and part segmentation (OSPS). Existing methods extract image-text alignment cues from cost volumes using a serial structure of spatial and class aggregations, which causes knowledge interference between class-level semantics and spatial context. The authors propose a parallel cost aggregation paradigm (PCA-Seg) to capture richer vision-language alignment information. Key components include an expert-driven perceptual learning (EPL) module with a multi-expert parser that extracts complementary features from multiple perspectives, and a coefficient mapper that adaptively learns pixel-specific weights to integrate complementary knowledge into a unified feature embedding. A feature orthogonalization decoupling (FOD) strategy further reduces redundancy between semantic and contextual streams. Experiments across eight benchmarks show state-of-the-art OSPS performance, with each parallel block adding only 0.35M parameters.

Paper Overview

Field: Computer Vision (CV) Authors: Jianjian Yin, Tao Chen, Yi Chen Published: 2025-03-18 arXiv: 2503.13840

Problem

Recent advances in vision-language models (VLMs) have drawn substantial attention to open-vocabulary semantic and part segmentation (OSPS). However, existing methods extract image-text alignment cues from cost volumes through a serial structure of spatial and class aggregations, leading to knowledge interference between class-level semantics and spatial context.

Proposed Approach: PCA-Seg

The paper introduces a simple yet effective parallel cost aggregation (PCA-Seg) paradigm that enables the model to capture richer vision-language alignment information from cost volumes.

Key Components

Expert-driven perceptual learning (EPL) module: Efficiently integrates semantic and contextual streams. It incorporates a multi-expert parser to extract complementary features from multiple perspectives.
Coefficient mapper: Adaptively learns pixel-specific weights for each feature, integrating complementary knowledge into a unified and robust feature embedding.
Feature orthogonalization decoupling (FOD) strategy: Mitigates redundancy between semantic and contextual streams, allowing the EPL module to learn diverse knowledge from orthogonalized features.

Results

Extensive experiments on eight benchmarks demonstrate that each parallel block in PCA-Seg adds only 0.35M parameters while achieving state-of-the-art OSPS performance.