Closure
Triangle
Tightly coupled substitutes or complements. Does quantization preserve the closure, or collapse the three items into one region?
Research proposal · Generative recommendation
Preserving heterogeneous local topology through RQ-VAE quantization
Generative recommenders compress item semantics into discrete Semantic IDs. This proposal asks what that compression does to the item graph — and whether curvature can be allocated, rather than chosen once, to keep the local structures that matter.
01Starting observation
RQ-VAE is usually trained on item semantics and judged by reconstruction, codebook usage, or downstream accuracy. The item graph it leaves behind is rarely the object of study.
In preliminary experiments, a semantic-only RQ-VAE still recovered part of the real item–item behavior graph after quantization. The recovery was not uniform: triangle, star, and chain neighborhoods were preserved to different degrees, and their retention rates correlated differently with downstream recommendation performance.
That unevenness is the starting point. If one item can belong to a triangle, a star, and a chain at the same time, a single geometry and a uniform reconstruction objective may not preserve those structures equally.
Closure
Tightly coupled substitutes or complements. Does quantization preserve the closure, or collapse the three items into one region?
Concentration
A hub with many leaves. Once codes are shared, is the hub retained, its leaves, or neither?
Ordering
A graded or sequential relation. Does a discrete code preserve the order, or only proximity?
Working hypothesis
An item participates in several local topologies at once. A single Euclidean codebook and a uniform reconstruction objective cannot preserve them equally. Geometry should become an allocatable resource: hyperbolic where hierarchy dominates, spherical where closure or cycles dominate, Euclidean as the stable default.
02Questions
How does RQ-VAE quantization reshape triangle, star, and chain structure, and which of these changes track downstream recommendation utility?
Can Euclidean, hyperbolic, and spherical biases be assigned per item and per codebook level to match heterogeneous local topology?
Under a fixed Semantic ID budget, does selective structure preservation outperform uniform neighborhood reconstruction?
03Approach
Build a KNN graph on pre-quantization item embeddings. Extract graphlet signatures and neighborhood descriptors, then measure the same structures after quantization.
Learn a per-item, per-level mixture over Euclidean, hyperbolic, and spherical components. Hyperbolic capacity for hierarchy, spherical for closure and cycles, Euclidean as the default.
Not every edge deserves equal weight. Identify behavior-relevant local structure and apply structure-preserving constraints selectively instead of reconstructing the full neighborhood.
Compare structure retention before and after quantization on real behavior graphs, then test generative recommendation at a fixed codebook budget with geometry and constraint ablations.
Separates semantic reconstruction from the behavioral topology a Semantic ID actually keeps.
Moves beyond a single global geometry by assigning geometric bias per item and per codebook level.
Tests whether selective structural constraints buy recommendation quality at equal discrete capacity.
04Plan
Quantify graphlet retention across RQ-VAE checkpoints and domains; correlate retention with downstream performance.
Prototype mixed-curvature codebooks and per-item, per-level allocation; track codebook utilization and stability.
Add selective structure-preserving objectives; ablate graphlet types and constraint strength.
Fixed-budget comparisons on real behavior graphs, multiple backbones, and a written analysis.
How do we define utility-relevant topology without leaking downstream labels into training?
Can mixed-curvature codebooks remain well utilized when geometry is reallocated across items?
Does graphlet retention transfer across domains and generative recommender backbones?
Working research proposal. The observation is preliminary and directional; no final benchmark result is claimed on this page. The sequence is designed so that each phase can falsify the next.
Contact
If you work on generative recommendation, hyperbolic representation learning, or discrete item representations — or if you see a failure mode in this plan — I would like to hear it.