Federated Relation Schema Induction: Learning Shared Type Signatures from Private Text Silos

Ontologies, the hand-built catalogues that say which kinds of things exist in a domain and how they may relate, are still mostly written by people. A typical rule is "a Patient can undergo a Surgery." Automating their construction has been a goal for two decades. The best automatic methods, however, need a lot of pooled text. In the domains that most need fresh ontologies, such as hospitals, insurers, and manufacturers, that text sits in separate institutions that are not allowed to share it. Federated learning addresses the pooling problem in general: clients train locally and send only model updates to a coordinating server. Yet federated work on knowledge graphs has only learned vectors for facts whose schema was fixed in advance. We propose to learn the schema itself, meaning the relation type signatures, federatedly.

Concretely, we take relation schema induction by coupled tensor factorization, a method that decomposes a table of (subject, object, relation) triples into interpretable entity categories. We split its parameters so that the parts describing which categories a relation connects are shared across clients, while the parts describing which entity belongs to which category never leave the client. The output is a small set of candidate signatures that a human ontology engineer reviews. The research question is whether this protocol recovers schemas close to what pooling the text would give, whether it beats each site working alone, and whether it beats the cheapest privacy-friendly shortcut: each site shipping aggregate type-level counts once.

Core Claim

Relation signatures are low-dimensional, relatively non-sensitive objects. They are also statistically hungry: they can only be estimated reliably when many entities from many sources are combined. We claim that iterating a federated factorization, which shares only category-level core matrices and hypernym-to-category maps and keeps entity-to-category memberships local, recovers most of the accuracy of centralized schema induction. It should clearly beat site-local induction when each silo holds a small, skewed slice of triples.

We further claim a structural advantage over one-shot count sharing. Local entity memberships let each client pass evidence between relations that share entities. That is the mechanism by which factorization overcomes sparsity. Type-level counts throw this signal away whenever an entity has no pattern-derived hypernym such as "cities such as Oslo."

Our second claim concerns disagreement. Sites genuinely disagree about schema: one hospital files OncologyPatient under inpatients, another under an outpatient protocol. For this case, a personalized version with a shared core plus a sparse per-client correction yields signatures that are both more accurate locally and still mappable to a common schema. Plain averaging would blur them.

Where This Sits

Two lines of prior work supply the method. Relation Schema Induction using Tensor Factorization with Side Information (SICTF) factorizes an OpenIE triple tensor jointly with a noun-phrase-to-hypernym matrix mined by Hearst patterns and a relation-similarity matrix from word2vec. It uses non-negativity so that the factors read as category memberships. SICTF reports 0.94 schema accuracy on StackOverflow against 0.8 for KB-LDA, at about 14 times the speed. We reuse its objective unchanged and only decide which factors travel.

Personalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph (PFedEG) builds a server-side graph of client affinity and sends each client an aggregate of its most similar neighbours, with a proximal term that keeps local embeddings near that personalized target. We borrow this personalization mechanism, but apply it to schema-level core matrices rather than instance embeddings.

PFedEG also reports that under semantic disparity, personalized federated training beat centralized "Collective" training. This finding motivates our second claim: pooling is not automatically the ceiling.

Several works constrain the design. Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation shows that communication cost dominates in federated embedding and cuts it by half through distillation. It assumes a fixed relation vocabulary, which is precisely what we are trying to induce. FedMKGC likewise assumes a predefined schema. Our shared payload is category-by-category-by-relation, which is independent of entity count, so communication is small by construction.

Collective ontology alignment (Helix) is the closest collaborative-ontology system. It grows a shared schema through users' manual linking actions on a centralized platform, with no learning and no privacy protection. We keep its "pay-as-you-go" human review, but place it after a privacy-preserving learned proposal instead of making it the whole mechanism.

The long-standing knowledge acquisition bottleneck described in Learning Ontologies for the Semantic Web is why the manual step matters. SICTF itself notes that humans must still name induced categories. We therefore measure engineer effort as an outcome, not an afterthought.

Research Program

Each client $k$ runs OpenIE on its private documents to get a sparse tensor $X^{(k)}$ over its own entities and a shared relation vocabulary. It also builds a local hypernym matrix $W^{(k)}$ from Hearst patterns. The factorization follows SICTF:

$$ \min_{A_k \ge 0,\; R \ge 0,\; B \ge 0} \sum_k \Big( \sum_r \| X^{(k)}_r - A_k R_r A_k^\top \|^2 + \lambda \| W^{(k)} - A_k B \|^2 \Big) + \mu \sum_{r,r'} S_{rr'} \| R_r - R_{r'} \|^2 . $$

The symbols are as follows. $A_k$ maps client $k$'s entities to $c$ latent categories and stays local. $R_r$ is the $c \times c$ core matrix for relation $r$; its large entries are the induced signature. $B$ maps hypernym words to categories. $S$ is the public relation-similarity matrix, and $\lambda, \mu$ weight the side information.

Clients alternate local updates of $A_k$ with gradient steps on $R$ and $B$. The server aggregates only these steps, through secure aggregation, optionally with Gaussian noise for differential privacy. Because $B$ is indexed by a shared hypernym vocabulary, categories are anchored consistently across sites. This addresses the permutation ambiguity that would otherwise make averaging factor matrices meaningless. We will test this anchoring directly.

The personalized variant replaces $R_r$ by $R_r + D^{(k)}_r$ with an L1 penalty on the per-client deviation $D^{(k)}_r$. It aggregates the shared part over a PFedEG-style affinity graph computed from the similarity of clients' $R$ updates. Sparse deviations remain interpretable: each nonzero entry is a named disagreement that an engineer can accept into the shared schema or keep local.

A final server step checks the shared signatures for simple logical conflicts before human review. Examples are a relation whose domain is inferred as two categories the seed ontology declares disjoint, or cyclic category nesting. This is in the spirit of the repair step in BERTMap. An engineer then labels the top signatures, and we record accept and reject counts and time.

We will also run a small leakage audit. It trains a membership-inference probe on the shared updates to test whether specific private triples can be detected, with and without noise.

Baselines

The naive reference point is site-local SICTF. Each client factorizes only its own tensor, and the resulting schemas are merged by matching categories on their hypernym profiles. It uses the same rank, the same side information, and the same hyperparameter grid chosen on a held-out client. We must beat it by a clear margin on the smallest clients, where sparsity should hurt most.

The upper reference is centralized SICTF on the pooled triples, run with identical settings. It is the strongest existing method for this exact task; on StackOverflow it reports 0.94 accuracy per the SICTF paper. We count federated SICTF as successful if it reaches at least 90 percent of centralized accuracy on the same split. We treat the personalized variant as a real advance if it matches or exceeds centralized accuracy on client-local evaluation under schema disagreement, which would mirror the PFedEG finding at the schema level.

The strongest cheap competitor is one-shot count sharing, a federated-analytics approach in the sense of A Survey on Federated Analytics. Each client sends a noised tensor of counts over (subject hypernym, object hypernym, relation), and the server factorizes this aggregate once. It receives the same privacy noise level and more total bandwidth than our iterative protocol, so the comparison cannot be won on budget. We beat it only if we are more accurate at equal privacy, with the gap growing as Hearst-pattern coverage of entities falls.

The ablation that isolates our key ingredient is federated SICTF without the shared hypernym anchor $B$. Here clients share only $R$, and the server must match categories after the fact. If this ablation performs as well as the full method, the anchoring claim is wrong.

We also include an LLM baseline in which each site runs a local open model through the modular OLAF pipeline to propose signatures, and the proposals are merged by label matching. No reported result was found on Lacuna for OLAF on MEDLINE, StackOverflow, or DBpedia-WebNLG, so we will report it ourselves.

Why not just…?

Why not just de-identify the text and run centralized induction? In clinical settings, de-identified text often still cannot legally leave the site, as Opportunities and obstacles for deep learning in biology and medicine discusses. Scrubbing names and qualifiers also removes the very noun phrases that Hearst patterns and OpenIE depend on. Where pooling is allowed, the centralized baseline is our ceiling, not a rival. PFedEG's result that personalized federated training beat collective training suggests that, under disagreement, pooling may not even be the ceiling. Our disagreement experiments test this at the schema level.

Why not just induce schemas locally and align them afterwards with BERTMap? Alignment can only connect categories that each site actually discovered. SICTF's accuracy comes from factorization pooling evidence across thousands of entities, and a small silo's factorization yields noisy, incomplete categories that no aligner can repair. The site-local baseline, merged with a BERTMap-style matcher, measures this directly. We expect the gap to shrink as client size grows; the size sweep will show where the crossover is. If local-plus-align matches us at realistic silo sizes, the federated protocol is not worth its complexity, and we will say so.

Why not just ship aggregate type-level counts once? This is the most serious threat. It is simpler, needs one round, and is easy to privatize. Its weakness is that it discards every entity lacking a pattern-derived hypernym, along with the cross-relation evidence carried by shared entities. We cannot argue that away; the count-sharing baseline, together with a sweep over hypernym coverage, settles it. If count sharing wins at high coverage and loses at low coverage, the honest recommendation is to use counts when coverage is high.

Why not just ask a local LLM to write the schema? An LLM needs no pooled corpus, which directly undercuts our premise. The file's evidence is mixed. On the Pertinence of LLMs for Ontology Learning found that Mistral 7B struggled with sub-class structure and that a modular pipeline extracted 77 object properties where the reference Pizza ontology has 8, at roughly six times the energy of end-to-end prompting. TRACE-KG, by contrast, reaches 97 to 99 percent compatible concept coverage and 84.8 percent compatible relation coverage on DBpedia-WebNLG, though with an LLM pipeline that a hospital could not run on private text through an external service. We run the local-LLM baseline and also feed LLM-extracted triples into our factorization. This tests whether the two approaches are complementary rather than competing.

Proposed Experiments

Four data sources ground the study. MEDLINE (about 50,000 abstracts, 2,499 triples) and StackOverflow (37,439 triples) come from SICTF. We partition them into 3, 5, and 10 clients, both uniformly and by topic skew, with shrinking client sizes. DBpedia-WebNLG's 19 domains, used by TRACE-KG, give a natural federation: each domain is a client, or domains are mixed with controlled overlap. Its ontology-held-out protocol lets us score induced signatures against a reference introduced only after induction, using compatible concept and relation coverage. For category hierarchy quality, we map induced categories onto the hand-built Tourism (293 concepts) and Finance (1,223 concepts) reference ontologies. FCA on centralized corpora reached F-measures of 44.69 and 38.85 there, per Learning Concept Hierarchies from Text Corpora using Formal Concept Analysis. These figures give a centralized anchor, though the task is not identical.

To create schema disagreement, we perturb the gold category assignments of a subset of clients, for example moving one subclass under a different parent, and score both global and per-client accuracy.

Metrics are SICTF-style signature accuracy judged against gold signatures, compatible coverage, number of engineer-accepted signatures per hour of review, bytes communicated per round, and membership-inference advantage at each noise level. Signatures are judged by two annotators, and we report agreement.

Decisive Checks

Three results would decide the proposal. First, federated SICTF must reach at least 90 percent of centralized accuracy on MEDLINE and StackOverflow at 10 clients, and beat site-local SICTF by a wide margin at the smallest client sizes. Second, it must beat one-shot count sharing at equal privacy noise, with the advantage increasing as hypernym coverage drops. If it does not, the iterative protocol is unjustified. Third, under injected disagreement, the personalized variant must improve per-client accuracy over plain federated averaging without lowering shared-schema coverage on DBpedia-WebNLG, and its sparse deviations must correspond to the injected disagreements more often than chance.

Failure on the first check means schemas cannot be learned this way. Failure on the second means simpler federated analytics suffices. Failure on the third means personalization at the schema level adds nothing beyond what PFedEG already showed for embeddings.