A context-aware framework that dynamically balances privacy and inference quality through semantic sketch collaboration between cloud LLMs and edge SLMs.
1University of Pennsylvania 2University of Hong Kong 3Jinan University
* Equal contribution † Corresponding author
LLMs excel at understanding natural language, but deploying them raises a fundamental tension between privacy protection and inference quality.
User prompts in sensitive domains (medical, financial) contain personal information. Sending full prompts to cloud LLMs exposes private data to potential breaches.
Running large models locally preserves privacy but is limited by compute, memory, and energy. Edge SLMs cannot match cloud LLM quality on complex queries.
Existing approaches apply uniform privacy noise to all tokens regardless of sensitivity, degrading utility unnecessarily for benign content.
PRISM executes in four stages, combining entity-level sensitivity profiling with adaptive privacy protection and semantic sketch collaboration.
The edge device analyzes the user prompt using NER (Named Entity Recognition) to extract entities and compute a category-weighted risk score. Each entity type (PERSON, LOCATION, DATE, etc.) carries a predefined sensitivity weight.
An entropy-regularized neural router maps sensitivity indicators to a probability distribution over three execution modes. Unlike hard thresholds, soft gating produces nuanced, context-aware routing decisions.
For collaborative paths, PRISM applies entity-level local differential privacy with two layers: (1) category obfuscation and (2) value perturbation. Privacy budget is adaptively allocated based on each entity's sensitivity weight.
The cloud LLM receives the perturbed prompt and generates a semantic sketch — a structured outline preserving task structure while hiding sensitive details. The edge SLM then refines this sketch using the original unperturbed context to produce the final response.
PRISM dynamically routes each query to the optimal execution path based on its privacy risk profile.
Low-risk queries like general knowledge questions are sent directly to the cloud LLM for maximum quality.
Moderately sensitive queries use adaptive LDP + semantic sketch collaboration for balanced privacy and quality.
Highly sensitive queries (e.g., medical records) are processed entirely on-device for maximum privacy protection.
Watch how PRISM processes a privacy-sensitive tourism query through the collaborative pipeline.
PRISM is validated on a synthetic, context-rich instruction dataset spanning diverse privacy scenarios.
Patient demographics, symptoms, diagnosis history, treatment plans
Travel plans, budgets, destinations, companion details, itineraries
Transaction histories, card disputes, account information, balances
Non-sensitive factual queries, benchmarking cloud-only quality
PRISM consistently achieves superior privacy-utility trade-offs across all evaluation scenarios.
If you find PRISM useful in your research, please consider citing our paper.
@inproceedings{zhan2026prism,
title = {PRISM: Privacy-Aware Routing for Adaptive Cloud-Edge
LLM Inference via Semantic Sketch Collaboration},
author = {Zhan, Junfei and Shen, Haoxun and Lin, Zheng and He, Tengjiao},
booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
volume = {40},
number = {33},
pages = {28150--28158},
year = {2026},
doi = {10.1609/aaai.v40i33.40041},
url = {https://ojs.aaai.org/index.php/AAAI/article/view/40041}
}