editorinchief@ijsreat.com +91 9840521421 Scholarly Open Access, Peer-reviewed, Refereed Journal e-ISSN 2583-1240 Since 2021
  1. Home
  2. Archives
  3. 2026
  4. Vol. 6 Issue 5
  5. Article
Original Article

SUSHILA: Scalable Upstream Synthesis for Hybrid Inference in Large-model Acceleration

  • Paper ID 388
  • Volume 6, Issue 5
  • Pages 97-109
  • Published Online: September-October 2026
Open Access Peer Reviewed Crossref DOI

Authors

  • Milind K. Patil1

1 Syncaissa Systems Inc, USA.

Article Information

Published Online
September-October 2026
Pages
97-109
Article Type
Original Article
Check for updates
check update

Abstract

View PDF

This paper starts from existing open-source weights, stacks new inference research on top of the optimization techniques already in use, and produces faster inference of the same model—without changing a single weight. Sushila.cpp is a free, local inference engine built on top of llama.cpp, the open-source engine behind Ollama, with SGLang on NVIDIA servers. Large language models are slow and expensive to run mainly because every generated token reads the model’s weights (for mixture-of-experts models, its active experts) from memory again. SUSHILA (Scalable Upstream Synthesis for Hybrid Inference in Large-model Acceleration) tests the published speed-up methods on each model once, right after its release, keeps those that make it faster at unchanged quality, and adds methods of our own. One of them, a landscape precomputed once per model, previews every next-token score cheaply, then computes only the best candidates exactly, reading 13–15% of the output layer with ≥99.75% agreement on the top token (7–70B models). Over eight GPU chat and coding models, Sushila.cpp achieves a geometric-mean speedup of 2.35× over Ollama (up to 4.05× on Llama-3.3-70B, on 160 benchmark prompts it never saw, with GSM8K accuracy unchanged within its error bars), measured on an NVIDIA A100 with Sushila.cpp’s SGLang serving path. Most of this gain comes from existing methods; our precomputed draft heads add 1–11% where a good head was already published and are most of the gain where none was: 3.64× on DeepSeek-R1-Distill-Llama-70B and 3.56× on Kimi-Dev-72B. Speed tests were done on one NVIDIA A100 80GB for chat and coding models (media mostly on one RTX 4090; engine-level tests also on CPUs), one user at a time; results may vary on other computers. Every result has its scripts and raw outputs in our public repository, and each chat and coding comparison reruns with one command.

APC from ₹1200 + 18% GST

Peer-reviewed open access₹1200 Indian (₹1400 with DOI)$80 USD internationalpaid after acceptance Peer-reviewed open access₹1200 Indian$80 USD international ₹1200 Indian · $80 intl

  • CrossRef DOI
  • ISSN 2583-1240
  • Open Access
  • Peer Reviewed
  • Plagiarism Checked
  • Google Scholar
Submit Paper