Serverless Architecture Patterns for Scalable Machine Learning Inference Pipelines
DOI:
https://doi.org/10.66838/sauc.6526Keywords:
Serverless Computing, Machine Learning Inference, Architecture Patterns, Cold Start Mitigation, Dynamic Batching, Event-Driven Pipelines, LoRA Adapter Serving, Cloud Cost Optimization.Abstract
Serverless Architecture Patterns for Scalable Machine Learning Inference PipelinesServing machine learning models in production is an exercise in handling uncertainty. Traffic arrives in bursts, models differ enormously in size and cost, and demand for a given model may be near zero for hours and then jump by an order of magnitude within seconds. Serverless computing, in which a cloud provider runs short-lived functions and managed services on demand and bills only for what is used, promises to match capacity to such demand automatically. Applying it to inference is not straightforward, however, because model loading causes cold starts, functions are stateless and limited in memory and duration, and accelerators are scarce. This paper presents a catalog of eight serverless architecture patterns for machine learning inference pipelines, together with guidance on when each should be used.....
Downloads
Global Statistics ℹ️
|
64
Views
|
15
Downloads
|
|
79
Total
|
|
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Authors retain copyright and transfer to the journal the right of first publication and publishing rights
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Those authors who publish in this journal accept the following terms:
-
Authors retain copyright.
-
Authors transfer to the journal the right of first publication. The journal also owns the publishing rights.
-
All published contents are governed by an Attribution-NoDerivatives 4.0 International License.
Access the informative version and legal text of the license. By virtue of this, third parties are allowed to use what is published as long as they mention the authorship of the work and the first publication in this journal. If you transform the material, you may not distribute the modified work. -
Authors may make other independent and additional contractual arrangements for non-exclusive distribution of the version of the article published in this journal (e.g., inclusion in an institutional repository or publication in a book) as long as they clearly indicate that the work was first published in this journal.
- Authors are allowed and recommended to publish their work on the Internet (for example on institutional and personal websites), following the publication of, and referencing the journal, as this could lead to constructive exchanges and a more extensive and quick circulation of published works (see The Effect of Open Access).
