Processing hardware running the ML model: a physical or virtual hardware platform, infrastructure executing the inference operations of the running trained model. Such a platform can consist of server clusters of CPU’s, GPU’s or specialized accelerators. GPU’s became a cornerstone for processing machine learning operations, as their parallel architecture accelerates matrix multiplications and linear algebra calculations, underpinning machine learning algorithms. As LLM’s require substantial computational resources, LLM’s are often trained with cloud-based servers fitted with GPU clusters. There may be many instances “*” of the processing hardware asset, dedicated to the processing of one machine learning model.
The system asset has the following defined methods:
- executeMachineLearningModelOperations() – a method, depicting the process of executing machine learning software operations for the purposes of inference.
- operateWithModelOperationalData() – based on the inference operations and conducted calculations – setup, update and utilize the temporary model’s operational data.
- storeOperationalData() – store the setup model’s operational data in RAM for further to enable further required inference operations.
The association between the system asset and the machine learning model operates with the following business asset:
- Model’s operational data: transient and intermediate data and calculations generated and executed upon, during the model’s operational, inference stage like target model’s parameters. The operational data is held in system memory during model’s operations, it may contain temporary results and sensitive information. In case of LLM’s, the calculations for the preprocessed input query involve calculations across model’s layers, attention mechanisms, processing of token embeddings.