]> Service Instance Deployment based on Integrated Network and Compute Metrics China Mobile
yaokehan@chinamobile.com
Telefonica
luismiguel.contrerasmurillo@telefonica.com
Qualcomm Europe, Inc.
jros@qti.qualcomm.com
opsawg The deployment of service instances across distributed interconnected edge-cloud environments can be optimized in terms of performance expectations and Service Level Objectives (SLOs) satisfaction when performed taken into account both network and compute metrics. In order to do so, this document primarily concentrates on existing standardized mechanisms, namely ALTO and CATS, to facilitate such integration of metrics. The ALTO protocol can be extended to expose compute metrics from a cloud manager to a network orchestrator or as part of the network and cost maps, enabling improved deployment of compute service instances based on joint awareness of both network and computing information. This document proposes protocol extensions, workflows, and operational considerations for ALTO enhancements using CATS metrics.
Introduction Applications such as artificial intelligence (AI) inference and cloud rendering require performance optimization based on joint awareness of both network and computing information. introduces the service lifecycle, including service deployment, service selection, and service assurance. The discussion and documentation of the service selection problem are currently being undertaken by the Computing-Aware Traffic Steering (CATS) Working Group, while the service deployment problem still lacks a dedicated venue for resolution. This document primarily focuses on the service deployment problem and leverages the flexibility of ALTO protocol extensions to enable joint awareness of both network and computing information for improved compute service instance deployment. , adopted by the CATS Working Group, defines three metric levels for all CATS-related metrics from the computing domain (e.g., cloud). These metrics are initially intended to support service instance selection for traffic steering, but they can also be exposed for compute service deployment. introduces a method for using ALTO protocol extensions for service deployment, but it does not cover CATS metrics or their encodings. This document provides supplementary information to , including protocol extensions for encoding CATS metrics, workflows, and operational considerations for compute service instance deployment.
Workflows The main entities involved in this solution are the ALTO server and the ALTO client. *ALTO Server: Deployed in the cloud manager and network controller, it is responsible for collecting various CATS metrics. *ALTO Client: Located in the service orchestrator, it is responsible for requesting and receiving information from the ALTO server, and for formulating compute instance deployment strategies based on the collected data. There are two basic implementation schemes. The interactions between the ALTO client and servers may vary depending on the chosen mode.
Request and Response shows the workflow under request and response mode. The ALTO client in the service orchestrator within the network domain first requests computing domain information from the ALTO server located in the cloud manager, and then requests network domain information from the ALTO server in the network controller. The ALTO server in the cloud manager collects CATS metrics, including various L0 metrics, calculates L1 and L2 metrics, determines and selects all or part of the metrics from L0, L1, and L2 to pass to the ALTO client, encapsulates the message, and sends the ALTO message to the ALTO client. The basis for the ALTO server in the cloud manager to determine which level of computing metrics to send is the request information from the ALTO client. The initial request from the ALTO client will clearly specify the requested level of CATS metrics (such as L2, L1, and/or L0), and for L1 and/or L0 metrics, it will specify whether to request all metrics at that level or specific ones (e.g., only the "compute type" L1 metric for L1, or only CPU utilization for L0). The ALTO server in the network controller obtains network domain information (such as link bandwidth, latency, etc.), encapsulates all the obtained information in an ALTO message, and sends it to the ALTO client. The ALTO client sends confirmation messages to both ALTO servers. The ALTO client in the service orchestrator calculates the compute instance deployment method based on the obtained computing domain and network domain information, and then sends the deployment strategy information to the ALTO server in the cloud manager, which notifies the cloud manager to deploy the corresponding computing instances.
Request-Response Mode | Request for network metrics | | <-----------------------------------------+ | | | +------+------+ | | |CATS metrics | | | | collection | | | +------+------+ +-------+-------+ | | |network metrics| | send ALTO message with CATS metrics | | collection | <-----------------------------------------+ +-------+-------+ | | | | ack | | +-----------------------------------------> | send ALTO message with network metrics | | +-----------------------------------------> | | ack | | +-----------------------------------------+ | | +----------+-------------+ | | |Make deployment strategy| | | | based on metrics from | | | | both sides | | | +----------+-------------+ | | | instance deployment notification | | +---------------------------------------->| | | | ]]>
Active Push shows the workflow under active push mode. The ALTO client establishes Server-Sent Events(SSE) long connections with the ALTO server in the cloud manager and the ALTO server in the network controller respectively. During connection maintenance, ALTO servers actively push ALTO messages to the ALTO client. The ALTO server in the cloud manager obtains CATS metrics, including various L0 metrics, calculates L1 and L2 metrics, determines and selects all or part of the metrics from L0, L1, and L2 to pass to the ALTO client, encapsulates the message, and actively pushes the ALTO message containing the selected computing domain information. The ALTO server in the network controller obtains network domain information, encapsulates it in an ALTO message, and pushes it to the ALTO client. The ALTO client sends confirmation messages and periodic heartbeat messages to maintain connection status. ALTO servers then reply with acknowledgment (ACK) messages if the connections are valid. The ALTO client formulates the compute instance deployment strategy based on the obtained information and sends the deployment strategy information. Note that in the active push mode, the ALTO server initially sends L2 level computing metrics by default. The ALTO client can carry requests for L1 and/or L0 level metrics (all or part) in the confirmation message, and the ALTO server will start pushing the requested metrics from the next cycle.
Active Push Mode | SSE connection | | <-----------------------------------------> | | | +------+------+ | | |CATS metrics | | | | collection | | | +------+------+ +-------+-------+ | | |network metrics| | push ALTO message with CATS metrics | | collection | <-----------------------------------------+ +-------+-------+ | | | | ack | | +-----------------------------------------> | push ALTO message with network metrics | | +-----------------------------------------> heartbeat message | | ack +-----------------------------------------> <-----------------------------------------+ ack | | heartbeat message <-----------------------------------------+ <-----------------------------------------+ | | ack | | +-----------------------------------------> | | +----------+-------------+ | | |make deployment strategy| | | | based on metrics from | | | | both sides | | | +----------+-------------+ | | | instance deployment notification | | +---------------------------------------->| | | | ]]>
Protocol Extention Examples The three-layer metric information can be defined by extending the ALTO endpoint cost service in RFC 7285, through extending the "cost-type" field. shows the example in JSON format:
cost-type extension
describes the encapsulation of CATS metrics as well as site information based on the ALTO protocol extensions in JSON format:
alto encodings
In the example above, the ALTO message carries metric information of two service endpoints. The ALTO server can define and set the content to be sent, choosing to send all levels of metric information for the corresponding endpoint, or select one level of metrics or a specific metric within a level.
Operational Considerations
Compute Instance Deployment Strategy The ALTO client in the service orchestrator formulates various computing service instance deployment methods based on computing and network information. The procedure of how to generate and deploy the strategy should also be considered. Firstly, determine the availability for instance deployment. The ALTO client must first verify whether a specific site is capable of hosting the service. Therefore, supplementary procedures may be required at the beginning of both workflows described in the previous sections. Upon receiving a service request, the ALTO client needs to notify the ALTO server in the cloud manager of the specific resources required for deployment (e.g., X CPU cores or Y GB of GPU memory). Secondly, determine the priority for instance deployment. If the ALTO client receives Level 2 (L2) metrics, it may perform a direct summation. If it receives Level 1 (L1) metrics, it may apply a weighted summation, for example: (Score of computing class L1 metric of a node from the cloud manager's ALTO server * weight1) + (Score of normalized network class L1 metric of a link from the network controller's ALTO server * weight2) If the ALTO client receives Level 0 (L0) metrics, the algorithm may involve applying a polynomial function over multiple metrics. After computation, the ALTO client sorts the results to determine the priority of instance deployment, with higher scores indicating higher priority. Thirdly, determine remaining resources after instance deployment. Once the compute service node for deployment is selected, the ALTO server completes the instance deployment, calculates the remaining resource availability, and notifies the ALTO client.
Deployment Considerations of ALTO Client and Server The ALTO server can be co-located with the network controller or cloud manager, or deployed separately. Similarly, the ALTO client can be co-located with the service orchestrator or deployed separately. The three-level metric framework provides flexibility in information exposure, allowing adaptation to different scenarios where the computing and network domains may belong to the same or different service entities. Dynamic updates of metrics should be considered to ensure the timeliness and accuracy of information for effective deployment decisions.
Security Considerations This document does not introduce new security risks beyond those inherent in the ALTO protocol. Security mechanisms specified in and related ALTO extensions (such as access control in ) should be applied to protect sensitive computing and network information, especially when computing and network domains belong to different service entities.
IANA Considerations This document has no IANA actions.
Acknowledgements
References Normative References Application-Layer Traffic Optimization (ALTO) Protocol Applications using the Internet already have access to some topology information of Internet Service Provider (ISP) networks. For example, views to Internet routing tables at Looking Glass servers are available and can be practically downloaded to many network application clients. What is missing is knowledge of the underlying network topologies from the point of view of ISPs. In other words, what an ISP prefers in terms of traffic optimization -- and a way to distribute it. The Application-Layer Traffic Optimization (ALTO) services defined in this document provide network information (e.g., basic network location structure and preferences of network paths) with the goal of modifying network resource consumption patterns while maintaining or improving application performance. The basic information of ALTO is based on abstract maps of a network. These maps provide a simplified view, yet enough information about a network for applications to effectively utilize them. Additional services are built on top of the maps. This document describes a protocol implementing the ALTO services. Although the ALTO services would primarily be provided by ISPs, other entities, such as content service providers, could also provide ALTO services. Applications that could use the ALTO services are those that have a choice to which end points to connect. Examples of such applications are peer-to-peer (P2P) and content delivery networks. Application-Layer Traffic Optimization (ALTO) Deployment Considerations Many Internet applications are used to access resources such as pieces of information or server processes that are available in several equivalent replicas on different hosts. This includes, but is not limited to, peer-to-peer file sharing applications. The goal of Application-Layer Traffic Optimization (ALTO) is to provide guidance to applications that have to select one or several hosts from a set of candidates capable of providing a desired resource. This memo discusses deployment-related issues of ALTO. It addresses different use cases of ALTO such as peer-to-peer file sharing and Content Delivery Networks (CDNs) and presents corresponding examples. The document also includes recommendations for network administrators and application designers planning to deploy ALTO, such as recommendations on how to generate ALTO map information. Informative References CATS Metrics Definition China Mobile Huawei Technologies Telefonica Qualcomm Europe, Inc. Huawei Technologies Computing-Aware Traffic Steering (CATS) is a traffic engineering approach that optimizes the steering of traffic to a given service instance by considering the dynamic nature of computing and network resources. In order to consider the computing and network resources, a system needs to share information (metrics) that describes the state of the resources. Metrics from network domain have been in use in network systems for a long time. This document defines a set of metrics from the computing domain used for CATS. Joint Exposure of Network and Compute Information for Infrastructure-Aware Service Deployment Nokia Bell Labs Telefonica Qualcomm Europe, Inc. Deutsche Telekom Service providers are starting to deploy computing capabilities across the network for hosting applications such as distributed AI workloads, AR/VR, vehicle networks, and IoT, among others. In this network-compute environment, knowing information about the availability and state of the underlying communication and compute resources is necessary to determine both the proper deployment location of the applications and the most suitable servers on which to run them. Further, this information is used by numerous use cases with different interpretations. This document proposes an initial approach towards a common exposure scheme for metrics reflecting compute and communication capabilities. Use of ALTO for Determining Service Edge Telefonica Nokia Bell Labs Qualcomm Europe Benocs Unicamp Service providers are starting to deploy computing capabilities across the network for hosting applications such as AR/VR, vehicle networks, IoT, and AI training, among others. In these distributed computing environments, knowledge about computing and communication resources is necessary to determine the proper deployment location of each application. This document proposes an initial approach towards the use of ALTO to expose such information to the applications and assist the selection of their deployment locations.