ARTICLES

This paper deals with the transmission power control problem in wireless networks. Such a problem represents a well known and relevant issue as it allows to efficiently manage the network's required energy and the interference experienced by end-users. With the widespread diffusion of smart devices, the relevance of this aspect further increased and has been identified as such also in 5G standards. The problem has been formalized as a Multi-Agent Reinforcement Learning approach (MARL) to guarantee scalability and robustness. These two aspects also drove the development of an original Distributed Average-Cost Temporal-Difference (TD) Learning algorithm. To adopt such an algorithm, a Markov Game formulation of the power control problem has also been derived. The effectiveness of the proposed distributed framework in reducing the total network's transmission power has been proved by means of simulations in a specific case study.

This paper presents a model predictive approach to the energy-aware control of tasks’ execution in an assembly line. The proposed algorithm takes into account both the need for optimizing the assembly line operations (in terms of the minimization of the total cycle time) and that of optimizing the energy consumption deriving from the operations, by exploiting the flexibility added by the presence of a local source of renewable energy (a common scenario of industries that are often equipped, eg, with photovoltaic plants) and, possibly, also exploiting an energy storage plant. The energy-related objectives we take into account refer to the minimization of the energy bill and the minimization of the peaks in the power injected and absorbed from the grid (which is desirable also from the perspective of the network operator). We propose a mixed-integer linear formulation of the optimization problem, through the use of H-infinite norms, instead of the quadratic ones. Simulation results show the effectiveness of the proposed algorithm in finding a trade-off that allows keeping at a minimum the cycle time, while saving on the energy bill and reducing peak powers.

Decision-making in a complex, dynamic, interconnected, and data-intensive industrial environment can be improved with the assistance of machine-learning techniques. In this work, a complex instance of industrial assembly line control is formalized and a parallel deep reinforcement learning approach is presented. We consider an assembly line control problem in which a set of tasks (e.g., vehicle assembly tasks) needs to be planned and controlled during their execution, with the aim of optimizing given key performance criteria. Specifically, the aim will be that of planning the task in order to minimize the total time taken to execute all the tasks (also called cycle time). Tasks run on workstations in the assembly line. To run, tasks need specific resources. Therefore, the tackled problem is that of optimally mapping tasks and resources to workstations, and deciding the optimal execution times of the tasks. In doing so, several constraints need to be respected (e.g., precedence constraints among the tasks, constraints on needed resources to run tasks, deadlines, etc.). The proposed approach uses deep reinforcement learning to learn a tasks/resources mapping policy that is effective in minimizing the resulting cycle time. The proposed method allows us to explicitly take into account all the constraints, and, once training is complete, can be used in real time to dynamically control the execution of tasks. Another motivation for the proposed work is in the ability of the used method to also work in complex scenarios, and in the presence of uncertainties. As a matter of fact, the use of deep neural networks allows for learning the model of the assembly line problem, in contrast with, e.g., optimization-based techniques, which require explicitly writing all the equations of the model of the problem. In order to speed up the training phase, we adopt a learning scheme in which more agents are trained in parallel. Simulations show that the proposed method can provide effective real-time decision support to industrial operators for scheduling and rescheduling activities, achieving the goal of minimizing the total tasks’ execution time.

In this work, a decision support system aimed at suggesting to critical infrastructure (CI) operators the optimal configuration in terms of deployed security functions Ali ties is presented. Two specific problems have been addressed: the security evaluation problem and the security configuration computation problem. Concerning the former problem, the framework provided by the Open Source Security Testing Methodology Manual (OSSTMM) has been retained and extended to capture innovative security features providing CI operators with a holistic insight on the system security level. Concerning the latter problem, the DSS has been provided with an optimisation framework based on a genetic algorithm (GA) for exploring the solution space; in this respect, three different implementations of the adopted GA have been developed and evaluated in realistic operation scenarios. Finally, the outputs of the DSS have been validated from a security point of view.

This paper deals with the problem of resource management in Multi-Access Networks. A Reinforcement Learning based hierarchical control strategy is presented. The main contribution of the proposed approach is its capability of simultaneously tacking the load balancing and QoS management problems in a scalable, dynamic and closed-loop way. The effectiveness of the proposed solution has been proved in a specific case study in the context of which the performances of the proposed algorithm have been compared with a standard load balancing controller.

This paper tackles the power control problem in the context of wireless networks. The development of intelligent services based on widespread smart devices with limited energy storage capabilities and high interference sensitivity is heavily bounded by the energy consumption required for communication. For addressing this issue, a decentralized control approach based on multi-agent reinforcement learning has been developed. The most interesting feature of the proposed solution consists in its scalability and low complexity. As a consequence, the proposed approach can be deployed in presence of sensor nodes with low processing and communication capabilities. Simulations are presented to validate the proposed solution.

Multi-Access Heterogeneous Networks introduced a step forward in modern communication networks allowing the provision of reliable and efficient broadband services. However, heterogeneous networks imply a burden of complexity in the integration, coordination and QoS management processes thus complicating the satisfaction of users' requirements. The aim of the present work is to address the above-mentioned issues by developing a mathematical framework for optimizing resource usage in 5G heterogeneous networks. More in detail, the optimization will take into account both the network's load and energy consumption simultaneously. The proposed approach, based on Model Predictive Control, will be compared with other control strategies for validation and performance comparison.

In the last two years, in Europe, 5G networks and services proliferated. The integration of 5G networks with other radio access networks is considered one of the key enablers for matching the challenging 5G Quality of Service requirements. In particular, the integration with high throughput satellites promises to increase the network performances in terms of resilience and Quality of Service. The present work addresses this problem and presents a user-aware resource allocation methodology for heterogeneous networks. Said methodology is articulated in two-steps: at first, the Analytical Hierarchy Process is used for deciding the network over which traffic is steered and, then, a Cooperative Game for allocating resources within the network is set up. Simulations are presented for validating the proposed approach.

This paper discusses the problem of assembly line control and introduces an optimal control formulation that can be used to improve the performance of the assembly line, in terms of cycle time minimization, resources' utilization, etc. A deterministic formulation of the problem is introduced, based on mixed-integer linear programming. A simple numerical simulation provides a first proof of the proposed concept.

The 5G-ALLSTAR project is aimed at integrating Terrestrial and Satellite Networks for satisfying the highly challenging and demanding requirements of the 5G use cases. The integration of the two networks is a key feature to assure the service continuity in challenging communication situations (e.g., emergency cases, marine, railway, etc.) by avoiding service interruptions. The 5G-ALLSTAR project proposes to develop Multi-Connectivity (MC) solutions in order to guarantee network reliability and improve the throughput and latency for each connection between User Equipment (UE) and network. In the 5G-ALLSTAR vision, we divide the gNB in two entities: 1) gNB-CU (Centralized Unit) and 2) gNB-DU (Distributed Unit) The gNB-CU integrates an innovative Traffic Flow Control algorithm able to optimize the network resources by coordinating the controlled gNB-DUs resources, while implementing MC solutions. The MC permits to connect each UE with simultaneous multiple access points (even different radio access technologies). This solution leads to have independent gNB-DU/s that contain the RLC, MAC and PHY layers. The 5G-ALLSTAR MC algorithms offer advanced functionalities to RRC layer (in the gNB-CU) that is, in turn, able to set up the SDAP, the PDCP and the lower layers in gNB-DU. In this regard, the AI-based MC algorithms, implemented in gNB-CU, by considering the network performances in the UE surrounding environment as well as the UE QoS requirements, will dynamically select the most promising access points able to guarantee the fulfilment of the requirements also enabling the optimal traffic splitting to cope with the connection reliability. In this paper, we present also an innovative AI-based framework, included within the Traffic Flow Control, able to address the MC objectives, by implementing a Reinforcement Learning algorithm in charge of solving the network control problem.

Driven by the business potentialities of the satellite industry, the last years witnessed a massive increase of attention in the space industry. This sector has been always considered critical by national entities and international organizations worldwide due to economic, cultural, scientific, military and civil implications. The need of cutting down satellite launch costs has become even more impellent due to the competition generated by the entrance in the sector of new players, including commercial organizations. Indeed, the high demand of satellite services requires affordable and flexible launch. In this context, a fundamental aspect is represented by the optimization of launch centers' logistics. The aim of this paper is to investigate and review the benefits and potential impact that consolidated operations research and management strategies, coupled with emerging paradigms in machine learning and control can have in the satellite industry, surveying techniques which could be adopted in advanced operations management of satellite launch centers.

The deep integration between the cyber and physical domains in complex systems make very challenging the security evaluation process, as security itself is more of a concept (i.e., a subjective property) than a quantifiable characteristic. Traditional security assessing mostly relies on the personal skills of security experts, often based on best practices and personal experience. The present work is aimed at defining a security metric allowing evaluators to assess the security level of complex cyber-physical systems (CPSs), as critical infrastructures, in a holistic, consistent and repeatable way. To achieve this result, the mathematical framework provided by the open source security testing methodology manual (OSSTMM) is used as the backbone of the new security metric, since it allows to provide security indicators capturing, in a non-biased way, the security level of a system. Several concepts, as component lifecycle, vulnerability criticality and damage potential - effort ratio are embedded in the new security metric framework, developed in the scope of the H2020 project ATENA.

This paper presents an optimization framework, based on Genetic Algorithms, for the control of the “security level” of a Cyber-Physical System (CPS). The security level is a quantity that has been studied in several industrial standards, among which we selected the Open Source Security Testing Methodology Manual (OSSTMM). The proposed optimization solution is validated on scenarios representative of real operations of a security evaluator, and numerical simulations report the performances obtained by the algorithm.

In this paper an approach to the many-to-many carpooling problem with automated passenger aggregation is presented. The proposed solution allows to optimally solve the related routing problem, by relying on a constrained shortest path algorithm, for users travelling within multiple transportation networks, thus enabling multi-modality, and exploits the users' availability to be aggregated into carpools. The mathematical model behind the proposed approach is illustrated. Then, an algorithmic procedure capable of reasonably coping with the complexity that arises in real-sized scenarios, often characterized by multiple heterogeneous data sources, is discussed. Finally, simulations are reported in order to evaluate the effectiveness and the performance of the proposed approach.

This paper presents a reference architecture and a control scheme for the aggregation and management of electric vehicle (EV) load at medium voltage level. The focus is put on the problem of EV load reprofiling, aimed at the procurement of active demand (AD) services to interested grid/market actors. The proposed approach achieves AD product composition always guaranteeing the respect of grid constraints as well as user constraints on the charging processes. Simulations are presented to illustrate the effectiveness of the proposed approach.