Optimization of Multi Agent System for Distribution Control of Distinct Heating System Using Improved Q-Learning Controller
DOI:
https://doi.org/10.32628/IJSRSET1229152Keywords:
Q- learning, multi agent, optimization ,controller, simulation, District heating systemAbstract
The aim of this research is the use of multi-agent systems for the optimization of a distributed control for district heating systems. A district heating system comprises of production units, a distribution network, and a host of consumer substations. The operations of district heating system usually involves conflicting goals, e.g., to satisfy customers and to minimize production costs. Hence the agent must be capable of optimizing between maximizing supply to substations and minimizing production cost. Current substations employ purely reactive devices, making local decisions without taking into account the global state. Moreover the substations determine the flow in all parts of the district heating system. The optimal operation of the district heating system is therefore limited to providing sufficiently high temperature and pressure to all customers by taking local measurement to achieve this goal without considering other factors such as cost of production and time. The approach studied in this research is to equip substations with software agents to form a multi-agent system using Q-learning. The study also shows that it is possible to control the trade-off between quality of service and degree of surplus production as well as the possibility of extending the system with new consumers without increasing production capacity. In another study, an experiment was conducted in a controlled physical environment, where two agent-based approaches were evaluated and compared to existing technologies. The experiment shows that it is possible to automatically load balance a small district heating network using agent technology.. Finally, a generalized formal characterization of the problem space under investigation is provided, i.e., production and logistics network management, together with a preliminary evaluation of the applicability of the suggested multi-agent system approach for this general problem area.
References
- Anaheim Parunak, et al (2014) “Agent-Based Modeling vs. Equation-Based Modeling: A Case Study and Users' Guide”, Multi-Agent Systems and Agent-Based Simulation, LNAI, Springer Verlag Vol. 1534,pp 10-25.
- Arntzen, B.C. and Trafton, L.L., (2011) “Global Supply Chain Management at Digital Equipment Corporation”, Interfaces, Vol. 25(1): pp 69-93.
- Arvastsson, L. et al (2017) “Stochastic Modeling and Operational Optimization in District Heating Systems”, Lund Institute of Technology, Sweden, ISBN 91-628-4855-0.
- Bellifemine, F. and Rimassa, G. (2017) “Developing multi-agent systems with a FIPA-compliant agent framework”, Software: Practice and Experience, Vol. 31(2), John Wiley & Sons, Ltd, New York (2001) 103-128.
- Bøhm, B., et al (2018), “Simple Models for Operational Optimization”, Report S1, NOVEM, ISBN 9057480212.
- Busoniu, R. Babuska, and B. de Schutter, (2008). “A comprehensive survey of multi agent reinforcement learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 2, pp. 156–172, View at: Publisher Site Google Scholar
- Clearwater, S.H., (2017.), “A paradigm for distributed resource allocation”, Xerox PARC, ISBN 981-02-2254-8, 1995.
- Canu, S., and Ding, X. (2010) “District Heating Forecast using Artificial Neural Networks”, International Journal of Engineering, Vol. 2(4).
- Cohen, M.A., and Lee, H.L. (2017) “Strategic Analysis of Integrated Production-Distribution Systems: Models and Methods”, Journal of Operation Research, Vol. 36(2): pp 216-228.
- Durfee, E. and Lesser, V. (2015), “Partial Global Planning: A coordination framework for distributed hypothesis formation”, IEEE Transactions on Systems, Man, and Cybernetics, Vol. 21(5) pp.1167-1183, 1991.
- Derksen, C and Unland, R. (2015) Fay, A. Structure and classification of unified energy agents as a base the systematic development of future energy grids. Eng. Appl. Artif. Intell., 41, 310–324.
- Erman, D.L., et al (2015) “The HER- SAY-II speech understanding system: Integrating knowledge to resolve uncertainty”, ACM Computing Survey, Vol. 12 pp.213-253, 1980.
- Fan, M., and Whinston, A.B., (2018) “Decentralised Mechanism Design for Supply Chain Organisations Using an Auction Market”, Addison-new deihi, ISBN 2-101-36749-9.
- Ferber, J. (2015) “Multi-Agent Systems”, Addison-Wesley, ISBN 0-201-36048-9.
- Fox, M.S. and Teigen, R. (2016) “Agent-Oriented Supply- Chain Managemen1t”, The International Journal of Flexible Manufacturing Systems, Vol. 12: 165-188, 2000.
- Johansson, C. and Towards, Y (2010). Intelligent District Heating. Ph.D. Thesis, Blekinge Institute of Technology, Karlskrona, Sweden,
- Jamal, B (2005). “A Pragmatic and Semantic Unified Framework for Agent Communication”, a PhD thesis, Université Laval, Québec,
- Wooldridge, M., (2018) “An Introduction to Multi-Agent Systems”, John Wiley & Sons, ISBN 0 471-49691.
- Wang, L. Liu, H. Zhang, and G. Xiao (2016)., “Fault-tolerant controller design for a class of nonlinear MIMO discrete-time systems via online reinforcement learning algorithm,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 46, no. 5, pp. 611–622, View at: Publisher Site Google Scholar
- Zhao, Z. Xia, and D. Wang, (2015). “Model-free optimal control for affine nonlinear systems with convergence analysis,” IEEE Transactions on Automation Science and Engineering, vol. 12, no. 4, pp. 1461–1468, View at: Publisher Site | Google Scholar
Downloads
Published
Issue
Section
License
Copyright (c) IJSRSET
This work is licensed under a Creative Commons Attribution 4.0 International License.