Developing an AI Model Registry and Lifecycle Management System for Cross-Functional Tech Teams
DOI:
https://doi.org/10.32628/IJSRSET25121179Keywords:
AI Model Management, Lifecycle Management, Cross-Functional Teams, Version Control, Model Deployment, Collaboration ToolsAbstract
This paper presents a comprehensive solution for managing AI models across their lifecycle through the development of an AI model registry and lifecycle management system. As AI continues to play a crucial role across industries, the complexity of managing models—from development to deployment—presents significant challenges, especially within cross-functional teams. These challenges include issues such as model versioning, metadata management, deployment inconsistencies, and communication breakdowns among data scientists, engineers, and business stakeholders. The proposed system addresses these challenges by providing a centralized platform that integrates features such as version control, metadata management, and automated deployment, thereby improving transparency and reducing the risk of deployment errors. Furthermore, the system fosters enhanced collaboration by integrating widely-used project management tools like GitHub, Jira, and Slack, ensuring that teams remain aligned throughout the model's lifecycle. By enabling continuous monitoring and incorporating automated model drift detection, the system ensures that AI models remain accurate and efficient post-deployment. This paper also explores the technical implementation strategy for the system, including the use of containerization, cloud-native infrastructure, and microservices architecture to ensure scalability and flexibility. The implications of this work extend beyond technical considerations, as it enhances collaboration, improves model quality, and accelerates deployment cycles. Future research directions include exploring automation in model updates, scalability in large enterprises, and the integration of additional tools and frameworks. This work provides a critical step toward optimizing AI model management, offering a scalable, efficient, and secure approach to managing AI models throughout their lifecycle.
Downloads
References
M. Chui and S. Francisco, "Artificial intelligence the next digital frontier," McKinsey and Company Global Institute, vol. 47, no. 3.6, pp. 6-8, 2017.
J. Bughin, E. Hazan, P. Sree Ramaswamy, W. DC, and M. Chu, "Artificial intelligence the next digital frontier," 2017.
S. GÜNDÜZ, "A Comparative Research on Artificial Intelligence-Driven Transformations in Business Management: Strategic Applications in Finance, Tourism, Healthcare, Retail, and Manufacturing Sectors," ULUSLARARASI AKADEMİK BİRİKİM DERGİSİ, vol. 7, no. 4, 2024.
P. Agarwal, S. Swami, and S. K. Malhotra, "Artificial intelligence adoption in the post COVID-19 new-normal and role of smart technologies in transforming business: a review," Journal of Science and Technology Policy Management, vol. 15, no. 3, pp. 506-529, 2024.
M. Javaid, A. Haleem, R. P. Singh, and R. Suman, "Artificial intelligence applications for industry 4.0: A literature-based study," Journal of Industrial Integration and Management, vol. 7, no. 01, pp. 83-111, 2022.
A. B. Rashid and A. K. Kausik, "AI revolutionizing industries worldwide: A comprehensive overview of its diverse applications," Hybrid Advances, p. 100277, 2024.
S. Abbasi and A. M. Rahmani, "Artificial intelligence and software modeling approaches in autonomous vehicles for safety management: a systematic review," Information, vol. 14, no. 10, p. 555, 2023.
I. H. Sarker, "AI-based modeling: techniques, applications and research issues towards automation, intelligent and smart systems," SN computer science, vol. 3, no. 2, p. 158, 2022.
Y. Gil et al., "Artificial intelligence for modeling complex systems: taming the complexity of expert models to improve decision making," ACM Transactions on Interactive Intelligent Systems, vol. 11, no. 2, pp. 1-49, 2021.
G. Dagnaw, "Artificial intelligence towards future industrial opportunities and challenges," 2020.
O. Michael, "of Artificial Intelligence," The Future of Small Business in Industry 5.0, p. 215, 2024.
M. Kejriwal, Artificial intelligence for industries of the future. Springer, 2023.
Y. Duan, J. S. Edwards, and Y. K. Dwivedi, "Artificial intelligence for decision making in the era of Big Data–evolution, challenges and research agenda," International journal of information management, vol. 48, pp. 63-71, 2019.
M. F. Tahir, "Role of AI/ML in decision making in software release management," 2024.
A. Soni, A. Kumar, R. Arora, and R. Garine, "Integrating AI into the Software Development Life Cycle: Best Practices, Tools, and Impact Analysis," Tools, and Impact Analysis (June 10, 2023), 2023.
W. Hummer et al., "Modelops: Cloud-based lifecycle management for reliable and trusted ai," in 2019 IEEE International Conference on Cloud Engineering (IC2E), 2019: IEEE, pp. 113-120.
R. U. Attah, B. M. P. Garba, I. Gil-Ozoudeh, and O. Iwuanyanwu, "Cross-functional team dynamics in technology management: a comprehensive review of efficiency and innovation enhancement," Eng Sci Technol J, vol. 5, no. 12, pp. 3248-65, 2024.
O. G. Berg, "Challenges and success factors in a cross-functional development team in a large-scale agile context: An exploratory case study," NTNU, 2024.
S. Schelter, F. Biessmann, T. Januschowski, D. Salinas, S. Seufert, and G. Szarvas, "On challenges in machine learning model management," 2015.
D. De Silva and D. Alahakoon, "An artificial intelligence life cycle: From conception to production," Patterns, vol. 3, no. 6, 2022.
S. Sinha and Y. M. Lee, "Challenges with developing and deploying AI models and applications in industrial systems," Discover Artificial Intelligence, vol. 4, no. 1, p. 55, 2024.
S. Passi and S. J. Jackson, "Trust in data science: Collaboration, translation, and accountability in corporate data science projects," Proceedings of the ACM on human-computer interaction, vol. 2, no. CSCW, pp. 1-28, 2018.
A. X. Zhang, M. Muller, and D. Wang, "How do data science workers collaborate? roles, workflows, and tools," Proceedings of the ACM on Human-Computer Interaction, vol. 4, no. CSCW1, pp. 1-23, 2020.
R. Avacharmal and S. Pamulaparthyvenkata, "Enhancing Algorithmic Efficacy: A Comprehensive Exploration of Machine Learning Model Lifecycle Management from Inception to Operationalization," Distributed Learning and Broad Applications in Scientific Research, vol. 8, pp. 29-45, 2022.
M. Elahi, S. O. Afolaranmi, J. L. Martinez Lastra, and J. A. Perez Garcia, "A comprehensive literature review of the applications of AI techniques through the lifecycle of industrial equipment," Discover Artificial Intelligence, vol. 3, no. 1, p. 43, 2023.
R. Ashmore, R. Calinescu, and C. Paterson, "Assuring the machine learning lifecycle: Desiderata, methods, and challenges," ACM Computing Surveys (CSUR), vol. 54, no. 5, pp. 1-39, 2021.
D. Nigenda et al., "Amazon sagemaker model monitor: A system for real-time insights into deployed machine learning models," in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 3671-3681.
L. Yang and D. Rossi, "Quality monitoring and assessment of deployed deep learning models for network AIOps," IEEE Network, vol. 35, no. 6, pp. 84-90, 2022.
S. Patchipala, "Tackling data and model drift in AI: Strategies for maintaining accuracy during ML model inference," International Journal of Science and Research Archive, vol. 10, no. 2, pp. 1198-1209, 2023.
F. Bachinger, G. Kronberger, and M. Affenzeller, "Continuous improvement and adaptation of predictive models in smart manufacturing and model management," IET Collaborative Intelligent Manufacturing, vol. 3, no. 1, pp. 48-63, 2021.
M. Yusuff, "Model Drift Monitoring: Continuously Tracking Model Performance Metrics to Detect Accuracy Degradation."
F. Bayram, B. S. Ahmed, and A. Kassler, "From concept drift to model degradation: An overview on performance-aware drift detectors," Knowledge-Based Systems, vol. 245, p. 108632, 2022.
K. Dubovikov, Managing Data Science: Effective strategies to manage data science projects and build a sustainable team. Packt Publishing Ltd, 2019.
L. E. Lwakatare, A. Raj, I. Crnkovic, J. Bosch, and H. H. Olsson, "Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions," Information and software technology, vol. 127, p. 106368, 2020.
A. Goyal, "Enhancing engineering project efficiency through cross-functional collaboration and IoT integration," Int. J. Res. Anal. Rev, vol. 8, no. 4, pp. 396-402, 2021.
T. Ahmad, J. Boit, and A. Aakula, "The role of cross-functional collaboration in digital transformation," Journal of Computational Intelligence and Robotics, vol. 3, no. 1, pp. 205-42, 2023.
S. U. Rahaman, "Breaking Silos: Architecting Cross-Functional Analytics Frameworks for Collaborative Insights."
W. H. Deng, N. Yildirim, M. Chang, M. Eslami, K. Holstein, and M. Madaio, "Investigating practices and opportunities for cross-functional collaboration around AI fairness in industry practice," in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 705-716.
N. L. Rane, "Multidisciplinary collaboration: key players in successful implementation of ChatGPT and similar generative artificial intelligence in manufacturing, finance, retail, transportation, and construction industry," 2023.
B. Saha and M. Kumar, "Investigating cross-functional collaboration and knowledge sharing in cloudnative program management systems," International Journal for Research in Management and Pharmacy, vol. 9, no. 12, 2020.
P. Goncharenko, "The Potential of Using Artificial Intelligence in Communications within Project Management," 2024.
F. Imran, K. Shahzad, A. Butt, and J. Kantola, "Digital transformation of industrial organizations: Toward an integrated framework," Journal of change management, vol. 21, no. 4, pp. 451-479, 2021.
M. Pöyhönen, "Human-AI Integration in Long-Established Organizations," 2024.
O. H. Olayinka, "Data driven customer segmentation and personalization strategies in modern business intelligence frameworks," World Journal of Advanced Research and Reviews, vol. 12, no. 3, pp. 711-726, 2021.
J. Jöhnk, M. Weißert, and K. Wyrtki, "Ready or not, AI comes—an interview study of organizational AI readiness factors," Business & information systems engineering, vol. 63, no. 1, pp. 5-20, 2021.
R. Bergmann, F. Theusch, P. Heisterkamp, and N. Grigoryan, "Comparative Analysis of Open-Source ML Pipeline Orchestration Platforms."
R. Kienzler and H. Kyas, "Tensorflow 2.0 and Kubeflow for Scalable and Reproducable Enterprise AI," in CS & IT Conference Proceedings, 2020, vol. 10, no. 1: CS & IT Conference Proceedings.
A. Choudhury, Continuous Machine Learning with Kubeflow: Performing Reliable MLOps with Capabilities of TFX, Sagemaker and Kubernetes (English Edition). BPB Publications, 2021.
D. G. Ancona and D. F. Caldwell, "Information technology and work groups: The case of new product teams," in Intellectual teamwork: Psychology Press, 2014, pp. 173-190.
J. Marttila, "ENHANCING MULTI-PROJECT MANAGEMENT THROUGH PRODUCT MANAGEMENT INTEGRATION," 2024.
R. Y. Gabrow, "Concurrent engineering, product life cycle management using cross-functional teams: a case study," Periodicals of Engineering and Natural Sciences (PEN), vol. 9, no. 2, pp. 842-857, 2021.
M. Shahin, M. A. Babar, and L. Zhu, "Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices," IEEE access, vol. 5, pp. 3909-3943, 2017.
W. Shen et al., "Systems integration and collaboration in architecture, engineering, construction, and facilities management: A review," Advanced engineering informatics, vol. 24, no. 2, pp. 196-207, 2010.
R. Van Der Lans, Data Virtualization for business intelligence systems: revolutionizing data integration for data warehouses. Elsevier, 2012.
A. Shrivastav, "THE ROLE OF AGILE METHODOLOGIES IN PRODUCT LIFECYCLE MANAGEMENT (PLM) OPTIMIZATION."
M. J. Goswami, "Optimizing Product Lifecycle Management with AI: From Development to Deployment," International Journal of Business Management and Visuals, ISSN, pp. 3006-2705.
E. Papagiannidis, I. M. Enholm, C. Dremel, P. Mikalef, and J. Krogstie, "Toward AI governance: Identifying best practices and potential barriers and outcomes," Information Systems Frontiers, vol. 25, no. 1, pp. 123-141, 2023.
M. Dumas et al., "AI-augmented business process management systems: a research manifesto," ACM Transactions on Management Information Systems, vol. 14, no. 1, pp. 1-19, 2023.
H. Lakk, "Model-driven role-based access control for databases," Citeseer, 2012.
M. Uddin, S. Islam, and A. Al-Nemrat, "A dynamic access control model using authorising workflow and task-role-based access control," Ieee Access, vol. 7, pp. 166676-166689, 2019.
T. Baumer, M. Müller, and G. Pernul, "System for cross-domain identity management (SCIM): Survey and enhancement with RBAC," IEEE Access, vol. 11, pp. 86872-86894, 2023.
A. Alabdulatif, N. N. Thilakarathne, and K. Kalinaki, "A novel cloud enabled access control model for preserving the security and privacy of medical big data," Electronics, vol. 12, no. 12, p. 2646, 2023.
S. O. Dyke et al., "Registered access: authorizing data access," European Journal of Human Genetics, vol. 26, no. 12, pp. 1721-1731, 2018.
A. Petrova, "Cloud Computing in the Age of Big Data: Storage, Analytics, and Scalability," Advances in Computer Sciences, vol. 6, no. 1, 2023.
D. Bayazitov, K. Kozhakhmet, A. Omirali, and R. Zhumaliyeva, "Leveraging Amazon Web Services for cloud storage and AI algorithm integration: A comprehensive analysis," Applied Mathematics, vol. 18, no. 6, pp. 1235-1246, 2024.
P. Mathur, "Cloud computing infrastructure, platforms, and software for scientific research," High Performance Computing in Biomimetics: Modeling, Architecture and Applications, pp. 89-127, 2024.
M. Haakman, L. Cruz, H. Huijgens, and A. Van Deursen, "AI lifecycle models need to be revised: An exploratory study in Fintech," Empirical Software Engineering, vol. 26, no. 5, p. 95, 2021.
S. Yajamanam Kidambi, "End-to-End artificial intelligence lifecycle management," Massachusetts Institute of Technology, 2022.
Y. Xie, "AI Model Lifecycle Management: Systematic Mapping Study and Solution for AI Democratisation," 2020.
A. K. Kordon and A. K. Kordon, "The AI-based data science workflow," Applying Data Science: How to Create Value with Artificial Intelligence, pp. 189-202, 2020.
E. A. d. Oliveira, M. L. Pimenta, P. Hilletofth, and D. Eriksson, "Integration through cross-functional teams in a service company," European Business Review, vol. 28, no. 4, pp. 405-430, 2016.
G. Abdiyeva-Aliyeva, "Application of ai in software engineering: Handling data management problems in production," Informatics and Control Problems, vol. 43, no. 2, pp. 94-101, 2023.
J. Paul, "How Software Engineering is Shaping AI's Future: The Tools and Practices Behind Smarter Systems," 2024.
D. K. Pandiya and N. Charankar, "INTEGRATION OF MICROSERVICES AND AI FOR REAL-TIME DATA PROCESSING."
E. Oye, E. Frank, and J. Owen, "Microservices Architecture for Large-Scale AI Applications," 2024.
D. V. A. Palli, "Monolithic vs. Microservices Architectures for AI-Integrated Applications," 2024.
E. Wolff, Microservices: flexible software architecture. Addison-Wesley Professional, 2016.
M. Abbasi, M. V. Bernardo, P. Váz, J. Silva, and P. Martins, "Adaptive and Scalable Database Management with Machine Learning Integration: A PostgreSQL Case Study," Information, vol. 15, no. 9, p. 574, 2024.
W. L. Schulz, B. G. Nelson, D. K. Felker, T. J. Durant, and R. Torres, "Evaluation of relational and NoSQL database architectures to manage genomic annotations," Journal of biomedical informatics, vol. 64, pp. 288-295, 2016.
O. Ur Rehman, "Frontend Module for the Management of Museums and Events," Politecnico di Torino, 2024.
H. Tuunainen, "The potential of generative artificial intelligence in leading a scalable agile enterprise by objectives," 2024.
H. S. Vudandapuram and R. R. Madireddy, "Evaluating Requirement Management Tool’s Features for Effective Collaboration in Distributed Software Development Teams," ed, 2024.
V. Ståhlberg, "Enhancing software development processes with artificial intelligence," Artificial intelligence (AI), 2024.
F. El Aouni, K. Moumane, S. Bendaouya, A. Ait Laasri, and Y. Wahi, "Development of a JIRA Plugin for Tracking and Managing Time Allocations for Tasks and Projects," in 2024 World Conference on Complex Systems (WCCS), 2024: IEEE, pp. 1-7.
O. AlHarbi, R. AlMalki, and N. AlYousef, "Advancing Project Management Methodologies: An In-Depth Analysis of Jira in Managerial and Developmental Contexts," International Journal of Technology Innovation and Management (IJTIM), vol. 3, no. 2, pp. 40-59, 2023.
S. R. Keshireddy, "AI Driven Strategies for Efficient Project Tracking and Delivery in Software Engineering Management," Research Briefs on Information and Communication Technology Evolution, vol. 9, pp. 228-249, 2023.
E. Kalelioğlu, Implementing Atlassian Confluence: Strategies, tips, and insights to enhance distributed team collaboration using Confluence. Packt Publishing Ltd, 2023.
E. Stoeckli, C. Dremel, F. Uebernickel, and W. Brenner, "How affordances of chatbots cross the chasm between social and traditional enterprise systems," Electronic Markets, vol. 30, pp. 369-403, 2020.
M. S. Farooq, Z. Kalim, J. N. Qureshi, S. Rasheed, and A. Abid, "A blockchain-based framework for distributed agile software development," IEEe Access, vol. 10, pp. 17977-17995, 2022.
B. Familiar and J. Barnes, "Business in Real-Time Using Azure IoT and Cortana Intelligence Suite," Apress: Berkeley, CA, USA, 2017.
B. Saha, "Evaluating the impact of AI-driven project prioritization on program success in hybrid cloud environments," Available at SSRN 5224739, 2019.
Z. Nishtar and J. Afzal, "A Review of real-time monitoring of hybrid energy systems by using artificial intelligence and IoT," Pakistan Journal of Engineering and Technology, vol. 6, no. 3, pp. 8-15, 2023.
V. U. Ugwueze and J. N. Chukwunweike, "Continuous integration and deployment strategies for streamlined DevOps in software engineering and application delivery," Int J Comput Appl Technol Res, vol. 14, no. 1, pp. 1-24, 2024.
P. S. Chatterjee and H. K. Mittal, "Enhancing Operational Efficiency through the Integration of CI/CD and DevOps in Software Deployment," in 2024 Sixth International Conference on Computational Intelligence and Communication Technologies (CCICT), 2024: IEEE, pp. 173-182.
A. MUSTYALA, "CI/CD Pipelines in Kubernetes: Accelerating Software Development and Deployment," EPH-International Journal of Science And Engineering, vol. 8, no. 3, pp. 1-11, 2022.
N. Dimonte, "Centralized Monitoring Infrastructure on Cloud: An Open Source Approach," Politecnico di Torino, 2024.
T. J. Akinbolaji, G. Nzeako, D. Akokodaripon, and A. V. Aderoju, "Proactive monitoring and security in cloud infrastructure: Leveraging tools like Prometheus, Grafana, and HashiCorp Vault for robust DevOps practices," World Journal of Advanced Engineering Technology and Sciences, vol. 13, no. 2, pp. 90-104, 2024.
P. Chintale, DevOps Design Pattern: Implementing DevOps best practices for secure and reliable CI/CD pipeline (English Edition). Bpb Publications, 2023.
V. V. R. Boda and J. Immaneni, "Optimizing CI/CD in Healthcare: Tried and True Techniques," International Journal of Emerging Research in Engineering and Technology, vol. 3, no. 2, pp. 28-38, 2022.
S. Banala, "DevOps Essentials: Key Practices for Continuous Integration and Continuous Delivery," International Numeric Journal of Machine Learning and Robots, vol. 8, no. 8, pp. 1-14, 2024.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Scientific Research in Science, Engineering and Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
