Beyond Code Assistance with GPT-4: Leveraging GitHub Copilot and ChatGPT for Peer Review in VSE Engineering

EasyChair Preprint no. 11192

Abstract

Most companies are Very Small Entities (VSEs), meaning they have fewer than 25 employees. Primarily domain specialists, these companies lack in-house expertise in important areas such as security and reliability engineering, process improvement, Quality Management (QM) and Systems Engineering (SE). VSEs struggle to adhere to Standard Operating procedures (SOP), and research has shown that contractual obligations to follow industry standards and best practices have little effect on actual engineering. This paper describes a case study that explored the potential of Large Language Models (LLMs) to support engineering best practices at a VSE by taking on the role of an expert peer in areas where the company had a skills gap. Aiwell, a Norwegian producer of building automation equipment, used ChatGPT, GitHub Copilot and GPT-4 to assess the quality of their system and stakeholder requirements. A GPT-4 foundation model with no additional training was given links to reference materials on requirements engineering produced by The International Council on Systems Engineering (INCOSE) and allowed to participate in discussions on the same digital collaboration platform as the human engineers. The study found that AI-assisted requirement reviews immediately and positively impacted the entire engineering process, supporting the feasibility of integrating advanced AI technologies in VSEs, even with limited training and resources. Participants highlighted the complementary nature of human intelligence and AI, where LLMs augmented human judgment through dialogue, leading to enriched engineering practices. Ethical and data privacy considerations also emerged as central themes, emphasising the need for proactive measures.

Keyphrases: Generative AI, Requirements Engineering, Systems Engineering, Very Small Entities

@Booklet{EasyChair:11192,
  author = {Roar Elias Georgsen},
  title = {Beyond Code Assistance with GPT-4: Leveraging GitHub Copilot and ChatGPT for Peer Review in VSE Engineering},
  howpublished = {EasyChair Preprint no. 11192},

  year = {EasyChair, 2023}}