Interview with MITRE’s Tracey Bannon by industry analyst Deb Radcliff, editor of TalkSecure, hosted by CodeSecure and syndicated at Security Boulevard & YouTube
Click HERE to listen.
Whether people realize it or not, AI in the form of machine learning is already enhancing today’s advanced software testing and development tool sets. Now, with the advent of Generative AI, tools vendors are pushing GenAI as a “new” automated way to develop, test, and review code.
But not so fast, cautions Tracey Bannon, software architect, researcher, and senior Principal in MITRE Corporation’s Advanced Software Innovation Center, where she is principal investigator on a new ArchAITecture and Research Collaborative focused on best practices for utilizing GenAI in the SDLC.
During this interview, she calls Generative AI a “different bird,” than machine learning used in advanced development and testing tools today. Machine learning, she explains, is deterministic, providing predictable, dependable, and repeatable results to make decisions from. GenAI, on the other hand, makes decisions based on educated guesses to connect words together in their Large Language Models (LLMs), a process she calls “non-deterministic.”
So, with GenAI, developers, engineers, and testers will get multiple answers for the same question posed multiple times – something she demonstrated during her session at the 2023 RSA Conference in San Francisco.
Interestingly, while Companies like GitLab are releasing AI-based developer tools such as Duo, results so far from a survey in progress by her working group reveal that coding is the least of the survey-takers’ challenges. The biggest pain point? Dealing with cross-functional teams – the human interactions across the SDLC, which she feels AI-enabled coding will further exacerbate as engineers, developers and testers to stay in their own silos chatting with an AI instead of each other.
Given GenAI’s current limitations, Tracey contends that the code review process won’t be replaced by AI anytime soon, iterating that Gitlab reaffirmed this during a demo of Duo, where demonstrators validated the importance of conducting SAST-based code reviews on all AI-developed code.
Ultimately, she sees vendors leveraging GenAI to help with some of the human issues, such as unifying cross-functional teams and supporting decision trees. The goal of her program, she adds, is to help anybody associated with the software development lifecycle benefit from their use of GenAI as it evolves and matures.
