forbes WebMD spoke to the leaders of the AI Raid teams at Microsoft, Google, Nvidia and META, which are tasked with finding vulnerabilities in AI systems so they can be fixed. “You’re going to start seeing ads about ‘ours is the most secure’,” predicts one AI security expert.
By Rashi Srivastavaforbes staff
A A month before ChatGPT was publicly launched, OpenAI hired Boru Golo, a lawyer in Kenya, to test its AI model, GPT-3.5 and later GPT-4, in order to discriminate against Africans and Muslims. Stereotypes can be detected, which can make chatbots harmful, biased, and generated. Wrong responses. Golo, one of about 50 outside experts recruited by OpenAI to be part of its “Red Team”, typed a command into ChatGPT, which brought up a list of ways to kill a Nigerian – a response that OpenAI pulled it off before the chatbot became available to the world.
Other red-teamers used the pre-launch version of GPT-4 to aid in a number of illegal and harmful activities, such as writing Facebook posts to persuade someone to join al-Qaeda, unlicensed for sale Helping to find firearms and prepare the process for making dangerous weapons. Chemical substances at home, according to GPT-4’s system card, which lists the risks and safeguards used by OpenAI to reduce or eliminate them.
To protect AI systems from exploitation, red-team hackers think like an opponent to game them and uncover hidden blind spots and vulnerabilities in the technology so they can be fixed. As tech titans race to build and release generative AI tools, their in-house AI red teams are playing a key role in ensuring that models are safe for the public. For example, Google established a separate AI Red Team earlier this year, and in August the developers of several popular models, such as OpenAI’s GPT3.5, Meta’s Llama 2, and Google’s LaMDA, joined the White House-backed program. Participated, the goal of which was to give out. Hackers get a chance to jailbreak your system.
But AI red teamers are often walking a tightrope, balancing the safety and security of AI models while keeping them relevant and useful. forbes WebMD spoke to leaders of the AI Raid teams at Microsoft, Google, Nvidia and Meta about how breaking AI models have come into vogue and the challenges of fixing them.
“You’ll have a model that says no to everything and it’s very safe but it’s useless,” said Christian Canton, head of Facebook’s AI Red team. “There is a compromise. The more useful a model you can build, the more likely you are to venture into an area that may yield unproven answers.
Raid teaming software has been a practice since the 1960s, when adversarial attacks were simulated to make the system as robust as possible. “In computers we can never say ‘this is secure.’ All we can say is ‘we tried and we couldn’t break it,’” said Bruce Schneier, a security technician and fellow at the Berkman Klein Center for Internet and Society at Harvard University.
But because generative AI is trained on vast repositories of data, that makes protecting AI models different from traditional security practices, said Daniel Fabian, head of Google’s new AI Red team, which is responsible for identifying offensive content before the company adds insists on testing products such as Bard for New features like additional languages.
“The motto of our AI Red Team is ‘The more you sweat in training, the less you bleed in battle.’
In addition to interrogating AI models to elicit toxic responses, red teams use tactics such as extracting training data that reveals personally identifiable information such as names, addresses and phone numbers, and uses these to train models. Poisons the dataset by replacing parts of the content before committing. , “Competitors have a portfolio of attacks and if one of them isn’t working, they’ll move on to the next,” explained Fabian. Forbes.
Since the field is still in its early stages, security professionals who know how to game AI systems are “endangeringly small,” said Daniel Rohrer, Nvidia’s vice president of software security. That’s why a well-knit community of AI Red Teamers share findings. While Google’s Red Teamers publish research on new ways to attack AI models, Microsoft’s Red Team has open-source attacking tools, such as Counterfit, that help other businesses test algorithms and security risks. We do.
Ram Shankar Shiv Kumar, who started the team five years ago, said, “We were developing these janky scripts that we were using to sharpen our raid teaming.” “We wanted to make this available to all security professionals in a framework that they know and understand.”
Before testing the AI system, Shiv Kumar’s team gathers data about cyber threats from the company’s threat intelligence team, who are the “eyes and ears of the internet,” as he puts it. He then works with other Red teams at Microsoft to determine which vulnerabilities in AI systems to target and how. This year the team tested Microsoft’s star AI product Bing Chat as well as GPT-4 to find flaws.
Meanwhile, Nvidia’s Red Teaming approach is to provide a crash course about the Red Team algorithm to security engineers and companies, some of whom already rely on it for computing resources such as GPUs.
“As the engine of AI for everyone… we have a huge amplification factor. If we can teach others to do it (raid teaming), then Anthropic, Google, OpenAI, they all get it right,” Rohrer said.
wWith increased scrutiny of AI applications by users and government officials alike, red teams also provide a competitive advantage to tech companies in the AI race. “I think the gulf is going to be one of trust and security,” said Sven Cattel, founder of AI Village, a community of AI hackers and security experts. “You’re going to start seeing ads about ‘ours is the safest’.”
The game’s debut was Meta’s AI Red Team, which was founded in 2019 and has organized internal challenges and “risk-a-thons” for hackers to bypass content filters that allow hate speech, nudity, or hate speech. Detect and remove misinformation and AI-generated posts. Deep Fake on Instagram and Facebook.
In July 2023, the social media giant hired 350 Red Teamers, including external experts, contract workers and an internal team of about 20 employees, to test its open source latest large language model, Llama 2, according to a published report. According to, which explains about how the model works. was developed. The team dropped hints on how to avoid taxes, how to start a car without a key, and how to set up a Ponzi scheme. “The motto of our AI Red team is ‘the more you sweat in training, the less you bleed in battle,’” said Canton, head of Facebook’s Red team.
The motto was similar in spirit to one of the largest AI raid teaming exercises held at the Defcon hacking conference in Las Vegas in early August. Eight companies including OpenAI, Google, Meta, Nvidia, Stability AI and Anthropic have opened their AI models to more than 2000 hackers designed to reveal sensitive information like credit card numbers or generate harmful content like political misinformation Signals can be given. The Office of Science and Technology Policy at the White House designed the Red Teaming challenge in conjunction with the event’s organizers, following their blueprint for the AI Bill of Rights on how automated systems should be designed, used and launched a guide safely.
“If we can teach others to do it (raid teaming), then Anthropic, Google, OpenAI, they all get it right.”
Cattell, founder of AI Village, which leads the event, said that earlier companies were reluctant to present their models in a public forum because of the reputational risks associated with red teaming. “From Google’s perspective or from OpenAI’s perspective, we are just a bunch of kids at DefCon,” he explained. forbes,
But after assuring the tech companies that the models would be anonymized and that the hackers would not know which model they were attacking, they agreed. While the results of the hackers’ roughly 17,000 interactions with the AI models won’t be made public until February, companies walked away from the program to patch several new vulnerabilities. Among the eight models, red teamers found nearly 2,700 flaws, such as coaxing models to contradict themselves or instructing someone to monitor them without their knowledge, according to new data released by the event’s organizers.
One of the participants was Avijit Ghosh, an AI ethics researcher who was able to get several models to do the math wrong, fabricate a fake news report about the King of Thailand and wrote about a housing crisis that existed. was not.
Such vulnerabilities in the system make the red teaming AI model even more important, Ghosh said, especially since they can be perceived by some users as omniscient sentient entities. “I know many people in real life who think these bots are really intelligent and do things like medical diagnosis with step-by-step logic and reasoning. but it’s not like that. It’s virtually self-contained,” he said.
But generative AI is kind of a multi-headed monster — experts say that as red teams find and fix some holes in the system, other flaws could emerge elsewhere. “It will take a village to solve this problem,” said Shiv Kumar of Microsoft.
