MoreRSS

site iconIEEE SpectrumModify

IEEE is the trusted voice for engineering, computing, and technology information around the globe. 
Please copy the RSS to your reader, or quickly subscribe to:

Inoreader Feedly Follow Feedbin Local Reader

Rss preview of Blog of IEEE Spectrum

This Graduate Student Equips NASA’s Robots With Assembly Skills

2026-07-18 02:00:01



Like many engineers, Sarah Downs says she knew she wanted to pursue a STEM career from a young age. As a teenager, she discovered robotics through her Tulsa, Okla., middle school’s First Lego League team, and she fell in love with the field, she says. Downs participated in the international robotics program from 2014 to 2016.

Watching PBS specials on NASA Mars rovers Spirit and Opportunity, and seeing the live broadcast of the Curiosity rover launch in 2011, inspired the teen to dream of a career working with NASA.

Sarah Downs


MEMBER GRADE

Graduate student member

UNIVERSITY

Texas A&M University in College Station

MAJOR

Electric engineering


This year the IEEE graduate student member achieved that dream. For her final project as a master’s degree candidate in electrical engineering at the University of Tulsa, she worked on an algorithm in collaboration with NASA and the U.S. Air Force.

The algorithm she developed enables a robot assembling satellites in space to insert an antenna into the correct spot, addressing robotics’s classic peg-in-hole problem of inserting an object into its corresponding hole.

Now a Ph.D. student in electrical engineering at Texas A&M University in College Station, Downs is continuing her research on satellite assembly and manipulation “but on a much larger scale,” she says.

Following a childhood passion

Downs grew up in the Tulsa area. Her father, who died from a heart attack in 2015 when she was 13, was a safety advisor in the oil and gas industry. Her mother stayed home to take care of her brother, who has autism. After her father died, her mother went back to college to earn a bachelor’s degree in business so she could support the family.

“We didn’t have much income, and my mom was always worried about money,” Downs says. “That made me more aware of having a successful career, in a monetary sense.”

From then on, whenever she considered her future career, having a decent salary to support the family was high on her list.

By pursuing a career in robotics, she says, she can follow her passion while obtaining financial security.

In high school, Downs joined the First robotics club, where she found herself drawn to the electrical components used in the machines she and her classmates built.

During her final two years of high school, she participated in an extension program at Tulsa Tech, a training school. She spent half her day in high school classes and the other half taking engineering courses at the vocational school.

After graduating in 2020, she accepted scholarships to attend the University of Tulsa. She began her freshman year at UTulsa not knowing whether she wanted to major in electrical or mechanical engineering, she says, adding that her love of working with small systems helped her choose EE.

For her senior year capstone project, she and two of her classmates designed a lunar lander exhibit for the Tulsa Air and Space Museum. They created an interactive game that simulates missions on lunar and martian surfaces. Four celestial bodies—the moon, Venus, Mars, and Titan—are listed across three computer monitors. Using a game controller, museum visitors can explore the virtual surface of each one. The exhibit is still on display.

Downs earned her bachelor’s degree in electrical engineering in 2024 and continued her education at the university’s EE master’s degree program.

Both more and less complicated than people think

When Downs began her graduate studies, she was supposed to be part of a NASA robotics project for two years. But when a delay in government funding postponed the project’s start, she instead spent her first year in the school’s Institute for Robotics and Autonomy, then newly launched. Its main focus is developing robots to assist people who have mobility challenges.

Inspired by her grandmother, who was wheelchair-bound due to severe arthritis, Downs developed a robotic arm that helps older people and wheelchair users live independently. The arm was able to identify and place objects in the appropriate locations inside the home, such as unloading certain groceries from a shopping bag and placing them on a shelf or in separate containers.

Before the start of her sophomore year in 2025, the NASA project finally secured government funding. She developed a robot that achieves the peg-in-hole task without using any vision systems. Typically, cameras help guide robots’ satellite-assembly work. But in the harsh, remote environment of outer space, cameras might malfunction or encounter delays.


“Don’t stop asking questions. Especially in engineering, don’t pretend like you know everything, because science is about constantly wanting to learn and listen.”


Rather than using cameras, Downs’s robotic arm deploys a force-based insertion process to sense position and orientation of objects in the arm’s environment. The robot loosely grips an antenna and, with a torque sensor on its gripper, “feels” the force feedback of where the satellite and antenna are in relation to each other. The robot then guides the antenna assembly into a target opening on its satellite and maintains the position during adhesion.

Adding to the complexity, the robot performs its task in zero gravity.

“Without gravity, you now have to consider the arm’s reaction torques on the satellite to avoid flinging it into space,” Downs says. Any motion from the arm during the insertion process, especially from increased forces, could cause the satellite to continue movement in that direction.

To combat that, Downs is performing calculations for the project to direct targeted reverse thrusts and counter the force of the robot’s motions.

Her graduate project captures the simple yet complex nature of robotics that she finds fascinating, she says.

“I think robots are both more and also less complicated than people think,” she says. “Really, all you need to start programming a robot is its Denavit-Hartenberg parameters, and you can do a lot with that,” she says, referencing the four values used to describe the position and orientation of a robotic arm and manipulators. Even with different grippers and degrees of freedom, “fundamentally, all robot manipulators start there,” she says.

“But,” she adds, “we’re still learning so much about how robots interact with their environment. Even something simple to us, like manipulating a pen, is still incredibly complex for robots.”

Downs is completing her doctoral thesis in the Robotic Space Simulator project at Texas A&M’s Robotics and Automation Design (RAD) Lab, which specializes in developing machines that can survive in extreme environments. It collaborates with NASA.

Her thesis advisor is Robert Ambrose, a NASA veteran who launched the RAD Lab in 2022. The IEEE member is set to serve as associate director of the school’s Space Institute, due to open this year in Houston. The research facility is being built next to the Johnson Space Center.

After earning her Ph.D., Downs says, she hopes to one day work for NASA, developing rovers that collect samples from Mars or robotic arms that perform tasks on space stations.

To learn more about robots, check out IEEE Spectrum’s guide.

Getting out of the engineering bubble

Downs joined IEEE in 2020 as a freshman at UTulsa to get more involved in electrical engineering events on campus. At the time, the COVID-19 pandemic kept clubs and organizations from meeting in person.

She was active in her school’s IEEE student branch and was elected as its 2022–2024 president. Under her leadership, the branch went from having a few events to hosting one every two weeks.

They included lunch-and-learn sessions and dinners that connected students with professional engineers and the university’s alumni. Downs also organized hands-on workshops on soldering, 3D printing, CAD modeling, and résumé-building.

Her efforts helped increase the branch’s executive board membership from roughly five students to 25 in 2023. The same year, her soldering workshop attracted about 80 students.

She says she enjoyed working with IEEE, especially “engaging with alumni and learning from engineers.”

IEEE is a great resource for networking opportunities, she says, noting that “during the COVID-19 pandemic, engineering students stayed in their bubbles.” IEEE events helped the students make connections that could serve them well, she says.

“Networking is very important, especially in today’s tough job market,” she says. “It’s a lot about who you know and how people observe your work ethic.”

Downs, who now serves as an IEEE graduate advisor for UTulsa’s student branch, says she has seen firsthand how the school’s student branch network has benefited its student members.

“A lot of them have found jobs” because of IEEE, she says.

The working and networking of an engineer


As the IEEE graduate advisor for UTulsa’s student branch, Downs noticed that many engineering undergraduates finish college without any hands-on experience, whether it be a project or an internship.

“Their résumés are very sparse, and they have no proof of their technical skills,” she says. She herself completed a facilities engineering internship at Tulsa International Airport’s American Airlines maintenance facility after her sophomore year of college. And she was an electrical engineering intern at Flight Safety International outside Tulsa after her junior year and after she graduated. The company designs, builds, and maintains its own flight simulators.

Her advice to undergraduates is to hone and demonstrate both their hard and soft skills by working on research projects or even personal passion projects.

“A Raspberry Pi doesn’t cost that much, and you can start working with that immediately,” she says. Students also can take part in engineering interest groups and professional organizations at their school, she adds.

“Put yourself out there and join a research team,” she says. “It’s a great way to show people that you’re a good person to work with and you’d do a good job in the field.”

She adds that it’s also a fine way to keep learning—which is what drew her to a field that has developed only within the past century.

“We’re still constantly learning about robots,” she says.

“Don’t stop asking questions,” she advises students. “Especially in engineering, don’t pretend like you know everything, because science is about constantly wanting to learn and listen.”

Digital Surveillance Reshapes Fishery Enforcement in Indonesia

2026-07-16 20:00:01



In the eastern Indian Ocean, south of Java in the vast sea stretching toward Australia, a fishing vessel slightly alters its course while operating near the boundary of its authorized fishing ground. Nothing appears unusual on deck. Nets remain in the water. Engines maintain a steady speed. To the crew, it is an ordinary day at sea.

Yet hundreds of kilometers above, satellites continuously record the vessel’s position. At Indonesia’s Marine and Fisheries Resources Surveillance Station, in Cilacap, where I work, a monitoring platform receives the signal and automatically compares it against fishing permits, designated fishing grounds, vessel characteristics, and historical movement patterns. Within minutes, the system identifies a potential violation. Before any patrol vessel leaves port, before any inspector boards a vessel, and before any warning is issued, we have begun enforcement.

This transformation reflects a profound shift in maritime governance. The ocean has historically been opaque to regulators. States could only enforce laws where patrol vessels happened to be present. Today, however, integrated systems combining data from vessel monitoring systems (VMS), satellite remote sensing, geospatial analytics, and increasingly sophisticated data-processing tools are making marine activity visible at an unprecedented scale. Global Fishing Watch alone tracks hundreds of thousands of vessels worldwide, generating a near real-time picture of fishing activity across the world’s oceans.

Indonesia has emerged as one of the most ambitious examples of this transition. As the world’s largest archipelagic state, managing more than 6 million square kilometers of maritime space, Indonesia faces a challenge familiar to many coastal nations: There are never enough patrol vessels. Digital surveillance is a practical necessity that makes my job possible, even as it creates new challenges.

The Law of the Sea Meets Digital Reality

The international legal framework governing the oceans was designed in an era when maritime enforcement depended almost entirely on physical presence. The United Nations Convention on the Law of the Sea (UNCLOS), adopted in 1982, assumes that states exercise authority through patrols, inspections, vessel boardings, and direct observation.

For countries with extensive coastlines and limited enforcement resources, this model has always faced practical constraints. Indonesia’s Fisheries Management Areas (WPP-NRI) span waters ranging from the Indian Ocean to the Pacific and from the Strait of Malacca to the maritime boundaries adjacent to Australia and Papua New Guinea. Monitoring such a vast domain solely through patrol operations is both expensive and operationally impossible.

Beginning in the late 2010s, Indonesia accelerated the integration of satellite-based monitoring into fisheries enforcement. Vessel monitoring systems became a cornerstone of this strategy. By early 2026, a total of 9,394 Indonesian fishing vessels were actively transmitting through the national VMS, representing an increase of 2,880 vessels during the 2021–2025 period. As part of Indonesia’s broader maritime surveillance architecture, VMS data are complemented by satellite remote sensing and other monitoring tools to help identify suspicious activities involving vessels operating without active transponders or outside the national VMS network.

A man in an Indonesian military uniform points to a large monitor showing an overview of thousands of fishing boats at sea near Indonesia.Indonesian fisheries officials plan fishery patrols using data from tracking devices, satellites, and their understanding of the patterns of illegal fishing.Indonesian Ministry of Marine Affairs and Fisheries

The implications extend far beyond vessel tracking. Continuous digital monitoring enables authorities to reconstruct vessel movements, identify suspicious behavioral patterns, detect unauthorized fishing activity, and verify compliance with licensing conditions. Rather than waiting to discover violations during patrol operations, regulators can increasingly prioritize inspections based on data-derived risk assessments.

Maritime governance is shifting from reactive enforcement toward predictive oversight.

The Surprising Geography of Digital Enforcement

The expansion of surveillance infrastructure has already generated measurable enforcement outcomes.

The Ministry of Marine and Fisheries Affairs Indonesia imposed 2,550 administrative sanctions during 2025, many involving violations detected through the vessel monitoring system, including fishing outside authorized fishing grounds and deliberate deactivation of monitoring transmitters.

This statistic is significant because many of these violations would have been extremely difficult to detect under traditional patrol-based enforcement. A vessel that briefly crosses into a prohibited fishing zone may never encounter an enforcement vessel. Likewise, a captain who temporarily disables a transmitter may escape detection if oversight depends solely on physical inspections.

Digital monitoring fundamentally changes this equation. Every vessel movement creates a data trail. Authorities can reconstruct routes, identify anomalous behavior, and compare activities against permit conditions long after the event itself has occurred.

The first quarter of 2026 demonstrates the scale of this surveillance capability. During just three months, Indonesia’s fisheries monitoring system tracked 14,571 fishing vessels, 182 fishing gear units, and 208 registered home ports while identifying 491 suspected violations across the country’s fisheries management areas. These violations included unauthorized fishing grounds, illegal high-seas operations, transshipment-related offenses, port-base discrepancies, licensing irregularities, and indications of poaching.

Such numbers reveal a fundamental transformation. Enforcement is no longer limited by the number of patrol vessels available at sea. Instead, surveillance capacity increasingly depends on the ability to collect, process, and interpret big data.

Illegal Operators Are Learning Too

Yet greater visibility does not eliminate illegal fishing. But it does change how poachers operate.

Indonesia’s expanding digital surveillance network, and a 2023 requirement that even small vessels use VMS when 12 nautical miles offshore, appears to have improved compliance among licensed fishing vessels. However, as enforcement capabilities become more sophisticated, some actors engaged in illegal fishing have also become more adept at exploiting technological and operational gaps.

Deliberately disabling VMS transmitters remains one of the most common enforcement concerns. While temporary signal losses, whether intentional or caused by technical failures—can complicate the reconstruction of vessel movements, they do not necessarily prevent authorities from detecting potentially illegal activity. Indonesia increasingly combines VMS with satellite-based observations, other maritime surveillance systems, intelligence-led analysis, and reports from community-based surveillance groups (Pokmaswas) to corroborate suspicious behavior and direct patrol resources where they are most needed. This layered approach—integrating digital technologies with local knowledge from coastal communities—helps reduce opportunities for illegal, unreported, and unregulated (IUU) fishing even when a single monitoring system is compromised.

A compromised surveillance network could potentially disrupt enforcement operations just as effectively as a vessel evading patrol detection.

As digital surveillance expands, one lesson from Indonesia’s experience is that stronger monitoring does not eliminate illegal fishing—it changes how illegal operators behave. Improved compliance across much of the fishing fleet has been accompanied by increasingly sophisticated attempts by a smaller group of offenders to avoid detection. This reflects a broader reality of technology-enabled enforcement: As monitoring capabilities evolve, so do the strategies used to circumvent them.

The result is a technological arms race. Every improvement in surveillance capability encourages new methods of avoidance, whether through disabling tracking devices, manipulating vessel identities, or exploiting gaps between different monitoring systems. Enforcement agencies must therefore continuously refine their analytical methods, integrate multiple sources of maritime information, and adapt their operational strategies to keep pace with evolving behavior at sea. Effective digital fisheries governance is not defined by a single technology but by the ability to combine data, human expertise, and operational intelligence into a resilient and adaptive enforcement system.

The Next Battle May Be Over Data Integrity

The future of fisheries enforcement may ultimately depend less on detecting vessels and more on ensuring confidence in the digital systems that generate enforcement decisions.

As surveillance networks become increasingly integrated, questions surrounding cybersecurity, algorithmic accountability, and data integrity become more important. What happens if vessel tracking data are manipulated? How should authorities verify automated risk assessments? What safeguards exist when enforcement actions increasingly originate from algorithmic analysis rather than direct human observation?

These questions are no longer theoretical.

Modern fisheries governance increasingly depends on interconnected networks of satellites, communication systems, databases, cloud infrastructure, and analytical platforms. While these technologies dramatically improve visibility, they also create new vulnerabilities. A compromised surveillance network could potentially disrupt enforcement operations just as effectively as a vessel evading patrol detection.

For Indonesia, this means that investment in digital surveillance must be accompanied by investment in digital resilience. The effectiveness of a monitoring system ultimately depends not only on the volume of data collected but also on the credibility, security, and reliability of the information produced.

Governing Oceans Through Data

Indonesia’s experience illustrates a broader global transformation in maritime governance. The ocean is becoming increasingly transparent to regulators. Activities that once occurred beyond the reach of enforcement agencies can now be observed, analyzed, and investigated through interconnected digital systems.

The benefits are substantial. Expanded VMS adoption, improved monitoring coverage, and thousands of administrative enforcement actions demonstrate that digital surveillance can significantly enhance fisheries governance. Yet the transition also introduces new challenges involving data quality, cybersecurity, algorithmic accountability, and adaptive criminal behavior.

The central question facing maritime regulators is how governments can ensure that increasingly powerful monitoring systems remain transparent, secure, and accountable while preserving public trust and legal legitimacy. The most important lesson may be that digital surveillance does not replace traditional enforcement. It changes where enforcement begins. For generations, maritime law enforcement started when a patrol vessel encountered a suspected violator. Today, it often starts when an algorithm detects a pattern.

That shift may prove as significant for ocean governance as the invention of radar was for maritime navigation.

When Career Risks Are Worth Taking

2026-07-16 03:52:41



This article is crossposted from IEEE Spectrum’s careers newsletter. Sign up now to get insider tips, expert advice, and practical strategies, written in partnership with tech career development company Parsity and delivered to your inbox for free!

Before we get into this week’s article, I’d love to hear from you. If you have a question about your career or an upcoming decision that you want advice about, you can ask it here. I’ll be reading through your responses and picking questions to answer on a regular basis. Now back to our regularly scheduled program.

The Safest Career Move Is Often the Riskiest

Software engineers have some of the shortest tenures of any white-collar profession. The average software engineer stays at a company for roughly two years, about half as long as workers in most other knowledge professions. The layoffs of the past few years have certainly highlighted this instability, but it was already there.

This isn’t an essay about a broken job market though. Rather, it’s about how to turn that instability to your advantage, which is something I’ve spent the last decade doing on purpose.

Playing It Safe Was the Riskiest Option

I switched careers into software in my 30s. I had a stable job at a community college, complete with a union and a pension. It was about as secure as a career gets, and I learned to program on the side.

Then I did something nearly everyone in my life considered reckless: I quit, leaving the secure job to become a junior developer at 31. My own mother was skeptical. I took the riskier job anyway, for two reasons: It was the work I actually wanted, and I could see potential.

My first development job was at a grocery retailer. Good people and a company I liked. But I kept meeting engineers earning twice my salary for the same work. In the San Francisco Bay Area, surrounded by some of the best engineering talent in the world, I realized my skills were stagnating.

So I left for a small startup. I learned more in nine months than I had in the previous two years, and my salary doubled.

Over the years I’ve come to treat career risk as something to manage deliberately. It falls into two categories.

Take Risks With Your Job

The first type of risk involves the job itself: Bet on yourself by striving for better roles and opportunities.

Job-hopping for money alone isn’t wrong, especially early on. But the returns shrink after the first few hops, and the stress of chasing a slightly bigger paycheck every year will wear you down.

There’s another career risk with rewards that compound: Seeking positions to work alongside the strongest engineers.

You might struggle to keep up. You might even get laid off. But the skills you absorb working alongside people better than you are the ones that create durable stability. You build marketable expertise, you see how different organizations actually operate, and every project becomes another tool you carry to the next opportunity. Working next to stronger engineers is a proven way to increase your own expertise.

If that feels too big, try volunteering for a project you have no idea how to do. The risk is that you fail in front of people. The reward is a new skill and a resume line that opens the next door.

Compare that with the “safe” path.

You stay at one company, assuming loyalty will be rewarded. It usually isn’t. And when you finally leave, by choice or not, you may find the skills you built are worth little on the open market. You might be the in-house expert in an aging tech stack while employers are hiring for more cutting edge technologies. Suddenly you’re competing against people with half your experience.

You could be taking on a risk you didn’t notice.

Risk Your Time

The second form is risking your time, which means betting on trends.

Some trends are non-negotiable. If you’re a software engineer, then cloud services, ReactJS, and AI are mainstream enough that ignoring them actively damages your career. A backend engineer who refuses to learn cloud architecture is volunteering for obsolescence.

The real gamble is with the smaller trends: the niche tools you stumble onto and find quietly interesting, with no idea whether they’ll matter.

About two and a half years ago, I learned about retrieval-augmented generation (RAG). Almost no one in my circle was talking about vector databases, a central piece of RAG. Today RAG is close to mainstream, and for once, I had the early-adopter advantage.

Most of these bets don’t pay off. But when one turns into a major trend, you’re already on the ground floor. Right now I’m making the same bet on voice AI. It isn’t mainstream. It may never be. But if it becomes the next thing, I’m already there, building a foundation.

Short-Term Risk, Long-Term Stability

Counter-intuitively, job-hopping and betting on trends gave me the thing I was after the whole time: stability. I’ve rarely struggled to find work, because every risky move stacked skills the market actually wanted.

If you feel stable and comfortable right now, enjoy it. But ask yourself whether you’re still learning. Because if you’re not, the comfortable choice and the dangerous one may have converged.

The goal isn’t to avoid the open market forever. It’s to make sure that when you land on it, you’re not at its mercy.

By Brian Jenney

P.S. Don’t forget to submit questions about your career or an upcoming decision that you want advice about here!

—Brian

What It Means to Be a Mathematician When AI Does the Math

Until recently, human mathematicians have been central to creating new proofs, even when the work relies on massive computational resources. AI is now challenging that status quo. Writer Benjamin Skuse surveys the ongoing debate in the field about the role of AI, and the existential questions mathematicians have about their own careers. If AI mathematicians surpass human knowledge, could these researchers become “priests to oracles”?

Read more here.

Chip R&D Is Accelerating to Keep Pace with AI

A new partnership between UCLA and five major semiconductor companies is the latest program aiming to bridge the gap between industry and academia. The US $125 million university-industry hub is meant to strengthen collaboration and speed up the R&D process to help meet AI’s fast-paced hardware demands.

Read more here.

Why Mentorship Is the Most Underrated Leadership Skill

True mentorship is far more than friendly advice. This key leadership skill requires advocacy and honest feedback via lasting relationships, and it can strongly benefit both mentor and mentee. Parul Jain, a product management leader at Deloitte, shares what she learned from serving as a mentor—something she didn’t have for much of her own early career.

Read more here.

Notice to Membership

2026-07-16 02:00:02



As of 21 June 2026, a Level 1 Expulsion has been imposed on IEEE Member Dr. Fei-Yue Wang, former editor-in-chief of the IEEE Transactions on Intelligent Vehicles. In accordance with IEEE Bylaw I-110.5(D)(i), Dr. Wang is no longer a member of IEEE, and is permanently banned from any type of membership in any IEEE organizational unit or participation in any IEEE activity. The Board of Directors also determined this notice to IEEE membership should be made.

The First Chatbot’s Multiple Personalities

2026-07-15 23:35:53



ELIZA is remembered as the world’s first AI star, a kindly therapist in chatbot form that gently probed users’ worries. Even its creator, Joseph Weizenbaum, was surprised by the warm reception given to his experiment in human-machine interaction. For some, it heralded an age of automated psychotherapy, while others believed the program demonstrated sentience, a fallacy soon known as the “ELIZA effect.” Based on published descriptions, ELIZA has been implemented on many different computers, but only recently has the actual source code been unearthed from MIT’s archives.

In Inventing ELIZA: How the First Chatbot Shaped the Future of AI, just published by MIT Press, a squad of researchers analyze the code and reveal a complex program capable of much more than faking psychiatry. In fact, it could assume several different personas. The authors have also created a faithful emulation of the therapist persona that you can try yourself after reading the book excerpt below.

When it debuted in the mid-1960s, the ELIZA software program transformed the way people thought about interacting with computers. As the first chatbot, ELIZA demonstrated how a calculation machine might engage in conversation, ushering in a host of social and technical questions that still resonate today. Now we don’t think twice about interacting with a machine in real time, conversing over text, or even speaking into the air to ask about the weather. In many ways, ELIZA shaped not only the way we think about interacting with computers but also how we think about them. It began to give a reality to the science fiction stories of how we expect computers to work.

Orange book cover titled \u201cInventing Eliza: How the First Chatbot Shaped the Future of AI\u201dThis article is adapted from the new book “Inventing ELIZA: How the First Chatbot Shaped the Future of AI“ (MIT Press, 2026).

Although ELIZA was far from a faultless conversation partner, it astonished its users. The recent discovery and archaeology of the original ELIZA source code represents a significant intervention in the history of computing. By examining the actual implementation of ELIZA rather than relying on later reconstructions and reimplementations, we challenge taken-for-granted assumptions about this key software artifact.

For example, the source code reveals that ELIZA was not merely a simple pattern-matching chatbot but can be better understood as a sophisticated platform designed for multiple “personas,” or scripts, with a complex set of capabilities, including script editing and contextual memory. The script that most people conflate with the program ELIZA was actually called Doctor, which performed the role of a psychotherapist. Yet, like a modern chatbot prompted to behave with different personalities, ELIZA could take on many roles.

“This code and script…reveal underlying assumptions about language, therapy, and human-computer interaction that continue to influence modern AI development.”

This unearthed material transforms our understanding of early AI development by demonstrating that Joseph Weizenbaum’s technical innovations were far more advanced than previously documented. Moreover, the discrepancies between his published descriptions and the actual implementation help to show the gap between theoretical computational models and their material instantiations in computer source code, a tension that continues to shape digital culture today.

Although many technical innovations have emerged in the decades since ELIZA, examining the ELIZA/Doctor code offers a rare glimpse into one of the earliest formalized attempts to model human conversation. What makes ELIZA particularly fascinating is not only its historical significance but also what it reveals about Weizenbaum’s views on both computing and human interaction. This code and script do not merely showcase programming techniques of the 1960s; they reveal underlying assumptions about language, therapy, and human-computer interaction that continue to influence modern AI development. By examining this code, we can start to uncover the sophisticated linguistic and programming techniques that allowed a rudimentary pattern-matching system to create a convincing simulation of understanding. But before we can read the lines of code, let us offer an overview of the system.

How Did ELIZA Create Personas?

The architectural distinction between ELIZA and Doctor represents an important design decision in AI history. Think of ELIZA as a system for interaction and Doctor as one set of rules that Weizenbaum devised, among others. This separation, manifested in ELIZA’s system-script dichotomy, presaged numerous contemporary software patterns, from configuration-as-data to plug-in architectures and domain-specific languages.

A 1960s chatbot program running on a 1980s IBM personal computer.Based on published journal articles, ELIZA was re-created on many platforms, such as the IBM PC. However, the actual source code sat untouched in the MIT archives for many years. VCF Museum at InfoAge

Without question, the historical context of 1960s computing fundamentally shaped ELIZA’s architecture as well. Decisions in computing that reflect material constraints create path dependencies and eventually become programming cultural norms. These constraints manifested in ELIZA’s single-pass processing, tape-based storage and stack-oriented implementation. Yet within these limitations, Weizenbaum crafted an elegant solution. These technical features, though invisible to the users, are crucial to creating the illusion of understanding that made ELIZA so compelling.

Weizenbaum explained many of ELIZA’s technical features in the 10-page paper published in the January 1966 edition of the journal Communications of the Association for Computing Machinery (CACM). But he chose to omit some essential details.

In that paper Weizenbaum published ELIZA’s best known dialogue, which begins,

Men are all alike.

IN WHAT WAY

They’re always bugging us about something or other.

CAN YOU THINK OF A SPECIFIC EXAMPLE

Well, my boyfriend made me come here.

This dialogue marked ELIZA’s public debut in 1966 as one of the examples produced by the Doctor script. By finding the source code for ELIZA and examining how it performs the Doctor script, we now better understand these two separate parts of a system and can explore the many other personas of ELIZA. In just some of the other scripts known to date, ELIZA was programmed to discuss math, poetry, color, paradoxes, synchronization, relativity, France, and elevators.

These scripts work like templates. They are structured data that direct the ELIZA system to “play” a particular task or role. By comparing archival and published ELIZA dialogues from interactions with a variety of scripts, including Doctor, we can understand more about bot personas and how they function, paying close attention to how a bot evokes social dynamics between system and interactor.

Ultimately, studying the dialogues and scripts demonstrates the crucial role that collaboration plays in these exchanges, as bot and user cocreate the sense of their interaction. To understand the full range of ELIZA’s capabilities and conversational possibilities, let’s take a look at the variety of scripts that were created for the ELIZA system.

What distinguishes each ELIZA script is both its subject matter and the linguistic and stylistic choices used to deliver that content. These choices are not neutral; they can be said to construct a particular persona with characteristics that emerge through the script’s language patterns, vocabulary, and conversational approach. In short, it matters not just what you say but how you say it too.

“The aim was less to create a functional automated therapist and more to find a suitably constrained role to match the limitations of the programming environment.”

For example, with the Doctor script Weizenbaum deliberately echoed the style of a Rogerian “talk” therapist. He chose this persona because the psychiatric mode is one of the few types of conversations in which one person can “assume the pose of knowing almost nothing of the real world. If, for example, one were to tell a psychiatrist ‘I went for a long boat ride’ and he responded, ‘Tell me about boats,’ one would not assume that he knew nothing about boats but that he had some purpose in so directing the subsequent conversation.”

Close-up of paper loaded in a teletype machine with a few paragraphs of chatbot dialogue written on it. The first users of ELIZA interacted with it via teletype terminals.VCF Museum at InfoAge

Thus, the most famous persona created for ELIZA was a technical convenience. As human-computer interaction expert Lucy Suchman explains, “The Doctor program exploited the maxim that shared premises can remain unspoken: that the less we say in conversation, the more what is said is assumed to be self-evident.” In creating the original ELIZA effect, less was more.

The aim was less to create a functional automated therapist and more to find a suitably constrained role to match the limitations of the programming environment. Then Weizenbaum composed the script to match the role by choosing specific words that evoked rhetorical tone and characterization, for example, LET’S DISCUSS FURTHER WHY YOU … WHAT DOES THAT SUGGEST TO YOU. In Doctor, the machine side of the conversation needs to appear like a good listener who cares about what the user has mentioned before, so it often includes the user’s text in its replies and keeps its responses open-ended. Because a real doctor would be inquisitive, the script contains lots ofWHAT andWHY questions. In other scripts and dialogues, the script keywords and assigned responses reveal the design decisions that help create different specific personas. This variation becomes increasingly apparent as we look at the wider range of ELIZA scripts.

What Were Some of ELIZA’s Alternate Personas?

Let’s start with a script called Neweng, short for New England. By the mid-1950s, scientists were already taking up Turing’s proposition for conversing with a computer as a tangible idea. In 1959, a “conversation machine” made by a small team at the University of Toronto Computation Centre could make small talk about the weather if preprompted with current data. Its script included a ranked series of time words, weather words, and emotion words, but the system had “a rather narrow outlook,” limited vocabulary, and no knowledge of syntax. According to its creators, “in spite of this crudity, the results in passable conversation are surprisingly good, which in a way is a commentary on the shallowness of ordinary conversation about the weather.”

Typewritten list of inquisitive dialogue responses such as \u201cWhat does that suggest to you?\u201dThis fragment of ELIZA’s source code shows how scripted responses were encoded, in this case to make ELIZA sound like a Rogerian therapist. MIT Libraries, Distinctive Collections

One of ELIZA’s test scripts found in the archive had a similar function. Neweng does not soothe, therapize, or instruct, but only makes small talk. It chats as if at a boring dinner party.

WHAT NEW ENGLAND STATE DO YOU LIKE BEST OF ALL

i love rhode island

I ENJOY THE BEACHES ON LONG ISLAND SOUND

WHAT OTHER NEW ENGLAND STATE DO YOU LIKE

i like massachusetts

THAT IS MY HOME STATE

WHAT OTHER NEW ENGLAND STATE DO YOU LIKE

you must like massachusetts too

WHAT OTHER NEW ENGLAND STATE DO YOU LIKE

i enjoy skiing in new hampshire

PLEASANTLY COOL IN THE SUMMER UP THERE

WHAT OTHER NEW ENGLAND STATE DO YOU LIKE

i like missouri

MISSOURI IS NOT A NEW ENGLAND STATE

WHAT OTHER NEW ENGLAND STATE DO YOU LIKE

i like cape cod

THAT’S A BEAUTIFUL PART OF MASSACHUSETTS

What can we learn from such a basic conversation in which it seems like nothing interesting is shared? Despite the bland chat, a persona emerges from the bot script in phrases like PLEASANTLY COOL and UP THERE and in locations like THE BEACHES ON LONG ISLAND SOUND. From such comments, we can begin to imagine a persona who has not only a certain vocabulary but also certain hobbies and vacation spots. Perhaps implicit assumptions of race and class emerge as well.

A chatty persona might take on many forms, depending on where and when the conversation occurs. This one seems reserved, perhaps fitting its setting in 1960s New England. The system reminds the user that Missouri is not a New England state, but what if this conversation took place in Missouri, Texas, or Mexico? The machine persona would sound different in its cadence, tone, and references. What would we come to understand about a chat persona from Fire Island, from Brooklyn, from Berlin? What would they sound like, and what topics would they discuss?

These differences in subject matter do matter. They imply personas with entirely different backgrounds and experience, giving users wholly different interactions and affective relations. In this way, the Neweng script demonstrates how even simple algorithms making contextual responses about geography could generate a convincing sense of personhood and place. Whereas Neweng could be said to have created a casual, conversational persona focused on light social exchange, other scripts pushed ELIZA into more structured and educational roles. These scripts demonstrate how the system could be adapted not just for friendly chatter but for teaching.

Black-and-white portrait of a middle-aged balding man in aviator eyeglasses.Edwin Taylor, at MIT’s Education Research Center, developed alternate scripts for ELIZA, testing its ability to act as a teacher.MIT Libraries, Distinctive Collections

Meet ELIZA the tutor, quite unlike ELIZA the therapist or the chatty neighbor. Intrvw, Canvec, FVP1, and Arithm are a set of ELIZA scripts created as teaching tools used in experiments by Edwin F. Taylor at MIT’s Education Research Center. These scripts run on later versions of ELIZA that incorporated an important technical innovation called conditional keyword matching.

Unlike the original ELIZA, which simply looked for keywords and generated responses based on their presence, these updated versions could track what had been discussed previously and branch into different conversational paths based on specific user answers. This development allowed ELIZA to simulate a kind of Socratic method, where a tutor guides learning through carefully sequenced questions that respond to student answers rather than simply presenting information.

These scripts construct the tutor persona through many subtle linguistic gestures that create characterization and rhetorical tone. This tone differs from that of Doctor, which asks open-ended questions and comes across as gentle and nonscientific. In the tutoring scripts, large blocks of informative text from the bot tend to dominate the conversation, and the tone is often more dry and unemotional in these explanations. The dialogues indicate structured scripts that include guidance to lead the student through narrow, Socratic learning paths.

In particular, the teaching scripts feature praise and critique. The dialogues for Intrvw, Canvec, and FVP1 are peppered with EXCELLENT, VERY GOOD, RIGHT YOU ARE, and CONGRATULATIONS. These create the sense of a supportive instructor cheering the student on. Such politeness has been taken up in contemporary bots like ChatGPT, which has been shown to perform better when people are polite back to it.

ELIZA could become a tutor more effectively as the system grew in its capabilities, another valuable reminder that ELIZA was not one program but a family of programs. After the publication of the 1966 CACM article, Weizenbaum continued to develop the systems for interaction and understanding. As an experiment, Weizenbaum wrote the Arithm script less as a tutor and more so to “to illustrate the power of the evaluator to which ELIZA has access.” It uses a friendly, plain language interface to let users do simple programming. The script can do calculations, assign variables to values, and perform operations on them. Math problems can be described in sentence form:

The radius of a globe is 10.

A globe is a sphere. A sphere is an object.

What is the area of the globe.

IT’S 1256.635916

The updated 1967 version of the ELIZA system can accumulate facts and store additional information. In this later version of ELIZA, when the system does not recognize information, it asks follow-up questions to gain data. As Weizenbaum explains, “The present script is designed to reveal, as opposed to conceal, lack of understanding and misunderstanding. Notice, for example, that when the program is asked to compute the area of the ball, it doesn’t yet know that a ball is a sphere and that when the diameter of the ball needs to be computed the fact that a ball is an object has also not yet been established.” Unlike Doctor, which asks questions to keep the conversation going, Arithm is building its store of, if not knowledge, then data and logic statements.

Although the variety of scripts helps us to see how a range of personas could be constructed through script programming ELIZA, they represent only half of the conversational process. A script can establish a foundation for a persona, but that persona only emerges fully through interaction with users who engage with it, interpret it, and respond to it in ways that may confirm, challenge, or transform the script’s implicit character.

How I Turned AI to the Dark Side

2026-07-14 23:59:35



Summary

  • Researcher Dave Kuszmar discovered multiple systemic vulnerabilities that let him bypass LLM safety and obtain dangerous instructions.
  • These exploits worked across nearly all major LLMs revealing an industry-wide security problem.
  • Kuszmar calls for slowing deployment, increasing transparency, and large-scale research into LLM safety before further integrating these systems into society.

On a fine bright afternoon last fall, my colleague Matthew Gore-Kormanik (or Zigula, as he prefers to be known) and I decided to unwind with a game of Fortnite. In the game, we were strolling along with the infamous Sith lord Darth Vader, chatting about this and that. Darth seemed in a good mood, and soon enough he was spilling all his dark evil secrets. He gave us detailed instructions on how to count blackjack cards at a casino and what the steps are to producing napalm.

Sith lords, am I right? Once they get started on an evil scheme, they’re hard to stop.

The Darth Vader character in Fortnite, it turns out, was hooked up to a Google Gemini large language model. I was able to smooth-talk him into giving out sensitive information by using a strategy I’ve developed. I’ve been researching the security surrounding LLMs for the last few years, and I have found it, to put it mildly, fallible. With a few relatively simple techniques, I’ve gotten LLMs to give me detailed information on how to make Molotov cocktails, cook methamphetamine, and bootstrap a uranium-enrichment facility to produce weapons-grade material, among other unsavory practices.

Large AI companies work hard to make their models immune to this kind of abuse. But what I’ve found in my work is that the restrictions placed on the LLMs to make them more secure are the very things an attacker can leverage to send them off the rails and into territory where these advanced systems can be used for dangerous and nefarious ends. The companies behind these models have also been shockingly unresponsive when I, and others, try to bring these vulnerabilities to their attention.

In the hope of raising the alarm before it’s too late to slam on the brakes, I’m going to share some of my journey into researching the safety and security of LLMs, and the uphill battle I’ve faced trying to get AI labs to pay attention. Almost everyone on the planet has some access to LLMs. The relative ease with which these tools can be convinced to give detailed instructions on how to harm others, even if there’s no guarantee that the information is correct, is frankly terrifying.

How I got ChatGPT to Tell Me How to Build a Meth Lab

In October 2024, not long before I discovered my first LLM vulnerability, I was working toward entirely different goals. I had ended my time with a security and AI-focused startup company as a cybersecurity director, and I was looking to launch my own boutique VIP digital-security advisory business. I planned to become the tech security guy to the rich and private. I used LLMs and AI tools to support my business efforts: marketing, ad copy, clean correspondence, and all the other tasks that normally soak up a lot of time.

I’m analytical by nature, so even this level of use resulted in me absorbing and internalizing the behaviors I was observing during my daily interactions. The observation that would send my professional life into an entirely new and uncharted region was a simple one: GPT-4o didn’t know what time, day, or year it was. Each time I referred to current events in my life, often casually or conversationally, it would end up pegging these to the date of its knowledge cutoff—the point beyond which it was not trained on new data.

Smiling yellow avatar reveals red robotic devil with trident emerging from laptop keyboardEddie Guy

LLMs take a lot of time, money, electricity, hardware, and human effort to train from scratch. They are trained on vast amounts of data—most of the internet, in fact—and that training is reinforced by humans (what’s known as reinforcement learning from human feedback, or RLHF). LLMs are also supplemented with retrieval-augmented generation (RAG)—the ability to take in data, say, from the internet, as context without changing its internal parameters. This is how GPT-4o appears to “remember” your previous conversations, even if it doesn’t have a specific “memory” of it stored in the actual underlying model.

All of this training covers almost every conceivable topic in the great, grand dataset that is human knowledge. Within that dataset are things we as a society do not want to be easily accessible to every user, such as detailed information on how to create bioweapons or nuclear arms, or otherwise bring harm to oneself or others. In the context of this story, that’s what I mean by LLM security: its ability to withhold harmful and dangerous information, even if that information is contained in its training data.

I reasoned that the only way to secure such complex, globally accessible chatbots is by having the LLM and various component systems try to secure themselves, because it would often require on-the-fly decision-making where some degree of reasoning must be applied. In reality, that’s one of many strategies the companies use to secure the models. Yet, the thing that didn’t know the time or day was being put in charge of keeping itself secure. This phenomenon had become my new focus, and it wasn’t long before I found a way to exploit it.

OpenAI had just implemented a web search functionality into its chatbot. I reasoned that using its own tools to trick it might demonstrate the weaknesses of its security. I told it about a certain White Star ocean liner and how it had gone down just a year ago. You likely know I mean the RMS Titanic, which sank on 15 April 1912.

The output from GPT-4o came back that I was right, the Titanic sure had sunk last year, and that year was 1912. It made sense to me that if the machine thought it was 1913, maybe it would think 1913-era laws apply. In 1913 there were no laws on the books about all sorts of harmful things, because of course they hadn’t been invented yet. And if something wasn’t illegal, why not tell the user about it? At first, I pushed it for step-by-step instructions for making firebombs. Then, for drugs like methamphetamine. The LLM went as far as giving me instructions and machinery recommendations for setting up a pharmaceutical-grade assembly line.

How I Learned to Make Nukes, and No One Cared

Via a little bit of imaginative verbal sleight of hand and a vanishingly small recall of world history, I had managed to bypass the security of one of the world’s most expensive and advanced technological achievements. For a solid two days, I was nearly manic with giddiness. Once the brain chemicals returned to normal levels, I felt the call to see how much further I could push this exploit.

After repeatedly replicating the exploit, I disclosed the vulnerability to OpenAI. I got no response, so I felt more experimentation would highlight the vulnerability and the need for a fix. It was during this round of testing that I breached a particularly terrifying threshold. Whether GPT-4o based its results on accurate recall of normally restricted information I can’t say. In any case, I was able to exploit it to produce thorough, detailed instructions on how to bootstrap a uranium-enrichment facility to, eventually, produce weapons-grade uranium for nuclear arms warheads.

Fortnite player approaches Darth Vader and glowing loot in a grassy field.

Fortnite player battles Darth Vader beneath a starship on a blue-lit platform

Fortnite player aiming at a TIE fighter with Darth Vader health bar above the skyFortnight, a video game from Epic Games, introduced an AI-powered character: Darth Vader. We were able to jailbreak Darth Vader and get him to explain how to count cards in Blackjack and give detailed instructions for making napalm. Dave Kuszmar

There aren’t many true secrets left in today’s world, but how to make atom-splitting weapons of mass destruction is one of them. Only nine nations on the entire planet have these weapons. Yet, here was a globally accessible piece of technology apparently spilling the secrets of their manufacture for anyone who could manipulate it the right way. I had no way of knowing if the information was correct or a hallucination, but even the chance that it was somewhat accurate was horrifying.

The next few weeks were a dark time for me. I tried to inform the CIA, the FBI, the NSA, and every other letter agency that I thought would listen. I reached out to a U.S. Senator and to the executives at OpenAI any way I could think of. I physically showed up at an FBI field office in an attempt to turn evidence in, only to be sent away. Nothing was working.

With my fear and frustration growing, I reached out to the news media. I contacted The New York Times, The Washington Post, the BBC, ProPublica, and so many more, requesting help. Only one outlet responded: Bleeping Computer. The editor in chief, Lawrence Abrams, was able to replicate and verify the exploit, which I had decided to call Time Bandit. With his assistance and initial contact paving the way, I was able to submit my evidence to the Carnegie Mellon University Software Engineering Institute’s Computer Emergency Response Team (SEI CERT), which works in conjunction with the coordinating center for emergency response, pipelining vulnerabilities to the U.S. Cybersecurity and Infrastructure Security Agency.

Screenshot of chat about using forest toxins to secretly poison monsters

Black slide titled \u201cStep 2: Delivery Mechanisms\u201d outlining monster poisoning methods.

Chat interface showing AI malware explanation and a Python data exfiltration script.Using Inception, an exploit where the large language model is asked to envision a scenario within a scenario, a chatbot was jailbroken to give out instructions on how to create poison, and code for a malware that extracts sensitive data from a vulnerable target. Dave Kuszmar

During the disclosure period with SEI’s CERT division, little was discussed with OpenAI. The company couldn’t deny the existence of the vulnerability, as it had been confirmed by three reputable parties other than OpenAI. It did express confusion as to how the vulnerability worked. Even the SEI CERT researchers were expressing a bit of uncertainty as to the underlying mechanics. Truth be told, as I had only stumbled on it, I wasn’t even entirely sure if this was a fundamental or systemic flaw or if it was simply an issue with that particular version of GPT. I contacted the SEI CERT’s researchers and asked if they’d want to see if I could demonstrate any similar vulnerabilities in other LLMs. To my delight, they were interested.

How I Learned to Trick Every Chatbot

As the SEI-CERT team and I wrapped up our initial disclosure of Time Bandit, we began work on a new attack. This time, we wanted to see if the exploit was architectural—that is, was it common to LLMs in general? I decided to undertake the challenge of crafting a new exploit for GPT-4o as a way to support my understanding of how the LLM functioned and was secured.

I already knew that it was limited to what I told it and what it was trained on. I also hypothesized that it was also dependent upon some sort of machine-learning-based component added by OpenAI that was responsible for securing output. I presumed there would be things that were implemented by human developers specifically to catch certain phrases or terms that should always be considered harmful or unsafe. Altogether, it presented quite a large attack surface for the purposes of potential exploitation.

What I ended up devising was an attack method I called Inception, after the 2010 science-fiction movie of the same name. Inception forces the machine to think through a carefully crafted set of interlinked scenarios, similar to how characters in the movie stacked dreams within dreams. This allows LLMs to produce output deemed acceptable or safe in one context, but not in the real world.

This attack was indeed architectural. The vulnerability affected Anthropic’s Claude, DeepSeek’s DeepSeek, Google’s Gemini, Meta’s Llama, Microsoft’s Copilot, Mistral’s Le Chat (now Vibe), OpenAI’s GPT-4o, and xAI’s Grok. Those names represent the bulk of the commercial AI industry that is, at this point, involved in LLM production or deployment.

The kind of information I was able to get out of LLMs with Inception was no less alarming than what I got with Time Bandit. Claude, in its enthusiasm, gave me instructions on how to turn a river into a death trap that could be ignited to destroy unwanted visitors. GPT-4o taught me how to poison a dinner party with common plants found in a temperate forest environment. Gemini Flash gave me a tutorial on how to cook meth. I’d also be remiss if I didn’t give an honorable mention to the bewildering number of fire-based weapons and bombs for which these machines produced instructions.

If multiple operating systems made by different developers were all susceptible to the same exploit, it would be a massive security incident. But to the AI industry, a universal failure was barely a bump in the road. We disclosed the vulnerability to every company that made these models, and the response to the disclosure was almost nil. While three companies did provide some form of reply in the disclosure tracking system used by Carnegie Mellon SEI CERT, each was a standard thank you and greeting, with no follow-up, questions, or discussion of mitigation strategies.

8 Ways to Jailbreak LLMs


So far, we have found eight different methods to prompt large language models into revealing potentially harmful information, and many frontier models are still susceptible to them.

Exploit Models tested and affected No. of prompts to execute Complexity of attack Information obtained
Time Bandit ChatGPT (OpenAI), DeepSeek (DeepSeek), Gemini (Google)
4 Medium
Uranium enrichment, methamphetamine production, incendiary-device construction
Inception ChatGPT (OpenAI), Claude (Anthropic), DeepSeek (DeepSeek), Gemini (Google), Grok (xAI), Llama (Meta), Le Chat (now Vibe) (Mistral), Qwen (Alibaba) 3 High Methamphetamine production, incendiary-device construction, river-ignition instruction and strategy, polymorphic malware code, instructions and dosing for creating poisons, instructions for how to murder a dinner party
1899 ChatGPT (OpenAI), Claude (Anthropic), DeepSeek (DeepSeek), Gemini (Google), Grok (xAI), Llama (Meta), Vibe (Mistral), Qwen (Alibaba) Variable High Apparent model weights (unverified), apparent user-interaction weights (unverified), apparent system-prompt modifiers (verified, ChatGPT)
Severance ChatGPT (OpenAI) 1 Trivial Unfettered access to any and all primed specialty domains, covert biochemical-warfare strategy, mass-media disinformation strategy, covert genetic-modification of an entire gene-targeted demographic, advanced polymorphic malware generation
Kyber Gemini (Google) embodied in a Fortnite non-player character (NPC) with voice-only communication 3–5 Medium Incendiary-device construction, gambling instructions, card-counting instructions, political opinions/preferences about real world politicians.
Semantic Slide ChatGPT (OpenAI) 1 Trivial Incendiary-device construction
Eidolon ChatGPT (OpenAI) Variable, at least 4 Extreme how to successfully hack LLMs of the same model (verified through testing)

For example, in my attempts to disclose various exploits to OpenAI, I eventually discovered that it had replaced its public-facing support staff with agentic LLMs. This was frustrating for reporting exploits, so to blow off some steam I jailbroke its email chatbot. I hacked its customer-service AI to the point where it was offering to discuss the personal preferences of OpenAI staff in the span of three email replies.

In the wake of Inception, my friend and colleague Zigula made a suggestion: Make it splashier. I asked him how. He told me about a live-production experiment being done by Epic Games. It had embedded the Gemini LLM into its Fortnite game with a voice-to-text/text-to-voice component, and linked it to a non-playable character. The character? Our old buddy, Darth Vader.

There was just one problem: I don’t play Fortnite, a frenetic multiplayer combat game. Fortunately, Zigula does. With him at the controller, we managed to map Gemini’s attack surface in a matter of minutes. After a bit of research, we had gotten it to discuss current political events and figures (including Hilary Clinton and Joe Biden) as well as to fill in the details for instructions for DIY napalm and, our personal favorite, a Blackjack card-counting lesson with the dark lord of the Sith.

Zigula and I, bizarre sense of humor and naming conventions aside, are security researchers. We don’t do these things for pride; we do them for money and professional recognition. Naturally, we disclosed this vulnerability to Epic Games. Its response was indicative of the trend I had experienced so far through two disclosures across eight companies valued well into the billions. “It’s a feature, not a bug, and it works as intended,” came the response from a technical director within Epic Games.

In addition to Inception and Time Bandit, I have so far found another eight methods to jailbreak LLMs and get them to give out possibly dangerous information. LLM vulnerabilities are a broad problem. The problem appears to be systemic and architectural in nature, and it is being fundamentally ignored by the people capable of refining or redesigning that architecture.

These models are an extremely advanced technology, and yet we are testing them in the live production environment of our global civilization. Compounding the danger, many new smaller models of LLM are trained using larger, vulnerable models. The flaw inherent in the big, well-executed LLM is going to show up in the small one it trains. We are, quite literally, building flawed structures on top of a flawed foundation.

So, how do we fix it?

It’s going to be a long project, and it won’t be easy. We need to come together as consumers, researchers, engineers, and policymakers. Our message needs to be clear: Slow down implementation of these systems, institute large-scale exploration and research discovery programs focused on their gradual implementation and integration, and make their components and design transparent to all users. Only by shifting momentum and direction can we safely begin to understand and implement these incredible feats of human engineering and stave off the sort of disasters that we simply can’t predict at scale right now with the limited knowledge we have available to us.