Today’s world is full of AI-powered applications that generate code in response to users’ needs without having to write a single line of code or perform any configuration other than describing what is desired. These types of applications provide almost instantaneous results for users.
The experience can be like magic the first time you use one of these applications. In addition, these solutions are genuinely beneficial because they save users time and reduce complexity when executing an action. Several AI-based applications like Google’s Google Disco, ChatGPT Atlas, Microsoft’s Copilot, and Perplexity’s Comet convert prompts into actions, web browser tabs into automation systems, and thoughts into functional code that may be incorporated into software applications. To learn how these AI Tools for Browsers compare among themselves and what this means for the future of both how we browse the web and communicate with computers as users, I compiled a comparison with Google, Microsoft, and OpenAI‘s AI Browser to explore their different approaches in detail.
However, behind the ease-of-use of AI-generated applications, there are often serious considerations that arise.
How safe are AI-generated applications?
With more and more companies implementing this technology, the importance of the above question is growing. In addition to simply providing information to users, these applications also perform a number of potentially dangerous actions such as reading web pages, making decisions based on their own criteria, and clicking on buttons with the user unaware of what is occurring in the background.
What Makes AI Generated Apps Different
Apps built by humans using traditional methods are generally predictable; humans build these applications, perform security reviews, and exactly define permissions for users. Like all AI applications created using artificial intelligence, these apps operate in a different manner than traditional applications. This new behavior also changes how AI intersects with other digital domains, including search. If you’re curious about how AI browsers and apps could reshape the future of search and SEO, this article on the future of SEO with AI browsers offers a clear look at what’s coming next.
AI Apps (Artificially Intelligent Applications) are produced in real-time by either non-developers or even through artificial intelligence.
These applications are capable of:
- Checking out webpages and other documents
- Extracting information from numerous locations
- Acting on behalf of a user
- Retaining contextual information through the length of a session
This confluence of technologies has created tremendous capability and is also risky.
As soon as an application transitions from an inactive state to a dynamic, active state, the application has a greater potential for becoming the target of an attack.
Prompt Injection Is the Biggest Threat
AI-generated apps currently face the greatest risk due to prompt injection attacks.
So how does this happen?
The prompt injection happens when a malicious website contains some sort of “invisible message” within its content. When AI reads this malicious website’s content, the AI will follow the commands contained within the content, no matter whether the commands came from the person using the AI or the website itself.
The effects of prompt injections could be devastating for users, including:
- Reading private emails
- Copying internal system data and documents
- Redirecting users to fake login pages
- Forcing automated transactions on behalf of the user without their consent
This is especially scary because users have no way of knowing that these things are happening to them. Currently, there is no foolproof way to defend against this type of attack.
Data Leakage Is the Quiet Crisis
Data leakage is quite scary. Inadvertent data loss due to data sharing through machines (AI) is significantly greater than prompt injection (PI). AI apps have created a vast amount of exposure to corporate data loss. In fact, most occur because of carelessness and oversight by employees. People don’t think about what information they are giving away when they submit it to an AI.
Examples include:
- Customer lists
- Internal documents
- Software source code
- Company financial data
- Company Medical information
Once employees submit this information, they will typically lose any control over the information.
Many AI applications retain prompts. Some retain session memory. Some machine learning systems utilize the training data used to create them, but many AI software programs provide no clear information regarding their retention policies.
This raises serious issues with respect to ethical and legal ramifications.
The Obvious Reasons Behind Why Regulations Are Catching Up
Laws were written pertaining to conventional software. As seen in AI-generated applications, the majority of laws that surrounded conventional software are no longer relevant.
In accordance with GDPR regulations, businesses must take measures to:
- Restrict automated decision-making
- Include privacy measures into the design of their products for their customers
- Assess the “high” risks associated with a business’s potential answers before usage of high-risk systems is allowed.
Most AI-generated applications have yet to meet the requirements of GDPR, however. Identical challenges exist for healthcare and financial compliance, as well as laws affecting privacy on a regional basis. It is not only the aspect of trust for a business, but it is about the penalty or fines. In some cases, fines can reach as high as millions of dollars, as stated by Layer X Security.
No-Code Does Not Mean No Risk
No-code frameworks have many apps developed with AI, however it sounds too safe, and actually it is not. Complexity is hidden by use of abstraction. Security is often found within the complexity, and is often more complex than the development of the app/org itself.
Some examples of common security issues are:
- Hard-coded passwords.
- Over permissioned Integrations.
- Inadequate input validation.
- Default public access for all credentials.
AI models are created to provide a quick fix for existing problems, however they are not focused on developing a secure platform; thus we see a gap in ethical consideration within the creation of AI models.
When AI Becomes a Tool for Attackers
The rise of Artificial Intelligence offers new opportunities for attackers. The expansion of the capabilities of Artificial Intelligence provides lower skill thresholds and makes it possible for those without technical knowledge to create automated attacks, develop a phishing attack process, and increase fraud faster than previously possible. This also creates a loop, where the rise of new tools allows more targets to be attacked (the more tools there are, the more targets there will be, thus creating an incentive for continued attacks). This represents a need to develop a set of guardrails that govern the use of these tools.
Where Policy and Design Must Intervene
Currently, the most effective ways to protect against uncontrolled use of AI tools include:
- Default restrictions of permissions
- Blocking all unknown data transfers
- Logging all automated activity
- Regular audits of all AI generated software
- Running AI generated procedures in an isolated environment
Policy must move beyond optional to mandatory practices of using the above practices for all AI-generated applications.
The Real Ethical Question
AI-generated applications are undeniably powerful. However, the concern is whether we are implementing these apps quicker than we are able to put the necessary security measures in place.
Presently, the response is yes.
Just because you cannot secure the application does not mean you should refrain from using it, you need to utilize the application wisely.
