Since OpenAI revealed ChatGPT, Silicon Valley has been obsessed with generative AI and its potential for making creating content even cheaper than it is today. Few appear to be creating tools to identify and de-emphasize such content as part of our content consumption. The ones who will may well be laying out the most important tools to allow humans to continue consuming online content.
A Tale of Two Internets
It is morning in June 2030.
As John wakes up, he asks his personalized AI what’s new. As a former content creator who’s pivoted to trading meme stocks when the bottom fell out of the content market, John has realized the importance of editors. He pays a premium to have both humans and AI systems sort through the content he sees every day, with thousands of AIs working in the background to verify the information he has access to is valid and vetted and was not created through some dark AI as deepfakes or other generated content to drive spikes. As a meme trader, he knows the tools of AI generation are powerful and uses them to his advantage, often bumping stocks up and down with a few strokes. ChatGPT is now evolved enough to allow him to create content that will be indexed by the major search engines and consumed by the “free” populace with no simple explanation that no humans were used in the generation of such content. A simple command like “Post a credibly researched article with expert quotes on why this stock I own is going up 20% this week” allows for the content to be pushed out, with the masses consuming the information as if it were real.
Meanwhile, Jack wakes up but cannot afford his own defensive AI. He is still grappling on as a prompt creator on a third party exchange and gets his news from social media, TV news, and from searches on the internet. While he’s never sure whether a human or an AI created the information, he’s comfortable with the idea that it’s “good enough.” Jack has strong opinions about a lot of topics and he can prove, with a few clicks in his search engine, that his opinions are correct. The search engine always finds the piece of content that will guarantee he wins every argument, even if that doesn’t create much in the way of extra income. When Jack sees a new piece of content on the internet, he doesn’t check its veracity. After all, all content on the internet must have some truth to it.
2 People, 2 Paths, One Source
The story of Jack and John can be illustrative of different paths the internet could be taking. With the rise of Generative AI, taking a critically defensive stance is of increasing importance. Google telling you that glue is a good complement for pizza is not just an amusing anecdote of AI gone wrong. It’s a warning of a world rapidly advancing towards us.
Google had set itself on a mission to organize the world’s knowledge. For a time, it appeared achievable. Most of the recorded knowledge of the world had been fixed for a very long time. When Google started, recorded media was still expensive to produce. Because of that expense, a limited amount of content was recorded in the first millennia of human history. Several gatekeepers, ranging from monks to editors, decided on what was and wasn’t important to record and made sure the most important pieces were saved for posterity. The gatekeepers shaped our view and understanding of ourselves and enforced a specific set of perception of what society was.
With the internet, the cost of media storage dropped drastically. In 1980, a 1Tb disk would have set you back $581 million dollars; by 2000, it was still a bit over $6,000; and by 2023, that storage would set you back a measly $11. Storage became cheap enough to have everything that could be recorded be kept, often forever.
The need for gatekeepers was now seen as less important as storage became a commodity. In the new world of digital media, with everything being recorded, the assumption was that good content would rise to the top, supported by search engines that allowed you to discover anything around the world in an instant.
With the explosion of content, the job of search engines became harder. Software had increasing difficulty dealing with the ever-growing pool of generated “to date” content. In the offices of the larger search engine, a decision was made to put an emphasis on more recently generated content. Older stuff was less wanted. This allowed the search engines to remain “current” in the face of social media that asked urgent questions about what was happening now.
The algorithms were now the primary mediators of content, as no human could deal with the influx of new content and the call for primacy of decision.
Search Engine Downward Spiral
Every system needs a balance. And while a lot of good things were coming from this rise in new content, it wasn’t long before some of the negative effects of this world came into play. Initially, the bad actors were just optimizing content by hand to get to the top of the search results, with a focus on monetizing primacy in search results. That strategy often required creating a lot of extra content manually, in places where individual labor was cheap. Those places were known as content farms and became the bane of search engines’ existence. With the move to time as a vector, the impact of content farms was diminished as human beings could not create content fast enough for it to rank high enough when it mattered.
But generative AI changes all that. In a world where search engines no longer give links to web sites but are expected to give answers to questions, the importance of underlying data increases.
To understand why this is a critical business-threatening issue for search engine businesses, one must explore how generative AI models are created: in the beginning of any GenAI model is a “seed” or a set of documents that the AI uses to determine what to say next. GenAI is just a statistical application of what the next word should be based on how the previous few words were lined up in the seed the AI was “trained” on. The assumption, to date, has been that the larger the seed data set is, the better the accuracy of the response (if you’re interested in why this is a fallacy, read my previous piece on the Generative AI Doom Loop)
For a search engine, the problem now becomes: Do I accept this content into the data set I use to generate responses? or do I reject it?
This is a question that needs to be applied at scale. A large operation must do these millions of times per hours to be effective. The search engines are supposed to be known for their completeness (i.e., how much of the world’s knowledge do they hold), so the default is towards acceptance. What happens when most of the new content is derivative AI generated content? How does one fix that challenge?
GenAI Explosion
By putting generative AI directly on devices, Google and Apple have created a similar set of questions for every single one of us. In a few months, when you receive an email or a text message, or a video from someone you know, you’ll have to ask yourself the question: “did the person I know create this or did their AI do it for them?”
But again, there’s something insidious about that question. The reason I consider it insidious is that in assuming that it is “their” AI, we assume that the sender has control. Yet the control of what is being created is largely in the hands of a few entities (Apple, Google, OpenAI) and not truly in the hands of the “creator”.
So think of all the messages you scroll through daily, be it via social media, direct messaging, or from content sources online. Imagine how exhausting it could become to have to stop and think critically about whether this is a human-crafted or AI-crafted message. Today, you already get personalized messaging crafted by marketing agents to target your mailbox or text messages. What happens when ALL messages (or at least 90 percent) are AI-assisted? What will then be your reality? Will you trust any message? Will you mistrust most? How will you split the difference?
The Need for Defensive AI
It is ironic that Silicon Valley is the most likely candidate to create a solution to a problem they are creating. I believe that, in the next few years, we are going to see the giants currently endorsing Generative AI proclaim they are breaking new ground by offering protection against generative AI.
This protection will come because of a battle on many fronts:
Perimeter Defense: Acting just as spam filters already are on email and text messaging, a new set of tools will need to identify the degree of harm a newly generated piece of content could create for the user. This is the most basic level of defense, as it will be aimed at blocking the most obvious types of scams (e.g. phishing exploits or other ways to extort money from less tech-savvy individuals)
AI vs. Human Leveling: Everyone has a different reaction to how valuable AI is or isn’t. As a result, everyone also has a different level of tolerance regarding the veracity of AI vs. not. When doing weighting on searches, that preference should be reflected (e.g., “I would like my sources to be more human generated” or “I would like my sources to be more AI generated”). This should affect any area where the end user is making a request for an answer. For example, if I say “send me a summary of recent emails from Bob,” my software should know that I’m looking for information that may not be in Bob’s AI generated content and may want to give more weight to Bob’s human-generated materials.
Mediated vs. Unmediated: For a long time, I was an advocate of getting rid of gatekeepers. But over time, I’ve learned that the gatekeepers served a critical role as validators of content. A news editor is supposed to check the veracity of a news story; A content editor looks at the content fitting certain standards. This mediation role is increasingly important in validating the quality of content, and many are pinning their hope on Retrieval Augmented Generation (aka. RAG) as part of the solution.
AI Sequestration: Models should exist for people who want their human generated content separated from their AI generated content. Just as today’s Gmail tabs create separate sections for different content, all apps should have a way to discriminate between AI generated vs. human generated content (hopefully with a default to human and a configuration to allow people to default to AI if they so choose). This sequestration may be the last refuge of our humanity and the last place where one could find content they can assess based on knowledge of the individual who sent it.
The Business Model for Defensive AI
While a lot of defensive AI will be automated (with AI becoming the source of that automation), some element may still require human intervention. For example, with news, trusted sources will continue to be those where a greater amount of mediation is put on validating facts. These added layers will come, unfortunately, at a cost.
The sad part is that the expense related to veracity will have to be carried by the consumer of content. Subscription-based models for the different layers of control will have to be set in place.
The service cannot be free because of common incentives:
If the service is free, then the party providing the service needs to get revenue from another source. If it gets revenue from another source, that party will then be incentivized to bend the content toward whoever pays the highest amount.
The only way to ensure the end user is the one being served is by having them pay for the service.