As you peruse the livestreaming videos on Taobao, China’s foremost e-commerce platform, at the ungodly hour of 4 a.m., you’ll notice an oddly bustling scene. While most people are sound asleep, diligent streamers are busily showcasing products to their cameras and offering midnight discounts.
These streamers are not genuine human beings; they are AI-generated replicas of actual streamers. As technologies for crafting lifelike avatars, voices, and movements become more sophisticated and affordable, the popularity of these deepfakes has surged across China’s e-commerce streaming platforms.
Today, livestreaming stands as the dominant marketing avenue for both traditional and digital brands in China. Influencers on platforms like Taobao, Douyin, Kuaishou, and others can strike enormous deals in mere hours. Top-tier influencers can move over a billion dollars’ worth of merchandise in a single night, elevating themselves to the status of royalty, akin to major movie stars. However, for smaller brands, the endeavor of training and retaining livestream hosts and navigating the intricacies of broadcasting entails substantial costs. Automating the process is a far more cost-effective solution.
Since 2022, a swarm of Chinese startups and major tech corporations have offered the service of fashioning deepfake avatars for e-commerce livestreaming. With just a few minutes of sample video and an expenditure of $1,000, brands can create a human streamer clone capable of working round-the-clock.
The Journey from Deepfake to E-commerce Synthetic media has been grabbing headlines since the late 2010s, particularly when a Reddit user named “deepfake” started swapping faces in explicit content. Since then, the technology has evolved, though the fundamental idea remains unchanged: with the right tools, faces can be generated or manipulated to mimic specific real individuals, performing actions the actual person has never done.
This technology has mostly been notorious for its negative applications, such as revenge porn, identity fraud, and political disinformation. While there have been efforts to commercialize it in less contentious ways, it has typically remained a novelty. Nevertheless, Chinese AI companies have now discovered a new and seemingly successful use case.
Founded in 2017, the Nanjing-based startup Silicon Intelligence specializes in natural language processing, particularly text-to-speech technologies like robocall tools. However, Sima Huapeng, its founder and CEO, asserts that his company recognized AI’s potential as a livestreaming tool in 2020.
At that time, Silicon Intelligence required 30 minutes of training videos to generate a digital clone capable of speaking and acting like a human. In the following year, this time was reduced to 10 minutes, then three, and now, a mere one minute of video is sufficient.
As the technology has advanced, the cost of the service has also decreased. Generating a basic AI clone now costs approximately 8,000 RMB ($1,100) for a customer. For more intricate and capable streamers, the price can climb to several thousand dollars, covering a year of maintenance in addition to the initial generation.
Once generated, the avatar’s mouth and body movements synchronize with the scripted audio. While scripts were once authored by humans, companies now employ large language models to generate them. Human workers now primarily input basic product information, such as name and price, proofread the generated script, and watch the digital influencer go live. In more advanced versions, the technology can identify live comments and provide matching responses from its database in real-time, creating the illusion of active communication with the audience. It can even adjust its marketing approach based on viewer numbers.
These AI clones for livestreaming are trained on common scripts and gestures found in e-commerce videos. Huang Wei, the director of virtual influencer livestreaming business at the Chinese AI company Xiaoice, explains that the company maintains a database of nearly a hundred pre-designed movements. For instance, when human streamers say, “Welcome to my livestream channel. Move your fingers and hit the follow button,” they inevitably point upward, as that’s where the “Follow” button appears on the screen of most mobile livestream apps. Similarly, when introducing a new product, they gesture downward, indicating the shopping cart where viewers can find products. Xiaoice’s AI streamers replicate all these customary techniques, ensuring that spoken and body language are in harmony.
Xiaoice, spun off from Microsoft Software Technology Center Asia in 2020, has consistently focused on creating more human-like AI, particularly avatars capable of displaying emotions. Traditional e-commerce sites often come across as sterile product displays to most customers. Livestreaming, on the other hand, fosters emotional connections between hosts and viewers, enhancing product introductions. After a successful pilot with select clients in the previous year, Xiaoice officially launched its service for generating digital clones costing less than $1,000 this year. Like Silicon Intelligence, Xiaoice only requires human streamers to provide a one-minute video of themselves.
Major tech giants are also dipping their toes into AI-generated livestreams. Alibaba, Tencent, Baidu, and JD have all introduced variations of similar services this year, allowing brands on their platforms to craft their AI streamers.
Marketing companies that employ numerous human streamers have also taken notice of this trend. Foshan Yowant Technology, a leading livestream marketing agency, has announced a strategic partnership with Xiaoice. Silicon Intelligence has also established a joint venture with the company behind Viya, formerly China’s “livestream queen.”
AI-generated livestreams’ surging popularity has also garnered the attention of video platforms like Douyin, China’s version of TikTok. Douyin seems to be more concerned with transparency than other tech giants, insisting that all AI-generated videos on its platform must be clearly labeled as such, operated by real humans. The platform has consistently prohibited the use of pre-recorded videos as livestreams. AI-generated livestreaming, with no pre-recorded footage but also minimal real-time human intervention, treads the line of this rule.
The Chinese government has introduced several laws in the past two years governing synthetic media and generative AI, which would apply to their use in e-commerce streaming. However, the impact of government and platform regulations remains uncertain, given that the technology is still relatively new and enforcement is challenging.
Silicon Intelligence’s next objective is to infuse “emotional intelligence” into AI streamers, allowing them to react to circumstances emotionally, such as becoming sad in the face of abusive comments or happy when products sell well. The company is also working on enabling AI streamers to interact and learn from each other.
Since its inception, the company has held an ambitious and somewhat daunting goal: to create “100,000,000 silicon-based laborers” by 2025. As of now, the company has generated 400,000 virtual streamers, signifying a long journey ahead.
0 Comments