Safeguarding Children From Immersive Tech Risks Over-Censorship

Efforts to protect children's safety on traditional social media platforms could unintentionally harm the burgeoning 3D world of augmented and virtual reality, according to a report released Tuesday by a Washington, D.C., technology think tank.

The report from the Information Technology & Innovation Foundation warns that legislative measures like the Kids Online Safety and Privacy Act (KOPSA), which has already passed the U.S. Senate and is under consideration in the House of Representatives, could lead to excessive censorship in AR/VR environments.

If KOPSA becomes law, AR/VR platforms might be pressured to enforce regulations as rigorously as conventional social media, potentially leading to unintended consequences, the report explained.

By granting the Federal Trade Commission (FTC) authority to determine what content is harmful on these platforms, the report argues, there's a risk of over-censorship. AR/VR platforms might either over-police content to avoid liability or preemptively censor material, which could include content important for children's education, entertainment, and personal development.

“One of our concerns with KOPSA is that it could pave the way for over-censorship by empowering the FTC to decide what qualifies as harmful,” said the report's author, Policy Analyst Alex Ambrose.

The Pros, Cons, and Pitfalls of AR/VR

The ITIF report highlighted that discussions around online safety frequently neglect AR/VR technologies. These immersive platforms promote social interaction and stimulate creativity and imagination, which are vital for children's development. Play, imagination, and creativity are essential components for healthy growth, the report emphasized.

However, the report also acknowledged the challenges in effectively mitigating the risks that immersive technologies pose to children. It noted that most current AR/VR systems are not designed for users under 13, leading to children navigating adult-oriented spaces. This exposure to age-inappropriate content can negatively impact children's mental and social development, fostering harmful habits and behaviors.

Addressing these risks will require a blend of market-driven innovation and thoughtful policy-making. The report pointed out that safety in the metaverse will largely depend on companies' design choices, content moderation practices, parental control tools, and trust and safety strategies.

While acknowledging the necessity of public policy interventions to address specific safety threats, the report noted that policymakers are already focusing on children's safety on "2D" platforms like social media. These regulations could extend to AR/VR technologies, but the report urged policymakers to first consider the ongoing safety efforts by AR/VR developers and ensure that these tools remain effective. When safety tools fall short, the report suggested that policymakers should concentrate on targeted interventions to address verified harms, rather than hypothetical risks.

"Most online services are striving to eliminate harmful content, but given the vast amount of content online, some will inevitably slip through the cracks," said Policy Analyst Alex Ambrose. "The challenges we face on current platforms, such as incitement to violence, vandalism, and the spread of harmful content and misinformation, are likely to persist in immersive environments."

"The metaverse will rely heavily on massive amounts of data, so it’s reasonable to expect that these issues might become even more prevalent than they are today," Ambrose added.

Designing for Safety

Lulham concurred with the report’s assertion that the design choices made by companies will significantly influence the safety landscape of the metaverse.

"In my perspective, the way companies approach online safety is crucial for establishing a secure digital environment for children," he remarked. "Given the current environment's inherent risks, companies have both the obligation and the capability to drive meaningful change."

He emphasized that user interface design serves as the first line of defense for safeguarding children. "Focusing on intuitive, age-appropriate designs can transform how children interact with online platforms," he noted. "By creating interfaces that naturally guide and educate users towards safer behaviors, we can greatly minimize harmful experiences."

Lulham also highlighted the critical role of content moderation. "The sheer volume of content necessitates a fundamental shift in our approach," he said. "While AI-driven tools are important, they alone are insufficient. I advocate for a hybrid model that combines advanced AI with human oversight to carefully balance protection and freedom."

Parental control tools, he argued, are often undervalued but essential. These tools should not be mere add-ons but integral components of the platform, designed with the same rigor as the core features. "I envision a future where parental controls are so seamless and effective that they become an essential part of family digital management," he added.

Lulham stressed that trust and safety strategies will set successful platforms apart from those that struggle. "Platforms that embrace a comprehensive approach, incorporating rigorous age verification, real-time monitoring, and transparent reporting, will set the benchmark for excellence," he declared. "Ongoing collaboration with child safety experts and policymakers will be crucial for companies dedicated to protecting young users."

"In summary," he concluded, "I foresee a future where 'safety by design' transcends being a mere catchphrase and becomes the core principle guiding every aspect of platform development."

The report underscored that children, as key participants in the metaverse, are pivotal to the market success of immersive technologies.

Balancing innovation with user safety in this emerging field will be a formidable task, the report acknowledged. It stressed that parents, companies, and regulators must collaboratively navigate the delicate balance between privacy and safety, while fostering engaging and groundbreaking immersive experiences.