Can Recommendation Algorithms Promote Self-Harm or Even Suicide?

This story is as shocking as it gets for AI and recommenders. A father claims that while using social media, especially Instagram, his daughter was bombarded with inappropriate images and that “some of that content is shocking in that it encourages self-harm, it links self-harm to suicide.” The girl in question went on to commit suicide.

Instagram/Facebook’s response has been comprehensive:

We will not allow any graphic images of self-harm, such as cutting on Instagram – even if it would previously have been allowed as admission. We have never allowed posts that promote or encourage suicide or self harm, and will continue to remove it when reported.

Instagram appears to admit that recommendation algorithms can promote negative messages and harm vulnerable people. The girl’s father is absolutely clear on this point:

“We are very keen to raise awareness of the harmful and disturbing content that is freely ­available to young people online. Not only that, but the social media companies, through their algorithms, expose young people to more and more harmful content, just from one click on one post.”

How teens are bombarded with Instagram posts glorifiying depression and suicide

A TROUBLED teen carves into her bloodied arm, a suicidal brunette screams into the camera and a girl slashes her sweet smile with a blade. These are just a few of the sickening posts on Instagram t…

Changes We’re Making to Do More to Support and Protect the Most Vulnerable People who Use Instagram

At Instagram, nothing is more important to us than the safety of the people in our community. Over the past month we have seen that we are not where we need to be on self-harm and suicide, and that…

Instagram Bans Graphic Images of Self-Harm After Teenager’s Suicide

Instagram and its parent company, Facebook, will no longer allow images of cutting and will make nongraphic photos of self-harm, such as healed scars, more difficult to find.