What’s Lost and Gained in the World of Narrated Journalism

Audm, an app that turns long-form articles into audio versions narrated by professional narrators, has had a similar trajectory to many successful startups that have come out of Silicon Valley. To start, a passion for a somewhat niche yet serviceable aspect of the human experience: in this case, the founders’ shared love for media and journalism but the lack of time to read everything they wished to. Columbia University alumni Christian Brink and Ryan Wegner incorporated the company in 2015 and launched the Audm app in 2016. 

In the summer of 2017, Brink and Wegner took part in Y Combinator, joining the startup accelerator program that helped launch Airbnb, Reddit, and Instacart. The beta version of the app included ten articles from publications such as ProPublica, New York Review of Books, London Review of Books, Outside. Now, narrated articles from Audm have become an indispensable part of issues from publications such as The New Yorker, Rolling Stone, and The Atlantic

Audm is among several long-form-to-audio apps that have emerged around the world in recent years. Though their premises are quite similar—curated, professionally-narrated articles from renowned magazines and newspapers—each has its own idiosyncratic features to entice consumers. The Ireland-based company, Noa (short for News over Audio), narrates primarily shorter articles from titles such as Harvard Business Review, The Economist, and The Independent. Unlike most audio subscription services, Noa offers a free listening option with limited “listening balances” and a premium subscription. Similarly, Curio, a UK-based company whose founders include a former BBC strategist, curates content relating to finance, technology, science, and foreign affairs, as well as work from independent writers such as Nir Eyal.  

Audm diverges from its competitors in that it is backed by one of the largest, most-respected publications in the world; in March 2020, the New York Times acquired Audm, and Brink and Wegner along with the rest of Audm’s staff joined the Times Company. Wegner described Audm as “a natural fit” for the Times, which already “has strong emphasis on audio as a part of their future.” 

Narrated journalism exists along the same spectrum as audiobooks and podcasts but is distinctive in both its craft and form. Perhaps the most striking difference is the speed at which narrated articles are produced. Usually, Audm can produce an article’s narrated accompaniment in anywhere from a few hours to a few days. Companies like Audm can move this rapidly because many of its narrators have at-home studios (which was de rigueur even before the pandemic). Narrator Eric Jason Martin shared that “The Plague Year”— Lawrence Wright’s robust, humanizing analysis of the first year of the COVID-19 pandemic that has a running time of around three and a half hours—was sent to him and recorded in a day and a half split between Christmas Eve and Day. 1 The quick split posed no problem for Martin, who added, “I come from a live entertainment background and I was used to working through the holidays, so I said, ‘Hey, I’m available.’” The article came out in The New Yorker two days later on December 27th. 

Audm narrators, most of whom come from the audiobook industry, are the best quality English-language narrators working today. In addition to their many awards, they bring a profound appreciation for long-form journalism and a masterful skill for conveying both objectivity and certainty to their work. Perhaps this is because many narrators have extensive backgrounds in related fields. Martin is an Audible best-selling author, a director, and a voice actor; Emily Woo Zeller, another Audm narrator I spoke to, is a performing artist who has a background dubbing anime and voicing video games.

When casting narrator articles, Audm’s team looks for a narrator who can best capture the subject and the author’s message. These casting conversations can last hours or days, which Wegner describes as the most subjective part of the narration process. Most narrators have a wide range, but some narrators’ voices, styles, and experiences can best lend themselves to particular types of articles. As Zeller explains, “​​I grew up speaking Chinese, so that’s kind of a specialty that I have. They often will give me articles that are written about Chinese topics, but not exclusively.”

There’s a real sense of intimacy and immediacy when something is read aloud, especially lengthy or personal stories, which long-form suits so well. In September 2020, The New Yorker published Jiayang Fan’s essay “Motherland,” 2 a piece about how Fan’s struggle to maintain adequate care for her ailing mother during the pandemic while defending her family from a media firestorm in the Chinese press surrounding her social media posts. When approaching such sensitive pieces, narrators work tirelessly to truly embody the writer’s intentions. “As a narrator, I try to be as authentic as possible to both me and to the author,” Zeller explains. “I try to be able to deliver it as honestly as possible, so if it’s a very personal moment that she’s sharing and I try to be available for that and without adding anything extra.”

This deep engagement with the author is a true talent because writers and narrators rarely communicate directly. Contacting the author is typically infeasible given the quick turnarounds for audio drafts. Nevertheless, for some publications and pieces, the writer’s own voice is so essential to a story that writers themselves are trained to narrate. Zetland, a Danish long-form media company, has been producing its own audio versions of articles read by their writers since 2017. However, they work on a much smaller scale, only publishing three or four stories a day, underscoring their company’s commitment to not adding to information overload. 

Wegner affirmed that Audm has no specific target audience. Rather, apps like Audm attract all types of listeners: from commuters to people who can’t find the time to read through their piles of magazines to people who simply like to listen to the news. Audm’s ultimate focus on making journalism accessible to all has built a large audience of over 34,000 subscribers. 

Long-form journalism occupies an essential place in understanding our world. Its stories cannot be commodified into a headline on social media but abridge the multi-hour commitment of a book. For some, the past eighteen months of sheltering in place have finally provided the space and time to start reading or listening to long-form. The pandemic led to major subscriber gains for the largest publications in the country with The Wall Street Journal and the New York Times gaining 380,000 and two million subscribers between 2019 and 2020, respectively. During the past year, essays and articles like Wesley Morris’ “My Moustache, My Self” 3 and Jesmyn Ward’s “On Witness and Respair: A Personal Tragedy Followed By Pandemic” 4 that processed universal complexities—identity, loss, race—on the page alongside readers. Likewise, long-form reporting provided many with a fundamental understanding of multifaceted crises such as Ed Yong of The Atlantic’s reporting 5 on healthcare workers and the development of the vaccine or ProPublica’s guides to the 2020 election season, 6 how to avoid evictions, 7 and who has received PPP loans. 8 Some stories need significant space to be told; long-form gives audiences the opportunity to understand a facet of the world in its full depth. 

For those who can spare the time and money to invest in long-form journalism, the benefits are immeasurably worthwhile, but the expense is unrealistic for many. Audm, for example, costs $7.99 per month or $69.99 per year, roughly on par with most major streaming services. Major newspapers, which include audio or narrated content, can run a subscription cost of over $100 each year each. The indispensable information of these outlets thus becomes inaccessible to many everyday Americans, especially those who are being reported on: refugees, the homeless or impoverished, the disenfranchised (and the list goes on) cannot afford to read articles about issues that pertain to their own lives. Some publications attempt to partially remedy this by allowing several free articles before putting up a paywall. The New York Times releases one narrated article every Sunday on their hit podcast “The Daily” that has millions of weekly listeners. Still, the most pressing stories of our time are largely unattainable. This inequity only augments the mindset of the well-to-do watching the world from above while the rest just experience it. 

When I asked Wegner where Audm fits on the growing continuum of the audio industry, he described Audm as “bridging the gap between podcasts and audiobooks.” Perhaps this outlook applies on multiple levels. Not only does narrated journalism combine aspects of both mediums in terms of its length or accessibility, but also in its cost to consumers. But, Audm and similar apps’ appeal is still not straightforward convenience. Their content is selective and time-consuming. Narrated journalism does not aim to assuage our need for immediacy. Rather, it provides something that is often missing from our news consumption: space. Space to sit with ideas, to get out of our heads and into someone else’s. Just for a moment.


Encoded Bias: How Pretrial Risk Assessment Algorithms Can Entrench Prejudice

Content warning: This introduction of this article mentions suicide.

In 2010, a young Black teenager named Kalief Browder was falsely accused of stealing a backpack with no physical evidence to support the accusation. He was sent to Rikers Island with bail posted at $10,000. Browder, unable to pay his bail and unwilling to plead guilty, spent three years in pretrial detention. After three years, the charges were dropped. However, these years were mentally taxing on Browder. Aside from losing crucial years of adolescent development, he also faced brutal treatment and abuse at the facility and was denied the mental health assistance he needed. After attempting to take his own life on multiple occasions over the years, he tragically died by suicide in 2015. 

Browder’s story shows just how impactful pretrial detention can be on a person’s life—and why we must exercise the highest level of discernment when considering whether to detain defendants pretrial, and how to arrive at that conclusion.

Browder’s experience with pretrial detention centered around the cash bail system. 1 Simply put, defendants can only leave pretrial detention if they have enough funds to post bail. Today, the criminal justice system is starting to phase out cash bail and instead use algorithms to determine whether a person should be detained or released pretrial. Many states have opted to build their own algorithms, but there are also leading nationwide assessments, such as COMPAS (Correctional Offender Management Profiling for Alternative Sanctions).

But what do these algorithms entail? Pretrial Risk Assessments (PTRA) determine the likelihood of recidivism, or the chance that a person will re-offend. One such example analyzes an individual’s age, past actions of misconduct, neighborhood, and more to come up with three different risk scores: risk that the individual will be committed for any kind of crime, be convicted for a violent crime, and be absent from court. This means that a person with a prior criminal record would be rated higher risk than a first-time offender. 

The algorithm then converts these risk scores into a release condition recommendation. A higher score indicates the necessity for stricter release conditions and vice-versa. The judge can dismiss the release condition recommendation if they deem the score to be unfit for the individual. While the algorithms are pitched as purely objective and free from human bias, the reality is not as rose-tinted.   

Algorithmic bias stems from a variety of factors. Primarily, the accuracy of an algorithm is based on its training data. Flawed datasets produce flawed algorithms. America’s sinister history of profiling minority communities, preventative policing, and mass incarceration reflect in the data—and increase the chance that a person of color is detained pretrial. Inadequate racial representation decreases the nuance in risk calculations. For example, training sets for facial recognition technology predominantly feature White and male individuals, a skew that has led error rates to be 34% higher for darker-skinned females than lighter-skinned males. 2  

Additionally, recidivism “risk factors” identified by the algorithm include demographic, socioeconomic, family, and neighborhood variables, which, based on structural issues, also work against racial minorities. 3 For example, minorities have historically been subject to gentrification and discriminatory housing policies that ultimately impact their evaluation by risk assessment algorithms. Furthermore, having prior convictions perpetuates a dangerous cycle. Beyond adding an additional risk factor, criminal records affect employment prospects and, thus, income and housing, all of which feed into the other variables that make one more likely to be detained. 

Another area of concern is that of judicial discretion. In many implementations of risk assessment algorithms, judges are given the ability to overrule the scoring—combining human prejudice and AI bias to multiply the effect on racial minorities. In Kentucky, expanded judge discretion eliminated the marginal increase in pretrial release seen previously. A 2019 study found that use of this discretion was more commonly used to set cash bail for Black defendants than for White defendants. 4

These concerns are not just theoretical; practical implementation of these algorithms has yielded troubling results. In Lucas County, Ohio, which started using risk assessment algorithms in 2015, pretrial detention rates increased. Fresno County Jail adopted the Virginia Pretrial Risk Assessment and saw that the pretrial jail population increased from 1600 in January 2012 to approximately 2000 two years later. 5 

Aside from an increase in mass incarceration, the overall accuracy of risk assessment predictions is called into question. In Cook County, Illinois, fewer than 1% of people released pretrial were rearrested for a violent crime despite being rated as a high risk of violence by the Assessment. 6

As nearly every sector of our society digitizes, it is worth examining whether algorithms have a place in the criminal justice system. I argue that there should not be a blanket ban on such technology, but instead a step back into the development process to increase transparency and eradicate sources of bias. The objective of removing human judgment from the decision-making process is valid, but the implementation of this goal needs work, as well-developed algorithms can be used successfully. A recent study of New York City found that algorithms could “far outperform judges’ track record”; 42 percent of detainees could be successfully released without increases in failure to appear in court. 7

Algorithms have power, and it is up to us to use that power for good. That begins with increased oversight over their development and use. Developers must seek solutions to eliminate bias and skew in training datasets, coupled with regular evaluations on accuracy. Currently, many PTRA are “black boxes”; information regarding internal software components—such as the weightage of different risk variables—are protected as proprietary trade secrets. This lack of transparency leaves room for concerns about the root of algorithmic decision-making, and thus it is imperative that development is subject to public accountability.

Another solution may solve a related issue: a lack of diversity in the tech field. A Columbia University study found that AI bias is not just dependent on datasets, but also on the diversity of data science teams. While individuals of different genders and races tended to be equally biased, homogeneous teams compounded biases. 8 This contrast shows the necessity of increasing access to the tech field among underrepresented minorities; there are direct impacts on larger systems in our society. Beyond hiring diverse programming teams, however, it is also important to seek insight from a broad range of stakeholders—from activists who have lived experiences with incarceration to professionals in the criminal justice system.
Mass incarceration is undoubtedly one of the biggest domestic issues America faces—its impacts bleed over into socioeconomic disparities, public health crises, and more. Kalief Browder’s story is just one of too many. Even a single week in pretrial detention can mean job loss, eviction, loss of child custody, and exacerbation of medical issues. And, the implications of risk assessments are not solely limited to a single instance of pretrial detention; those who are detained are 1.3 times more likely to recidivate. 9 The simple fact that current algorithms exacerbate inequities in our criminal justice system should be enough to make us reconsider their use.


Floor or Stepping Stone: The Potential and Pitfalls of Minimum Basic Income

Forty years ago, the minimum wage guaranteed a decent standard of living, including food, rent, and transportation. Now, Americans working just as many hours at the same pay level find these essentials almost out of reach. In forty more years, as automation continues to take root, it may be considered a privilege to even have a job, let alone afford the essentials. At the same time, the spectre of the super-wealthy looms over our society. The continued consolidation and automation of jobs benefit the same select few magnates, and the increasingly complex tax system makes it difficult to redistribute that wealth. It’s clearly time to re-level the playing field.

The Cliff Effect

“People view economic success and wellbeing in life as … a product of choice, willpower, drive, grit, and gumption,” says Nat Kendall-Taylor, CEO of FrameWorks. 1 “When we see people who are struggling,” he says, those assumptions “lead us to the perception that people in poverty are lazy, they don’t care, and they haven’t made the right decisions.”

Unfortunately, such assumptions don’t capture the whole picture. There is no place in America where a family supported by one minimum-wage worker with a full-time job can live and afford a 2-bedroom apartment at the average fair-market rent. 2 Moreover, the Government Accountability Office reports that 70% of the 21 million people receiving Medicaid or SNAP benefits work full time. 3 As the gap between cost of living and actual wages has grown exponentially, federal assistance programs have become a crucial safety net for working families across the country. 

To qualify for assistance programs like Medicaid or SNAP, citizens must prove that they live at or below the adjusted federal poverty level. Given the political stigma surrounding investment in welfare programs, poverty levels are often set lower to target only those in serious need of assistance. For example, a family of three would need to make just $1,810 a month to qualify for SNAP, the equivalent of a 40-hour a week job paying just over $11. A $1 hourly wage increase could disqualify the entire family from their benefits. 

These stringent conditions imposed upon such subsidies and benefits prevent people from rising up the economic ladder by inadvertently trapping their recipients under an income ceiling. These benefits, such as controlled rent, food assistance, and free childcare, are too materially valuable, so families pass up opportunities for advancement in the workplace to keep them, thus “staying in the same place on the cliff” to avoid slipping off.

As shown above, a family of three in Virginia lost actual income when their earnings increased from $10,000 to $38,000. 4

Even while these workers might be staying put at the knife’s edge of benefits eligibility, the economy around them is constantly changing. Increased automation in factories has consolidated manufacturing jobs, while self-checkout systems in grocery stores now require employees to have some technical knowledge to assist customers. As market forces demand more skills or offer fewer hours, workers in the subsistence position find themselves stagnating or even falling behind. If they lose their job, they might be hard-pressed to find a new role. 

Solution: An Income Floor

Minimum Basic Income, or an unconditional income floor, would allow people to grow without the fear of a steady income being taken away from them. Looking beyond a monthly scramble for money to pay for rent and food, people will be given crucial breathing room to improve their lives more tangibly. An unconditional source of income would serve as a crutch for people to actively improve their situation through education and skill-development without fear of losing that crutch through this process.

With Universal Basic Income (the umbrella term for unconditional federal aid) gaining popularity over the past couple of years, experimental trials have allowed us to gather data on the efficacy of the policy. The Stockton Economic Empowerment Demonstration or SEED is the first mayor-led income floor demonstration in America. Launched in February 2019 by former Mayor Michael D. Tubbs, SEED gave 125 randomly selected residents $500/month for 24 months. The participants were at or below Stockton’s median household income level of $54,614. 5

While research spanning the full trial will be released in 2022, the early qualitative and quantitative data available is very positive. Notably, almost none of the money was spent on restricted products like drugs or tobacco, contrary to the widespread misconception about the usage of donated money. 

MBI recipients used the majority of their monthly payments on basic needs and consumer goods.5

Additionally, those who received the monthly benefit went from part-time to full-time employment at more than twice the rate of those without the benefit, and recipients were also healthier, showing less depression and anxiety. 

As one participant, Tomas, put it: “With the SEED money, there’s no need for the hustle. I have the time to actually sit there and actually take opportunities for myself, like if I felt like I wanted a new job or a different job, I was able to make it. I didn’t have to worry about it.”

Critics worried that people would be disincentivised to work when given unconditional handouts. 6 However, another study conducted in Finland supported the data in Stockton, refuting the critics’ assumption. 7 People on basic income worked an average of 78 days between November 2017 and October 2018, which was six days more than those on unemployment benefits. 

An umbrella review by Stanford’s Basic Income Lab could find no evidence to support the critics’ worries. 8 The report, which compiled and critically examined 16 reviews on Universal Basic Income, concluded that “Findings are generally positive that UBI type programs alleviate poverty and improve health and education outcomes and that the effects on labor market participation are minimal.”

Other skeptics of UBI have raised that an income floor could reduce the number of workers doing necessary minimum-wage/unattractive jobs. By reducing people’s desperation for income, workers would be given the choice to weigh their wages against the job’s personal fulfillment. This could lower the number of participants in necessary menial labour. 

Thankfully, automation can help fill in the gaps, increasing efficiency in the bottom rungs of the ladder without displacing people from jobs. This may sound futuristic, but it is well within our technological means to have a robotic foundation for the vast majority of minimum-wage jobs given their generally repetitive and algorithmic nature. Automated vacuums would clean our floors, checkout-free supermarkets 9 would make grocery shopping human-optional, and self-driving trucks could pick up our garbage, 10 all while we humans could spend our lives following more fulfilling, enjoyable careers.

For those who are permanently employed, Minimum Basic Income will not be sufficient to live on, as it’s not designed to replace jobs. However, with the economic progress driven by the lower-income population’s education, it is likely that new jobs will open up. Automation could also convert some demanding, full-time work into part-time and the income floor will step in to cover the gap in wages.

The Cost of a Minimum Basic Income

All this sounds utopian, but as always, one callously pragmatic question brings us crashing back down to reality. 

Who will pay for it?

If an income floor of $500 a month is given to all Americans, it would carry a hefty federal price tag of $2 trillion annually. However, this isn’t much more expensive than the current welfare system, which encompasses roughly $718 billion at the state and local level 11 and $1 trillion through federal Social Security. 12

To prevent a potential inflation spike due to this massive cash infusion, we need to employ policy and economic tools in redistributing funds across the population. Just this week, House Democrats proposed a new tax plan to offset spending for their own ambitious $3.5 trillion spending bill focused on the social safety net. The plan calls for a top corporate tax rate of 26.5% and a top individual tax rate of 39.6%. Moreover, the proposal includes a 3% surcharge on individual income above $5 million and an increase in the capital gains tax to 25%. 13 Clearly, targeted changes to the tax code could yield major revenue to fund proposals like Universal Basic Income. If legislators are actually committed to reducing income inequality, progressive taxation is the perfect starting point.

Moving funds from the one-percenters to the rest by using the top bracket of progressive tax wouldn’t make too much of a dent in the wealthy’s pockets. It would even boost the overall economy, since “based on standard fiscal multipliers established by Moody’s Analytics, every extra dollar going into the pockets of a high-income American only adds about $0.39 to the GDP. By contrast, every extra dollar going into the pockets of low-wage workers adds about $1.21 to the national economy.” 14In context, if we applied this progressive tax plan to America’s 650 billionaires’ income during the pandemic and moved those funds into a Minimum Basic Income program, the GDP would be boosted by $300 billion. UBI payments will cycle back into the economy through increased consumption, supercharging the economy rather than stagnating it.

Academics have developed alternative models of taxation to fund UBI proposals. If income tax was enacted as shown in the table above provided by the Roosevelt Institute, the respective income floors would be funded for (“child allowance” refers to $250/month/child). 15 This zero-sum money transfer across the population “turns out to be a positive sum in the macro-simulation, thanks to the increase in aggregate demand and therefore in the size of the economy.” according to the same report’s brief.

While UBI would be a serious investment, it’s clearly a feasible one. More progressive taxation would surface the majority of the funds for the program, and a few years of deficit spending would yield major gains to the GDP and America’s industries. 

Why now?

The pandemic has identified gaps in our public health infrastructure, our political system, and our society as a whole. The recovery could offer us a clean slate, a rare opportunity to refresh our policies on a deeper, more meaningful level, anticipating current and future economic issues rising with automation. 

The distribution of stimulus checks shows that raising federal funds for widespread financial aid is logistically possible. 16 People have also used the money responsibly, despite the fact that the cheques were given to higher income households too.

“For all the talk of revenge spending and pent-up demand for travel, you wouldn’t know it by seeing just 13 percent of stimulus check recipients indicating that any of the money would be spent on discretionary activities or nonessential items,” said Greg McBride, chief financial analyst at Bankrate. 17 

As stimulus cheque usage could simulate how Minimum Basic Income would be used in practice by the American population, this responsible spending pumping money back into the system from the roots is highly promising.

Finally, unlike COVID-19, automation won’t be a surprise. In the midst of this economic disaster, we realised we have to prepare for the next one, making it especially key to legislate on automation-responsive social policies like UBI now.

The Verdict

UBI isn’t just an opportunity for recovery. It’s an opportunity for economic regeneration — to limit inequality, improve quality of life, and prepare our society for a more digital tomorrow. After decades of regression for America’s working class, Universal Basic Income would be a proactive step towards bold, poverty reduction rather than a retroactive attempt to provide the bare necessities. We won’t just be rebuilding the economy post-pandemic, but renovating it.


Your Next Interviewer May Be a Machine: “Moneyball” in Corporate Hiring

Paul DePodesta revolutionized the world of baseball when he suggested using strictly quantitative data to select players for a team. Forget a player’s confidence level, his pitching style, and his past trades. Forget his body-mass index and the fact that he has an ugly girlfriend (which, according to scouts in “Moneyball” 1, means the guy has no confidence). Instead, focus only on one algorithmically curated number: the player’s quantitative value. 

DePodesta’s fictionalized character in the 2011 film “Moneyball,” Peter Brand, saw numerical value in players that no one else did. And it worked. He and Billy Beane, the Oakland Athletics team manager, put together a chaotic team for the A’s with an incredibly low budget and set a record by winning twenty games in a row. 

“People are overlooked for a variety of biased reasons and perceived flaws — age, appearance, personality,” Brand says in his pitch to Beane, explaining mathematician Bill James’s theory of sabermetrics: combining baseball and statistics. This empirical approach ignores a player’s personality, whether that be his horrible attitude at the plate or his stunning passion to improve himself every day. Sabermetrics would have turned away all those inexperienced, promising backyard players that they show in inspirational baseball movies. But the game of baseball for players is already cold — they wake up one morning wearing a Giants jersey only to find themselves flying across the country in a Cubs jersey that same night. 

A purely numerical approach to player recruitment makes sense given the high stakes of Major League Baseball. But what if we brought this same hiring approach to corporate America? What if hiring employees becomes an algorithmic process designed to select the best team solely based on numbers? What if your next interviewer was a computer?

The ‘Computerized Interview’

Given the increase in four-year college degrees and internships in recent decades, the number of qualified candidates for any job opening is on the rise. Forced to deal with a growing applicant pool, companies schedule up to three interviews per candidate and often fail to reach all candidates. To reach more candidates and speed up the interview process, recruiters have turned to computerized interviews. 

Most companies, such as Goldman Sachs, Hilton, Target, and Amazon, facilitate computerized interviews through prompt-based software, where candidates receive a question and record their answers. 2 While initially, human evaluators reviewed these responses, some companies have recently integrated AI graders that score according to a behavioral rubric. 3 At Great Western Bank, a regional bank based in South Dakota, applicants walk through an interactive assessment where they handle customers’ requests for money withdrawals, account openings, and more. They interact with difficult requests and angry customers. Their responses are recorded and scored later by a bank official. The Bank reported that not only did the process reduce “useless” personal interviewing, but also ensured that those hired were less likely to be fired or quit within 90 days. The creative nature of the interview gave candidates a more accurate glimpse of the job for which they were applying.4 Hilton credited the process to shrinking their hiring process from six weeks to five days. 5

Introducing AI Scoring

Initially, AI scoring for computerized interviews was introduced as another method to save time for recruiters by directly presenting them with an evaluation of candidates. Platforms such as HireVue have taken it a step further, claiming that automated scoring also reduces implicit biases that humans may act on when hiring. Their mission is to closely follow the 1978 Uniform Guidelines on Employee Selection and “aid in the achievement of equal opportunity for everyone regardless of gender, ethnicity, age, or disability status.” 

Humans have prejudices apparent only after looking at hiring statistics. According to Freakonomics and a field experiment on labor market discrimination, a white-sounding name on a resume is 50% more likely to get a callback than a black-sounding name with an identical resume. 6 This is analogous to older baseball players being overlooked by baseball scouts as “out of their prime” and slow at running the bases, regardless of their true performance. 

HireVue wishes to eliminate such bias in hiring via fair algorithms…but what constitutes a “fair” algorithm? Nathan Mondragon and his team of Industrial-Organizational psychologists at HireVue believe that a structured, algorithmic interview is one of the best and fairest predictors of job success.7 They pledge to use diverse datasets, hire a diverse team, and consistently review their algorithms for bias. Mondragon offered some insight into the algorithm, stating that a 30-minute interview with around six questions can provide 500,000 data points used in the calculation of a person’s score. For instance, the AI may assess how a candidate’s face moves to determine their level of excitement about a task.

“Those ‘Facial Action Units,’ Mondragon said, can make up 29 percent of a person’s score; the words they say and the ‘audio features’ of their voice, like their tone, make up the rest.

False Promises

The technology hasn’t been around long enough for its statistics to be published, but problems have already arisen. In 2019, the nonprofit Electronic Privacy Information Center filed a complaint against HireVue, claiming that their attempt to have AI score video interviews led to “unfair trade practices.” In response, HireVue banned facial expression monitoring during the interview process. 9 

Facial monitoring bias is already a hot topic in the news as we know that facial recognition is accurate less frequently for people of color than it is for their white counterparts. 10 Several other components of HireVue’s technology have been marked as features that could use improvement (understanding accents, response length, word diversity, etc). The allegations of bias aren’t isolated to HireVue; in fact, they’re criticisms levied at scoring algorithms across sectors. Critics have long questioned whether bias-free scoring is feasible at this stage of technological development. In 2015, Amazon was caught in a scandal when its intelligent hiring program unfairly rated female job applicants. Due to the male-centered data of 10-years of employee history the algorithm trained on, the program penalized resumes with the word “women’s” and applicants who attended all-women’s colleges. “In effect, Amazon’s system taught itself that male candidates were preferable… Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said.” 11

Black Box

Although hiring algorithms have become more diffuse, they remain a black box. Technologists can control for bias in one area and accidentally develop a new bias in the algorithm. AI scorers deprive candidates of crucial feedback, as candidates cannot ask for areas of improvement after an interview. Moreover, even when companies detect bias, they’re reluctant to share their dataset in fear of divulging trade secrets. 

“HireVue offers only the most limited peek into its interview algorithms, both to protect its trade secrets and because the company doesn’t always know how the system decides on who gets labeled a ‘future top performer.’” 12

If we continue to use these black box algorithms, our best path is a completely representative dataset collected only of factors that judge an individual’s capability, not his or her background or identity. That way, no matter which factors the algorithm latches onto, it will be fair.

But is this what we need? To hire employees without understanding where they come from and who they are? Are we moving towards a reality where one’s abilities are all that matter, and not one’s determination, enthusiasm, and grit? Will we automatically choose the employee with a stellar resume who came from money over the candidate who rose from poverty to success simply because he can speak more about his past work experiences? The algorithms take into account enthusiasm and word syntax, but these factors do not encompass one’s overall life story, passion, or true love for the job. Enthusiasm can be faked and rhetoric can disguise a lack of expertise. Ultimately, a computer cannot yet connect with a human on a non-quantitative level. That personal connection is what so many people rely on when interviewing. 

“I’d personally do better when I can see the person I’m interacting with,” shared a recent interviewee for a summer internship which used HireVue, “If you’re awkward in front of the camera, that doesn’t mean you’ll be awkward in the workplace or in front of an interviewer.” The applicant noted that she felt slightly uncomfortable that she had to stare back at herself instead of being able to watch her interviewer’s reactions to her responses.

The Future of Computerized Interviews

The promise of AI scoring is incredibly appealing: humans are biased, so if they are taken out of the equation, the process will no longer remain biased. But, that’s not quite how it works. The historical data fed into the system and the algorithms written by humans mirror the implicit biases prevalent in our society. Companies such as HireVue do their best to ensure their algorithms and data are unbiased, but that technology simply does not exist yet. To this day, facial recognition algorithms often misidentify black faces. Courtside algorithms assign higher recidivism scores to people of color. Women are penalized in the hiring processes for being women. Candidates rejected by computerized hiring systems do not know why. We aren’t at the point where we can trust technology to be perfectly fair, and yet HireVue interviewed more than 5.6 million people around the world in 2020. 13

This number will continue to rise. More companies will turn to automated hiring either out of convenience or in pursuit of a more ethical method of evaluation. Data scientists and psychologists should continue to curate AI, but in the meantime, society should have a broader conversation about which aspects of our lives we should automate. Data collection — automate it. Product testing — automate it. Medical diagnoses — automate it. But hiring employees? For machines to decide whether applicants can thrive at a company, computers need to develop the cognition to understand humans, their stories, passions, and aspirations. Until then, we can use them to augment, but not replace human interviewers.

Computers may augment human interviews by facilitating immersive assessments which can be objectively graded. A computer program could assess a cashier’s ability to make change or a grocery bagger’s ability to pack a bag. For more in-depth behavioral interviews, prompting software could be used for biographical questions, allowing applicants to quickly recall details from their resumes while saving in-person interview time for more story-based questions. 

Secondly, in order to reduce the responsibility of the AI, automated scorers should not make definitive decisions. For the most part, the technology can be used for preliminary screening to weed out unfit applicants who fail to complete the interview. For companies that hold two to three interviews per candidate, the computerized interview can handle the first round and provide a preliminary judgement. This still cuts down time and cost. 

Finally and arguably most importantly, there needs to be transparency. Even if the technology is 100% fair, until there is transparency, there will always be doubts in the mind of interviewees. Companies should communicate feedback with interviewees, perhaps via a rubric scored by the algorithm. Last year, Illinois’s governor signed a first-in-the-nation law requiring employers using AI-based hiring to explain “how the artificial intelligence works and what general types of characteristics it uses to evaluate applicants” in addition to obtaining consent. 14

Taking a “Moneyball” approach to hiring may make it easier to navigate large pools of applicants. But, the lack of transparency and allegations of bias associated with these tools make it imperative that we implement them carefully. Until we fully understand these technologies, computerized interview tools should augment, improve, and explain.


The Media’s Role in Shaping the Narrative Surrounding Athletes’ Mental Health

On a quiet Tuesday afternoon, I sat at my computer with the TV buzzing in the background. My dad was working in the rocking chair, and my mom skimmed a fashion magazine from the sofa. It was Olympics season—the one thing that gathered my family in the living room where we could all tune in to the fun and games.

The words “Simone Biles,” “showstopper,” and “must-see TV” drew my eyes to the bigger screen, where I calmly—then disbelievingly—watched Biles’s “very uncharacteristic” vault performance. 1 “Wow…” My reaction echoed that of the commenter’s. “Was that serious?” came from my mom who had tossed aside her reading. My dad, the most athletic but the least engaged sports fan out of the three of us, missed the six pivotal seconds in gymnastics history. 

“What happened?” 

Exactly—what happened? Simply put, Biles had made a mistake on the vault, executing 1½ twists instead of the planned 2½ twists and stumbling forward on the landing. 2 Normal, especially in high-pressure and high-technique situations, right? Except it wasn’t simple or normal, not for the most decorated gymnast of the generation, and certainly not for the media, which opened fire on her Olympic ‘fail.’

This isn’t the first time digital media—including online news sources, social media, and other internet-based platforms—has bashed professional athletes for a less-than-satisfactory performance. Titles such as “EPIC athletic fails” are constantly trending on YouTube, and nearly everyone can point to one famous (or is it infamous?) mistake in sports. But, widespread speculation and attention on the mental health reasons behind Biles’s mistake is relatively new in the sports world. This spotlight is paving the way for a broader conversation on the importance of mental health for athletes and for all people in general,

Traditionally, media reports of athletes have focused entirely on physical wellness and ability, fostering an absolute correlation between physical health and athletic performance. The media narrative around sports injuries has been heroic and sympathetic—from reporting Tiger Woods’ historic 2008 U.S. Open win on a broken leg to detailing Kobe Bryant’s Achilles tear in 2013. 3 4 Physical injuries are recognized, analyzed, and normalized. Rehab is considered a commonality and an integral aspect of playing and watching sports. Until very recently, the state of an athlete’s physical wellbeing was almost synonymous with performance excellence and overall wellbeing. 5 Mental health, consisting of our “emotional, psychological, and social well-being,” is an important factor of our overall health, yet its presence is almost nonexistent in the public realm of sports. 6

This year, instances of elite athletes withdrawing from competitions due to mental health reasons have ignited the debate surrounding the definition and importance of mental well-being for athletes. 

In May 2021, tennis superstar Naomi Osaka refused to participate in the French Open press conferences and withdrew from the tournament due to mental health concerns. Her move initiated a wave of media criticism and peer support, broaching the long-ignored conversation about the importance of athletes’ mental health. After declining to face the media which gave her “huge anxiety,” Osaka was fined by the tournament’s organizers who refused to relax the traditional press duties. 7 In a statement withdrawing from the French Open, Osaka confessed to experiencing “long bouts of depression” since the U.S. Open in 2018 and specifically cited media scrutiny for making her feel “vulnerable and anxious.” Osaka’s assertive actions to prioritize her mental health, however, were labeled narcissistic, with some online platforms even calling her withdrawal due to mental health reasons “an excuse.” The wave of outrage following Osaka’s boycott reveals exactly the problem with the media—that its bias, subjectiveness, and ambiguity compounded by its massive influence magnifies every action and word of the victim, damaging one’s self-esteem and mindset.

The wide spectrum of attitudes in response to Osaka’s message brings to light the complexity of the role mental health should play for professional athletes. A joint statement issued by the Grand Slam tournaments called Osaka out for “Code of Conduct infringement,” raising the moral question of whether an athlete is obligated to perform his/her duties when mental wellness is at stake. 8 Other athletes expressed understanding for Osaka but backed the media, revealing a lack of acceptance and emphasis for mental health in sports. 13-time French Open champion Rafael Nadal said, “I understand her, but without the press…. we will not be the athletes that we are today.” In a vehement attack at Osaka, one article criticized Osaka for launching a reality show, a Barbie, and starring on the cover of the SI swimsuit issue after saying she was too introverted to talk in front of the media. 9

On a more optimistic note, an outpouring of love and unconditional support from several media sites defended Osaka, criticizing the public outrage surrounding her withdrawal that carried no thought of the pressure she faces. 10 Fans also took to internet platforms and social media to express their understanding and praise, opening the conversation on mental health to netizens and the general public. At a time when mental health is gradually being destigmatized, especially in communities of color, Osaka’s announcement showed others that it’s OK to not be OK. 11 Public acknowledgment of mental pressures the media imbues, especially by media sources themselves, have great potential to shift the discussion from “should we care about athletes’ mental health?” to “how can we care for athletes’ mental health?” 

Just weeks after Osaka’s withdrawal from the French Open, legendary gymnast Simone Biles stumbled on the vault at the Tokyo 2020 Olympics and withdrew from several events in a similar move, citing a need to focus on her mental health. “It’s honestly petrifying trying to do a skill but not having your mind and body in sync,” Biles explained. 12 “10/10 do not recommend.” Biles pointed out that mental struggles translate into physical struggles, which have a direct impact on athletic performance. The potential danger of serious injury when athletes compete with an unhealthy mindset brings much-needed prioritization of mental health for athletes. While fans and media reporters are disappointed by Biles’s withdrawal, the attitude toward her actions has largely been neutral and accepting, if not positive and supportive. 13

Perhaps after Osaka’s controversy, people were more accepting of Biles’s reason for stepping down from competition. If anything, the emphasis that mental health is health on the media’s part shows the progress society is already making toward prioritizing overall wellbeing. One article concedes “although it’s tempting to want a clear answer on why exactly Biles dropped out, we shouldn’t be surprised by the murkiness surrounding her decision. The truth is always more complicated than it seems. Mentally, physically, and emotionally, Biles had a lot on her plate.” 14 Encouragement and even praise came from teammates and other athletes in the industry. Former Olympic gymnast Nastia Liukin said that she will “forever be so proud of [Biles] for setting such a great example.” 15 Moreover, new sponsors like Athleta have backed Biles, describing her as a strong female athlete and a role model for female empowerment. 16

A national, even global conversation on mental health and athletic performance opened by professional athletes has the potential to inspire emerging athletes, sports fans, and netizens alike. Curious about the effects of media, especially social media, on the mental health of student-athletes performing in an increasingly digitized and informed world, I sought an exclusive interview with one of Stanford’s own rising stars. In an interview with incoming Stanford freshman and USA U19 Women’s National Soccer Team member Haley Craig, we discussed media culture, balancing social pressures, and the importance of a strong support system, among other things. 

Craig committed to playing soccer at Stanford during her freshman year in high school, a statement of dedication few others her age could make and stick to. Aside from training to become a professional soccer player, Craig is also a huge advocate for mental health and plans to major in International Relations with a minor in Psychology. 

“Soccer has played a huge role in my life,” says Craig. “My end goal is to play professional soccer and law school is something I’m thinking about. I want to make a difference in the world, maybe with my platform through soccer.”

Even without considering the influence and attention of the media, being a student-athlete means balancing two very busy schedules for Craig, who is already on campus for pre-season training. “Balancing school and soccer is definitely tough, but we have a ton of resources,” Craig admits. “Using my resources and my teammates [helps]. It’s school first. It’s always school first.” 

When asked about any positive, neutral, or negative media publicity she has experienced, Craig’s answer was optimistic and conscious. While she hasn’t received the global attention of millions like celebrity athletes Osaka and Biles have, her experience with and attitude toward digital media provides insight into the potential, and the potential danger, of the massive media influence today.

For Craig, the media attention she received has boosted her confidence and given her motivation to train harder. “I haven’t had a ton of spotlight at Stanford, but back home [in Michigan] there was a lot of positive light. There’d be articles in the local paper about me and my friends doing cool things with our athletics. I’ve had a positive experience thus far, which is great for my mental health. I can’t imagine [how I would feel] if there was negative talk toward me,” Craig says.

However, Craig has witnessed the injustices in the treatment of athletes like Biles and Osaka, whom she believes have every right to thrive outside of their sports. She finds the rumors and speculation unacceptable, believing strongly that athletes are more than their sports. Her attitude towards social media, the popular form of online presence for young people like ourselves, is more skeptical and cautious.

Craig says, “Honestly, I hate social media. I’m not big into it. I use Instagram for networking. I have a Twitter account for sports. I don’t have Tiktok. I don’t want Tiktok. I feel like social media takes away the connection you make with people. It kind of creates a false reality of everyone’s lives.” 

I couldn’t agree more. While there are alternative ways to reap the benefits of social media, the superficiality created often causes overthinking, comparison, and unhealthy mindsets. For overcoming stress and anxiety, Craig relies on her support system she has built over the years, including family, coaches, teammates, and mental health professionals. She has had a very positive ongoing experience with her own therapist and encourages everyone, no matter their state of mental health, to see a therapist. 

As we discussed Biles’s performance and decision to withdraw from her signature events at the Tokyo 2020 Olympics, Craig reflects on past experiences when she wasn’t in the mental headspace, emphasizing the importance of having a supportive coach.  

For Craig, not being in the right mental headspace makes it hard to play a game in front of an audience and perform at one’s peak. Luckily, she’s had really supportive coaches who will not put players in a situation where they’re not mentally ready to play. Her experience reveals the resources and protection athletes can seek to alleviate stresses posed by the outside world. Sometimes, having the people closest to you accept you for taking a step back counters the perceived animosity and expectations of the media or audience. Craig shares one example of the importance of having a support system. After having nose surgery in April, she wore a face shield but wasn’t mentally ready to go back due to fear of getting injured. Her coaches supported her and didn’t make her play, instead letting her sit on the bench because they knew Craig wasn’t going to be her best self on the field.

Finally, I asked Craig what she thinks about the role the media should play in the lives of professional athletes. Her response factored in perspectives from both sides, striking a balance between privacy and publicity. 

“It’s fair to want the full perspective on any situation. You want all sides of the story. But I think that a lot of the time we treat athletes as if they are not human. It’s the same with celebrities—we kind of treat them as if they are a thing, not a human being,” Craig says, “But [they] have a life outside of their sport. I feel like just taking into account that everyone is a human being and knowing that they’re putting themselves first, which you’re allowed to do and be selfish sometimes, especially when it comes to safety.” 

Like Craig, many emerging athletes are prioritizing their mental health and other pursuits (in her case, academics) alongside the sport they play. In the age of digital media when biases, assumptions, and entire lies can be made and brought into popular belief through online networks, it is especially important to raise awareness on the mental well-being of publicized figures. 

While it is clear that the media’s scrutiny and attention negatively affects the mental health of athletes who are put under a skewed spotlight, there remains debate on whether and how change should be enacted. It is evident that digital media, when employed correctly, can be a driver of positive change in destigmatizing mental health for athletes. The wide-reaching influence of media, consumed by sports fans and the general public alike, has potential to spread the prioritization of mental wellbeing to non-athletes as well, fostering a global movement for self-care and wellness.

In a nutshell, mental health is health. Just because mental health isn’t as visible or quantifiable as physical health doesn’t mean it has less of an impact on the optimal performance and abilities of athletes. Athletes, like all human beings, should have their mental health needs prioritized alongside their physical health needs. Digital mass media can use its widespread influence to spearhead a shift in attitude toward acknowledgment and acceptance of the importance of athlete’s mental health.


Instagram Infographic Activism: One Step Forward or Two Steps Back?

Around five years ago, the typical teen scrolling through their Instagram feed would have encountered brightly saturated vacation photos, Tumblr-esque fashion inspiration posts, cute animal videos, or cheesy inspirational quotes over a galaxy background. I personally remember trying to capture artsy, colorful snapshots of my Starbucks Frappuccino and slapping a vignette filter over my sunset just like any other 13 year old back then. 

Nowadays, Instagram looks quite a bit different. The platform has evolved into a favored space for activists to share their perspectives and for youth to receive these messages. These new posts range from bright pink slideshows promoting gender equality to tribute artwork for victims of police violence to lists of resources and petitions supporting various causes. The variety of posts is endless and readily repostable for the average Instagram user. 

In the wake of the Black Lives Matter protests in the summer of 2020, “Instagram infographic activism” has become a phenomenon that has changed the focus of the app.  Traditionally a platform dedicated to aesthetic vacation and lifestyle pictures, it now offers a space for activists to share information through aesthetic templates and ten slide posts. The new wave of activism has drawn both acclaim and criticism, ranging from scorn at perceived “performative activism” to praise about how these campaigns elevate marginalized voices to the general public. Perhaps the answer to how we perceive this new type of activism lies somewhere in between. Instagram infographic activism should be dependent not only on the actual content that is posted, but on how users choose to interact with it, both online and in real life. 

The roots of this transformation sprung from the need of activists for a platform to amplify their message digitally. Surprisingly, these activists told NBC News that Instagram was the perfect platform to share content and organize in-person events. 1 In part, this is because Instagram is widely used among teenagers for entertainment, education, and information. Rather than reading publications or watching the news, a teenager is much more likely to be exposed to new information while scrolling through their feed. Similarly, young people are accustomed to hearing about local issues through word of mouth on Instagram, making it an ideal platform to organize protests and events.  

Raising awareness on platforms such as Instagram allows activists and ideas to reach audiences that normally would not be exposed to these social issues and allows youth who are limited in traditional avenues of protest and activism, such as protests, to get involved online. Although their contributions might be limited at this stage, their presence is essential in the success of a movement. In fact, as a study from New York University’s Data Science Center confirms, peripheral participants (defined as the “immense majority of users who surround the small epicenter of protests”) are essential to maximizing the reach of messages from the core participants of a movement. 2 Overall, the researchers found that these peripheral participants aren’t as useful to a movement individually as a core participant, but the numbers they provide help spread the message far beyond its initial audience. Since Instagram provides a way to increase these peripheral participants to a young, eager audience, it has become the ideal platform for activists to invigorate a new audience and build a robust movement. 

Additionally, Instagram activism provides a more accessible avenue for activists to share their experiences than traditional venues for discourse such as newspapers, academia, and publications. This is especially applicable to marginalized voices and perspectives. Historically, for instance, media outlets have been unable to show full clips of violent police interactions due to sensitivity guidelines. These censored clips dulled activists’ messages and made it difficult to communicate the severity of the situation. Beyond that, many local news outlets prefer to stay apolitical, meaning it’s hard for activists to utilize this media without being censored for potentially polarizing opinions. Instagram and other social media platforms provide an undivided space for activists to share specific calls to action, primary source footage, and commentaries on policy change without violating the terms of service. 

The more expansive opportunities for expression afforded by Instagram and other social platforms have resonated especially with communities of color. In a survey by the Pew Research Center, black people were more likely than white people to believe that social media sites are important for getting involved politically and expressing their opinions. 3 In fact, most black and Hispanic users believed that social media was important for their political engagement. Specifically, around half of black social media users said that “these platforms are at least somewhat important to them as a venue for expressing their political views or for getting involved with issues that are important to them.” The use of social media activism through #BlackLivesMatter allows people to share their own experiences with police brutality while @blackivystories on Instagram detail the microaggressions and racism black students face on Ivy League campuses. Instagram essentially allows the unfiltered truth to be exposed with primary source clips and stories directly from affected communities. Having this ability to discuss, organize, and share content online freely rather than passively observing sustains activist communities. Social media essentially allows people to create affinity networks that are much smaller and intertwined than traditional media audiences

Although Instagram activism has certainly grown in popularity, critics have raised a number of concerns about it as a form of organizing. One of the most prominent and justifiable concerns is the spread of misinformation and misrepresentation online, an issue that platforms such as Facebook and news publications have grappled with already. When someone creates an infographic, there are no systems of checks to ensure the information they’re presenting as fact is actually true. If an infographic containing misinformation becomes viral and gets reposted all over social media, many people may get misled about the cause that the post is advocating for. Even when a creator is not intentionally trying to spread misinformation, posts that try to boil down complex geopolitical situations into 10 slide posts tend to oversimplify and misrepresent them. 

Similarly, when a page posting infographics isn’t transparent with their identity, it can lead to outrage. @soyouwanttotalkabout, which was created in response to Black Lives Matter protests in the summer of 2020 on Instagram, was criticized for speaking out on race issues and using the name of a book by a black author, all while not being transparent about the fact that the account operator was white. 4 This type of misrepresentation is harmful as it lends a large audience to voices that aren’t actually part of the affected community. In an interview with a Stanford ‘25 student, Julia Biswas, who started her account @activismgirl with the intent of informing people about issues that mattered to her and talking to people around the world, I asked her how she prevents the spread of misinformation through her own graphics and page. She said she always cites her sources and conducts thorough research before making her infographics, but if someone points out something that may not be true, she’ll always amend her captions to reflect it. Misinformation can harm activists’ cause by hurting their credibility, so they have an incentive to forward the most accurate information. Beyond that, these posts are supposed to serve as primers on subjects, not the sole source. It’s the responsibility of the user to actively engage and research on their own time. 

As a social media user herself, Julia says when she sees an infographic, she’ll always go to the article the infographic used and read through it to actively fact check and prevent misinterpretation. Just like how we expect users to read petitions before signing and vet GoFundMes before donating, we should expect users to examine the information they’re seeing on the platform. After all, it is not the fault of the platform that people abuse misinformation. Essentially, their initial attention to an issue can be sparked from an infographic, but the brunt of the reading, educating, and understanding falls on their shoulders, not the person creating the infographic.

Beyond that, critics often characterize infographic activism as “performative” and unable to create actual change.  In some cases, the true intent of influencers and brands sharing and creating infographics about social issues is debatable. Some merely want to increase their own engagements and create a positive public perception as someone who cares. However, this shouldn’t discredit the fact that these infographics do have the ability to educate and drive people to create action in the real world. Julia argued that one cannot consider themselves an “activist” just from reposting online, stating that she believes “using Instagram to advocate and inform can only work if both those who make the posts and those who read them are open to growth. In her words, true activists are “always willing to learn.” 

Social media solely serves to provide a platform for users to disseminate content: in this case, infographics or other media that activists want to share. But, organizers and the audience shoulder the responsibility in converting online engagement into in-person action. While the infographic serves as the first step in identifying the problem, organizers should forward opportunities to affect change, including fundraisers, meetings with local officials, and petitions. Audience members should take an active role in further researching, both verifying the claims of the infographic and seeking out next steps. In a study from last year, researchers in the Netherlands focused on the relationship between online activism and offline collective action. 5 They discovered that online activism and offline activism actually have a positively related and intertwined relationship because online activism tends to encourage people to take action in real life. Evidently, online activism seems to serve as a stepping stone into real activism, but it’s an important step for those who don’t have experience with traditional forms of activism.  Once people are exposed to social issues online that may resonate with them, they’re more likely to act on it offline, bolstering the overall movement. 

Instagram infographic activism has ushered in a new era for social movements and activism across the world. While it’s a great tool to ignite change, activism-related content on Instagram should be just that: a starting point. From there, organizers and audience members alike should work to transition their online passion for a better world into offline action.


Linguistic Revival: How Japan Restored the Native Ainu Language with “AI Pirika”

Every two weeks, a minority language dies. Regrettably, humanity doesn’t just lose the unique evolution of that language when it fades; the unique meanings and forms of expression encoded within the language die as well. Language is the medium for cultural heritage in all of its forms — from medicinal knowledge to geography to communities’ humor, love and memory. In short, the loss of a language is an erasure of centuries of rich tradition. 1

While language extinction isn’t a new phenomenon, language loss is occurring at a breakneck pace in the twenty-first century. At least half of the estimated 7,000 languages spoken across the world today are likely to disappear within the century. 2 But, recent technological breakthroughs are offering new hope to affected communities. Applications of natural language processing (NLP) and computational linguistics may offer a pathway to preserve endangered languages from extinction.

AI Pirika is one particular application aimed at reviving the critically endangered Ainu language spoken by the Ainu people, the indigenous inhabitants of Hokkaido in northeastern Japan and a string of islands to the north of Japan, called “Ainu Moshiri,” or “Land of the Ainu.” 

Known mostly for their tradition of catching and raising bear cubs as members of their families, the Ainu possess an ancient spiritual relationship with the natural world. These communities believe in the spirit of the sacred bear, worshipping and caring for bears in hopes of yielding good fortune for their families. From the vibrant geometric designs of their clothing to the mouth tattooing women undergo to signify their coming-of-age, traditional Ainu culture is significantly different from Japanese culture. In fact, their indigenous language is, much like the people themselves, of unknown origins. Ainu possesses no relationship to any other language, making it one of only 129 language isolates in the world. 3

In April 2019, researchers launched AI Pirika to preserve the unique language isolate of the Ainu people along with the vibrant cultural heritage imbued within the language. The project is a 5-year collaboration between the Society for Academic Research of Ainu Culture (SARAC), AI expert Professor Kenji Araki of Hokkaido University, and other technical and Ainu collaborators. Dubbed “Pirika,” which means “pretty girl” in Ainu, the virtual agent will be a hybrid between a virtual chatbot and speech recognition engine. 

Tomoyuki Hanaoka, MD, PhD of SARAC found his passion for the project through the linguistic story of his colleague, Tokuhei, an Ainu man in his 70s who knows Ainu words but cannot speak the language fluently. Tokuhei was born in Kushiro, the Eastern part of Hokkaido, to an Ainu mother and a Wajin, or ethnically Japanese, father. Because neither his grandparents nor his mother spoke the Ainu language before him, Tokuhei was not exposed to conversation in his ancestral tongue. The cycle was amplified by his grandparents’ desire to assimilate into Wajin culture. 

Tokuhei’s fraught linguistic story is a microcosm of the broader story of Ainu assimilation during the Meiji era. When Japan colonized Hokkaido in the 1850s, the Ainu were massacred by genocide and disease, dispossessed of their traditional land, and forcibly relocated to the mountainous barren area in the island’s center in which traditional subsistence living was nearly impossible. Even today, they need explicit permission from authorities to fish in their traditional land. 4 

Laws outlawing Ainu customs pressured the remaining Ainu people to assimilate under the banner of Japan’s mythological homogeneity. Japanese settlers banned the Ainu language and dictated Ainu children to attend Japanese schools. Not only did the assimilation policy force Ainu to use the dominant languages and customs of the Wajin, but it also resulted in significant education gaps, socioeconomic disparity, and rampant bigotry. To this day, Ainu often conceal their identity when seeking jobs or marriage to avoid discrimination. Ainu culture was consigned to display as an exotic tourist attraction or an object of anthropological research. Adding insult to injury, Japan’s illusory ethnocultural homogeneity has left the Ainu struggle blind to the global eye, along with those of Japan’s wealth of ethnic minorities. 5 Only 2 native Ainu speakers remain as a result of this systematic cultural erasure. 6 With only one surviving dialect, the Ainu language faces tenuous odds on the road to preservation. 

Against these odds, however, the Ainu have prevailed against Japan’s oppressive forces through recent activism on an international scale. In a matter of decades, the Ainu have shed their dependence on government and engaged in political mobilization to win national legislation promoting and protecting their culture. From launching periodicals proclaiming their indigeneity to founding the Hokkaido Ainu Association, the Ainu built collective solidarity during Japan’s postwar period. With global indigenous rights activities as an accelerator during the era of global human rights in the 1970s, Ainu activists allied with indigenous communities abroad and leveraged international pressure to further their domestic aims following the United Nations General Assembly’s acceptance of the United Nations Declaration of the Rights of Indigenous Peoples (UNDRIP) for which Japan voted in favor. After calls for recognition “as an indigenous people… with their own unique language, religion, and culture” immediately following the adoption of the UNDRIP and a strategically timed Indigenous Peoples’ Summit to precede the 34th G8 summit scheduled for Hokkaido, the Ainu successfully elicited the Japanese government’s 2008 declaration recognising the Ainu as an indigenous people. 7 Beyond the hard-won recognition of their indigeneity, Ainu continue to campaign to win back their ancestors’ right to fish, for the repatriation of ancestral remains and for the freedom to perform rituals necessitating land access. 8

AI Pirika is thus part of a larger movement to revive Ainu culture. In particular, “[t]he goal of AI Pirika is to revive the Ainu speaker,” said Dr. Hanaoka. “Unfortunately, the Ainu native speaker is almost extinct. We are working with the feeling of reviving the dead.” Dr. Hanaoka and the AI Pirika team hope that the system will survive as an Ainu speaker in the future to contribute to Ainu language education programs and activism, and other applications of NLP to the thousands of minority languages spoken ​​worldwide.

Compared to other NLP applications, AI Pirika features a novel verbal element absent in existing AI systems for language preservation which have largely taken the form of chatbots, such as “Reobot,” a Facebook Messenger chatbot created to preserve the Indigenous language of New Zealand. 9 “[W]e aim to develop a voice dialogue system at the same level as Native, considering that the Ainu language originally has no written language,” said Dr. Hanaoka. The plan to enable both text and voice communication will make the system more accessible to surviving speakers of the exclusively oral language, as well as inspire verbal developments in other systems.

Mainstream NLP applications also rely heavily on written dialogical data, but since obtaining a sufficient amount of conversation transcripts, literature, and other written documentation in the endangered Ainu language is impossible, AI Pirika creators have taken a creative approach to encoding the language. To combat the data deficit, Professor Kenji Araki of Hokkaido University developed a Spoken Dialogue Method Using the Inductive Learning Method Based on Genetic Algorithm with Sexual Selection (SeGA-ILSD) for AI Pirika which allows the system itself to generate dialogue data through selective mating and mutation of genetic algorithms. “Selective mating is a method of generating different sentences by dividing and connecting two different sentences starting from a certain point, and mutation is a method of replacing randomly selected words at random positions with a certain probability,” explained Professor Araki. By snowballing existing data, the algorithm grows the language resource base until sufficient for native-level conversation.

Granted, the dialogue data automatically generated by the system SeGA-ILSD is bound to contain errors. But, the system is programmed to refine its data pool through a user-led feedback process. “If the system makes a mistake, the accuracy will be improved by the selection process of the genetic algorithm based on the mistake pointed out by the user who is using the system,” said Professor Araki. The selection process then improves the accuracy of the entire system by lowering the priority of the rules used to generate incorrect sentences.

But the genetic algorithm alone does not suffice for native-level accuracy. To tune the system further, Professor Araki incorporated the sexual selection theory into AI Pirika. In sexual selection theory, each rule has a sexual distinction; a female rule prefers a male rule. The purpose of this preference is to facilitate selective mating between rules and generate rules that include many common parts. Through this algorithmic process, the words in the rules evolve to possess commonalities, and it is through these small commonalities that AI Pirika learns new and superior rules from examples. “By using such a mechanism, we aim to realize a highly accurate dialogue system even in languages ​​with few language resources such as Ainu language,” said Professor Araki.

Beyond functionality, AI Pirika will be an open access tool available to the public, broadening its impact as a tool for cultural revival. “We want many people to use AI Pirika,” said Dr. Hanaoka. “We plan to open it to the public online for this purpose.” Lowering the barrier to access preservational tools like AI Pirika will increase its reach to prospective Ainu language learners. 

Though AI Pirika is an effective tool to preserve the vanishing Ainu language, there are ethical implications of academic research involving indigenous communities — even in the name of preservation — that merit consideration. 

For one, academic research on indigenous communities can be a detriment to healing from intergenerational trauma. Ainu already have a long painful history with anthropological exploits launched on their ancestral remains post-colonization. Some 1,653 Ainu remains are held at Japanese universities to this day, and Ainu rights advocates have demanded the return of these remains to their lineal descendants. In 2009, the Japanese government proposed to transfer all unidentified Ainu remains from universities to a memorial hall to be built in Shiraoi, Hokkaido by 2020. This plan was met with disdain by Ainu advocates who were not involved in this decision. 10

Technological documentation and research on the Ainu languages may thus risk perpetuating this cycle of exploitative academic research if ethical standards are not defined and upheld. In particular, language research can be exploitative if indigenous people lack agency in the oral history preservation process. As stated in Article 31 of the Declaration on the Rights of Indigenous Peoples, “Indigenous peoples have the right to maintain, control, protect and develop their cultural heritage, traditional knowledge, and traditional cultural expressions, as well as the manifestations of their sciences, technologies, and cultures, including . . . oral traditions.” 11

The creators of AI Pirika are striving to make contributions toward ending this cycle by adopting a more mindful approach to language preservation. Recognizing the diverse views Ainu have toward the application, AI Pirika’s technical collaborators are working to integrate emotion and spirituality, specifically the Ainu’s unique bear-worshipping animism, into the system: “[Some Ainu] think that AI cannot reproduce the spirituality of the Ainu and show a negative attitude toward our project,” said Dr. Hanaoka. 

“On the other hand, some Ainu people expect the Ainu language to be protected by current technology. We think they have a strong desire to retain the Ainu language by any method. We will respect the former feeling and include the spirituality of the Ainu in the AI Pirika system.”

Seeking and incorporating community preferences is a step towards participatory engagement, but co-creation and partnership are even closer. “We spend a lot of effort looking for collaborators,” said Dr. Hanaoka, who recruits technical cooperators to program this system and other collaborators to translate and record the Ainu language. 

Given the dwindling native Ainu speaker population, however, finding native Ainu partners for the project has posed a challenge. “As mentioned above, there are almost no native speakers in Ainu. Therefore, it is not easy to make a collaborator,” said Dr. Hanaoka. “I asked an Ainu woman in her 50s, who had heard the Ainu language in her childhood, to cooperate in recording and started the project. However, due to the health reasons of the collaborator, the partnership ended.” For now, the team is working with those familiar with the Ainu language who can supervise the linguistic side of the project while conducting outreach for native Ainu collaborators, particularly young Ainu eager to learn the language. 

Neighboring concerns around participatory engagement are those regarding virtual preservation as an intrinsically limited form of preservation––a bandage slapped onto a broader issue of cultural erasure due to systematic oppression that merits direct reparations. On the issue, Dr. Hanaoka said, “[Since] the Ainu native speaker is almost extinct[,] I think it is more realistic to leave an environment where the Ainu language can be spoken by AI.” While the Japanese government must make strides toward reparations, any effort to retain the Ainu language is leagues better than the imminent and altogether extinction it faces. 

As for revitalizing the oral Ainu tradition via education, AI Pirika holds promise. “I believe that talking with AI Pirika will increase the chances of using Ainu language, which will increase the number of Ainu speakers,” said Dr. Hanaoka. Steps must be taken, however, to bridge the tech divide to ensure equal access to technology for language minorities, as the absence thereof may defeat the purpose of virtual preservation for those it hopes to serve most. In all preservation efforts, an agentic and collaborative approach is also crucial to upholding indigenous rights outlined in the UNDRIP. Commitment to these practices raises hope in imbuing technological preservation efforts with the indelible spirit of the Ainu.


Why We’re Seeing Déjà Vu with Vaccination Passports

‘Le pass ç’est la division (‘The pass is dividing us’) read a sign amid a crowd of protestors waving the French tricolor as they marched down a street on 22 August 2021—beginning the sixth week of protests in the country. Twitter filled with images and live feeds of citizens as they began striking against the COVID-19 Health Pass approved by its government.

Meanwhile, in its former colony, Morocco, the local government banned residents without a vaccine passport from visiting the tourist hotspots of Casablanca, Marrakech, and Agadir.  In the North African country with only 35% of its population vaccinated, vaccine passports were introduced nationwide in June 2021 in the form of physical certificates and digital QR Codes. 1

France and Morocco are just two of many around the world considering vaccine passports as the next step in their reopening process. Whether or not they may be the right move forward, vaccine passports are here to stay — especially as new variants cause fresh outbreaks and the questions about how to resume global travel and trade still loom. Yet the waves of opposition in different countries have hindered the implementation of these passports. In France, protests were triggered due to the perceived unfairness in imposing greater restrictions on unvaccinated persons. Similarly, protestors in Canada and Australia found the pass relegated the unvaccinated into a separate underclass and effectively forced them to take the vaccine to access greater liberties. 2 3

Governments have tied vaccine passports with existing physical IDs and passports, to use as identification for individuals to travel or enter events and establishments. In particular, some countries have used vaccination passports alongside electronic identification, with mobile applications or QR codes to ease usage of the pass.

Interestingly, passports, physical ID, and electronic identification prompted similar concerns during their introduction. Yet, countries have been able to implement some of these tools successfully regardless. Perhaps we can find the key to implementing vaccination passports from the experience gained by administrations in crafting previous forms of identification. Specifically, clarifying data protection regulations, ensuring accessibility to all communities, and provisions for vulnerable populations should form some of the pillars of implementation strategies. However, vaccination passports will have to surmount similar challenges of their predecessors, such as ethical issues and the question of global standardization. 

While the vaccination passport is a novel policy tool due to the pandemic, it is part of a larger trend of digitizing forms of identification. Since the advent of electronic payment and digital signatures, the use of electronic identification has grown exponentially. First developed in the form of a card with an RFID microchip in 1999, electronic ID was introduced by banks and governments used to access social services and for authentication in Finland. 4 Before the pandemic, the development of NFC (Near-Field Communication), allowed some states to transfer electronic identities to mobile applications. Countries such as Germany, Guatemala, and Kazakhstan already had in place electronic identity cards and affiliated systems before the pandemic. Some countries such as Singapore and Turkey had developed Singpass and e-Government Gateway, respectively, as one-stop platforms for citizens to access public services.

With such digital infrastructure in place, many governments saw that transferring the vaccination passports to the aforementioned platforms could ease the usage of the pass in everyday activities. Be it entering a restaurant or clearing an immigration checkpoint, adding a vaccine certification to existing digital systems could expedite the process of verification, as compared to producing a physical card or certificate.

In China, the government introduced International Travel Health Certificates through mobile apps, AliPay and WeChat. Singapore allowed its citizens to access vaccination records through its contact-tracing app, TraceTogether, itself linked to the national identity system, Singpass. In the UK, individuals could access their vaccination passports using the NHS App, which is also used to book appointments at healthcare institutions. 

These are a few of the several digital forms that vaccination passports have taken. With such platforms, states can also move a step further to digitize other social services. A digital identity would ease citizens in obtaining other government services, including paying for taxes and healthcare bills.

Yet, they would have to first overcome the hurdle of implementing vaccination passports. One of the many concerns with the tool is that of violation of privacy. While implementation of vaccine passports may vary from state to state, the fundamental basis of the pass is the storage of personal healthcare information. How this information is stored and who has access to it largely depends on the authority collecting such information. Without proper regulation or privacy laws in place, individuals are skeptical if such information is handed over to corporate entities or leaked out.

For instance, according to a Harris Poll, 80% of American respondents were worried that their personal information was being endangered by the use of a vaccination pass. With such fears, the buy-in for vaccination passes piloted in states has been significantly low. A 16% sign-up rate for New York’s Excelsior Pass and a 7.5% rate in California as of July 2021 is a testament to the lukewarm response to vaccination passports. 5 This is especially pertinent in countries where vaccination centers are run by and information is held by private healthcare providers or pharmacies. In the US, with Walgreens and CVS collating data on its customers who have walked in to receive vaccination, what regulations are there to govern its use? 6 7

To assuage these fears, governments need to dictate how data is managed and who has access to data. While many states try to etch out what data protections look like, the EU has a potential example with its General Data Protection Regulation Law (GDPR), first written in 2016. The law, which governs the use of data regarding any individual in the EEA, stipulates the rights of the data subject, the role of data controllers and processors, as well as associated penalties. 8 Preceding this, the EU had implemented eiDAS (electronic IDentification, Authentication and trust Services), a regulation that stipulates how electronic identification can be used within EU states. This regulatory framework facilitated the development of digital identification for citizens of the EU countries to perform transactions and access public services. As a precursor to GDPR, it also enshrined privacy and user consent as key principles in the management of handling digital personal information.

When implementing the EU Digital COVID-Certificate, the EU cross-applied the same laws governing the use of personal information. Since June 2021, citizens of EU Countries have been using Digital Certificates to travel smoothly between countries. 9 As other countries begin using digital passports, they should consider setting up data protection regulations first or streamlining it in the case of countries with varying laws across different territories. 

For vaccination passports to be successful, it also requires significant buy-in from all residents, especially underprivileged communities. The case of photo ID in the US highlights the issue of accessible identification. 10 Because photo ID is not mandatory, the federal and state governments directed existing agencies to produce photo IDs upon request for civilians. Government offices are disproportionately more accessible to wealthier communities, thus making it far more difficult for low-income residents to obtain an ID. Without this ID, marginalized residents may find it difficult to buy a gun, access healthcare, or in some states, vote. 

The difficulties of getting a vaccination passport are irrelevant, however, if an individual does not get a vaccine in the first place. In June, MPs from the Public Administration and Constitutional Affairs Committee in the UK found vaccination passports “would likely disproportionately discriminate against people on the basis of race, religion and socioeconomic background”. 11 The fear that minority communities, where vaccine hesitancy is higher, may not have access to vaccination passports and the greater freedoms that come with it is very real.  In March this year, the Office for National Statistics announced findings showing that vaccination rates among elderly Black African, Black Caribbean, and Pakistani and Bangladeshi groups were 58.8%, 68.7%, and 75% respectively, markedly lower than the national average of 90%. 12

To ensure that vaccination passports do not become another source of inequality along wealth and racial lines, it is important to ensure that all citizens are given the opportunity to get vaccinated and the accommodations necessary to do so. If there is hesitancy amongst groups, it is all the more important to conduct educational campaigns so that no community is left behind. A successful case of this was observed in France, where making centers more accessible in poorer neighborhoods has increased vaccination rates. For instance, the region of Seine-Saint-Denis saw the vaccination rate exceeding that of the national average shortly after pop-up immunization centers with translators were set up. 13 

Some individuals, however, cannot get vaccinated because they are immunocompromised or are medically unable to take the vaccine. Vaccine passports would isolate them further from the rest of society. As such, while it is important to prevent unvaccinated individuals from being exposed to more dangerous variants, governments should do their best to ensure that differentiated regulations do not affect daily livelihoods in preventing people from working or going to school. 14 In doing so, vaccination passports can become less of a divisive tool, and account for the vulnerable.

On the other hand, there is a group of individuals who may not take the vaccine due to personal choice or religious obligation. With these individuals, there is the question of the ethical basis of differentiated restrictions. At the current moment, the U.S. federal government has introduced a partial mandate that compels employees of most large corporations to be vaccinated or face weekly testing.

Such restrictions may prompt accusations of implicitly forcing individuals to get vaccinated. Such a dilemma is not exclusive to vaccination passports — it also preceded the introduction of the much older travel passports. In the aftermath of World War I, countries met in 1920 to set out ways for the world to return to its pre-war state. High on its agenda was the abolition of passports to ensure freedom of movement. 15 Proponents of this principle claimed that passports would unfairly limit the right to movement of individuals who chose not to get a passport. However, the countries finally agreed to an International standardized passport to be used, recognizing the security risks that were eminent at that time. 

Today, governments have to choose whether the risk of infections is a more significant issue than protecting the freedoms of unvaccinated individuals. Even if governments would like to implement differentiated restrictions to increase uptake of the vaccine, it may not be successful. Inspection of the Green Pass in Israel found that tiered restrictions according to vaccination status “resulted in antagonism and increased distrust among individuals who were already concerned about infringement on citizens’ rights”. 16

Amongst the various models that countries are piloting, the use of ‘Vaccination or Regular Testing’ adopted by Singapore in particular sectors may provide a middle ground. As per its namesake, employees who are not vaccinated are required to take a Polymerase Chain Reaction (PCR) or Antigen Rapid Test (ART) twice a week. While testing may be inconvenient, it would still allow for unvaccinated individuals to work in person, while protecting the rest of the employees. 17

Perhaps the most significant challenge that lies ahead for vaccination passports is that of standardization across different countries. While the development of an international standardized accreditation is still premature, it would be essential in the future to facilitate global travel and trade.  As different governments have adopted varying levels of restrictions, different models of vaccination passports—digital or physical—as well as different approvals by health authorities may also hinder travel and authentication of immunization status.

Conventional passports follow standards from the International Civil Aviation Organization, making it easier for immigration authorities to process travelers from other countries. The International Air Transport Association has announced it is trialing a Travel Pass that mimics a conventional passport, a digital pass that contains testing and vaccination records. While 43 airlines have adopted its trials, it has not been adopted by most governments. As of April this year, the WHO still had not supported the use of vaccination passports, as they were not confident that vaccination prevented transmission and associated ethical issues. 

Ultimately, the vaccination passport will have to face many hurdles before it becomes a key to more relaxed domestic restrictions and smoother global travel. The opposition toward the pass is bound to continue as states further embrace digital identity. Yet, we opt into identity cards, social security numbers, or biometric passports, because they give us the ability to access social services and other goods. While we are accustomed to such forms of identification, they faced similar concerns, of exclusion, privacy, and accessibility when they were introduced. Perhaps what we need then, is time — to fine-tune the passport to accommodate as many of us, to communicate the safeguards that come with it, and to be accustomed to it being a new global norm.


Windows Into AI: How New Interpretability Tools Might Enable Trustworthy AI

“The clock is ticking,” sighs Chief Nursing Officer McCabe, blinking away tears as she recalls a sick patient’s wait for COVID-19 treatment. 1 Her patient was not unique. Since March 2020, surging case counts have made many hospitals overflow, forcing some medical staff to choose which patients had to wait for treatment. 2 Traditionally, staff made these choices themselves; more recently, these decisions have been increasingly informed by machines.

During early waves of the COVID-19 pandemic, many hospitals turned to artificial intelligence (AI) tools to decide in which order they would treat patients. 3 Around the world, healthcare facilities started to use software that quickly scanned patients’ chest x-rays and made predictions about which patients had COVID-19. If these algorithms predicted that someone had COVID-19, that patient would be moved up in a waitlist.

In important ways, this decision helped. It partly made up for staff shortages, likely saving lives.

But along with these benefits, the rushed implementation of AI tools into COVID-19 prognosis brought a major downside: it introduced algorithmic bias into life-and-death decisions. 4 While confirming individual cases of bias would require further research, it’s likely that patients were denied quick access to life-saving treatments on the basis of their race or gender.

We may be tempted to respond by abandoning these algorithms outright, but that would be very costly and likely counterproductive as an approach to supporting vulnerable communities. After all, understaffed hospitals without rapid prognosis tools will need to leave more patients — disproportionately minorities — with dangerously long wait times. 5 Before abandoning an AI tool, we should consider whether it’s feasible to sufficiently improve it and impose minimum requirements.

For years, engineers’ understanding of leading AI algorithms has been limited — constraining their ability to reduce algorithmic bias — but recent research discoveries suggest new ways forward. 6 By offering new ways to make sense of AI tools’ inner workings, these findings suggest novel ways engineers can identify and eliminate certain forms of algorithmic discrimination. 

In light of these possibilities, now is a time for researchers, governments, and industry to act. They should take this opportunity to invest in our tools for interpreting AI and, when feasible, implement high standards based on these tools. On top of their immediate benefits, such actions would also build our institutions’ experience with using AI interpretability tools to improve AI — experience that will help prepare us to address other AI challenges.

As we wait for AI interpretability methods to mature, algorithmic bias will continue to cause harm, so policymakers and industry leaders should complement long-term investments with practices that will reduce bias today. Among other steps, they should expand access to and use of diverse datasets, implement existing methods for detecting and reducing algorithmic bias, and diversify engineering staff.

Before we get too far into these solutions, let’s take a closer look at the problem.

Opaque AI and its Downsides

Users might assume that the engineers who design AI algorithms surely know how to eliminate their bias. But that is not always the case, because today’s leading AI algorithms are machine learning (ML) models — they learn on their own, and even engineers often don’t know precisely what they’ve learned.

As a refresher, cutting-edge ML models are a kind of AI algorithm inspired by the human brain. They’re structured as neural networks, which partly means they’re made up of many small pieces (“neurons”) that send signals to each other. When someone runs an ML model, information starts at one column of neurons, is passed and processed through a bunch of intermediary columns, and gets churned out at a final column. Collectively, these neurons can do things like read a scan of someone’s lungs and output whether the patient has COVID-19. To succeed at this, neural networks benefit from having many neurons (e.g. millions of them), and from having neurons connect to each other in just the right ways — a result achieved through automated learning. 6

The complex structure of machine learning models — and the involvement of automated learning — often means that, although an ML model produces meaningful results, no one knows how it gets there. Engineers can follow each small step of the algorithm, but a big-picture understanding is typically absent. As a rough analogy, it’s as if someone had learned how to make a cake, but we didn’t understand the steps in the recipe, or the ingredients.

Engineers’ limited knowledge about ML models means that, when something is wrong with an algorithm (e.g. it’s biased because of biased historical data), they might not know how to fix it. Building on our previous analogy: if someone made a cake that tasted funny, and you didn’t understand the recipe, teaching them how to fix it (without messing up the cake in other ways) would be hard.

Going back to COVID-19, the opaqueness of machine learning models has made it harder for developers and users of prognosis tools to identify and eliminate problems. And there have been plenty of problems. A study published in Nature reviewed 62 ML models designed to detect COVID-19 using chest scans. 7 Researchers report a bleak finding: “none of the models identified are of potential clinical use due to methodological flaws and/or underlying biases.” Flaws in the ML models they studied included demographically unrepresentative data, a problem that has historically contributed to major inequities in healthcare provision. 8

The life-and-death limitations of current ML models are especially clear in the context of hospitals’ triage. But similar problems crop up in other uses of AI; whenever complex ML models are used — whether that’s to screen job applications, 9 to inform court sentencing decisions, 10 or to identify military targets 11  — engineers and operators lack a human-interpretable understanding of precisely how these algorithms make decisions. This will cause growing harms as high-stakes applications of AI expand. 

With little insight into ML models, people can only try to correct errors retroactively. But in high-pressure settings like hospitals, time is often too short for such retroactive error correction. In these critical decision situations, avoiding algorithmic harms requires that we have enough insight into ML models to address their flaws proactively.

Ways Forward – Recent Discoveries in Understanding AI

Fortunately, some AI researchers have now spent years developing techniques to better understand what machine learning models are doing. Some of these techniques aim for post-hoc explanations. In other words, they try to provide case-by-case explanations for AI tools’ decisions, after each decision is made. Another branch of work in understanding AI is research in AI transparency: trying to understand how these algorithms work in a more general way.

Within this second branch of work — AI transparency — recent research has revealed the existence of what are called “multimodal neurons” within some image labeling ML models. 12 In short, these neural networks seem to have neurons that detect abstract concepts. Going back to our cake analogy, researchers are learning to identify some of the fancier ingredients involved in algorithms’ “recipes.”

For example, a single neuron in a model seems to track the concept of Halloween. It activates in response to photos of the word “Halloween.” 13 That’s not all; it also activates in response to a bunch of sights that are conceptually related to Halloween: the word “haunted,” horror movie poster fonts, spooky masks, the word “spooky,” gravestones, and Jack O’Lanterns. 

There are plenty more exciting findings. Other neurons in this ML model seem to keep track of other abstract concepts, including the emotion of surprise, Lady Gaga, and history.

(Researchers figured this out by asking questions like: Out of the photos and text in our data, which most activate a specific neuron? And out of all possible inputs — even if they’re not in actual data — which most activate a specific neuron?)

Similarly, at least some of these image classification models seem to be identifying group identities that are federally protected from discrimination, including male, female, and elderly categories. These findings are very recent, so there is likely still much to learn. Still, they point to new tools for identifying and mitigating algorithmic discrimination.

Multimodal neurons may make it possible to directly, precisely detect whether a model is making decisions on the basis of a characteristic that is federally protected from discrimination, such as sex or race, and to reduce or even eliminate those biases.

The finding is promising in part because researchers identified these neurons in the step right before ML models’ outputs (in the second-to-last column of neurons). If researchers found something similar in neural networks that are applied to other tasks — such as COVID-19 prognosis — this would be very informative: the connection between one of these neurons and the neural network’s output would tell us exactly how categories such as “female” influence an algorithm’s recommendations. 

At its best, that would enable both accountability and improvement. Engineers could look at the connection between such neurons and an algorithms’ output to directly assess how the algorithm uses gender information. They could then tweak these connections so that certain demographic information influences the output in desired ways. For example, engineers could set the connections between a “female” neuron and an algorithm’s output to have zero weight, making neurons that identify women have zero influence on the algorithm’s final recommendation.

That wouldn’t be a miracle cure; identifying the relevant neurons could be very time-intensive, or even impossible for ML models that are too simple to have neurons that straightforwardly track relevant demographic categories. Still, it would be a powerful addition to engineers’ arsenals of tools for countering algorithmic bias. Unlike some blunter approaches, this approach zeroes in precisely on the parts of an algorithm that are discriminatory, so it has the potential to more thoroughly eliminate algorithmic bias, while leaving other useful parts of algorithms intact. 14

How Government and Industry Can Help

To help mitigate algorithmic bias in healthcare, researchers should use methods similar to those used by Goh et al. to: try to identify representations of protected categories in a wide range of ML models in high-stakes applications, modify existing AI tools so they represent protected group identities in more interpretable ways, and test ways to address related biases while minimizing downsides. 15 They should also seek ways to distinguish between eliminations and obfuscations of these biases — removing clear representations of protected categories, for example, may make certain kinds of discrimination happen through proxies, rather than eliminating them. 

As a start, governments, businesses, and academic institutions should support such work. The National Science Foundation could issue a “Dear Colleague” letter encouraging and committing to funding high-quality grant proposals of this variety. Advanced research projects agencies could fund additional projects in this area. AI labs could direct more of their researchers to apply these techniques. And research universities could encourage the same. 

Once researchers’ techniques here have matured sufficiently for best practices to be clear, industry or governments should establish these as industry standards. AI developers should proactively refine and implement cost-competitive approaches to identifying and eliminating algorithmic discrimination.

If industry fails to take this initiative, advocates and governments should use refined AI interpretability tools to enforce existing anti-discrimination laws in the context of AI. After all, established civil rights law applies to AI-supported healthcare decisions just as much as it applies to plain old healthcare decisions. 16 Depending on the technical landscape, other regulatory options — like creating a review process for algorithms before they’re implemented in high-risk settings — could also be promising. AI developers might even support such regulation; it could prove to be a low-cost way to increase (deserved) trust in their products. 

Throughout the above, stakeholders should ensure that AI transparency research — not only the creation of case-by-case explanations, although that is also valuable — features prominently in efforts to create trustworthy AI.

As we invest in AI transparency, we should also recognize and address its limitations. The above techniques remain far from shovel-ready, and they will take years to mature; in the meantime, we should take additional, more reliable steps to tackle the immediate problems of algorithmic bias. Following researchers’ recommendations, government and industry actors should improve the quality, accessibility, and especially the diversity of COVID-19 databases. 17 18 They should ensure that AI developers train ML models with such representative data, and that engineers use promising practices that have already been tried for reducing algorithmic bias (such as testing algorithms for bias before use and modifying the parts that most contribute to unfair outcomes). 19 On top of this, relevant employers should diversify the staff that creates and uses these algorithms.

Beyond AI in COVID-19 Prognosis

The technological and institutional tools we develop to address algorithmic bias in healthcare could serve as a model and inspiration for progress on other AI issues. Similar techniques could help address algorithmic bias in other AI applications, such as job application screening software. In addition, better understanding the inner workings of ML models could help us predict when some ML model will fail to generalize what it has learned to a new environment. And more tentatively, AI transparency tools might help us directly access information that is implicitly stored in ML models — learn what machine learning models have learned, and use that knowledge to help solve problems.

As Chief Nursing Officer McCabe said, the clock is ticking. AI is likely to continue playing increasingly influential roles in healthcare decisions and other high-stakes areas. Whether we gain the insight into ML models to make this influence fair and beneficial is up to us.


Unlocking the IP Safe: Waiving Intellectual Property Rights for COVID-19 Vaccines

Each year, global pharmaceutical companies invest billions of dollars in developing novel drugs and medical technologies to cure the world’s most pressing illnesses. In exchange for their research efforts, these companies retain intellectual property rights to their technologies in the form of patents. While these proverbial “safes” for intellectual property can protect pharmaceutical companies from copycat competitors, they also limit the products’ reach to a select group of consumers and countries. The sweeping effects of COVID-19 and dramatic success of vaccines for the virus have ignited a new debate about intellectual property rights in medicine, as many question whether the integrity of intellectual property outweighs the potential to save lives.

Vaccines and other disease prevention technologies received global patent protection through a World Trade Organization agreement in 1995. Per this agreement, companies holding the patents hold the exclusive right to manufacture, sell, or use their vaccines for 20 years. In times of global health crisis, these proprietary rights give pharmaceutical companies the financial upper hand in negotiating the price and sale of these vaccines. As of May 5th, 2021, Pfizer had earned $3.5 billion in profits from its COVID-19 vaccine because countries raced to outbid each other for the limited doses. 1 The bidding war, however, magnifies the divide between wealthy and middle or low-income countries. To keep their profits steady, multinational pharmaceutical companies must continue to look to the highest bidder, and middle and low-income countries often do not have the economic capacity to be the highest bidder. Experts estimate that this unequal auction approach to vaccines will prolong the delivery of two billion vaccine doses to middle and low-income countries. 2 The proprietary rights lead countries to overspend, overbuy, and ultimately, overlook other countries’ needs for vaccines because of the artificially limited supply. 

In October 2020, with the calls for vaccines intensifying in the developing world, the South African and Indian governments proposed an initial waiver to the World Trade Organisation. If this waiver was passed, member nations would not have to enforce the rights to the vaccine tech and would allow every state to take appropriate measures in obtaining COVID-19 vaccines in a legal manner. 3  Almost immediately, opinions were divided. Emmanuel Macron, the President of France, strongly favored the intellectual property waiver. He cited its benefit in increasing total vaccinations and the potential for a waiver to create “partnership with the poorest countries.” The priority, today, is certainly to give doses,” he emphasized. Meanwhile, pharmaceutical manufacturers fought against the waiver. The International Federation of Pharmaceutical Manufacturers and Associations (IFPMA) argued, “Waiving patents of COVID-19 vaccines will not increase production nor provide practical solutions needed to battle this global health crisis. On the contrary, it is likely to lead to disruption.” Their alternative? “The only way to ensure quick scaling up of and equitable vaccine access to all those in need remains pragmatic and constructive dialogue with the private sector.”

The waiver did not pass. Currently, production of vaccines is concentrated in high-income countries while production by middle-income countries has been occurring at a substantially slower rate via licensing or technology transfer agreements. Vaccine inequity between higher and low-income countries is a reality. At the time this article was written, the least wealthy 52 countries had 3.3% of the world’s vaccines, but 20.5% of the world’s population. 4

Living in India over the last year, there was never a particular moment where my parents and I outright called the disparity “vaccine inequity,” but the knowledge of its existence loomed over all our conversations about a potential return to normalcy and the acknowledgement that the duration of the pandemic was likely to be different, depending on where you were located.

As Moderna and Pfizer published their first clinical trial results, I remember the initial feeling of excitement and relief that all of my friends and family felt on hearing about the success of these vaccines. However, reality soon sunk in for us, as we acknowledged that the third world was probably low on the totem pole of priorities for most of these pharmaceutical companies.

A few months later, the second wave of COVID-19 infections hit India. By the 1st of May, 2021, India recorded its largest peak of 412,262 new coronavirus cases in 24 hours. More than terrified, everyone was resigned. There were a couple of weeks where every day came with the death of a colleague, relative or another acquaintance. Oxygen shortages became the norm, and friends and relatives with oxygen cylinders were delivering their oxygen to people they knew on an hourly basis. It was utter chaos, and the end was nowhere in sight.

Meanwhile, Israel announced it had vaccinated 50% of its population, the US administered its 100 millionth vaccine dose, and pharmaceutical companies, including Pfizer and Moderna, wrote a joint letter to the Biden Administration opposing the IPR waiver proposed at the World Trade Organisation. These two events were ridiculously infuriating for so many of us in the epicenter of the pandemic. In our eyes, Israel and the US were on their way to ensuring a return to normalcy. Moreover, vaccine hesitancy in the US was unbelievable to so many people I knew. They were ready to pay thousands of rupees to help their parents and elderly relatives get the vaccine, and there were so many people treating the vaccines like they were inconsequential. 

The disjoint between high vaccine enthusiasm in the developing world and hesitancy in countries replete with vaccines is a global phenomenon. A study published in Nature in July found vaccine hesitancy was 20% lower in developing countries than in places like the US and Russia. 5 What most of us could not and would not stand was the ignorance of pharmaceutical companies to write a letter opposing an IPR waiver while the second most populated country in the world was destroyed by a virus they knew how to control. 

I firmly believe that a waiver for IPR is key to combating the pandemic today. The principle of patents is to emphasise creating useful new ideas for future generations by reducing the rate of diffusion of useful new ideas in the short run. However, during the backdrop of a global pandemic, this exchange cannot be tolerated. Right now, we will ideally enable the creation and distribution of vaccines and other treatments, not reduce its pace. Providing drug companies with the ability to slow developments down by ensuring that competition never comes into the picture and simultaneously hiking prices for the sake of upholding profits is possibly amongst the worst ways of ensuring a global recovery and return to normalcy. 6 These are extraordinary circumstances and when the world delves into chaos, the preexisting systems of law and order need modifying if we are to survive and make progress.