The Power Of Coding In Computer Made Easy

Road to the IGF: Sisi Jiang's LIONKILLER

This interview is part of our Road to the IGF series. You can find the rest by clicking here.

LIONKILLER casts the player as Hua Mulan, conscripted into into the First Opium War against the British Empire. Through their choices, the player can chase their dream of running a flower shop, uncover conspiracies, or kiss a girl. But will these personal acts of rebellion change the history that's unraveling before them?

Gamasutra spoke with Sisi Jiang, sole developer of the Excellence in Narrative-nominated title, to talk about the power of finding your own means of fighting even if your fate cannot be changed, hoping the player can be stirred to feel and think through story, and the beauty that still comes from less-than-ideal development circumstances.

I'm Sisi Jiang, and I am the sole developer. Which means that any bugs you might encounter are fully mine. Sorry about that!

I come from roleplaying forums that were mostly run by girls who migrated from Neopets. I learned to code by taking apart and reassembling the code of better designers (with their stated permission, of course). Beyond that, I’ve never taken a single computer science course. My major in college was history, which probably helped in the historical research for LIONKILLER. I’m hoping that my nomination can show young designers that you don’t need an engineering background to make video games.

You know how marginalized fans are often told “If you don’t like the big franchise, then go make it yourself?” That was my reaction to Disney’s Mulan (1998). I saw it for the first time four years ago, and I was horrified that it had been turned into a “girl power” kind of story. I mean, that’s Disney, but I grew up with the ballad. For me, the story was about sacrificing a girl because the old man was more valuable to the family. I don’t think any amount of individual girl power can override living in a patriarchal society.

I used Twine, Photoshop, and a lot of online tutorials.

I know people are expecting me to say that I started with a political agenda (people will probably still say this), but the honest truth of it is that I have a history degree. I prefer researching modern history to classical history. The reason is because modern states have much better bookkeeping, so there were a lot more details to draw from when creating this game.

China gets invaded by different countries, depending on the version of Mulan. My academic background was mostly in the 19th-20th centuries. Guess who was the most powerful empire back then! I needed an actually threatening villain, so I started following the breadcrumbs to the First Opium War. Once I got there, I started feeling that the wars were linked to the larger British strategy of trying to conquer all of Asia. British India would produce raw materials (opium) and provide the soldiers (sepoys).

I linked LIONKILLER to the campaigns in Central Asia by making the British officer a veteran of the first Anglo-Afghan war (look up “the Great Game”). My secret is that I had to cut 40 percent of the original game for scope and budget reasons, but I wanted to bring in a more colorful cast of characters. More intrigue! More unlikely historical alliances! Unfortunately, scope cuts are a painful reality of game development. It still blows my mind that people can play one route in a scoped-down game and think that’s the canon LIONKILLER.

The branching itself wasn’t that hard, I think the real issue was trying to create the illusion of choice while keeping scope creep under control. Like, if I branched too early and closed the branch too late, I’d create too much of a burden for content needs. I think I eased the pressure once I saw the playtest comments and realized that most players don’t really think about the alternative paths other than the one that they’re currently on. And most people only play the game once.

Choice is really fragile in LIONKILLER. I mean, the whole story revolves around a conscription. Why choose if your decisions still lead to a war that you didn’t sign up for? Why choose anything if your fate is still etched into the stars? I think that question is relevant to a lot of young people who have been pigeonholed into careers they didn’t want.

LIONKILLER presents players with private moments of rebellion that are meant to keep their hearts human. I mean, you can’t actually desert with the Brigadier-General’s horse, but you could make an unsuccessful attempt. You’ll be punished for it, but you’re using the game as a tool to express your dissatisfaction. That was important to me because I always considered Mulan a story about filial piety, even at the expense of selfhood. As the designer, I wanted to make it clear that the player’s anger and discomfort mattered.

There were a lot of choices that didn’t affect the outcome in part one because I thought that it was important as a part of a war story. The main character had a whole life ambition. They want to open a flower shop, or they want to run away with a girl. I authored those things, but the player gets to pick the dream that the character has. That’s important. Just because a hero is being called to some greater good doesn’t mean that their dreams just went away. Players usually feel very upset when they think that a roleplaying game ignored all of their choices. I used that psychological tendency as a way to convey the feeling of loss.

Choices also affect your ability to get different endings in the game. There’s a route that rewards you for being consistent, and another that rewards you for being bold. There’s a “game over” for charging into a dangerous situation unprepared. Try to get multiple endings!

This situation isn’t really unique to LIONKILLER; it’s a question that every narrative designer in the industry has to grapple with. I have a lot more freedom, but every game is inherently collaborative. Even if I wrote a completely linear story with straightforward characters, I can’t control how players think and feel inside of their heads. My advice is to embrace that. Create characters who sink their meat hooks into your player’s heart; create content that doesn’t let go without dislodging some buried thoughts from their dusty brains. Mulan is not an avatar, they’re a vehicle for transporting you into no-win situations.

I see my characters as tools, and not purely extensions of my ego. That doesn’t mean that they’re objective or bias-free (Mulan is a passive-aggressive complainer). It just means that they’re props that I use to create the narrative. The character of Hua Mulan is not a person, the player is a person. As a designer, I know that I have trapped my player into terrible situations. That’s why I always use the game’s narrative voice to comfort, rather than condemn their choices. I was the one who wrote those choices!

I give the player absolute narrative control in a moment in part three where I ask the player to type in their opinion of the British Empire. The game acknowledges and canonizes the player’s response on the next screen. I took a risk, since the player could still force Mulan to state: “May the sun never set on the British Empire.” However, it would be more powerful if the player chose to condemn the empire on their own terms. My player has always been my partner-in-crime, and that input screen is my ultimate expression of trust.

I have some friends who have trouble reading certain colors on computer screens, so I ran my new color choices past them for accessibility reasons. Purple indicates that there is more text on the page, while pink indicates that you’re moving to a new passage. The crimson on a few passages in part three indicate that the British officers are speaking. My next project focuses on UI even more, and the reason is because I didn’t have the budget to hire an artist. UI can be the visual interest if you’re not a game artist.

I did have a line of dialogue that looks super distorted, and it’s when the British officer is yelling at Mulan in English. I thought it was fun because Chinese and Arabic are so frequently represented as random scribbles. I bet most players have never seen English represented like that before.

What I do as a narrative game designer is that I allow players to express their emotions through the computer (a tool). I can’t put any feelings into somebody who didn’t already feel that compulsion. So a player can feel sad about a scene that made another player angry. Dialogue that made me feel cathartic might be painful for another. I don’t know what path players took in the game, and I don’t know the personal path that the player took in life. These two paths create far too many possibilities for me to account for. I’m not a god, I’m just a shady fortune-teller with tricky sleights-of-hand.

I do have one! It wasn’t planned and it wasn’t written in the most ideal circumstances (welcome to game development). You get this ending by coming clean about your gender and asking these men for their solidarity. By the way, these men have betrayed the Brigadier-General’s trust. They have skipped their military duties, and they have been lying to their superiors. All for their own survival. By any society’s metric, they are untrustworthy. But Mulan was also a liar. Can a liar condemn others who have lied? It’s a paradox.

I’m not telling other queer people to forgive unreliable people in their lives. This game will never judge the player for withholding their heart. This game doesn’t ask you to forgive these men. They don’t deserve forgiveness. But earning their trust and agreeing to save them is the only way to secure their cooperation. And sometimes, being vulnerable in the den of lions is what it takes to save other things that you love.

Using AI, people who are blind are able to find familiar faces in a room

Paralympics in Brazil

Project Tokyo was born out of a challenge, in early 2016, from senior leaders at Microsoft to create AI systems that would go beyond completing tasks such as fetching sports scores and weather forecasts or identifying objects. Morrison said creating tools for people who are blind and with low vision was a natural fit for the project, because people with disabilities are often early adopters of new technology.

“It is not about saying, ‘Let’s build something for blind people,’” Morrison said. “We are working with blind people to help us imagine the future, and that future is about new experiences with AI.”

Morrison and her colleague Ed Cutrell, a senior principal researcher at Microsoft’s research lab in Redmond, Washington, were tapped to lead the project. Both have expertise in designing technologies with people who are blind or with low vision and decided to begin by trying to understand how an agent technology could augment, or extend, the capabilities of these users.

To start, they followed a group of athletes and spectators with varying levels of vision on a trip from the United Kingdom to the 2016 Paralympic Games in Rio de Janeiro, Brazil, observing how they interacted with other people as they navigated airports, attended sporting venues and went sightseeing, among other activities. A key learning, noted Cutrell, was how an enriched understanding of social context could help people who are blind or with low vision make sense of their environment.

“We, as humans, have this very, very nuanced and elaborate sense of social understanding of how to interact with people – getting a sense of who is in the room, what are they doing, what is their relationship to me, how do I understand if they are relevant for me or not,” he said. “And for blind people a lot of the cues that we take for granted just go away.”

This understanding spurred a series of workshops with the blind and low vision community that were focused on potential technologies that could provide such an experience. Peter Bosher, an audio engineer in his mid-50s who has been blind most of his life and worked with the Project Tokyo team, said the concept of a technology that provided information about the people around him resonated immediately.

“Whenever I am in a situation with more than two or three people, especially if I don’t know some of them, it becomes exponentially more difficult to deal with because people use more and more eye contact and body language to signal that they want to talk to such-and-such a person, that they want to speak now,” he said. “It is really very difficult as a blind person.”

Ed Cutrell, a senior principal researcher with Microsoft’s research organization in Redmond, Washington, is a co-leader of Project Tokyo. On his desk are several modified Microsoft HoloLenses that project researchers are using to help people who are blind and with low vision learn who is where in their social environment. Photo by Dan DeLong. A modified HoloLens

Once the Project Tokyo researchers understood the type of AI experience they wanted to create, they set out to build the enabling technology. They started with the original Microsoft HoloLens, a mixed reality headset that projects holograms into the real world that users can manipulate.

“HoloLens gives us a ton of what we need to build a real time AI agent that can communicate the social environment,” said Grayson during a demonstration of the technology at Microsoft’s research lab in Cambridge.

For example, the device has an array of grayscale cameras that provide a near 180-degree view of the environment and a high-resolution color camera for high-accuracy facial recognition. In addition, the speakers above the user’s ears allow for spatialized audio – the creation of sounds that seem to be coming from specific locations around the user.

Machine learning experts on the Project Tokyo team then developed computer vision algorithms that provide varying levels of information about who is where in the user’s environment. The models run on graphical processing units, known as GPUs, that are housed in the black chest that Grayson carted off to Regan’s house for the user testing with Theo.

One model, for example, detects the pose of people in the environment, which provides a sense of where and how far away people are from the user. Another analyzes the stream of photos from the high-resolution camera to recognize people and determine if they have opted to make their names known to the system. All this information is relayed to the user through audio cues.

For example, if the device detects a person one meter away on the user’s left side, the system will play a click that sounds like it is coming from one meter away on the left. If the system recognizes the person’s face, it will play a bump sound, and if that person is also known to the system, it will announce their name.

When the user only hears a click but wants to know who the person is, a second layer of sound that resembles an elastic band stretching guides the user’s gaze toward the person’s face. When the lens’ central camera connects with the person’s nose, the user hears a high-pitched click and, if the person is known to the system, their name.

“I particularly like the thing that gives you the angle of gaze because I’m never really sure what is the sensible angle for your head to be at,” said Bosher, who worked with the Project Tokyo team on the audio experience early in the design process and returned to the Cambridge lab to discuss his experience and check out the latest iteration. “That would be a great tool for learning body language.”

Peter Bosher, middle, an audio engineer who is blind who worked with the Project Tokyo team early in the design process, checks out the latest iteration of the system at Microsoft’s research lab in Cambridge, UK, with researchers Martin Grayson, left, and Cecily Morrison, right. Photo by Jonathan Banks. Prototyping with adults

As the Project Tokyo team has developed and evolved the technology, the researchers routinely invite adults who are blind or with low vision to test the system and provide feedback. To facilitate more direct social interaction, for example, the team removed the lenses from the front of the HoloLens.

Several users expressed a desire to unobtrusively get the information collected by the system without constantly turning their heads, which felt socially awkward. The feedback prompted the Project Tokyo team to work on features that help users quickly learn who is around them by, for example, asking for an overview and getting a spatial readout of all the names of people who have given permission to be recognized by the system.

Another experimental feature alerts the user with a spatialized chime when someone is looking at them, because people with typical vision often establish eye contact to initiate a conversation. Unlike the bump, however, the chime is not followed by a name.

“We already use the name when you look at somebody,” Grayson explained to Emily, a tester in her 20s who has low vision and visited the Cambridge lab to learn about the most recent features. “But also, by not giving the name, it might draw your attention to turn to somebody who is trying to get your attention. And by turning to them, you find out their name.”

“I totally agree with that. That is how sighted people react. They capture someone out of the corner of their eye, or you get that sense, and go, ‘Cecily,’” Emily said.

The modified HoloLens the researchers showed to Emily also included an LED strip affixed above the band of cameras. A white light tracks the person closest to the user and turns green when the person has been identified to the user. The feature lets communication partners or bystanders know they’ve been seen, making it more natural to initiate a conversation.

The LED strip also provides people an opportunity to move out of the device’s field of view and not be seen, if they so choose. “When you know you are about to be seen, you can also decide not to be seen,” noted Morrison. “If you know when you are being seen, you know when you are not being seen.”

The U.S.-China tech dispute breeds suspicion in Silicon Valley

The first time 35-year-old Zhang Lelin, a software engineer from China, felt uncomfortable in Silicon Valley was last year when he heard that a “huge team” of FBI agents had raided the home of one of his Chinese neighbors.

The target, Zhang Xiaolang, was charged with stealing trade secrets when he left Apple to join a Chinese electric car start-up and is now awaiting trial. Zhang Lelin, who works at Amazon, says the drama had been “like a movie” and wonders if it was engineered to “send out a message” to other Chinese tech workers.

His worries have been compounded not only by other high-profile cases of alleged Chinese intellectual-property theft, but by the increasingly antagonistic relationship between Washington and Beijing. The darkening climate has rattled Chinese companies, investors and — most of all — employees in Silicon Valley.

“For years, Silicon Valley ran on three main currencies: code, connections and cash. Chinese engineers churned out plenty of the first, and Chinese venture capitalists brought an infusion of the third, making the valley a relatively easy place for them to blend in,” says Matt Sheehan, author of The Transpacific Experiment, a book on Chinese and Californian collaboration.

Events of the past two years have changed the equation. “First, [the trade war] put a stop to venture capital flows, then it began complicating things for the big companies, and now it’s filtered all the way down to individual researchers,” says Sheehan.

This is not the first time that the U.S. government’s fears over technology have placed undue suspicion on immigrants with a Chinese connection. In 1999, Taiwanese-American scientist Wen Ho Lee, who worked in the Los Alamos nuclear bomb-design laboratory, was arrested on suspicion of spying for Beijing and placed in solitary confinement for nine months. The charges could not be proven because they were largely based on his friendliness with Chinese scientists.

Physics professor Paul Chow shows his objection to the arrest and detention of Dr. Wen Ho Lee on a charge of mishandling classified data during his work at Los Alamos National Laboratory in 2000.

(Los Angeles Times)

But the climate of suspicion has ramped up considerably in recent years, fueled by trade war tensions, the controversy over telecom-equipment supplier Huawei, and a string of cases involving Chinese nationals being accused in the U.S. of stealing trade secrets.

One proposal from the Commerce Department is particularly chilling for Chinese nationals in the valley, since many believe it could lay the legal basis for U.S. tech companies to segregate them or even fire them. The Commerce Department has said it wants to expand existing export controls on so-called “dual-use technologies,” which have both civilian and military applications, to what it calls “emerging” and “foundational” technologies. The agency is now looking into categories such as artificial intelligence and microprocessor chips.

Since U.S. rules consider even conversations about technology between Americans and foreign nationals as “deemed exports,” tech companies may have to apply for licenses for their foreign workers from countries outside of NATO. Chinese nationals are classed as the most sensitive, alongside Russians and Iranians.

In 2018, Commerce more than halved the number of Chinese nationals it granted export licenses to, awarding only 350 compared with 771 the previous year — the biggest proportional decrease of any country.

As a result, several Silicon Valley companies’ trade compliance and HR teams are already laying out plans for segregating their Chinese-national employees from the rest of their researchers, according to Dan Wang, analyst at Gavekal Dragonomics in Beijing.

Kevin Wolf, an export controls lawyer and the assistant commerce secretary during the Obama administration, however, thinks such concerns are premature, “because the new controls are likely to be narrowly tailored to address specific national security threats, as required by the export control statute.”

Libo Weng, an engineer at a semiconductor company in California, says it is already becoming more difficult for Chinese engineers to progress at work.

“From conversations with colleagues and friends, I think employers are more careful about letting Chinese engineers rise into positions where they can come into contact with core technologies — AI, chips — in which the U.S. is more advanced than other countries. It was not as severe as this before Trump came into power,” says Weng.

Chinese nationals and first-generation immigrant families make up a significant portion of Silicon Valley. According to census data, 17% of the population speaks a form of Chinese at home in the Silicon Valley counties of Santa Clara and San Mateo. The valley’s longstanding ethnic diversity and the reliance of its tech companies on Chinese workers mean that it’s still welcoming to Chinese talent.

University Avenue in Palo Alto. According to census data, 17% of the population speaks a form of Chinese at home in Santa Clara and San Mateo counties.

(Noah Berger / For the Times)

But few Chinese engineers have made it to the very top of U.S. tech companies, and the tensions between the U.S. and China have prompted many to examine where they stand in their workplace. In particular, Chinese engineers complain that the precarious nature of their work visas means they feel the need to cling to their employers, regardless of unfair or poor treatment.

Under the H-1B visa program for talented workers, which sustains most foreign nationals at tech companies, an employee is given just 60 days after being fired to find a new job, apply for and receive a new H-1B visa — after which they have to leave the country. Many feel the window is impossibly tight.

Employees at Amazon, Google and PayPal all cited heightened visa insecurity as the main pressure point facing Chinese staff. In 2018, the approval rates for H-1B visas started to fall, reaching 75.4% in the last quarter of the year, compared with 94.5% in the same quarter in 2015.

At the same time, processing time has taken longer as more applicants are sent through a second “request for evidence” stage. Five years ago, Zhang Lelin’s renewals took less than a month, but now they take two, he says, blaming the delay on his previous work on AI and computer chips.

He says one of his friends was stuck in China for seven months waiting for a visa renewal. “We all have the same background, so this kind of thing is random and could fall on your head any time,” he says.

These tensions have taken their toll. In September, a 38-year-old Chinese engineer for Facebook, Qin Chen, committed suicide by jumping from a building at Facebook’s Menlo Park headquarters. A week later, hundreds of engineers — mostly Chinese — from companies across the valley gathered on the campus to chant: “Chinese lives matter, Zuckerberg!”

Some spoke of the need for a better work environment, and others highlighted the fragility of Qin’s immigration status. Despite apparently living in the U.S. for eight years, he did not have a green card and his entire family depended on his work visa.

Facebook said at the time that it was providing mental health and suicide prevention support to employees, and also added that bullying was not tolerated at the company.

Other Chinese engineers complain that Silicon Valley has an ethnic glass ceiling, and interviewees, many of whom are naturalized American citizens, compared their status with the relative success of Indians in large tech companies.

“In the U.S. many Indian managers have become big companies’ CEOs, but in many ways there’s still a bottleneck for Chinese people,” said Hans Tung, managing partner of GGV Capital, at a recent conference in Beijing. Turning to Eric Yuan, the founder of Zoom, he asked: “What do you think can be done?”

Yuan is Silicon Valley’s best-known Chinese immigrant success story. His video conference app Zoom, which listed last April, is valued at almost $21 billion. Yuan responded that many Indian engineers were not only proficient at technical work, but also at understanding business models and management.

“It’s Chinese culture: It emphasizes obedience and modesty, not confidence,” says Sophie Xu, a PayPal employee who left China for Canada and then the U.S. 17 years ago. “There’s no wild ambition, we love book smarts.”

At an evening discussion organized by the Silicon Valley Innovation Entrepreneurship Forum, which hosts some of the biggest gatherings of Chinese tech talent in the area, a panel of high fliers dispensed advice on how to organize one’s time and succeed at work.

“These are the rules of the game in the U.S.,” says Zhang Yue, a Google engineer who also organizes a Chinese new year’s gala and a local athletics club.

“Unlike a Chinese company, you don’t need to stay in the office until after your manager leaves — what matters is you get it done. And saying what you’ve done can be even more important than doing it,” says Zhang.

Chinese companies and the government are keen to take advantage of the worsening climate to lure talent home.

“The long-term impact of the trade war will make it easier for China to attract talent back,” says Tim Li, who has spent the past five years recruiting people from Silicon Valley to return to China, so-called “sea turtles.”

“There are also pull factors from China,” he adds, referring to China’s domestic tech boom. Li recently posted an advertisement for his services under a LinkedIn post by a Silicon Valley returnee to Beijing that had gone viral.

 After a Chinese engineer committed suicide by jumping from a building at Facebook’s Menlo Park headquarters in September, hundreds of engineers from companies across the Valley gathered on the campus to chant: “Chinese lives matter, Zuckerberg!”

(David Butow / For The Times)

The post was titled, “To those Silicon Valley people who want to return [to China], but don’t dare.” It argued that working for a mature U.S. firm meant a comfortable lifestyle with limited opportunities for advancement, while returning to China meant a more cut-throat working environment, but one where people could have a greater impact.

“The U.S. has displayed strong anxiety and discomfort over China’s rise. ... China’s internet industry will surpass the U.S.’ on many fronts,” the post said. “If the long-term trend is like this, I think you should join the wave and take advantage of it.” The author, Wei Xu, recently left Facebook for a Beijing-based start-up.

Some, however, are more cautious. Zhang Lelin fears that Chinese engineers jumping from U.S. to Chinese firms might trigger suspicions within their company or the FBI over IP theft. “That makes me hesitate when considering a Chinese firm,” he says.

China’s government-sponsored talent recruitment agencies have also spotted the repatriation opportunity. But they are also more cautious in operating in the U.S. now because they fear increased scrutiny from the U.S. government, according to one former recruiter.

Instead, adds Li, China’s private sector is moving quickly. “The semiconductor industry here is shrinking, whereas in China it’s expanding — there are lots of laid-off workers to be picked up,” he says.

Zhang Lelin says Chinese employees working at U.S. tech companies are “cannon fodder”: “We get hurt by the U.S.-China dispute, but we don’t decide the outcome.”

He reflects on how, during World War II, the U.S. government forcefully relocated more than 120,000 Japanese-Americans to internment camps, a decision that had little to do with national security.

The first such camp was built in Manzanar, Calif., a few hundred miles from where most tech companies are now based. For most workers in the valley, that dark chapter of U.S. history is in the past. But for some, it is not quite as far removed as it once seemed. “Look at history,” says Zhang Lelin.

© The Financial Times Ltd. 2020. All rights reserved. FT and Financial Times are trademarks of the Financial Times Ltd. Not to be redistributed, copied or modified in any way.