By Isabel Galwey

Some of the most famous animations of the socialist period in Mainland China were made using the Shanghai Animation Film Studio’s ink painting animation techniques. Masterpieces such as Little Tadpoles Look for Mama (1960) and Feelings of Mountain and Water (1988) brought traditional paintings to life through painstaking frame-by-frame animation. Since the 1980s, however, this technique went out of fashion for various reasons, including the labor-intensive process required to produce even a short piece of ink-painting animation using analogue methods.

However, in the past two decades, strides in digital technologies have breathed new life into the ink-painting animation medium. Some of the most exciting work using ink-painting techniques in recent years has taken place in Hong Kong, where animators and software developers joined forces to produce both technological and aesthetic innovations.

In August 2021, I had the opportunity to speak with Dr Nelson Chu, creator of ink painting software Expresii (寫意), who gave an illuminating talk at the Inaugural Conference of the Association of Chinese Animation Studies in Spring 2021. At the conference, he discussed some of the applications of Expresii and how it has evolved over the years (Interested readers can find the follow-up post on his blog here). This article revisits some of the topics discussed in Dr Chu’s presentation, including the past and current applications of his software for animation and his thoughts on the future of digital art creation.

As well as Dr Chu, I was lucky enough to sit down with animators Angela Wong and Anita So of Studio Biped (二足步行動畫有限公司), both of whom have been pushing the boundaries of Dr Chu’s software in their latest project, Find Find (爪抓). We discussed the experimental animation methods that they developed, the challenges and opportunities of working with ink-painting animation, and their future hopes for the development of the medium in Hong Kong and elsewhere.

Origins of Expresii

Dr Chu first developed ink simulation software in the early 2000’s as a response to a gap in the market at that time. He is a keen artist himself but studied computer science at university: “many people would say that in art it’s hard to make a living,” he explained, “so I did computer engineering so as to have better job security later.”

Dr Chu noticed that the existing brush simulation software was designed on stamp technology rather than replicating the actual brush shape and fluid dynamics of a traditional Chinese ink brush. He decided to research and develop an alternative. After some years of work, he published his findings in a paper at ACM SIGGRAPH (the Association for Computing Machinery’s Special Interest Group on Computer Graphics and Interactive Techniques), which attracted interest from international software companies like Adobe. He later landed a full-time job working for Microsoft in the US, simulating Western art media like oil and pastel. After resigning from Microsoft, he started building a new paint program called Expresii, which currently focuses on ink painting.

According to Dr Chu, the key features of Expresii which set it apart from other software are: the brush, which is able to capture the nuances of hand movements; the powerful fluid simulation; and the “deep Zoom” capability, which combines the scalability of vector graphics with the organic appearance of raster images (those composed of individual pixels).

Video 1: Expresii Official Release

As Dr Chu’s digital ink-painting methods attracted attention in the scientific community, artists and filmmakers also began to take notice. Hong Kong special effects studio Menfond Electronic Arts (萬寬電腦藝術設計) used the software for sequences in their blockbuster A Chinese Tall Story (情癲大聖, 2005). It was also used by the US-based creative agency Psyop, and by a Beijing-based studio for a sequence in the opening ceremony of the Beijing Olympics. For the most part, Dr Chu was always keen for artists and practitioners to exchange ideas and use the software in new ways: “studios all around the world contacted us… We had mutual respect. They trusted us, and we trusted them.”

As the Expresii ink painting software continued to develop, one creator, Angela Wong, decided to experiment with a new application for the software: in recreating the effect of ink-painting animation.

Collaborating with Animators 

Angela Wong first used ink painting software to animate a number of short frame-by-frame animations. She was working at the SIGGRAPH festival in Macau, where Nelson Chu was exhibiting the Expresii software. Her first ever digital ink-painted animation was a run cycle for a fluffy-tailed red squirrel. Her early Expresii animations also included swallows in flight and the atmospheric short film Please Mind the Gap (請小心月台空隙, 2018).

Video 2: Swallow Animation Process  

Video 3: Please Mind the Gap

However, it was the squirrel which went on to become the basis for the development of the Red Squirrel Mai, which was originally conceived by the Rooftop Animation Studio as the first fully-digital ink-painting feature film. However, using Expresii for a large-scale animation project proved to be extremely difficult. To start with, the sophisticated ink simulations and rendering in Expresii required computers with good graphics (GPU) processing power. The workspace was also not set up to include a timeline, onion skins or other staples for the animation pipeline. This meant that the animators had to complete a rough frame-by-frame version of the animation and then import it into Expresii, which was extremely labor-intensive.

Video 4: Squirrel Animated

Secondly, it was difficult to train animators to work with the software. Angela Wong recalls that it took two to three months to train each person to properly use Expresii for animation. Still, despite the setbacks, the team managed to create some beautiful ink-painting animated sequences, even incorporating 3D camera work into the shots.

Video 5: Red Squirrel Mai Trailer

Challenges and Contradictions

As the experience of Rooftop Animation demonstrated, ink-painting animation is extremely difficult, and Angela Wong observed that in some ways, it even seems like a contradiction in terms. xieyi (寫意) ink painting, especially, which has a lot more energy and takes much of its aesthetic from imprecision, is challenging to combine with an animated workflow that requires consistent character models and movements. baimiao (白描) ink painting, a style which is more detailed and less impressionistic, would have been easier to streamline for animation production, but Nelson’s software was developed with the bold and free xieyi style in mind.

In the end, after an immense amount of work, the Red Squirrel Mai project moved in a different direction. However, Dr Chu, Angela Wong and Anita So continued to experiment and use trial and error to find a solution to their ink-painting animation conundrums. Angela Wong and Anita So emphasized the importance of taking time out to reflect and develop a more efficient solution—one that truly took advantage of the capabilities of the Expresii technology.

“Programming,” says Angela Wong, “isn’t like traditional animation. You can’t just push through and ‘draw it out’. You have to step back and wait for the solution.”

Multidisciplinary Innovation and Problem Solving

Dr Chu always emphasized that his software was a work in progress, and that he was keen to allow artists to “have the privilege to use my Beta features,” if they were useful. He worked with Angela Wong and Anita to adapt the software to make it easier to use for animation. Likewise, Angela Wong and Anita continued experimenting with Nelson’s software, trying to figure out ways that the software could be used in a more efficient way. In the end, forming a new studio—Studio Biped—and beginning on a new animation project allowed Expresii’s use in the animation workflow to take on a new lease of life.  

Eventually, they hit upon a technique: using After Effects to build a rig (a sort of framework for an animated character, like a puppet that can be moved) and then programming Expresii to make brush strokes which followed the rig’s movements. With the financial backing of the Hong Kong Animation Support program, Studio Biped applied this technique to their first short film, Find Find  (抓爪, 2022), which follows the adventures of an octopus exploring the ocean, including some man-made objects which have made their way down to the seabed.

Video 6: Find Find

Studio Biped were even able to program the software to add slight jitters of dryness, in order to simulate the organic textures of ink painting. This workflow massively speeded up the animation process, avoiding the labor-intensive methods used to apply Expresii to animation in earlier years. After a collaborative process, Dr Chu and Studio Biped were able to find an innovative and multidisciplinary way of creating digital ink-painting animation.

Adaptation was also a key element of the story that Anita So and Angela Wong hoped to tell in Find Find. For Anita So, what was important in the narrative was that it should emphasize nature’s ability to adapt. When they were first developing the idea, Anita So recalled, some of their mentors suggested that they should tell a heart-rending tale of an octopus which is suffering in a polluted ocean. Instead, they decided to tell a realistic yet optimistic story of an octopus taking a piece of flotsam—a plastic cup—and making it into its home.

Just as they turned away from cliché in their storytelling, Studio Biped also decided to take a different approach to the visual style for the animation itself. Although they were using the Expresii software, they didn’t attempt to directly emulate the “traditional” ink-painting aesthetic in Find Find, either. The ocean setting gave them many opportunities to try out Dr Chu’s software in novel ways. Although they borrowed certain aesthetic conventions, such as the use of long horizontal and vertical pans, rather than giving the feeling of a 3D space, and the use of finely inked patterns and textures in the seascape backgrounds, they did not market the film as an ink-painted animation. Instead, they were combining elements of traditional ink-painting with their own ideas and going in a new direction.

For himself, Dr Chu has also been happy to see his software being used for a variety of projects, which in turn pushes the software forward. In the future, he hopes that he will be able to expand its functionality to include other techniques such as the bloom effect in Western watercolor, or perhaps adapt it for pastels. Meanwhile, Studio Biped are considering branching into game development, as well as further developing Expresii’s compatibility with After Effects to optimize their animation workflow and make it more accessible to others interested in making ink-painting animation.

The Future of Ink-painting Animation in Hong Kong

Dr Chu’s developments with Expresii are far from the only attempts to bring ink painting into the digital age. The Shanghai Animation Film Studio itself is currently working on a new feature-length ink-painting animation (though not yet using Expresii), and there have been many other attempts to digitize the art form in recent years.[1] After the long process of trial and error that led to ink-painting animation in Hong Kong, do Dr Chu and Studio Biped think that there are any steps that could be taken by institutions and organizations to make research, development, and production easier in the future?

Dr Chu suggested that more funding into research and development in higher education would be helpful. “When I show you my work, it’s always pushing boundaries. I want to do something better, I want to solve problems like, how do you get the best of both worlds, with raster [and vector], for example.” He suggested that the government could also give support in the form of grants or tax breaks for researchers and creators at the intersection of technology, the arts and business. He also points out that in animation, research and development does feed directly back into the end product, giving it a clear outcome and impact: “just like Pixar. Renderman is a software product they use themselves to do all those movies… A lot of the research projects come from actual needs in the production, and they actually use the software.”

Similarly, Studio Biped emphasized that the ink-painting animation process requires a lot of man- and computer-power, and it is important to know that this kind of innovation cannot happen overnight. They suggested that a possible route for support would be separating Hong Kong’s existing Animation Support Program, there should be grants available for dedicated research and development into animation technologies. Sometimes, there is a gap between investment for specific productions, more academic research, or fine art practice. Animation, they observe, can somehow slip through the cracks—especially for creatives who may not know how best to navigate the funding “game.” 

Both animators and software developers understand the necessity of taking things systematically, and step-by-step, and they are reluctant to employ buzzwords or hot topics like “big data” or “artificial intelligence” to try and procure funding. Dr Chu is wary of such hype: “I think, people should really go back to the basics and be realistic.”

He emphasized the fact that the software is a work in progress and any progress made will be incremental, taking time and effort to develop. While he is skeptical of the influence of virtual reality and artificial intelligence on the animation industry, he did concede that the rise in GPU made powerful tools like Expresii accessible to more people than would have been thought possible just a few decades ago. When he was just starting out, he recalls, “they were saying, if you use fluid dynamics, there’s no way that you can get it done in real time. But I kind of had faith, I still want to try. If such a solution doesn’t really exist, hopefully I can find the second-best thing to do and still make some progress.”

Aside from research, education is also important. Animators and software developers can pass on their knowledge to students interested in multidisciplinary careers, as well as training them to use software such as Expresii, which in turn will spur further development of the program itself. Studio Biped and Dr Chu also hope that once the finished version of Find Find is released, it will play a role in convincing others of the viability of digital ink-painting animation, whether to invest in it or to try it out themselves. The potential of animation itself to inspire viewers to learn more about the production process—whether from a creative or technical perspective—should not be underestimated. New technologies have given ink-painting animation a new lease of life. The combined passion and dedication of artists and scientists will usher ink painting into the digital age.

Video 7: Expresii Demo Ashitaka 

[1] For more information on other recent ink-painting animation, see Chen Hailu’s article “CGI Ink-Painting Animation in Contemporary China, 1989-2019,” Association for Chinese Animation Studies, July 24, 2020.

Bio:

Isabel Galwey is a second year MPhil student in the Division of Humanities at the Hong Kong University of Science and Technology. Her research focuses on the digital turn and its impact on the medium of animation in Hong Kong. Prior to beginning her graduate studies, Isabel completed a BA in Chinese Studies at the University of Oxford and Peking University, from which she graduated with a first, and worked at a UK-based media company. She has contributed as a translator to the volume Chinese Animation and Socialism: From Animators’ Perspectives (Brill, 2021) and written for Global Storytelling: Journal of Digital and Moving Images (Michigan Publishing, forthcoming) as a reviewer. She will present a paper on 1950s left-wing animated filmmaking in Hong Kong at the 2022 Society for Animation Studies conference. She also makes her own animation.