Context and Lexicon​ — A Brief Discussion on the Possibilities of Technology in Future Art and Design Education

(Speech by Gong Jiawei at the 2021 Shenzhen Design Week – China Future Design Education Forum)

Both art and technology serve to define our lived world, continually reshaping it. The technologies within art not only challenge our perceptions but also constantly push our understanding of nature to new levels. Since ancient times, art and technology have shared a mutually dependent and collaborative relationship. The continuous exploration in new materials, processes, and programs has, from the very beginning, provided artists with original means of expression.

Historically, there are countless examples of artists engaging in scientific research, with Leonardo da Vinci being one of the earliest prominent figures. Although many of his scientific findings were not widely recognized until nearly 150 years later, Leonardo (1452–1519) was undoubtedly a polymath whose work spanned multiple disciplines including civil engineering, chemistry, geology, geometry, fluid mechanics, mathematics, mechanical engineering, optics, physics, and zoology. As an engineer, his ideas were far ahead of his time—conceptually inventing the parachute, helicopter, armored vehicle, centralized solar energy utilization, calculator, the basic theory of plate tectonics, and the double-hulled ship.

In the 1660s, British physicist Isaac Newton conducted a series of experiments with sunlight and prisms, proving that white light is composed of seven visible colors. This discovery laid the foundation for later developments in color theory among the Impressionist painters. At the same time, the invention of metal tube-based painting pigments allowed these painters to venture outdoors and capture the ever-changing interplay of light and color. British photographer Eadweard Muybridge’s pioneering research in motion photography propelled early cinema, while Walt Disney revolutionized the film industry by employing moving images in animated films and releasing the first sound-synchronized animation. Nam June Paik, the father of video art, demonstrated another possibility of the fusion of technology and art with his installation Electronic Highways 1995—an artwork composed of 47 channels, 313 displays, neon lights, and steel structures.

Observing contemporary life, it is clear that in the 21st century technology has deeply integrated into our daily existence. Take mobile technology as an example—it provides speed and convenience, yet it incessantly occupies and drains our life energy. As London illustrator Andrew Rae depicted, the fantastical monsters appearing on mobile screens have become a metaphor for modern life.

Regarding technology’s impact on art, a 2013 Pew Research Center survey in the United States found that both individuals and art institutions generally believe that the Internet and social media have increased participation in art, making the experience more interactive and diversifying audiences. Respondents also largely felt that the Internet plays an important role in broadening the boundaries of art. However, most art institutions also noted that mobile devices, ringtones, and text messages significantly disrupt live performances, and that technology has fostered the expectation that “all digital content should be free.” Although opinions differ on whether technology negatively affects the attention of live audiences, there is consensus that technology has not “diluted” art; rather, by opening new avenues for art participation and critique, it has transformed artistic expression. The digital world, as an immensely democratizing force, bridges gaps between the rich and the poor, the educated and the uneducated. The ubiquity of computers—in homes, schools, or local libraries—has rendered issues that once seemed critical as less significant.

Current and emerging technological developments that are likely to impact art design and education include immersive media and artificial intelligence. Immersive media encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR replaces real space by constructing a virtual or digital environment, immersing users completely into the digital world through space projection or head-mounted displays and sensors; AR blends real scenes with computer-generated digital content by overlaying virtual information onto live images; and MR experiences, which may include holograms or holographic projections, offer an even deeper level of interactive engagement than AR.

A concept that has recently garnered much attention is the metaverse—a virtual world constructed through VR and AR technologies where people can socialize, work, and entertain themselves. Facebook CEO Mark Zuckerberg believes that the metaverse represents the future direction of the Internet and his company; meanwhile, Microsoft founder Bill Gates made an important prediction in his “2021 Year in Review” blog, suggesting that within the next two to three years most virtual meetings will transition from 2D camera images to three-dimensional spaces featuring digital avatars.

The application of immersive media in art also reveals remarkable potential. In 2019, the Culturespaces Foundation transformed an exhibition space into a vivid painting using 140 state-of-the-art laser video projectors and 50 speakers, making visitors feel as if they had stepped into a Van Gogh artwork—a multisensory, musical, and aesthetic experience accessible to art lovers of all ages. Similarly, teamLab’s interactive digital installation, Water Particles in an Oil Tank, uses a massive waterfall formed by water cascading inside a tank; when visitors stand on the installation, they act as a rock that diverts the flow of water. The work continuously changes based on the audience’s actions, aiming to create a hyper-subjective space where viewers become the center of the experience, freely partitioning, folding, or merging the space.

Another example of technology’s impact on art is the application of artificial intelligence (AI). The AI Portraits online platform, developed collaboratively by the MIT-IBM Watson AI Lab and Politecnico di Milano, utilizes information from over 45,000 15th-century classical paintings to “create” portraits from uploaded photographs. By analyzing image features and backgrounds, the platform determines the optimal style for the portrait, generating a unique 4K image in the style of classical masters. Interestingly, the platform also noted a scarcity of smiling or laughing subjects in the portraits—since traditional portrait painters rarely depicted smiles—thus offering us a new perspective on art history.

In the realm of AI and art creation, Christie's sold the first AI-generated artwork at a major auction in 2018 for $432,500. Created by the French trio Obvious, the work was intended to explain and popularize artificial intelligence through art. Although some pointed out that the code generating the painting was largely written by another artist and programmer, this event nonetheless marked the debut of AI art on the global auction stage. At the same time, 19-year-old Robbie Barrat shared his Generative Adversarial Network (GAN) algorithm under an open-source license, further advancing AI’s application in artistic creation.

Today, it is easier than ever for artificial intelligence to generate images that look realistic but are entirely fictitious. To meet the needs of digital designers, Icons8 created Generated Photos, a repository of 2.6 million AI-generated human faces that are free of copyright restrictions, suitable for websites, mobile apps, emails, landing pages, user avatars, and more. The diversity of facial features and consistent lighting effects provided by this resource make it remarkably simple to integrate multiple individuals into the same project.

NVIDIA’s GauGAN2, named after the Impressionist painter Paul Gauguin, generates realistic images from segmentation maps—labeled sketches that describe scene layouts. Artists can design landscapes using text, brushes, fill tools, or a combination thereof, and style transfer algorithms allow creators to apply filters to transform a daytime scene into a sunset or convert a real image into a painted style. Meanwhile, NVIDIA’s Vid2Vid Cameo employs deep learning models to enhance video conferencing experiences; users can submit a refined 2D photograph or cartoon avatar before a call, and an AI model synthesizes a realistic talking-head video from the static image using generative adversarial networks, mapping facial movements in real time to capture dynamic changes.

Furthermore, well-known Photoshop introduced an AI-powered “Neural Filters” feature in its 2021 version. For example, the “Smart Portrait” filter can alter the subject’s age, expression, pose, and skin tone by analyzing the portrait and adjusting facial features to convey emotions such as happiness, surprise, or anger. It can even shift the direction of the eyes or head, modify lighting angles, adjust hair density, alter the intensity of a smile, or transfer makeup from one photo to another—thereby achieving effects such as aging.

Technology has provided art with unprecedented methods and tools; it seems that today’s art and design practices can no longer do without technological support. From another perspective, the concept of STEM—science, technology, engineering, and mathematics—has been widely discussed in Western educational systems, often used to articulate school policies or curriculum choices. As an idea, STEM attempts to construct an integrated curriculum spanning all four disciplines, enabling students to solve problems using interdisciplinary knowledge. This signifies a shift away from traditional learning methods that focus on memorization and recitation; instead, students are rarely provided with a clear-cut solution and are expected to independently seek answers using what they have learned, a process that demands creativity, flexible thinking, and a deep understanding of multiple fields. Although science, technology, engineering, and mathematics have been interrelated since ancient times, the term “STEM” did not gain widespread recognition until the early 21st century—and only after 2005 did it truly begin to develop momentum, eventually forming its own core group within the U.S. Congress and frequently making headlines from 2008 onward.

The United States and Europe have long been promoting an increase in STEM graduates to meet the growing demand in related professions. Educators and professionals in STEM fields have advocated for the introduction of STEAM (science, technology, engineering, art, and mathematics) into K–12 curricula, adding an artistic component to the existing STEM foundation. It has become increasingly clear that art can broaden the scope of STEM education and application—a solid grounding in art can provide additional impetus for innovation and fearless creativity. John Maeda, the president of Rhode Island School of Design, is a prominent proponent of integrating art into STEM, believing that opening the door to art helps transform critical thinking into critical creation. Studies have shown that students with a strong foundation in the arts tend to perform better overall. For example, the University of Florida found that high school students who studied art for four years scored, on average, 98 points higher on the SAT than those who studied for half a year or less; students who took music appreciation scored 61 points higher in the verbal section and 42 points higher in the math section.

Although the STEAM movement is still in its formative stages and is gradually being organically integrated into STEM, some artists are already approaching the challenge of integration from a different angle, striving to bring art and science closer together. For instance, NASA once sought the assistance of origami artists to help design radiation shields for spacecraft, aiming to protect future astronauts from cosmic rays and electronic component damage.

Returning to the theme of “Context and Lexicon,” we can say that the ever-changing modern technologies continuously provide us with new cognitive contexts, prompting artists, designers, and educators to seek new creative vocabularies to interpret and define these emerging sociocultural experiences. Meanwhile, technological advancements themselves gain recognition and validation in contemporary society precisely through such cultural interpretation and redefinition.