Machine Learning and Art (1 of 2)

Machine learning and deep learning are increasingly disrupting all sectors of society, making it possible to improve artificial intelligence, halt the spread of malware, among many other benefits. However, scientists are not the only ones interested in it. So are artists…

As part of a series of blog posts, CMF Trends met with Google engineer Damien Henry during the Google I/O 2016 developers conference.

Damien Henry leads the Cultural Institute Experiment Team (CILEx), part of the Google Cultural Institute. This team uses modern tools—including machine learning—for artistic purposes.

Machine learning is an IT field in which tasks are not programmed directly by a human being. Instead, an algorithm learns to complete the task on its own. Instead of explaining to a computer how to recognize the picture of a dog, a developer can provide the computer with millions of images—some showing dogs, others not—and the computer will analyze them and learn how to recognize a dog.

Damien Henry also co-invented with David Coz the Google Cardboard virtual reality glasses.

Interview with Damien Henry

Damien Henry at Google I/O 2016
Photo credit: Maxime Johnson

CMF Trends: What is the CILEx?

Damien Henry: The CILEx is a small team at the Google Cultural Institute that experiments along three axes. The first axis involves engagement, i.e., everything and anything that enables people to come into contact with culture. The second axis consists of organizing the information to which the Google Cultural Institute has access. Discovery involves finding ways to present things such as to help users find their way among millions of works. The third axis involves analyzing the data to determine, for example, if the use of colours evolves over time.

For the past year now, we have been operating a small residence where artists are invited to work with us. We are seeking to determine what happens when we provide artists and creative programmers access to our tools. Up to now, it is truly surprising how diverse the results have been.

CMFT: How can machine learning be useful for artists?

DH: The CILEx team uses a highly pragmatic approach. We use existing technology and try to apply it to various artistic fields. And we have no preconceived ideas.

For example, we have used machine learning to categorize millions of paintings. But categorizing represents only a very small portion of what neural networks can accomplish. For example, paintings can be placed in three-dimensional spaces for the purpose of determining if the images are similar or different and so forth.

It is also possible to create works from deep learning algorithms. That’s exactly what has done Mario Klingemann, a code artist who is residing with us at the moment.

CMFT: You use a lot of paintings for machine learning purposes. Can machine learning also be used with other art forms?

DH: Machine learning is also starting to be used to analyze and process video content. For example, an algorithm is able to itself colorize black and white films.

But, until further notice, images dominate. It must be said, however, that a lot of research is being conducted on images and that it remains the field where existing tools work best.

But that could also be applied to other fields. Studies are being led in an attempt to use machine learning on 3D objects for example. When searching for scientific articles on deep learning, new articles are located practically each and every week.

CMFT: Will machine learning make it possible to develop a better understanding of art?

DH: One must be very humble when it comes to machine learning and art. If we today have access to all of these works, it’s because generations of curators saw to preserving them in the first place.

People have been studying these paintings for hundreds of years. It would be pretentious to contend that an algorithm would automatically give us access to new knowledge.

That being said, these are new tools. It is therefore not impossible for someone who has good knowledge of art and who uses these tools to make a new discovery.

However, the tool itself will never discover anything; the experts will. It’s a good time to undertake a doctorate on art and machine learning seeing as there’s lots to accomplish in that regard.

CMFT: Is machine learning within everyone’s reach?

DH: Yes. The machine learning community has always been very open and open-source tools abound. Anyone who is motivated and curious does not need to be a good programmer to use machine learning. It’s easy to understand the concepts and try things out. Experimenting is also easier than it once was because the algorithms are quicker than they used to be.

Although machine learning has existed for a long time, it’s the progress accomplished with respect to computers, tools and the community that today creates conditions that make it possible for anyone to try it out.

CMFT: Will machines one day be able to create art themselves?

DH: Generative art, whereby software applications are used to create art automatically, has existed for a long time and machine learning could be used to improve its techniques.

But there’s always a starting point, and that’s human will. A computer can be programmed to write words, but it’s a human being that initially coded the program.

Art and machine learning: where to start?

As pointed out by Damien Henry, with a vibrant community, open-source tools and increasingly rapid software, everything is in place for anyone who knows even the slightest bit about programming to master machine learning.

Here are four resources that the engineer recommends to artists and programmers who are interested in discovering this technology:

  • AMI Initiative: a program bringing together artists and Google engineers to realize projects using machine learning

  • Project Magenta: the publications of a team of researchers on using machine learning to create art

You can also read a second post with examples of art and machine learning experiments carried out by CILEx team residents.


Maxime Johnson
Maxime Johnson is an independent journalist who specializes in the analysis and observation of new technology. He writes a column for the Métro newspaper and for L’Actualité magazine. He also collaborates with several magazines including Protégez-vous and can be heard on the radio, namely during ICI Radio-Canada Première’s La sphère show.
Read Bio