Tarot Card Generation with Machine Learning

Idea & Inspiration

There is a certain kind of magic in choosing through a process that seems seemingly random. Tarot cards are one example, where there is a series of cards and we try to divine meaning through cards that we choose at random. If the idea of Tarot Cards is to generate meaning through a randomized chaos, I figured that this would be a good idea to machine-generate a deck of tarot card images and their meanings. Another benefit of using Tarot Cards is the established rhythm of how the cards are drawn — they are usually portrait, a bit long in format, and the names are usually written at the bottom. They have a certain mystical atmosphere, and I was hoping that synthetic mediums would be able to make sense out of the card images that I would feed it.

---

Process

This project initially started out as an experiment related to my Electronic Rituals, Oracles and Fortune Telling class. However, I decided to explore further since I was starting to learn so much from this project.

For this project, I collected about 700 tarot card images from the internet (source links [1], [2], [3] and [4]) and used StyleGAN on RunwayML to train a tarot card model. Then, I compiled interpretations of readings and ran them through GPT-2 on RunwayML to generate new meanings. Finally, for the names, I settled on having a list of feelings that would pick 22 of them out of random. The number 22 comes from the major arcana of Tarot Cards. There were many pitfalls that I encountered throughout this process, I will list them below.

Attempt #1 at Training StyleGAN

Attempt #2 at Training StyleGAN

Attempt #3 at Training StyleGAN

Attempt #4 at Training StyleGAN

---

Attempt to Train GPT-2

In an attempt to train GPT-2 on ‘fortune-telling’ sentences, I collected data from fortune cookie message archives (link [1], [2], [3]), proverb saying collections from the internet, and Tarot reading data (from Allison Parrish). Unfortunately, it ended up only being about 167KB’s worth of text (it takes a LOT of text to be over 1MB, I realized!), and the GPT–2 ended up only repeating the data that it already had.

So in the end, I just ended up using the same dataset but just selecting random samples of meanings (instead of generating them).

---

Final Outcome

After being disappointed with earlier attempts, I tried to debug the dataset — meaning that I tried to have a smaller quantity of samples that are more similar to each other in terms of artistic quality and resolution. I scraped this Japanese Blog that had four editions of the Rider-Waite-Smith tarot card deck, and added in one more set that I had. It wasn’t quite 500 images, but I hoped that it would be enough. Also, I found that the StyleGAN in Runway had a second, improved version. So I tried it out.

Here is a gist of the scraping Jupyter Notebook

To my pleasant surprise, the model turned out the most successfully than I expected! It produced much more variety than the previous models, and more predictable styles with less blobs. Still not varied enough, but a bit better than the previous attempts.

---

Name: The Disdain

Keyword: ['Satisfaction', 'choice', 'option']

Light: Experiencing transcendent joy

Shadow: Responding to constructive criticism with defensiveness

Name: The Restless

Keyword: ['stamina', 'conclusion', 'worry']

Light: Searching for the truth

Shadow: Spending money carelessly

Name: The Shut Down

Keyword: ['collaboration', 'motivation', 'Enthusiasm']

Light: Honoring limits

Shadow: Obsessing on errors and overlooked details

Name: The Safe

Keyword: ['difference', 'skill', 'rejection']

Light: Delivering exactly what others have asked for

Shadow: Using a barbed tongue to upset others

Name: The Wooden

Keyword: ['nurturing', 'science', 'sneakiness']

Light: Letting go

Shadow: Avoiding work that needs to be done

Name: The Present

Keyword: ['disaster', 'disruption', 'certainty']

Light: Hoping for the best

Shadow: Avoiding work that needs to be done

Name: The Frozen

Keyword: ['acquisition', 'Wealth', 'attraction']

Light: Exercising authority

Shadow: Rejecting information that suggests your intuitions are misguided

Name: The Humiliated

Keyword: ['Advancement', 'reflection', 'initiation']

Light: Investing time in learning or teaching a difficult task

Shadow: Looking out for yourself while allowing harm to come to others

Name: The Calm

Keyword: ['Speed', 'cycles', 'completeness']

Light: Starting your own business

Shadow: Taking a fatalistic approach to life

Name: The Agitated

Keyword: ['charm', 'Revival', 'withdrawal']

Light: Allowing someone to use his or her own methods to get a job done

Shadow: Debilitating passion

Name: The Brave

Keyword: ['Variance', 'fantasy', 'learning']

Light: Telling jokes

Shadow: Taking unnecessary risks as a means of proving your fearlessness

Name: The Aggravated

Keyword: ['openness', 'illumination', 'revolution']

Light: Being cheered on by the crowd

Shadow: Refusing to handle stress in healthy ways

Name: The Irritated

Keyword: ['prosperity', 'Assessment', 'grace']

Light: Starting a savings plan

Shadow: Obsessing on death and dying

Name: The Invigorated

Keyword: ['wit', 'oppression', 'reversals']

Light: Receiving the perfect gift at the perfect time

Shadow: Seizing every new idea that comes your way without question

Name: The Clenched

Keyword: ['dreams', 'purity', 'destruction']

Light: Blazing your own trail

Shadow: Habitually discounting input or feedback from others

Name: The Releasing

Keyword: ['discipline', 'Union', 'cooperation']

Light: Experiencing intense joy

Shadow: Adopting a point of view and refusing to reconsider your conclusions, even when presented with refuting evidence

Name: The Weary

Keyword: ['attention', 'irrationality', 'health']

Light: Making sacrifices

Shadow: Using cheap illusions to dazzle others

Name: The Compassion

Keyword: ['assumptions', 'investigation', 'combination']

Light: Taking care of the small details

Shadow: Employing sarcasm

Name: The Itchy

Keyword: ['dissatisfaction', 'illumination', 'diplomacy']

Light: Understanding the meaning of life

Shadow: Using clever insults to undermine the confidence of others

Name: The Tight

Keyword: ['fervor', 'training', 'fulfillment']

Light: Bearing fruit

Shadow: Putting excessive emphasis on appearances

Name: The Burning

Keyword: ['overwhelming emotion', 'community', 'rejection']

Light: Picking yourself up by your own bootstraps

Shadow: Ignoring healthy approaches to life

Name: The Relaxed

Keyword: ['Authority', 'Enlightenment', 'Fertility']

Light: Winning a competition

Shadow: Torturing yourself with regrets

---

Final Thoughts

I’m happy with where it ended up, although I can’t get over my frustration of wanting the images to be more different and varied. It seems the data is running into the problem of overfitting, when it trains for more than 4,000 steps. If only I had more tarot cards and data!

The latent space for the last model

Generating Type from SVG data with GPT2

Here’s another idea for generating type data… what if we give it vector coordinates using with GPT2?

Rough idea of pipeline

1. Get Data via MyFonts API (PNGs of Glyphs)

2. Batch Image Trace .PNG to .AI via Adobe Bridge Script

3. Convert .AI to SVG via Illustrator Script

4. Process SVG data via Python to get rid of White backgrounds (#FFFFFF)

---

… Long story short, all the files were good to go, and the model ran on Runway.

I’m not quite sure why it didn’t work out the way I thought it would.

Here’s what I was able to get from a GPT2 SVG Data.

Most of the images looked like the above — not quite comprehensible beyond the black and white.

Style GAN of Sans Serifs

I’ve been making a bunch of progress on training Style GANs via Runway ML!

I also have to give a lot of credit to Kevin Yeh for helping me out a lot for navigating the MyFonts Developer API and OpenCV (and pretty much lots of other life things)

The pipeline for generating this StyleGAN was as follows:

1. Download the dataset in PNG form through MyFonts API

2. Process the images via OpenCV in Python

3. Upload the dataset to Runway ML to train model

First Attempt:

Not great. Turns out the 500 or so images that I gave for training the StyleGAN model was too low res (about 100px tall), and it produced a random looking set. The optimal for this model is a 1024px square.

First (disastrous) attempt

Latent space walk for the first draft… it’s a ghost!

Second Attempt:

I trained the model on the same images, except modified the Python scripts to give me a much higher res than before, about 700 px high or so. It’s not 1024, but the results were much better. The initial model from Runway is trained on faces from Flickr, so you can see an uncanny series of images where faces turn into letters…

Second attempt! Better with better resolution images

Third (and latest) Attempt:

This time, I continued training the above model with many, many more images — about 24,000 in fact! As Yining mentioned in class, getting and debugging the dataset was where the majority of work was.

Process for gathering dataset:

1. Figure out ids of all fonts under the category of ‘sans-serif’

   • About 6,000 families, with each style the number it turns out to be over 50,000!)

2. Download sample images with the word ‘ASH’ rendered

   • Label the downloaded images with the original name and style

3. Filter out undesirable data

   1. Filter by name labels

      • Didn’t want a slant factor (e.g. Obliques and Italics)

      • Mislabeled Categories (e.g. slab serifs, handwriting, serifs)

      • Overly Decorative Categories (e.g. inline, shadow, layered)

   2. Filter Manually (sadly by scrolling thumbnails)….

      • Wrong data (e.g. dingbat fonts, tofu characters, inaccurate encoding)

      • Anything that didn’t get filtered via name

4. Process image data (via OpenCV Python)

   1. Get rid of surrounding white space and crop it by bounding box

   2. Make image 1024 x 1024 by scaling & adding white pixels as necessary

      • During this process, also filter out images that are too high or too wide that might skew result

In the end, I ended up with about 24k sample images.

Voila! And this is what it currently looks like:

Latest model, trained on ‘ASH’

Latent Space walk for the above model

---

Some thoughts for next time:

Perhaps the result is too uniform, maybe I filtered out too much or should’ve added more categories to make it more interesting?

Idea in Progress...

Here are just ideas that I’m jotting down after Week 3 of Synthetic Media class.

This idea was sparked by Gene Kogan’s work, A Book From the Sky and the following example using Runway ML shown in class:

I’ve been working to train a GAN machine learning model to draw letterforms based on a set of input typefaces with @runwayml. Here is an interpolation of it drawing an S. #machinelearning #gan #ai #generative #runwayml pic.twitter.com/sLPLp45Xrf

— Leon Butler (@leonbutler) November 27, 2019

The ‘magical wish’ I’ve always had is — what if a preview of a typeface could be generated from a test sketch (e.g. handgloveshuman, adhesion)?

The sketch I showed Yining to consult on the method.

Datasets are where things get tricky — I know that perhaps I could probably use the googlefonts library and use drawbot or something to automate getting font pixel data. (not sure if vector data would work, but bitmap data seems it might be more easy? I could be mistaken)

If the googlefonts library doesn’t work, I think I could also use the preview renderer from MyFonts via scraping.

screenshot of page

Fortunately, you can manipulate the online renderer via changing the text. This link was working as of February 2020:

https://render.myfonts.net/fonts/font_rend.php?id=698b4dc0fdc0696a2cc7851e4d7065c8&rt=A&rs=32&w=1500&sc=2&nie=true&fg=000000&bg=FFFFFF&ft=ccmp%2Cmark%2Cmkmk%2Cliga%2Cclig%2Ckern%2Cpalt&nf=1

With manipulating these values, you can change the sample text, and background/text color.

A problem is though, that the fonts have unique IDs and there’s seemingly no easy way to get them:

Inspecting a myfonts page via Chrome Inspector

So, it seems that if I search the ‘id’ using a font_rend prefix I can grab quite a few ids. Of course, doing this one by one seems too annoying…

Perhaps this is where making a scraping script like Sam Lavigne’s Flickr Scraper will come in handy. Not sure how this will work for myfonts yet; gotta dig through old files….

What Machine Learning Pipeline to use?

This is the big question. Talking to Yining, she mentioned that there would be two ways to go about it — one being StyleGAN and the other being a Style Transfer.

The Style GAN would allow me to explore a vector space, but be limited in terms of giving visual input (rather, I would have to try to build some way of navigating the vector). In short, it’s limited in terms of generating new content.

If i wanted to give a visual input and have the machine generate other characters that look like the input, an easier way to go about it would be a Style Transfer. It seems that this might be the way to go for what I want to create.

Trying out Runway ML

I’ve been eying Runway ML since I took Gene Kogan’s Style Transfer workshop a couple years ago. Fortunately, Yining Shi’s class on Synthetic Media is based on Runway ML and gave me a chance to futz around in Runway and see what it’s capable of.

Eventually, I’d like to learn what I could do with font data, like Nan’s project on Machine Learning Font.

So far, my time with Runway has been figuring out the interface and what it’s capable of.

Some, like the Cartoon GAN, is like a highly stylized Style Transfer:

Others do similar things, but with other content (like letters via the Shape Matching GAN)

Here’s one on Text to image generation, via AttnGAN. It generates text descriptions of scenes into images.

View fullsize

View fullsize

View fullsize

Still, others like the Ukiyo GAN I had a hard time figuring out what to do with. It gave me a vector and its options, but I wasn’t quite sure how to go about manipulating it in a way I that was meaningful for me.

Some models didn’t work at all or only barely, like the GLOW model. I was confused — was it because of my surroundings and background? My face wasn’t properly being recognized?

Thoughts: I would like to be able to create something meaningful that goes beyond the superficial.

I had a hard time making something that had meaning to me, and I think that’s where my creativity was blocked. It always felt like I was stealing someone else’s output, and it didn’t feel like it was ‘mine’ on an intellectual level. Most of the existing models are very specific in their output (e.g. Ukiyo), and I think training my own model would go a long way since the subject/output isn’t too flexible per model.