Artificial Intelligence tools have received a lot of hype in the news recently.
You may have heard about the AI writing tool, ChatGPT, which successfully passed written exams from prestigious law and business schools. College professors are wringing their hands, wondering if they’ll be able to trust a student’s term paper ever again.
Then a piece of AI-generated artwork slipped past judges at the Colorado State Fair last fall and won a first-place ribbon.
The world is debating. Is it fair or ethical for artificial intelligence to write an exam paper or win an award? Is the Robot Apocalypse upon us? Or is there still hope for humanity?
What do all these new AI tools mean for authors? Is it “cheating” to use AI software to write your book or design your book cover? Will authors who do NOT use AI tools lag behind their competitors?
I did a rundown of AI tools for authors last year and a deep dive into Sudowrite, one of the better AI writing tools. With all the buzz about AI recently, I thought it would be a good time to revisit the topic.
Then I saw that one of our Novel Marketing listeners, Danny F. Santos, was doing incredible things with AI and had posted about his experience on AuthorMedia.social.
Danny writes high fantasy novels and has used AI to design book covers and help craft the content for his Aeonlith Series. I asked him about his experience using AI.
Thomas Umstattd, Jr.: I first saw you were dabbling with AI when you asked for feedback on your eight book covers. You didn’t tell us they were AI-generated, but you asked everyone to vote on which cover was best.
How did you create such good-looking book covers using AI tools?
Danny F. Santos: It takes a little while to understand the language of AI prompts.
Thomas: By “prompts,” you mean the descriptive text you type in the text box of the AI generator. Users input their ideas into the text box, and the AI generator outputs the art.
There’s no magic button that can read your mind and produce a great cover. You must learn how to talk to the computer in a way the computer can understand. You must inspire it. The spark of creativity that ignites the fire still exists in the human mind. The real skill is in creating prompts the computer can interpret.
How do you create a prompt to get the AI going in the right direction?
Danny: You basically describe what you see in your mind and write it out.
The more complex aspect is that if you say you want a character to have red eyes, the AI will think you want the image to be red. It’ll make everything red, even though you might want the character wearing blue clothes.
Much of it is trial and error until you get something approaching what you’re looking for. It will never give you exactly what you’re looking for because it generates so many possibilities.
I used Midjourney to generate the art because it’s more illustrative. It looks a bit more like a painting. In my genre, that style usually performs best.
Each AI tool has its own flavor. I believe DALL-E produces more of a stock photo look.
Thomas: As I looked at your covers, I could tell you were exploring.
You knew you wanted your protagonist, a red-haired girl, on your cover but didn’t know what other elements you wanted.
How do you then choose which AI-generated cover you’ll use?
Danny: My original idea was to narrow down the choices through groupthink. After that, I planned to do an A/B split test using Facebook advertising.
After the voting, I narrowed it down to two covers, but I forgot I had another cover I wanted to introduce into the competition. I decided to add the third cover to my Facebook ad campaign and run an A/B/C split test.
The cover that wasn’t part of the original voting was the one that received the most clicks by a large margin.
Thomas: In an A/B/C split test, Facebook will show your target audience all three versions of your cover. Then they will monitor which version gets the most clicks.
You can see how many impressions and clicks it received and the clickthrough rates for each. Facebook will eventually decide which is performing best and just show that ad.
In my experience, the split-test winner is not necessarily the one I like best. It’s usually the one that readers like the best. A split test is a great way to get rapid feedback.
How much did you spend on the ads for the split test?
Danny: I spent $50-$60. It was enough to get about 1,000 views per cover, which was my goal.
Thomas: That is such an effective strategy. For $50, I don’t know why every author doesn’t run split-test ads.
I remember talking with the marketing director at one of the largest publishing companies in the country. When I told him about this technique of split-testing covers, he had never heard of it.
I was dumbfounded. They chose their covers based on the highest-paid person’s opinion, and they didn’t use any testing. Even though they had spent $50,000 on the advance, they didn’t spend $50 to see if what they had chosen was the best-performing cover. It boggles my mind.
Indie authors may believe they don’t have access to the best marketing tools, but in reality, indies may be using better techniques than the biggest traditional publishing companies.
Danny: I had five covers for my second book in the series, and none of them were doing very well. The split test for book one got a 7.5% clickthrough rate, but everything else got 3%.
I figured I’d try and generate more covers and start testing those against what I had. I created about ten new covers to test. The cover that won wasn’t even part of the original grouping that I asked people to pick from.
Thomas: Since you were using AI, it was financially feasible to generate ten new covers. All your covers look illustrated. They look hand-drawn by a human even though AI generates them.
Having an illustrator draw ten different covers would cost you a lot of money. Even traditional publishers wouldn’t be willing to spend what that would cost. And yet, by using AI tools, you could affordably generate new covers. You didn’t need thousands of dollars to get it done. You simply went back to Midjourney and fiddled with your inputs.
How did you change your prompt inputs from the first grouping to the second?
Thomas: What did you change from the first group of covers, which weren’t working, to the second wave? What words did you change?
Danny: To be honest, I didn’t use any different words. I used the exact same prompt every single time until an image worked. All the images looked very different.
Many people think you press a button and get what you want, but much of what Midjourney generates is unusable. AI is notoriously terrible with hands, and it’s a little bit terrifying. Sometimes you’ll see someone with 12 fingers, so you can’t use that. I come from a graphic design background and have illustrative abilities, so I could fix that, but it’s a lot of work.
Sometimes if you’re really unlucky, you’ll have an arm turning into a torso with a head growing out of it. It gets grotesque.
You have to spend hours and hours generating AI images so that there will be a few good images to choose from.
Thomas: Initially, you input a paragraph describing the scene. “I want two people on a book cover, and the girl has red hair and a blue shirt.” Then you copied and pasted that same paragraph, clicked “generate” again, and got different images.
Some authors make the mistake of only clicking “generate” once. They might describe the scene, click generate, and get the monstrosity of a hand with 12 fingers. After one click, they determine AI is dumb, and everyone recommending it is stupid. If they had just clicked “generate” one more time on that same input, they might have seen a good version the next time.
It requires a real shift in thinking. In the past, it was very expensive to contract an artist. The structure of the design was fitted around maximizing the illustrator’s time.
Using AI requires the same kind of shift in thinking that happened when we moved from printing on paper to publishing on the web.
Paper was expensive, so the design techniques involved squishing as much information on a page as possible so you didn’t waste paper.
For example, we’d design the text to wrap around the photo so that we use all the available space on the paper. Magazines and newspapers used text-wrapping.
When people started designing websites, they used the same text-wrapping technique, but it wasn’t necessary. Space is unlimited online, and you don’t pay per page.
The limiting factor in web design is people’s attention. White space helps reset people’s attention and provides some visual breathing room. People will spend more time reading your article if you don’t wrap the text.
That shift in thinking is often difficult because those old rules existed for old reasons. AI is changing the rules. It’s like going from a typewriter to a word processor. Editing a typewritten draft was painful and expensive, but with a word processor, editing is comparatively easy.
If AI generates a new image every time, how do you keep your character consistent from cover to cover?
Danny: One of the downsides of using AI is that there is no way to do that. I’ve tested it quite extensively. It doesn’t generate the same image even if you describe the hooked nose or full lips. You just have to generate a lot of images and hope it looks similar.
I have a reader magnet on my website, and the male character on that cover is supposed to be the same male character on the second book cover. It’s similar enough, but there is no way to make it exactly the same.
In some ways, the best way for authors to use AI is to A/B test several covers on Facebook and then go to an illustrator and say, “Paint this image as close as you can, and this is what the character should look like.” For each subsequent book cover, you’d return to the same illustrator. When you provide a designer with the AI image, they’ll have the color palettes and composition and can create a much better final product than the AI could.
Thomas: You’re basically using AI to generate an in-depth design document.
You can take your AI-generated image to an illustrator and ask them to fix certain elements the AI got wrong.
It’s important to note that the mainstream use of AI won’t make humans obsolete. In the United States, there are twice as many job openings as people looking for jobs. If the robots are coming for our jobs, they’re moving slowly.
Past technological revolutions may have made some jobs obsolete, but they have also shifted jobs around. Often, technological advances create more and different kinds of jobs.
AI isn’t necessarily putting illustrators out of their jobs, but it is changing their jobs. An illustrator used to talk to a client to figure out what they wanted, and those conversations were often points of frustration. The client and designer each had a mental picture of the design, and as they discussed their mental images, they seemed to be on the same page. But when the designer painted the image he’d imagined, the client was often unhappy because it didn’t match the picture in their own head.
AI-generated images could potentially help reduce some friction while still leaving room for the creativity and skill of the designer.
How does AI handle typography?
Thomas: Your covers have strong typography. I assume that you didn’t do this with, Midjourney. Did you add it later?
Danny: Yes. I did the typography.
As I mentioned, I come from a graphic design background. I did a lot of graphic manipulation for the actual images. The images generated by AI weren’t ready to stick on a cover.
One of the prompts I used was “movie poster” and “book cover.” Wording like that will give you compositions that look great. Unfortunately, they’ll also have gibberish lettering. It’s weird. The images look best with those prompts, but you have to remove the nonsensical lettering in Photoshop. It’s not layered, so you have to manipulate it manually.
Thomas: By the time we get the next two or three versions of DALL-E, they’ll have figured out how to remove gibberish letters from the book cover.
Danny: I’m still waiting for five-finger hands, so hopefully they’ll do that too.
The AI-generated images made the protagonist too short, and there was too much white space at the top, so I had to fix that.
The faces weren’t exactly right either. To upscale the faces, I found a script called GFPGAN that you load into your own Google Drive. It can be used to upscale old photos. I saw a YouTube video where someone used it with DALL-E to clean up faces, and it looked really good.
Even the resolution sizes aren’t great for print, so I had to use another AI program to upscale the resolution, making a low-resolution image high-resolution. There are many free services that upscale AI images.
Creating a book cover using AI-generated art requires a lot of understanding and work. Editing the image requires many additional steps before you have a suitable image.
Thomas: When using AI, it’s important to understand you can and should use multiple AI tools for different tasks. Danny used one tool to create the initial image, another to generally upscale that image, and a third tool to specifically upscale the faces.
It’s like going from using hand tools to power tools. Once you start using power tools, you have a power drill, a power saw, and a power screwdriver. Just because each tool uses a battery doesn’t mean you use them in the same way. AI is a set of tools.
What tips do you have for someone who wants to use Midjourney to design a book cover?
Danny: Midjourney is great for developing ideas. If you’re trying to create a new character, you can input characteristics like “black hair and blue eyes,” and it’ll create a character for you.
If you want to write a scene set in a tavern, you can input “low-lit tavern candles on the table,” and it’ll create a tavern environment. You just describe what you’ve seen. It’s a great little shortcut if you have writer’s block.
Thomas: In film, they use the same technique and call it storyboarding. A storyboard is like a comic book picture of the scene. Alfred Hitchcock was famous for storyboarding the whole movie ahead of time. He actually thought the filming was boring cause he’d already solved all of the scenes in the storyboard.
Hiring a storyboard artist is expensive. But you can use Midjourney to create storyboards for your scenes. It will help you write a detailed scene.
Midjourney doesn’t replace you or do your work, but it helps you do better work than you’d have done otherwise. You know how to describe a tavern, but Midjourney gives you more ideas and helps you think differently.
How did you use ChatGPT to write your book?
Danny: I used ChatGPT for writing a separate four-book series that had been noodling around in the back of my head. I knew the main character and had some story beats, but I hadn’t had time to outline the series or do the world-building.
I wondered how I could use ChatGPT to get the series written and out the door.
I spent two days playing around with it, and by the end of the weekend, I had the main outline for four books. It was incredible. What normally takes me a couple of months to brainstorm was done in two days. It was mind-blowing.
What did you learn from playing around with ChatGPT?
Thomas: Wow. What do you mean by playing around with it? What were you doing?
Danny: You have to figure out how to talk to the AI. For example, if you want an outline, you have to ask it to “provide an outline based on the following synopsis.” If you say, “outline the following synopsis,” it might write exactly what you wrote, which is not helpful.
But if you provide a detailed outline, it’ll fill in some details in interesting ways you didn’t expect. Some ideas make sense for how the story should go, but most of the stuff it generates isn’t useful.
Most of the time, it’ll inspire something else in your mind. You may not like the words it generated, but you may like the direction it took, and you can modify that.
The great thing about ChatGPT is that it remembers what you’ve said or written before. You can tell it, “I don’t like this, rewrite it, and include these details.” You’ll just keep developing it with more detailed guidance.
Thomas: It’s like having an enthusiastic brainstorming partner who loves your ideas and riffs off them. Even though most of their ideas aren’t helpful, they’re enthused to participate in the process.
Danny: Everyone knows Alexandre Dumas wrote the Three Musketeers and The Count of Monte Cristo. But people don’t realize those books were co-written with Auguste Maquet.
Maquet was never given credit for the 18 books they wrote together. He tried to sue the publisher, but the courts voted against him. He never got his name on those books, but he was the one who outlined the stories.
I’m approaching ChatGPT in a similar way. It’s my Auguste Maquet, and with its help, I developed the story. I’ll change things as I write because that’s the writing process.
Does ChatGPT write the chapter based on the outline, or do you?
Thomas: Do you ask ChatGPT to use the outline to generate a chapter, or do you write the chapter on your own?
Danny: You could use it to generate a chapter. I usually tell it to generate ten chapter titles. It’ll give you ideas about breaking that story into ten chapters and how the book could be outlined.
I’ll probably write more than ten chapters, but the chapter titles it generates will provide a sequence. I tried using it to write the chapters, but the prose is awful, and the dialog is too on-point. ChatGPT uses no subtext.
Thomas: I mentioned that ChatGPT was able to pass college exams, but when you read the article, you’ll see it received a C grade. It’s not that great.
Future versions of ChatGPT will be better, but they will never replace the beauty a human brings to a piece of writing, especially when it comes to subtext and saying two things at once.
You could use AI to write a book, but you’ll have to constantly judge the quality and accuracy of the content it produces.
Much of what you’re doing is judging. You have to know what good prose and good book covers look like. You looked at hundreds of terrible covers to get those eight dazzling covers. Then you judged which eight you’d post on AuthorMedia.social. You couldn’t have tested hundreds of covers on Facebook.
Danny: No. AI provides a good, quick way to sketch things in, to rough it in. Michelangelo’s first hour sculpting David only gave an approximation of what a human would look like. He had to take it the rest of the way himself.
Learning how to talk to AI takes a lot of work and research. It doesn’t have a context unless you provide one.
If you ask it to write a story without giving it direction, it’ll generate a princess in a tower because that’s all it knows. It won’t add any of that creative spark that makes a story pop, nor will it have a good voice. People read books because they like the author’s voice. You cannot get that from ChatGPT or Sudowrite.
Thomas: Both tools provide rudimentary writing.
How do you become fluent in AI-speak so you can help the computer understand what you want?
Danny: Midjourney has an entire community behind it. In the community tab, you can see what other people use for prompts. As people share their prompts alongside the image Midjourney produced, you start to understand why Midjourney is creating an image in a certain way.
For instance, if you include “volumetric lighting” as part of the prompt, those images will have a smoky background, as if things are disappearing into the mist. You can catalog that for future use when you need an image with that effect.
If you want something to look more real, you can use a term like “ultra-realistic.” My best advice is to observe what other people are doing. Test it out and see what the outcomes are. If it doesn’t work, just play with it, and figure out why it gave you what it did.
Thomas: Midjourney’s community is run through Discord. I think that’s one reason Midjourney is winning the market share battle, especially with authors. You can use Midjourney for free and get a certain number of prompts free, but you have to publicly share your prompts and the results so other people can learn too.
Danny: As you use it, your vocabulary develops and becomes a little more second-nature. It’s literally just learning the language so you can speak to AI.
ChatGPT is a little easier to talk to because you can just ask it questions.
You can input a synopsis or chapter and ask ChatGPT questions about it, such as, “What are three other things the main character could be doing in this scene?” It will generate three ideas for you.
If you’ve written a boring scene, ChatGPT will provide some concepts you can play around with. Maybe none of them will work, but it’ll give you something to consider.
AI and Ethics
Thomas: That brings up the ethical questions. Technically you can say, “Give me a cover inspired by Rembrandt,” and it will give you a Rembrandt-style image. Rembrandt has been dead for a long time, and his work is in the public domain. People probably won’t be mad about your Rembrandt-style cover.
But you could also choose a living artist and say, “Write a Danny-Santos-style book, and if ChatGPT had been fed Dany Santos books in the past, it would produce something similar. However, Danny F. Santos wouldn’t get any money or credit.
Some artists are suing Deviant Art, which has made its art library available to train these AI algorithms.
What are your thoughts on the ethics surrounding AI?
Thomas: How would you feel If I published a Danny-Santos-inspired book cover?
Danny: This is sticky territory. I don’t have a good answer because it’s so new. I hope people use it as a tool to inspire creativity and not to copy other authors.
I don’t want to write a Brandon Sanderson book. I want to write a Danny F. Santos book. At some point, you might get the AI to spit out a Sanderson novel and put your name on it. I do not do that.
Thomas: While the application is new and weird, the ethical framework is not. Do unto others as you would have them do unto you. It’s a good ethical heuristic that’s helpful in this context.
Would you want somebody using AI to steal your style? Probably not.
Do you mind if somebody uses AI where your work is included in the millions of pieces of writing it’s pulling from? Probably not. But if it’s specifically copying Thomas Umstattd’s style, that feels more like a violation.
The Golden Rule is a good guide for navigating the new weirdness.
Danny: Works fully generated by AI are not covered under copyright. Several court cases have addressed the issue. The most famous one was the monkey that took a picture of itself. PETA wanted the monkey to get the photo credit, but the courts ruled that only humans could have copyrights.
If you create an entirely AI-generated book, you can’t copyright it under your name.
Thomas: I hope that will discourage people from going 100% AI because that’s certainly not what we recommend.
Don’t flood Amazon with a bunch of 100% AI books. ChatGPT very recently rolled out a tool that can detect AI text. It’s a reverse-engineered version of ChatGPT that uses the predictive algorithm to guess the likelihood of whether that text was AI-generated. It may set the world on fire because people will see whether social media comments are real or AI-generated. It will be a little mind-boggling when people realize how much of their online interaction is with bots. That’s my hot take. You can fact-check me on that a year from now.
Approach AI the same way Danny did. If you get a terrible image or paragraph, don’t view it as a failure. View it as learning how to use the tool.
The more you learn about how to interact with AI tools, the easier they’ll be to use. You don’t want to wake up one day to find that everybody else can write books faster and better than you because they know how to use the tools, and you’re playing catchup.
Danny: I’m glad you brought up speed because I ran some numbers.
A tool like Sudowrite helps me write a first draft 16% faster. It’s not much faster because you must wait for it to generate ideas. It’s pretty quick, but you still have to wait. It’ll generate several ideas, and sometimes they’re unusable. You can’t use AI to plow through a first draft.
However, when I use dictation tools, I’m 63% faster. You’ll write books faster using dictation than you will using current AI tools.
Thomas: I see authors using Sudowrite (Affiliate Link) to help with the brainstorming and dictation to help with speed.
Dictation, which is also AI, turns your voice into text. It’s simpler AI that’s far more popular. Many authors are playing around with dictation right now.
When a technology is new, there are many different ways to use it.
For instance, 100 years ago we knew flying was possible because we had two competing technologies for flying: zeppelins and airplanes. Both were incredibly dangerous for people.
Airplanes were 100% fatal, but people had survived a zeppelin crash (the Hindenburg). At that time, people thought that zeppelin technology would win the flight-technology race. Today, the Goodyear blimp is the only well-known zeppelin.
Which do you think has a better impact on the quality of the writing?
Danny: Sudowrite is better. My favorite tool in their suite is “Show Don’t Tell.”
In my first drafts, I typically tell more than I show. When I finish the draft, I can select the “telling” and ask Sudowrite to “show don’t tell.” It provides some descriptive wording, and I edit the description according to my voice.
Thomas: ChatGPT may be a fad, and dictation may win out. But I think ChatGPT will get faster. The prompts will get better, but the dictation technology will also improve.
Give it a try. Play with it. Just don’t write it off.
Connect with Danny F. Santos and check out his book covers at his website, DannyFSantos.com.
“Have you ever been curious about Asian and Pacific Folklore? Join Hawaii’s Goddess of fire, Pele, in her search for sanctuary. Confront the Philippine’s shape-shifting vampire. Fight beside Samoa’s Goddess of War. Embrace exile with the Goddess of Bats. Battle the dreaded Wendigo. And witness Hawaii’s Ocean Goddess get vengeance against her sister, Pele.
These collected stories influenced cultures, formed beliefs, explained the unknown, and spread throughout the world to become Legends from the Pacific. Get your copy on Amazon or LegendsFromThePacific.com now.”
Legends from the Pacific is also a podcast! You can listen at LegendsFromThePacific.com.
Kamuela’s heritage (part Asian/Native Hawaiian) and history (raised in Honolulu) make him the ideal host for his podcast, Legends from the Pacific.”
You can become a Novel Marketing Patron here.