A Wichita State professor is developing an app that aids visually impaired people in reading graphic novels. English professor Darren DeFrain has been working on the app, Vizling, for five years. Users are able to move their fingers across their screen and get audio feedback in order to “read” the word bubbles and “see” the illustrations.
“That’s where we came up with this combination of using haptics and some visual linguistics to kind of allow people to use their finger to go across the cell phone and get some feedback, hear an audio version,” DeFrain said.
Recently, the National Endowment for the Humanities awarded DeFrain $150,000 to continue developing his app, which is already on app stores. They previously gave DeFrain $100,000 in 2021.
DeFrain said that this award gave him validation for all the hard work he and many others have put into this project over the years. It showed him that more people share his passion for aiding the visually impaired.
“You work on something, and you work on something, and you hope that it’s going in the right direction,” DeFrain said. “… The money’s necessary for the project, but I mean, the validation is at least as important for us to want to keep going with it.”
Vizling utilizes audio and haptic feedback, small vibrations triggered by touches on the screen, to help readers visualize the story on their screen. As the reader moves their fingers from panel to panel, descriptive audio plays. Haptic feedback helps guide the finger on where to go on the screen.
Defrain said one issue on his mind when developing the app was the subjective nature of art, including graphic novels
Using a Batman comic as an example, he explained the problem at the heart of the project: In a top panel, Batman says he is going to hit Joker on the head, and then in the next panel, he hits Joker on the head.
“But that’s me interpreting that whole thing, which really undermines the reading process,” DeFrain said.
DeFrain said reading in that way is closer to a math problem, which has a definite answer, but reading, and art in general, is extremely subjective. DeFrain wanted to maintain that subjectiveness as much as possible when translating a comic to audio.
How the project started
DeFrain experienced temporary blindness from an adverse reaction to prescribed medication five years ago. After missing one day of work, he was back at it the next day with bandages covering his eyes and help from his wife and students to continue his class’ education.
“This was such a WSU experience,” DeFrain said. “Students were like, ‘Well, it’s alright; we’ll read the things out loud.’”
After about two weeks, DeFrain’s vision returned.
Shortly following this incident, DeFrain began teaching a graphic novels class. He wanted to make his classes as accessible as possible, and, despite not having a visually impaired student, he started to contemplate the difficulty of reading comics for those individuals.
Traditional books have audiobooks, and movies have descriptive audios. While those may not be perfect solutions for those mediums, DeFrain said there is a severe lack of accessibility for graphic novels.
There, the idea for Vizling was born.
Rob Gerlach, executive director of tech transfer and commercialization at WSU, joined the Vizling team after connecting with DeFrain through E-Launch. Gerlach’s job, as DeFrain put it, was to help them “learn all the things we didn’t know we needed to learn.”
Gerlach said his role is to help develop the technology required for added features on Vizling. A key upcoming feature is an in-app audio description creator.
While Vizling is currently available in most app stores, DeFrain said it is ever evolving as he and his associates continue to add features. The biggest upcoming step is implementing a way for publishers to create their own audio description formats for their graphics.
Graphic novels and beyond
DeFrain and his team think Vizling has potential to help visually impaired people access more art and information, not just graphic novels.
Gerlach used an example of an art museum when describing how a person or business will be able to create their own audio descriptions in-app. The museum could upload the art onto Vizling, and then add in their own audio descriptions, including descriptions of textures and smaller details based on where individuals touch the art piece on their screen.
The implications of that technology could reach not only museums but also workplaces and schools.
DeFrain has a vision of Vizling’s capabilities expanding to include graphics such as charts and graphs to aid visually impaired people in the workplace and classrooms.
“It’s something we’re applying beyond comics,” DeFrain said. “It’s been for any multimodal text, which is just something that you encounter online, like, all the time now in your classes.”
Vizling and its team have dedicated themselves to creating a more accessible way for visually impaired people to enjoy the wonderful art of graphic novels. The team has come far but still looks to the future to continue their development in making the best app they possibly can.
Gerlach said the amount of energy DeFrain and other partners have continually put into the project “makes you want to root for them and jump on the team.”
