Artificial Intelligence and Vision

Screening children for eye problems, diagnosing corneal and retina disorders and developing digital glasses for patients with limited vision are among Bascom Palmer’s many artificial intelligence initiatives that are contributing to the future of vision care.

In the world of fashion, image is everything. The right look turns heads and captures the attention of social media influencers. At Bascom Palmer Eye Institute, images of the eye play an even more important role, helping clinicians diagnose, treat and manage diseases that affect vision.

Now, Bascom Palmer’s physicians and scientists have teamed up to combine the power of artificial intelligence (AI) with advanced three-dimensional (3D) optical imaging to create the world’s first autonomous artificial intelligence system for corneal tissues – a remarkable advance that could dramatically improve vision care around the globe.

“Artificial intelligence is creating a revolution in ophthalmology, and Bascom Palmer is in the forefront,” said Mohamed Abou Shousha, M.D., Ph.D., associate professor of clinical ophthalmology.

Abou Shousha, who holds secondary appointments in electrical and computer engineering and in biomedical engineering, leads a team of 15 clinicians, engineers, trainees, and doctoral students at Bascom Palmer’s Artificial Intelligence and Computer Augmented Vision Laboratory, located within the Institute’s William L. and Evelyn F. McKnight Vision Research Center. The team has developed AI software that can analyze high-resolution optical coherence tomography (OCT) images and determine with exceptional accuracy whether the corneal tissues in the front of the eye are healthy or not.

Bascom Palmer’s new AI system can provide accurate early diagnoses of vision disorders that change the shape and health of the cornea, such as keratoconus, Fuchs’ dystrophy and eye infections. It can also better detect dry eye syndrome, a condition that affects an estimated one in eight U.S. adults but lacks an objective and accurate diagnostic technique.

“We are looking at a number of conditions on the corneal tissues and surface of the eye,” said Sonia Yoo, M.D., a professor of ophthalmology and a specialist in corneal and external diseases. “Our goal is to take those OCT images, feed them into our diagnostic suite and use this leading-edge system in patient clinics. A trained technician can then use those images for highly sophisticated diagnoses, regardless of geographic location, which allows us to share Bascom Palmer’s expertise globally.”

Advanced Corneal Diagnostic Software

Bascom Palmer’s autonomous AI system incorporates three leading-edge technologies, beginning with advanced OCT imaging. Although OCT technology has been available for more than two decades, ophthalmologists have primarily focused on the retina and macula in the posterior part of the eye. “One reason is that there has been little ability to produce clinically relevant information from a basic cross-sectional image of the cornea,” said Abou Shousha.

To improve that capability, Abou Shousha developed Corneal Microlayer Tomography (C-MLT), an advanced OCT tool that provides 3D images of the individual layers of the cornea. “C-MLT is the only available diagnostic system that can show all layers of the cornea and permit easier recognition of findings that differ from normal conditions, which improves substantially upon existing methods for diagnosis of important corneal diseases,” he added.

Abou Shousha and Yoo have recruited more than 700 Bascom Palmer patients for a clinical study of using C-MLT technology. To date, they have obtained more than 200,000 images of different stages of eye disease as well as images of eyes with no disease.

After collecting that information, the data are fed into a powerful computer that uses machine learning protocols. Models using neural network algorithms were created to build an autonomous AI system. “Any neural network needs to be taught, and it’s only as good as its training,” said Abou Shousha. “In this case, our AI system now has an accuracy of over 99 percent in diagnosing many corneal conditions.”

For instance, the AI system is able to recognize the early stages of keratoconus, a progressive eye disease in which the cornea thins and bulges to form a cone-like shape, leading to loss of vision. Because of the distorted shape of the cornea, individuals with keratoconus should not have refractive (LASIK) surgery. Currently, the standard of care is to screen every patient for keratoconus before LASIK surgery, and the AI system will permit accurate detection of keratoconus even in the early stage of development.

Bascom Palmer’s system can also quickly evaluate patients facing cataract surgery for Fuchs’ dystrophy, and forewarn the surgeon and the patient of a heightened risk of post-operative swelling of the cornea. In some cases of corneal swelling, a patient might need a corneal transplant – placing a tissue graft from a donor – to address the underlying condition, in addition to the cataract procedure.

In 2015, Bascom Palmer received a grant from the National Institutes of Health (NIH) to develop this screening technology also for detecting corneal graft rejection. “Using C-MLT imaging we are able to see graft rejection two months before it becomes apparent to an ophthalmologist,” said Abou Shousha. “That lets us treat the problem at an earlier stage with better patient outcomes.”

A Cost-Effective Device

Along with the C-MLT and AI application, Abou Shousha’s team is developing a cost-effective, high-performance OCT device powered by Bascom Palmer’s diagnostic software that can be used for patient examinations almost anywhere in the world.

With this equipment, a clinician could check an individual’s eyes by capturing images with the OCT device. Those images would be uploaded to the diagnostic suite – an application housed in a cloud-computing environment. The AI system would then send back the results to the clinician quickly, along with any recommendations for treatment and follow-up.

Currently, not every physician has the specialized experience to interpret OCT images and make a diagnosis, which is why automating the process and screening with a low-cost device and AI software is a game-changer for vision care.

“We are giving international vision care providers the ability to benefit from the expertise of our specialists at Bascom Palmer,” said Yoo. “It extends the reach of our corneal vision care to underserved and remote communities throughout the world.”

Identifying Pathogens

Along with detecting dry eye and corneal disorders, Bascom Palmer’s autonomous AI system may also lead to faster identification of microbes that infect the eye.

“It can take several days or longer to obtain results from cultures taken from a patient’s infected eye,” said Darlene Miller, D.H.Sc., M.P.H., research professor and technical director of Bascom Palmer’s Ocular Microbiology Laboratory. “Infections by many microorganisms have the same presenting signs and symptoms, making it difficult to determine the right treatment.”

One of the most common conditions is a corneal ulcer (also known as keratitis), an infection of the cornea caused by trauma to the cornea or improper use of contact lenses. A prompt, accurate diagnosis is critical because keratitis can lead to a corneal scar or blindness if not treated appropriately. “Unfortunately, many clinical diagnoses are incorrect,” said Miller. “As a result, some patients take unnecessary antibiotics or the wrong medication for their eye conditions.”

Bascom Palmer’s researchers and clinicians are in the early stages of using AI to improve the diagnostic process. “We are gathering images and data, and building a neural network,” Miller said. “Our goal is to develop a mobile platform where clinicians around the world could take images of the cornea, upload them to our system and get diagnostic results with a high degree of certainty, long before the diagnosis and treatment would traditionally have begun.”

So far, the dataset includes images of the cornea of more than 1,000 patients with corneal ulcers, information about the microorganisms obtained from the microbiology laboratory, and clinical information from electronic health records (EHRs). Identities have been removed from the data to protect patient privacy. “Although a challenging project, it can be life-changing for patients, and a revolutionary advance in corneal ulcer care,” Miller said.

A Collaborative Approach

For Abou Shousha, the autonomous AI system is the culmination of 10 years of collaborative research and development which has produced 28 U.S. and international patents to date. “We have a core multidisciplinary group at the University of Miami evaluating evolving applications of this technology,” he said. “This work involves colleagues from the biomedical and the electrical & computer engineering departments, collaborating with Bascom Palmer’s scientists and physicians.”

While the AI system is well on its way to advancing the diagnosis and care of diseases that affect vision, it has other potential medical applications as well. For instance, one of the early signs of rejection in kidney, lung or heart transplants is a thickening of the endothelial cells that line the interior surface of the patient’s blood vessels.

Currently, physicians typically insert a needle into a transplanted organ to capture those vascular endothelial cells and check for rejection. Rejection of a corneal transplant can be determined by visualizing the endothelial cells of the transparent cornea by OCT and AI technology. It is hoped to find a way to apply the AI technology to the vascular endothelium of organ transplants.

This is just one of the many exciting research programs at Bascom Palmer that benefit from grants and private philanthropy, Abou Shousha added. “There is real value in our collaborative approach to vision research. It’s one of the main reasons that Bascom Palmer will continue to make lasting contributions to the field of vision.”

Many of Bascom Palmer physicians and scientists are applying machine learning and artificial intelligence in their efforts to improve vision care. “AI is now woven into the fabric of our diagnostic, treatment and research programs,” said Eduardo C. Alfonso, M.D., director of Bascom Palmer.

Swarup Swaminathan, M.D., assistant professor of clinical ophthalmology, is looking to expand the use of artificial intelligence in the diagnosis and monitoring of glaucomatous disease, as are other glaucoma specialists at Bascom Palmer.

Richard Lee, M.D., Ph.D., associate professor of ophthalmology, is using advanced ocular imaging technologies with AI to assess glaucoma risk and to detect glaucoma progression. In conjunction with Anna K. Junk, M.D., professor of clinical ophthalmology, and Sanjoy K. Bhattacharya, Ph.D., professor of ophthalmology, Lee is seeking to advance understanding of the pathophysiology of pseudoexfoliation glaucoma to develop directed anti-glaucoma therapy.

Partnering with Microsoft to Diagnose Global Vision Problems

When Lilian Lee-Ferland was diagnosed with glaucoma, she wanted to know whether her vision would deteriorate quickly, slowly, or at all. Fortunately, she came to Ranya Habash, M.D., medical director of technology innovation and an assistant professor of clinical ophthalmology, who is leading the Institute’s partnership with Microsoft to develop a worldwide screening and assessment system.

“We are using artificial intelligence to improve our diagnosis of serious eye problems such as glaucoma, using patient records from Bascom Palmer and other sites around the world,” said Habash. “Through machine learning, we are teaching Microsoft’s AI application to recognize glaucoma, macular degeneration, diabetic retinopathy, melanoma, and myopia, just as an experienced ophthalmologist would.”

“Our algorithm integrates data like Lilian’s age, family history, eye pressures, visual fields, and optical coherence tomography to give a real-time prediction of her glaucoma risk. This is a far more personalized, accurate, and customizable approach to medicine.”

“I feel more confident about my future, thanks to this new technology,” Lee-Ferland said.

A Global Consortium

In 2016, Bascom Palmer and Microsoft co-founded a global consortium called the Microsoft Cloud+AI Network for Eyecare, launched in collaboration with several of the largest eye institutes in the world. The goal was to use artificial intelligence to diagnose, to prevent blindness, and to deliver eye care services worldwide through remote diagnostic capabilities and telemedicine services. “Our focus was to develop algorithms that identify various eye diseases on a global scale, so early treatment can prevent vision loss,” said Habash.

Artificial intelligence can make these associations and integrate data within seconds or minutes, while it would take humans years to collect those data and identify patterns within them. While research papers typically draw on findings from 25 to 30 patients, AI-powered technology can analyze the data on millions of patients at one time.

In the first two years of the project, data and images from the electronic health records system were used for machine learning, powered by Microsoft’s Azure platform. The first development was for predicting and tracking the progression of refractive errors, such as nearsightedness, farsightedness, and astigmatism in children. “Uncorrected refractive errors are a leading cause of blindness in the world. It’s incredible that one of the world’s leading causes of blindness can be prevented if a child simply has a pair of glasses,” said Habash. Easily treated refractive vision problems can make it difficult for children to learn in school and hamper their ability to enjoy sports and other outdoor activities.

Habash presented this myopia model at HIMSS 2018, one of the world’s largest health IT conferences, alongside Microsoft Corporate Vice President Anil Bhansali and Raghu Gullapalli, executive director of emerging technologies at LV Prasad Eye Institute in India. Afterwards, the Indian state of Telangana, with more than 35 million people, agreed to screen children routinely for visual impairments using AI.

Diagnosing Retinal Diseases

Diseases such as glaucoma, macular degeneration, and diabetic retinopathy steal vision. They also add to the nation’s burden of healthcare costs, which makes early diagnosis a national imperative. However, there is limited access to vision screening and care.

“This motivated our team to develop an AI algorithm for classifying and diagnosing retinal diseases by using a neural network and a large data set of more than 86,000 images,” said Habash.

Harry W. Flynn, Jr., M.D., the J. Donald M. Gass Chair in Ophthalmology, worked with Habash in labeling the retinal images in order to train the machine learning system. “It is exciting bringing AI into our clinical retinal practices and could be a dramatic step forward in worldwide screening procedures,” he said.

Flynn added that early detection of retinal diseases results in prompt referrals to ophthalmologists. “The combined use of telemedicine and AI will allow application of earlier and better treatment of patients with vitreoretinal disease.”

Retina specialist Luis J. Haddock, M.D., an assistant professor of clinical ophthalmology, is also exploring innovative surgical and diagnostics technology for retinal diseases using artificial intelligence.

A Breakthrough Machine Learning Algorithm

Most recently, Habash is working with Microsoft’s Cloud+AI for Healthcare team to develop screening algorithms for other eye diseases including age-related macular degeneration, diabetic retinopathy, glaucoma, and ocular melanoma. “We’ve recently made a huge breakthrough, developing an algorithm that detects several retinal diseases with 88 percent accuracy – better than most eye care providers,” Habash said. “We’ve demonstrated that prediction of multiple retinal diseases can be achieved on a single model with high accuracy. That’s a game-changer.”

According to the World Health Organization, at least 2.2 billion people worldwide have blindness or visual impairment, at least 1 billion of whom could be treated to reduce vision loss. “Retinal diseases such as macular degeneration and diabetic retinopathy are insidious because patients don’t see a problem until damage has been done. This also presents a tremendous healthcare cost to society,” Habash said. “The economic burden of eye disorders and vision loss in the United States is $139 billion and increasing each year, which is why automated AI screening for early diagnosis and treatment is critical.” According to Habash, limited access to healthcare, rising medical costs, lack of insurance or transportation, and other financial and personal constraints restrict early diagnosis of eye disease.

Using Bascom Palmer’s Multi-Disease Retinal Algorithm, Powered by Microsoft Cloud+AI Platform, patients can be screened on a large scale for very little expense, since the diagnosis is generated by the AI algorithm rather than an ophthalmologist. “What makes our software unique is that it can diagnose several retinal diseases at once, including the severity of those diseases, and can be used on any device ubiquitously,” said Habash. “I foresee our technology on any device, in every healthcare clinic, anywhere in the world. That’s how we can help the most people.”

Currently, Habash and the Microsoft team are working through the U.S. Food & Drug Administration (FDA) submission process to achieve Breakthrough Device Designation. Breakthrough designation indicates that the FDA views the algorithm as a potential solution for an unmet medical need, which helps accelerate their regulatory review process. Habash envisions the algorithm will enhance and not replace the doctor-patient relationship. Since widespread screening is fast, accurate, and inexpensive, patients could gain access at local eye clinics, primary care offices, or even a neighborhood pharmacy. “Imagine walking into your local Walgreens or CVS where you are able to check your blood pressure at one kiosk, and have a retinal scan at another. The scan could show, for example: ‘your photos suggest moderate macular degeneration. Please see your ophthalmologist in the next three weeks.’ This allows patients to become proactive and for doctors to provide treatment in a timely manner.”

The project’s next algorithms will focus on Alzheimer’s disease and retinopathy of prematurity (ROP). Researchers have found amyloid plaques in the brains of patients with Alzheimer’s disease. Because the same plaques can be identified on the inner surface of the retina using optical coherence tomography (OCT), an AI algorithm for the OCT scans could serve as a screening tool for dementias. As for ROP, access to pediatric ophthalmologists who screen for this disease can be sparse. According to Habash, “using an algorithm to screen automatically from the neonatal unit would help us treat infants in time to save their vision.”

Now that the foundation has been set, Habash and her Microsoft colleagues are inviting other institutions to join the growing network. “With Bascom Palmer’s physicians and scientists collaborating with others around the world, there’s an even more robust set of geographic, ethnic, and socioeconomic data to power the algorithm. It’s Bascom Palmer’s commitment to unite the world and help patients through technology,” said Habash. This collaboration includes academic institutions, pharmaceutical companies, device manufacturers, electronic medical records vendors, and other technology companies. “The Microsoft Cloud+AI Network for Healthcare is one of the biggest research initiatives in history,” Habash said. “We are proud that Bascom Palmer is paving the way for better vision care around the world.”

In Bascom Palmer’s Artificial Intelli-gence and Computer Augmented Vision Laboratory, researchers are taking the gift of sight into the future with digital glasses that can expand a patient’s limited field of vision or correct double vision. Now in the development stage, this revolutionary technology could open the door to a higher quality of life for patients with severe vision loss.

“It was too stressful for me to enjoy going shopping because I was always worried about bumping into people,” said a Bascom Palmer patient with limited peripheral vision who tried out the digital glasses. “Soon, I’ll be able to go to crowded places and see who’s next to me.”

Sharing their clinical and scientific knowledge, Abou Shousha’s collaborative AI team is building an augmented vision system straight out of “Star Trek.” It combines an augmented reality headset that contains tiny cameras and a video display with an advanced software algorithm that quantifies and compensates for visual field and double vision impairments.

First, the camera captures a wide-angle image of the environment in front of the patient. The image is uploaded to a cloud computing system where an AI application makes adjustments to compensate for lost vision and sends the image back to the patient in real time. Based on each user’s unique vision profile, the software displays customized corrective images of the scene being viewed through the digital glasses in real time, enhancing the user’s vision and awareness of the surrounding environment. As the patient moves around, the AI software automatically remaps and redisplays images being viewed by the patient in order to supplement the intact visual field.

“Our patients’ desire for greater safety and mobility, as well as restoring their ability to watch TV or read in comfort, are important benefits they have experienced from our augmented vision program,” said Abou Shousha.

A New Dimension in Visual Aids

Bascom Palmer’s vision augmentation software application takes visual aids that are currently available into a new dimension. Unlike some other magnifying devices, prisms and special lenses, digital glasses can be worn anywhere for several hours at a time in order to augment an individual’s vision.

“We trained our system using a database with visual field data from thousands of patients with glaucoma, stroke, or retinitis pigmentosa – diseases that decrease the field of vision,” said Abou Shousha. “It is the only available method to autonomously detect and correct the user’s vision defects using AI algorithms trained on real patients.”

In an ongoing clinical trial at Bascom Palmer, more than 100 patients affected by these diseases have tested the diagnostic and augmented vision abilities of the digital glasses. A prototype version was able to augment peripheral object awareness in 78 percent of the patients. It also improved mobility by facilitating the identification of moving objects outside the reduced field of vision.

After using the prototype digital glasses on a visual “obstacle course” in the laboratory, a patient said, “I wasn’t able to throw a football with my grandkids because the ball would disappear in my blind spots.” Another patient, William Benson, liked the comfort of the lightweight headset, adding he was excited to retain much of his peripheral vision. Looking ahead, patient Thomas Wilson said, “It gives me hope that these digital glasses will help all the younger people who are losing their vision to disease.”

A Worldwide Need

Along with augmenting a patient’s lost vision, the AI application being developed at Bascom Palmer also includes powerful new capabilities for diagnosing, screening and monitoring the progression of vision-stealing eye diseases.

Throughout the world, more than 500 million people are losing their central or peripheral vision from different diseases and disorders. Glaucoma and retinitis pigmentosa are two diseases that lead to a loss of peripheral vision, which makes it difficult to see objects to the side, above, or below a person’s central vision.

Age-related macular degeneration is the leading cause of loss of central vision among people age 50 and older. Another serious condition is diabetic retinopathy, which affects vision in about a third of individuals with diabetes. Additionally, millions of stroke patients also have irreversible visual impairments.

A Powerful Diagnostic Capability

“Our digital glasses incorporate an automatic self-administered diagnostic testing capability including a simpler application for tracking visual field impairments,” said Abou Shousha. “Instead of the traditional visual field diagnostic test in which the patient clicks a handheld button when seeing a flash of light, our application tracks the movements of the eye to look at the light. It is therefore faster and more accurate.”

Abou Shousha said the diagnostic software in the digital glasses can measure a visual field of 100 degrees, substantially better than the current standard of care. “Clinical studies conducted at Bascom Palmer have demonstrated that our autonomous visual field test is accurate, reproducible, and easier to use than existing technology,” he said. “With 8 million visual field tests conducted annually, this new screening and diagnostic application could be a valuable addition to the market.”

Advancing Vision Technology

‘It’s very rewarding to surprise our patients by making an unexpected contri-bution to their lives,” said Abou Shousha. “Being able to help our patients is why we are all here at Bascom Palmer.”