American College of Physicians: Internal Medicine — Doctors for Adults ®


Is the time right for speech recognition software?

Despite recent breakthroughs, experts warn the technology still requires a lot of physician time

From the September 1998 ACP-ASIM Observer, copyright й 1998 by the American College of Physicians-American Society of Internal Medicine.

By Howard Wolinsky

When John H. Kaicher, ACP-ASIM Member, began using speech recognition technology four years ago, dictating patient notes directly into the computer was a somewhat painful process. Dr. Kaicher, an internist in Aurora, Colo., had to speak very slowly and leave unnatural pauses to give the software time to register each word. He couldn't dictate much faster than 90 words a minute—by comparison, normal speech approaches 120 words a minute—and the software was accurate only about 90% of the time.

Since then, however, speech recognition technology has undergone some dramatic advances. Today, when Dr. Kaicher dictates directly into his computer, he can speak at speeds of up to 160 words per minute, without having to insert pauses. In addition, the software is accurate about 98% of the time.

For speech recognition technology, this ability to speak naturally, known as continuous speech, is a major advance. Continuous speech is so critical that its availability is expected to help sell one million voice recognition packages this year, up from the 300,000 sold last year.

To date, however, most physicians have not taken advantage of advances in speech recognition technology. Figures are not available on how many doctors use voice recognition, but as David Owen, director of product management at Lernout & Hauspie, a vendor of speech recognition software, noted, "Medical use has not pushed the envelope."

Why? Many physicians think that speech recognition technology remains too difficult to learn and use. Instead of jumping on the voice recognition bandwagon now, they are waiting until the technology can produce significant savings or additional revenue.

According to some physicians who are experienced users of the technology, that may not be a bad plan. They say that unless physicians are willing to commit the time to learning how to use the software, voice recognition software is probably not a good investment—yet.


The biggest obstacle faced by physicians who use speech recognition technology is the time they must spend to become comfortable with the software. When Dr. Kaicher began using voice recognition software, for example, he had to spend an extra hour each day just working through the quirks of the program and inputting new patient names, medications and medical terms. Dr. Kaicher became so interested in beefing up his software's medical vocabulary that he created and now sells several vocabulary packages that physicians can use with their speech recognition software. (See "Speech recognition software: resources," for more product information.)

Even so, Dr. Kaicher still spends 20 to 30 minutes extra each day on his dictation, in part because he enjoys working with the software, but also because he now creates more detailed patient reports.

For physicians who don't have Dr. Kaicher's expertise or passion for computing, however, speech recognition technology may consume even more time. Users emphasize, for example, that voice-recognition software is not designed to be used right out of the box. After an initial training period, physicians can expect to spend an additional 20 to 30 minutes a day teaching the software to accurately recognize their speech patterns. The more time users spend correcting and teaching the system, the better it gets at recognizing words.

Interestingly, physicians say that they actually learn to enunciate more clearly to increase their software's accuracy. If you use a lot of "ahs" and "uhms" when you speak, a fine-tuned speech recognition program will include them in your patient notes. Once you get tired of deleting them from your notes, users say, you'll begin to stop using them in your dictation.

But what really consumes valuable physician time is checking computer-generated transcriptions for errors that voice recognition systems seem to regularly produce. While today's software is easier to use, it is not error-proof, and physicians must still review transcriptions for errors, just like physicians who use a transcription service. When they add up the time spent working with the software, dictating notes and then checking for errors, some physicians find that they are spending just as much time—if not more—using voice recognition software as if they had sent their notes to a dictation service.

Stanley N. Schwartz, FACP, an infectious-disease specialist in Tulsa, Okla., who uses IBM's ViaVoice Gold software to dictate his notes and letters, said that because of errors, the software doesn't save time—"not by a long shot." He said that the technology's main problem is still accuracy, and that correcting errors takes time. Worse yet, he said, "Every error is perfectly spelled, so your spell-checker won't pick them up."

While many physicians assume that speech recognition software's biggest problem stems from recognizing complex words and medical jargon, longer words actually give the software more time to predict the correct word. Instead, most errors tend to appear with shorter words such as "the," "and", "a" and "but."

To illustrate, Dr. Schwartz gave an example of some of the more outrageous errors he's seen recently:

  • "She had the typical murmur of my true love pro laps." The correct phrase was "mitral prolapse."
  • "When his fever broke, he felt rather dead." Dr. Schwartz actually said "rather bad."
  • "She had pork tolerance to exercise." The right phrase "was poor tolerance."

Dr. Schwartz said that on a consultation letter with 400 words, there are typically at least 20 corrections. "Voice recognition is in its infancy for medical use," he said.


While users tend to agree that voice recognition software does not generally save time, they say that the technology can save money by allowing physicians to slash—or even eliminate—transcription costs. When Dr. Kaicher started using speech recognition software, for example, he was able to eliminate his transcription service and save more than $500 a month in transcription fees. Instead of having to review a transcriptionist's work filling in blanks and making corrections, he has an instant copy of his notes ready to go into the patient's chart.

Stephen S. Epner, MD, a general internist in Park Ridge, Ill., said that while he spends more of his own time using the software than he would sending his notes out for transcription, he saves money. He said that transcription costs average from 11 cents to 13 cents per line and $15 for a three-to-four-page report.

While Dr. Epner said that cutting transcription costs already justifies his using the software, he anticipates more benefits. "The software continues to learn my cadence, and I continue to learn to pronounce words in a way to improve performance," he said. "The payback will become greater as the software and hardware become more powerful over time."

Even some vendors, however, admit that the cost-savings may not entirely justify the purchase and use of speech recognition on the part of physicians. Joe Orlando, worldwide manager of speech products for IBM, said that early adopters of the technology are enthusiastic but typically have trouble justifying the expense. "They really can't quantify what it means to them yet, and neither can we," he added.


As speech recognition software advances, users and vendors alike are finding solutions to some obstacles. Doctors, for example, have learned to save time by creating "macros," boilerplate copy for commonly used phrases and paragraphs. "If I wanted to have a normal heart examination on my report, I could say 'normal heart' and an entire macro with a three- or four-line description of a normal heart examination would come up," Dr. Epner said. "It saves a considerable amount of time."

Physicians and vendors are also building customized vocabularies to make the software more useful for physicians. Dr. Kaicher, for instance, has developed Medi-Terms NS, vocabulary-building software that plugs into software from Dragon Systems Inc. The $20 Medi-Terms NS contains thousands of words in modules designed specifically for use for specialties such as internal medicine.

Other vendors are creating solutions to help physicians make the most of speech recognition technology. William Meisel, editor and publisher of the newsletter, "Speech Recognition," noted that the software vendor Lernout & Hauspie has introduced a new feature it calls "natural language processing" that allows physicians to move text by using simple voice commands. Users can give commands such as "move this word back three spaces" or "insert a table here." "If a doctor is going to use the technology for multiple things, whether it be creating a letter to a patient that he wants to format a little bit or doing his own editing by voice, that might be an important feature," Mr. Meisel said.

Some new programs also allow doctors to scan existing notes and identify the words they most commonly use. The software then adds these words to its dictionary, helping to eliminate errors.

The future

Because of the lukewarm reception from the medical community, speech recognition vendors are also trying to come up with new ways to attract physician interest. For physicians who don't always have a computer handy, for example, there are digital recorders that allow physicians to dictate their notes wherever they happen to be and then later plug the recorder into the computer and let speech recognition software transcribe their notes. The recorders are being sold now for between $100 and $300.

The consensus, however, is that it's going to take more than such relatively minor gains to bring doctors on board. Vendors are looking for the "must-use" application, the so-called "killer app," that will compel physicians to become regular users.

Some predict that HCFA's evaluation and management (E/M) documentation guidelines, which require physicians to provide extensive documentation, may motivate more physicians to try speech recognition software to help organize their notes. A system from Lernout & Hauspie, for example, contains 150 templates that prompt physicians to "fill in the blanks" to ensure that they fill in reports completely. If a patient is being seen for a lung problem, for example, the system will generally ask a series of questions regarding the eyes or other organs to ensure that the record is as complete as possible. Templates are available for primary care, cardiology, emergency medicine, pathology and radiology.

"These knowledge bases are especially vital in the managed care environment," explained Mr. Owen, the company's director of product management. "They save physicians a lot of time on paperwork and give them more time for patient care."

Speech recognition systems that also know what elements of a patient exam need to be documented may help prevent physicians from being fined for insufficient documentation during audits. HCFA has said it will become more aggressive in auditing documentation and will deny payments or penalize physicians who bill the agency without sufficient documentation in patient notes.

Michael Ginsburg, MD, creator of VoiceDoc, a product that uses IBM speech recognition technology, said that because products like his help physicians provide all the necessary documentation, they ultimately help them choose the right billing code. But some, such as editor Mr. Meisel, think that even more sophisticated solutions are needed to entice doctors. "If a doctor dictates 'a patient needs a blood test in six months to check liver function,' the system should be smart enough to automatically mail a reminder to the patient or give a list to the secretary to call the patient to remind him to come in for a blood test," he said. "This will generate revenue as well as better patient care."

Howard Wolinsky is a Chicago-based freelance medicine and technology writer.

Speech recognition software: resources

For more information on speech recognition software, try the Speech Recognition Update Web site (, which provides industry news and detailed information on products. In addition, here is a list of some of the leading vendors of speech recognition technology:

This is a printer-friendly version of this page

Print this page  |  Close the preview




Internist Archives Quick Links

Not an ACP Member?

Join today and discover the benefits waiting for you.

Not an ACP Member? Join today and discover the benefits waiting for you

ACP offers different categories of membership depending on your career stage and professional status. View options, pricing and benefits.

A New Way to Ace the Boards!

A New Way to Ace the Boards!

Ensure you're board-exam ready with ACP's Board Prep Ace - a multifaceted, self-study program that prepares you to pass the ABIM Certification Exam in internal medicine. Learn more.