Visual recognition of medication

Researchers at the Advanced Digital Sciences Center (ADSC) have demonstrated an application that helps patients, caregivers, and medication packagers verify that the correct drugs are being administered, through the use of computer vision techniques to recognize pills and capsules. Different prescription drugs can look very similar, as shown in the photo at right, and the same drug is often produced in different sizes, shapes, and colors by different manufacturers. Over time, different suppliers may win the contracts to fulfill a pharmacy’s orders, so that the appearance of a particular medicine may change regularly, even if it comes from the same pharmacy. As a result, automated recognition of pills and capsules can provide additional assurance that the intended drug is being ingested.pills

ADSC’s pill and capsule recognition application targets two different settings: human users and packaging machines. In the former case, a caregiver or patient can put a sample of the pill or capsule that she would like to identify onto the palm of her hand, snap a photo with her camera phone, and invoke a mobile application that compares the photo to an online database of images of existing medicines. The application then returns the top match or matches. When the application is used with a pill and capsule packaging machine, a camera is mounted directly onto the machine. The pill or capsule images captured by the camera are matched against a database of known medicines, to verify that what has been loaded into the packaging machine is in fact what was requested and expected.

pill packersADSC’s recognition approach employs a novel approach to identifying the shape of a pill or capsule. Compared to traditional shape analysis approaches, ADSC’s new method is faster and more accurate for geometric objects such as pills and capsules. Further, ADSC’s automated method is easier for users than the traditional web-based approach to identifying pills. For example, to narrow down the possible matches for a pill by its color using the pill identifiers at Drugs.com and WebMD.com, the user must choose between similar colors such as peach and pink, tan and beige, yellow and gold, maroon and red, and white and off-white. Narrowing down by shape can be equally baffling, with options such as barrel, oblong, and rectangle. Online pill identifiers also let users type in the imprint (letters and numbers) on the pill, if it has one. This is more work for the user than just snapping a photo and letting the image analysis algorithm determine the shape and read the imprint. Further, ADSC’s approach is the only practical option when there is no human in the identification loop, as with a pill packaging machine.

ADSC plans to license the pill recognition approach to a Singapore company in the near future. The company will commercialize the technology for use in hospitals to complement pill packaging machines by automating the inspection of sachets made by the machines.

Pill recognition is an application of ADSC’s basic research on recognizing objects in video and still images, a key feature for augmented reality.