Increasing sophistication of robots in all corners of our society led to the explosion of scientific research in this area. Practical implementation of computer vision was achieved only by the established corporations, such as Google, Tesla, and others. However, applications of augmented reality, object tracking and recognition, stereo vision, feature extraction, pattern recognition has been exploding in media storage, retrieval and search, robotics and toys, and many other areas.
Applied Research Group scientists and engineers has worked with a variety of customers to bring advanced computer vision capabilities that rival that of industry leaders.
The services include selecting the right hardware implementation for your project, choosing appropriate software algorithm and optimizing the full hardware/software stack for latency or energy consumption.
Industrial Augmented Reality
Augmented reality glasses, despite Google efforts, gain popularity only in niche segments. Given the limitations of the size and processing power, a new crop of specialized augmented reality glasses appeared, with software and hardware optimized for specific use cases.
Aligned Research Group has partnered with a leading medical school in Eastern Europe to work on the next generation surgical workflow software that will enable doctors to do a virtual reality pre-operative planning and perform a surgery with augmented information about the patient and patient body delivered through the specialized augmented reality glasses. This video describes the project:
As a part of this project, we are working with a manufacturer of the augmented reality glasses designed specifically for surgeons performing open surgeries to overlay hidden information about blood vessels and nerves obtained from the CT and MRT scans during preoperative planning. If you are working on a customized augmented reality glass, and would like to add or optimize the software for the glasses, we can help.
Unlike during Apollo 12 mission in 1969, when to make a stereo image astronaut Al Bean took the first picture while resting his weight on his right foot and the second after shifting to his left, stereo vision can now be implemented with an off-the-shelf stereo cameras, as well as built as custom solutions.
Stereo vision is typically used to locate an object in space, as well as measuring its dimensions and spacial location. Compared to other methods of distance metering such as laser and echolocation, stereo vision has the advantage of being a passive instrument and does not need to transmit any energy outside. This is useful both for energy consumption and to avoid detection, if needed.
We applied stereo vision expertise to the task of live liver registration with markers attached to the liver for an augmented reality surgical system. Stereo vision is essential to correctly identify the location of the surgical instrument during the surgery. Object tracking section below has more information about different marker types.
Object Detection and Tracking
Object detection and tracking, whether active or passive, is actively used in robotics, warehouse and storage automation, image-guided surgeries and augmented reality. In addition to the location and size of the object determined by object trackers, spacial orientation is frequently important. The image shows how 6 degrees of freedom (6DoF) are typically tracked for both the location and the angular orientation of an object.
In augmented reality applications, object tracking allows correct overlay of the additional information based on the location and direction of the object.
Object tracking in a video stream typically involves monitoring an object’s spatial and temporal changes in the stream, including its presence, position, size, and shape. This is usually done by matching the target region in successive frames in an image sequence. Object detection and object tracking processes are closely related as tracking usually starts with detecting objects, while detecting an object repeatedly in subsequent image sequence is often applied to verify tracking.
To simplify object tracking and improve its performance, markers are frequently employed. A set of markers provide algorithmically recognizable color or shape. In the table below, we compare three major types of markers: a passive 2D tracker, a passive 3D tracker, and an active IR marker, with typical use cases and advantages and disadvantages.
Passive 2D marker
Advantage: Convenient, inexpensive.
Disadvantage: Small working area, difficult to track with a single stereo camera, small viewing angle (~30 deg), wider areas often require multiple stereo cameras.
Passive 3D marker
Advantage: Large working area, which can be covered by a single stereo camera.
Disadvantage: Several markers are required for good precision, which makes the setup complicated. More expensive than 2D marker-based setups.
Active marker (IR)
Advantage: Excellent recognition in a video stream, good noise tolerance. Does not require additional IR illumination.
Disadvantage: Is a separate electronic module that requires power supply. The strength of the signal deteriorates as the battery runs out.
Aligned Research Group has experience processing images produced with all three types of markers. We examined a variety of cameras from different vendors and in various configurations and light conditions. The right choice of the camera model is critical for the tracking solution to operate reliably within the target environment.
With Aligned Research Group, you can design your products that require computer vision and bring to market faster. Contact us to discuss your computer vision needs and requirements. Please contact us with details of your project.