Artificial Intelligence can convert 2D images into 3D

Using deep learning techniques, the team from University of California, Los Angeles (UCLA) devised a technique that extends the capabilities of fluorescence microscopy, which allows scientists to precisely label parts of living cells and tissue with dyes that glow under special lighting.

In a study published in the journal Nature Methods, the scientists also reported that their framework, called "Deep-Z," was able to fix errors or aberrations in images, such as when a sample is tilted or curved.

Further, they demonstrated that the system could take 2D images from one type of microscope and virtually create 3D images of the sample as if they were obtained by another, more advanced microscope.

"This is a very powerful new method that is enabled by deep learning to perform 3D imaging of live specimens, with the least exposure to light, which can be toxic to samples," said senior author Aydogan Ozcan, UCLA chancellor's professor of electrical and computer engineering.

In addition to sparing specimens from potentially damaging doses of light, this system could offer biologists and life science researchers a new tool for 3D imaging that is simpler, faster and much less expensive than current methods.

The opportunity to correct for aberrations may allow scientists studying live organisms to collect data from images that otherwise would be unusable.

Investigators could also gain virtual access to expensive and complicated equipment, said researchers.

"Deep-Z" was taught using experimental images from a scanning fluorescence microscope, which takes pictures focused at multiple depths to achieve 3D imaging of samples.

In thousands of training runs, the neural network learned how to take a 2D image and infer accurate 3D slices at different depths within a sample.

Then, the framework was tested blindly - fed with images that were not part of its training, with the virtual images compared to the actual 3D slices obtained from a scanning microscope, providing an excellent match.

The researchers also found that Deep-Z could produce 3D images from 2D surfaces where samples were tilted or curved.

"This feature was actually very surprising," said Yichen Wu, a UCLA graduate student who is co-first author of the publication. "With it, you can see through curvature or other complex topology that is very challenging to image."