The Problem

Since the start of the OpenScope project, we have aimed to turn the microscope we developed into a fully automated, autonomously operating system. As such, it could not lack the autofocus feature. Autofocus is especially useful in certain experimental setups such as:

• time lapse imaging - to account for stage drift/settling with time

• monitoring of motile samples

• focus stacking - for imaging 3-dimensional objects

Or last, but not least, autofocus just facilitates the work of the person, operating the OpenScope. Manual focusing, although not very challenging, might still be fiddly, especially for the inexperienced user. This is when the autofocus algorithm comes to help.

The Solution

The strategy of the autofocus algorithm is to calculate the focus score for each frame imaged while gradually changing the sample-objective distance, until the score is maximized. This corresponds to the most sharply focused image. The physics behind the process is very simple: image of an object, viewed through a lens, is produced at a particular distance from the lens; at any shorter/longer distance the light rays from a single point on the object do not converge into a point, and therefore the image is blurred. The focus score can therefore be simply regarded as a mathematical function with a single maximum. However, multiple methods exist for calculating the focus score. The challenge was to find the most robust one. Here is what we tried:

• Tested out focus scores based on variance since this was recommended by both sources [1][2] on artificially blurred out images (Gaussian blur with different standard deviations). This worked well. Variance was chosen as the focus measure to implement since it is quick to evaluate, and numpy has a function that does this already.

• Implemented an autofocus method for multiresolution search (described in the textbook [2], section 16.3.3).

• Implemented DWT (Discrete Wavelet Transform) to decompose image to low resolution versions. This was expected to allow for faster computation and autofocusing. In practice, there was no significant computational advantage from this, so low-res images were not used in the final algorithm.

• Implemented a hill climbing algorithm to locate an interval, within which to locate most focused position. This assesses the focus scores of individual frames.

• Implemented the Gaussian approximation to predict in focus position from the interval. This assumes that the image is formed from paraxial rays only, which is a good approximation in the case of microscopic imaging.

• Implemented the Fibonacci search algorithm. It turns out that this is the optimal search algorithm for the situation of unimodal functions [2].

• Implemented the parabola approximation to find in focus position.

The performance of our final algorithm varies, depending on the starting point of the search. A typical processing time is around 40s, which allows for autofocus during live-stream imaging through the Webshell.

The autofocus algorithm we have developed is open for further improvements and performance enhancement. Ideas for future development include a threshold value to indicate the event of reaching max focus, and hence to stop the search.

The Autofocus algorithm we have developed can be found in the full software package. It was developed solely by Rajiv, with useful feedback and advice from the rest of the Software team.

References:
[1] Firestone, L., Cook, K., Culp, K., Talsania, N. and Preston, K. (1991). Comparison of autofocus methods for automated microscopy. Cytometry, 12(3), pp.195-206
[2] Wu, Q., Merchant, F. and Castleman, K. (2008). Microscope image processing. Amsterdam: Elsevier/Academic Press.