Computer Vision and Imaging (extended) [06-30241] Assignment 1
This assessment is summative and contains three parts. In Part 1, you will answer theoretical questions. In Part 2, you will perform local image feature detection and extraction, add and remove image noise to images using MATLAB. In Part 3, you will work on image segmentation.
Your answer must be submitted to Canvas before the deadline in the form of a single zip archive file containing:
1. Your answers to the questions in prose and diagrams. This should take the form of a single PDF document with the answers for each question using the provided LaTeX template.
2. Your code and any accompanying files necessary to execute the code for any programming questions as specified in the LaTeX template.
and a separate PDF document with the answers for Turnitin checking (two files in total). Some or all of the text of each question is emphasised using italics. This emphasis indicates a question that must be explicitly answered or a task that must be completed.
Question 1.1 [4 marks] Consider the following image matrix and kernel (Figure 1). Produce the resulting convolved image matrix for the central dashed region. What affect does this kernel have? (Consider both this image and larger example images)
Figure 1: An example Image and Kernel
Question 1.2 [5 marks] Another pair of filter kernels are shown in Figure 2. What are their responses to uniform brightness? Are the filters isotropic or anisotropic and explain the terms. Which filter would you use for finding edges? What kind of edges would you use it to find?
Figure 2: A pair of filter kernels
Question 1.3 [5 marks] Is it possible to use the intrinsic and extrinsic camera properties to transform from a point in the camera 2D pixel coordinate system to a point in the world 3D coordinate system, and why?
Question 1.4 [6 marks] CCD/CMOS cameras contain arrays of sensors which convert photons into a proportional charge at each point. The number of sensors can range from a few hundred thousand to many million. Taking a 1cm2 6 MegaPixel imaging array as an example, what is the limiting factor to increasing the number of sensors to increase the resolution of the image captured?
Local image feature extraction and matching
Question 2.1.1 [7 marks] Add gaussian (mean = 0.3, variance = 0.5), salt & pepper (noise density = 0.75) and speckle image noise to dom.jpg using the imnoise function. Show the images using subplot and add title to each image separately (See example in Figure 3.). Discuss your observations.
Figure 3: Example result of 2.1.1.
Question 2.1.2 [7 marks] Implement minimum, maximum, median and mean filters for image noise removal. Remove the previously added noises from the images with Gaussian and Salt&Pepper noise using 5×5 structuring element. Show the results using subplot and add title to each image separately (See example in Figure 4.). How well did they work and why? Discuss your observations.
Figure 4: Example result of 2.1.2.
Question 2.2 [13 marks] Extract features from the Salt&Pepper image (from Question 2.1.1) and the denoised images (from Question 2.1.2, note that this is not the original image), compare results and visualize them. In case you could not complete 2.1.2. you can use the original image for this Question, but points will be deducted. Display all of your results similarly to Figure 5., indicate it in the title if you used the original image. Discuss your observations.
Figure 5: Example result of 2.2.
Question 2.3. [13 marks] Use the dom full.jpeg and dom part.jpg images. Perform image reg- istration and display your result similarly to Figure 6.
Figure 6: Example results of 2.3.: (1) All of the matches including outliers, (2) Inlier matches, (3) Recovered image
Image segmentation task
“Image segmentation is a commonly used technique in digital image processing and analysis to partition an image into multiple parts or regions, often based on the characteristics of the pixels in the image. Image segmentation could involve separating foreground from back- ground, or clustering regions of pixels based on similarities in color or shape. For example, a common application of image segmentation in medical imaging is to detect and label pixels in an image or voxels of a 3D volume that represent a tumor in a patients brain or other organs.” – Matlab Image Segmentation
OTSU’s method (matlab function graythresh) is an example of an Matlab Segmentaion func- tion: “OTSU’s method chooses a threshold that minimizes the intraclass variance of the thresholded black and white pixels”.
Question 3.1 [12 marks] Image ‘cells.png’ is an example of cells tagged with green fluorescent protein (GFP), imaged using a fluorescence microscope. Firstly, use the OTSU’s method of segmentation (See Matlab function graythresh) with no pre-processing or filters. Then segment the cells using ANY method/s you consider appropriate, with any pre-processing and/or filters. Justify the choice of your best performing method (or a sequence of methods), list the values of all the parameters used and justify their choices. Compare your results to the results of using the OTSU’s method and comment on the reasons for the observed differences in results. If other methods produce worse results than the default OTSU’s method, then please explain why you think this is.
Please note that some groups of cells overlap. Separating them is quite hard so is acceptable to leave the most difficult ones clustered together and count them as one. There will be extra marks for those students who manage to separate all the cells.
Figure 7: Cells tagged with GFP, imaged using a fluorescence microscope.
Figure 8: An example of the various ways a cell segmentation could be displayed. This does not necessarily have to be reproduced for this question, but is here as a guide. Note, this image is not the same as the original cells.png.
Question 3.2 [8 marks] Use either the default OTSU’s method, or your best method. For vi- sual verification superimpose the outlines of the cells on the original image. Apply any pre- processing (e.g. smoothing) or post-processing operations (mathematical morphology oper- ations such as erosion, dilation, opening, closing, etc.) to get as smooth as possible – but as accurate as possible – boundaries of the segmented cells. Explain the purpose and the outcome(s) of any operation that you have used.
Question 3.3 Compute the cell statistics from the default OTSU method:
Question 3.3.1 [4 marks] Area (in pixels) of each cell
Question 3.3.2 [4 marks] Mean brightness (in green channel) of each cell
Question 3.3.3 [2 marks] Mean and standard deviation for the area and brightness for all the cells in the image
Question 3.4 [10 marks] A group of E-coli cells (‘Ecoli.png’) have been tagged with the GFP and red flourescent protein (RFP) (figure 9). Repeat Questions 3.1, 3.2, and 3.3 on the image of E-coli. Only segment the cells tagged with the GFP., i.e. do not include any RFP tagged E- coli cells. Individual cell statistics do not have to be shown, but average cell statistics should be presented.
Figure 9: E-coli Cells tagged with GFP and RFP.