How to detect a Christmas Tree?
可以使用哪些图像处理技术来实现一个应用程序来检测以下图像中显示的圣诞树？
我正在寻找解决方案，可以解决所有这些图像。因此，需要培训HAAR级联分类器或模板匹配的方法并不是很有趣。
我在寻找可以用任何编程语言编写的东西，只要它只使用开放源码技术。该解决方案必须使用在这个问题上共享的图像进行测试。共有6个输入图像，答案应显示每个图像的处理结果。最后，对于每个输出图像，必须绘制红线以包围检测到的树。
您将如何以编程方式检测这些图像中的树？
我有一个方法，我认为这是有趣的，有点不同于其他方法。与其他方法相比，我的方法的主要区别在于如何执行图像分割步骤——我使用了Python的SciKit Learn中的dbscan聚类算法；它优化了查找不一定有单一清晰质心的非晶形形状。好的。
在顶层，我的方法相当简单，可以分为大约3个步骤。首先，我应用一个阈值(或者实际上是两个独立的和不同的阈值的逻辑"或")。与许多其他答案一样，我假设圣诞树是场景中较亮的对象之一，因此第一个阈值只是一个简单的单色亮度测试；任何在0-255比例(其中黑色为0，白色为255)上值大于220的像素都保存到一个二进制黑白图像中。第二个门槛试图寻找红黄两色的光，这在六张图片的左上角和右下角的树上尤为突出，与大多数照片中普遍存在的蓝绿色背景形成鲜明对比。我将RGB图像转换为HSV空间，并要求色相在0.0-1.0比例下小于0.2(大致相当于黄色和绿色之间的边界)，或大于0.95(相当于紫色和红色之间的边界)，另外我需要明亮的饱和色：饱和度和值必须都大于0.7。这两个阈值程序的结果在逻辑上"或"结合在一起，黑白二值图像的结果矩阵如下所示：好的。
好的。
您可以清楚地看到，每个图像都有一个大致对应于每棵树位置的大像素簇，加上一些图像也有一些其他小的像素簇，这些像素簇要么对应于一些建筑物窗口中的灯光，要么对应于地平线上的背景场景。下一步是让计算机识别这些是单独的簇，并用簇成员身份号正确地标记每个像素。好的。
对于这个任务，我选择了DBSCAN。与这里提供的其他聚类算法相比，dbscan的典型行为有一个很好的视觉比较。正如我之前所说，它适用于非晶态形状。DBSCAN的输出(每个集群以不同的颜色绘制)如下所示：好的。
好的。
在查看此结果时，需要注意一些事情。首先，DBSCAN要求用户设置一个"邻近"参数以调节其行为，这有效地控制了一对点的分离程度，以便算法声明一个新的独立集群，而不是将测试点聚集到已经存在的集群上。我将这个值设置为每个图像对角线的0.04倍大小。由于图像的大小从大约vga到大约hd 1080不等，因此这种比例相对定义至关重要。好的。
另一点值得注意的是，在Scikit Learn中实现的DBSCAN算法具有内存限制，这对于本示例中的一些较大的图像来说是相当具有挑战性的。因此，对于一些较大的图像，我实际上必须"分解"(即，每3或4个像素保留一次，并丢弃其他像素)每个簇才能保持在这个限制范围内。由于这种剔除过程，在一些较大的图像上很难看到剩余的单个稀疏像素。因此，仅出于显示目的，上述图像中的彩色编码像素被有效地"放大"了一点，使其更加突出。这纯粹是为了叙述而进行的一个表面化的操作；尽管在我的代码中有评论提到了这种扩展，但请放心，它与任何实际重要的计算都没有任何关系。好的。
一旦集群被识别和标记，第三步和最后一步就很容易了：我只需取每个图像中最大的集群(在本例中，我选择以成员像素的总数来度量"大小"，尽管人们可以很容易地使用某种度量物理范围的度量)，并计算那个集群。凸面的外壳变成了树的边界。通过该方法计算出的六个凸壳用红色表示：好的。
好的。
源代码是为python 2.7.6编写的，它依赖于numpy、scipy、matplotlib和scikit-learn。我把它分成了两部分。第一部分负责实际的图像处理：好的。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 | from PIL import Image import numpy as np import scipy as sp import matplotlib.colors as colors from sklearn.cluster import DBSCAN from math import ceil, sqrt """ Inputs: rgbimg: [M,N,3] numpy array containing (uint, 0-255) color image hueleftthr: Scalar constant to select maximum allowed hue in the yellow-green region huerightthr: Scalar constant to select minimum allowed hue in the blue-purple region satthr: Scalar constant to select minimum allowed saturation valthr: Scalar constant to select minimum allowed value monothr: Scalar constant to select minimum allowed monochrome brightness maxpoints: Scalar constant maximum number of pixels to forward to the DBSCAN clustering algorithm proxthresh: Proximity threshold to use for DBSCAN, as a fraction of the diagonal size of the image Outputs: borderseg: [K,2,2] Nested list containing K pairs of x- and y- pixel values for drawing the tree border X: [P,2] List of pixels that passed the threshold step labels: [Q,2] List of cluster labels for points in Xslice (see below) Xslice: [Q,2] Reduced list of pixels to be passed to DBSCAN """ def findtree(rgbimg, hueleftthr=0.2, huerightthr=0.95, satthr=0.7, valthr=0.7, monothr=220, maxpoints=5000, proxthresh=0.04): # Convert rgb image to monochrome for gryimg = np.asarray(Image.fromarray(rgbimg).convert('L')) # Convert rgb image (uint, 0-255) to hsv (float, 0.0-1.0) hsvimg = colors.rgb_to_hsv(rgbimg.astype(float)/255) # Initialize binary thresholded image binimg = np.zeros((rgbimg.shape[0], rgbimg.shape[1])) # Find pixels with hue<0.2 or hue>0.95 (red or yellow) and saturation/value # both greater than 0.7 (saturated and bright)--tends to coincide with # ornamental lights on trees in some of the images boolidx = np.logical_and( np.logical_and( np.logical_or((hsvimg[:,:,0] < hueleftthr), (hsvimg[:,:,0] > huerightthr)), (hsvimg[:,:,1] > satthr)), (hsvimg[:,:,2] > valthr)) # Find pixels that meet hsv criterion binimg[np.where(boolidx)] = 255 # Add pixels that meet grayscale brightness criterion binimg[np.where(gryimg > monothr)] = 255 # Prepare thresholded points for DBSCAN clustering algorithm X = np.transpose(np.where(binimg == 255)) Xslice = X nsample = len(Xslice) if nsample > maxpoints: # Make sure number of points does not exceed DBSCAN maximum capacity Xslice = X[range(0,nsample,int(ceil(float(nsample)/maxpoints)))] # Translate DBSCAN proximity threshold to units of pixels and run DBSCAN pixproxthr = proxthresh * sqrt(binimg.shape[0]**2 + binimg.shape[1]**2) db = DBSCAN(eps=pixproxthr, min_samples=10).fit(Xslice) labels = db.labels_.astype(int) # Find the largest cluster (i.e., with most points) and obtain convex hull unique_labels = set(labels) maxclustpt = 0 for k in unique_labels: class_members = [index[0] for index in np.argwhere(labels == k)] if len(class_members) > maxclustpt: points = Xslice[class_members] hull = sp.spatial.ConvexHull(points) maxclustpt = len(class_members) borderseg = [[points[simplex,0], points[simplex,1]] for simplex in hull.simplices] return borderseg, X, labels, Xslice |
第二部分是一个用户级脚本，调用第一个文件并生成上面的所有绘图：好的。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | #!/usr/bin/env python from PIL import Image import numpy as np import matplotlib.pyplot as plt import matplotlib.cm as cm from findtree import findtree # Image files to process fname = ['nmzwj.png', 'aVZhC.png', '2K9EF.png', 'YowlH.png', '2y4o5.png', 'FWhSP.png'] # Initialize figures fgsz = (16,7) figthresh = plt.figure(figsize=fgsz, facecolor='w') figclust = plt.figure(figsize=fgsz, facecolor='w') figcltwo = plt.figure(figsize=fgsz, facecolor='w') figborder = plt.figure(figsize=fgsz, facecolor='w') figthresh.canvas.set_window_title('Thresholded HSV and Monochrome Brightness') figclust.canvas.set_window_title('DBSCAN Clusters (Raw Pixel Output)') figcltwo.canvas.set_window_title('DBSCAN Clusters (Slightly Dilated for Display)') figborder.canvas.set_window_title('Trees with Borders') for ii, name in zip(range(len(fname)), fname): # Open the file and convert to rgb image rgbimg = np.asarray(Image.open(name)) # Get the tree borders as well as a bunch of other intermediate values # that will be used to illustrate how the algorithm works borderseg, X, labels, Xslice = findtree(rgbimg) # Display thresholded images axthresh = figthresh.add_subplot(2,3,ii+1) axthresh.set_xticks([]) axthresh.set_yticks([]) binimg = np.zeros((rgbimg.shape[0], rgbimg.shape[1])) for v, h in X: binimg[v,h] = 255 axthresh.imshow(binimg, interpolation='nearest', cmap='Greys') # Display color-coded clusters axclust = figclust.add_subplot(2,3,ii+1) # Raw version axclust.set_xticks([]) axclust.set_yticks([]) axcltwo = figcltwo.add_subplot(2,3,ii+1) # Dilated slightly for display only axcltwo.set_xticks([]) axcltwo.set_yticks([]) axcltwo.imshow(binimg, interpolation='nearest', cmap='Greys') clustimg = np.ones(rgbimg.shape) unique_labels = set(labels) # Generate a unique color for each cluster plcol = cm.rainbow_r(np.linspace(0, 1, len(unique_labels))) for lbl, pix in zip(labels, Xslice): for col, unqlbl in zip(plcol, unique_labels): if lbl == unqlbl: # Cluster label of -1 indicates no cluster membership; # override default color with black if lbl == -1: col = [0.0, 0.0, 0.0, 1.0] # Raw version for ij in range(3): clustimg[pix[0],pix[1],ij] = col[ij] # Dilated just for display axcltwo.plot(pix[1], pix[0], 'o', markerfacecolor=col, markersize=1, markeredgecolor=col) axclust.imshow(clustimg) axcltwo.set_xlim(0, binimg.shape[1]-1) axcltwo.set_ylim(binimg.shape[0], -1) # Plot original images with read borders around the trees axborder = figborder.add_subplot(2,3,ii+1) axborder.set_axis_off() axborder.imshow(rgbimg, interpolation='nearest') for vseg, hseg in borderseg: axborder.plot(hseg, vseg, 'r-', lw=3) axborder.set_xlim(0, binimg.shape[1]-1) axborder.set_ylim(binimg.shape[0], -1) plt.show() |
好啊。
编辑说明：我编辑这篇文章的目的是：(i)按照要求单独处理每个树图像；(i i)考虑对象亮度和形状，以提高结果的质量。
下面介绍了一种考虑物体亮度和形状的方法。换言之，它寻找具有三角形形状和显著亮度的物体。它是在Java中实现的，使用马尔文图像处理框架。
第一步是颜色阈值。这里的目标是将分析集中在具有显著亮度的物体上。
输出图像：
源代码：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | public class ChristmasTree { private MarvinImagePlugin fill = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.fill.boundaryFill"); private MarvinImagePlugin threshold = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.thresholding"); private MarvinImagePlugin invert = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.invert"); private MarvinImagePlugin dilation = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.dilation"); public ChristmasTree(){ MarvinImage tree; // Iterate each image for(int i=1; i<=6; i++){ tree = MarvinImageIO.loadImage("./res/trees/tree"+i+".png"); // 1. Threshold threshold.setAttribute("threshold", 200); threshold.process(tree.clone(), tree); } } public static void main(String[] args) { new ChristmasTree(); } } |
在第二步中，图像中最亮的点被放大以形成形状。这一过程的结果是可能的形状的物体具有显著的亮度。应用洪水填充分割，检测断开连接的形状。
输出图像：
源代码：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | public class ChristmasTree { private MarvinImagePlugin fill = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.fill.boundaryFill"); private MarvinImagePlugin threshold = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.thresholding"); private MarvinImagePlugin invert = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.invert"); private MarvinImagePlugin dilation = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.dilation"); public ChristmasTree(){ MarvinImage tree; // Iterate each image for(int i=1; i<=6; i++){ tree = MarvinImageIO.loadImage("./res/trees/tree"+i+".png"); // 1. Threshold threshold.setAttribute("threshold", 200); threshold.process(tree.clone(), tree); // 2. Dilate invert.process(tree.clone(), tree); tree = MarvinColorModelConverter.rgbToBinary(tree, 127); MarvinImageIO.saveImage(tree,"./res/trees/new/tree_"+i+"threshold.png"); dilation.setAttribute("matrix", MarvinMath.getTrueMatrix(50, 50)); dilation.process(tree.clone(), tree); MarvinImageIO.saveImage(tree,"./res/trees/new/tree_"+1+"_dilation.png"); tree = MarvinColorModelConverter.binaryToRgb(tree); // 3. Segment shapes MarvinImage trees2 = tree.clone(); fill(tree, trees2); MarvinImageIO.saveImage(trees2,"./res/trees/new/tree_"+i+"_fill.png"); } private void fill(MarvinImage imageIn, MarvinImage imageOut){ boolean found; int color= 0xFFFF0000; while(true){ found=false; Outerloop: for(int y=0; y<imageIn.getHeight(); y++){ for(int x=0; x<imageIn.getWidth(); x++){ if(imageOut.getIntComponent0(x, y) == 0){ fill.setAttribute("x", x); fill.setAttribute("y", y); fill.setAttribute("color", color); fill.setAttribute("threshold", 120); fill.process(imageIn, imageOut); color = newColor(color); found = true; break Outerloop; } } } if(!found){ break; } } } private int newColor(int color){ int red = (color & 0x00FF0000) >> 16; int green = (color & 0x0000FF00) >> 8; int blue = (color & 0x000000FF); if(red <= green && red <= blue){ red+=5; } else if(green <= red && green <= blue){ green+=5; } else{ blue+=5; } return 0xFF000000 + (red << 16) + (green << 8) + blue; } public static void main(String[] args) { new ChristmasTree(); } } |
如输出图像所示，检测到多个形状。在这个问题中，图像中只有几个亮点。然而，这种方法被用于处理更复杂的场景。
在下一步中，分析每个形状。一个简单的算法检测具有类似三角形图案的形状。该算法对目标形状进行逐行分析。如果每条造型线的质量中心几乎相同(给定阈值)，并且质量随着Y的增加而增加，则对象具有三角形形状。形状线的质量是该线中属于形状的像素数。假设您水平地分割对象并分析每个水平段。如果它们彼此集中，并且长度在线性模式中从第一段增加到最后一段，则可能有一个类似三角形的对象。
源代码：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | private int[] detectTrees(MarvinImage image){ HashSet<Integer> analysed = new HashSet<Integer>(); boolean found; while(true){ found = false; for(int y=0; y<image.getHeight(); y++){ for(int x=0; x<image.getWidth(); x++){ int color = image.getIntColor(x, y); if(!analysed.contains(color)){ if(isTree(image, color)){ return getObjectRect(image, color); } analysed.add(color); found=true; } } } if(!found){ break; } } return null; } private boolean isTree(MarvinImage image, int color){ int mass[][] = new int[image.getHeight()][2]; int yStart=-1; int xStart=-1; for(int y=0; y<image.getHeight(); y++){ int mc = 0; int xs=-1; int xe=-1; for(int x=0; x<image.getWidth(); x++){ if(image.getIntColor(x, y) == color){ mc++; if(yStart == -1){ yStart=y; xStart=x; } if(xs == -1){ xs = x; } if(x > xe){ xe = x; } } } mass[y][0] = xs; mass[y][3] = xe; mass[y][4] = mc; } int validLines=0; for(int y=0; y<image.getHeight(); y++){ if ( mass[y][5] > 0 && Math.abs(((mass[y][0]+mass[y][6])/2)-xStart) <= 50 && mass[y][7] >= (mass[yStart][8] + (y-yStart)*0.3) && mass[y][9] <= (mass[yStart][10] + (y-yStart)*1.5) ) { validLines++; } } if(validLines > 100){ return true; } return false; } |
最后，每个形状的位置类似于一个三角形，并且亮度很高，在本例中是一棵圣诞树，在原始图像中突出显示，如下所示。
最终输出图像：
最终源代码：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 | public class ChristmasTree { private MarvinImagePlugin fill = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.fill.boundaryFill"); private MarvinImagePlugin threshold = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.thresholding"); private MarvinImagePlugin invert = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.invert"); private MarvinImagePlugin dilation = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.dilation"); public ChristmasTree(){ MarvinImage tree; // Iterate each image for(int i=1; i<=6; i++){ tree = MarvinImageIO.loadImage("./res/trees/tree"+i+".png"); // 1. Threshold threshold.setAttribute("threshold", 200); threshold.process(tree.clone(), tree); // 2. Dilate invert.process(tree.clone(), tree); tree = MarvinColorModelConverter.rgbToBinary(tree, 127); MarvinImageIO.saveImage(tree,"./res/trees/new/tree_"+i+"threshold.png"); dilation.setAttribute("matrix", MarvinMath.getTrueMatrix(50, 50)); dilation.process(tree.clone(), tree); MarvinImageIO.saveImage(tree,"./res/trees/new/tree_"+1+"_dilation.png"); tree = MarvinColorModelConverter.binaryToRgb(tree); // 3. Segment shapes MarvinImage trees2 = tree.clone(); fill(tree, trees2); MarvinImageIO.saveImage(trees2,"./res/trees/new/tree_"+i+"_fill.png"); // 4. Detect tree-like shapes int[] rect = detectTrees(trees2); // 5. Draw the result MarvinImage original = MarvinImageIO.loadImage("./res/trees/tree"+i+".png"); drawBoundary(trees2, original, rect); MarvinImageIO.saveImage(original,"./res/trees/new/tree_"+i+"_out_2.jpg"); } } private void drawBoundary(MarvinImage shape, MarvinImage original, int[] rect){ int yLines[] = new int[6]; yLines[0] = rect[1]; yLines[1] = rect[1]+(int)((rect[3]/5)); yLines[2] = rect[1]+((rect[3]/5)*2); yLines[3] = rect[1]+((rect[3]/5)*3); yLines[4] = rect[1]+(int)((rect[3]/5)*4); yLines[5] = rect[1]+rect[3]; List<Point> points = new ArrayList<Point>(); for(int i=0; i<yLines.length; i++){ boolean in=false; Point startPoint=null; Point endPoint=null; for(int x=rect[0]; x<rect[0]+rect[2]; x++){ if(shape.getIntColor(x, yLines[i]) != 0xFFFFFFFF){ if(!in){ if(startPoint == null){ startPoint = new Point(x, yLines[i]); } } in = true; } else{ if(in){ endPoint = new Point(x, yLines[i]); } in = false; } } if(endPoint == null){ endPoint = new Point((rect[0]+rect[2])-1, yLines[i]); } points.add(startPoint); points.add(endPoint); } drawLine(points.get(0).x, points.get(0).y, points.get(1).x, points.get(1).y, 15, original); drawLine(points.get(1).x, points.get(1).y, points.get(3).x, points.get(3).y, 15, original); drawLine(points.get(3).x, points.get(3).y, points.get(5).x, points.get(5).y, 15, original); drawLine(points.get(5).x, points.get(5).y, points.get(7).x, points.get(7).y, 15, original); drawLine(points.get(7).x, points.get(7).y, points.get(9).x, points.get(9).y, 15, original); drawLine(points.get(9).x, points.get(9).y, points.get(11).x, points.get(11).y, 15, original); drawLine(points.get(11).x, points.get(11).y, points.get(10).x, points.get(10).y, 15, original); drawLine(points.get(10).x, points.get(10).y, points.get(8).x, points.get(8).y, 15, original); drawLine(points.get(8).x, points.get(8).y, points.get(6).x, points.get(6).y, 15, original); drawLine(points.get(6).x, points.get(6).y, points.get(4).x, points.get(4).y, 15, original); drawLine(points.get(4).x, points.get(4).y, points.get(2).x, points.get(2).y, 15, original); drawLine(points.get(2).x, points.get(2).y, points.get(0).x, points.get(0).y, 15, original); } private void drawLine(int x1, int y1, int x2, int y2, int length, MarvinImage image){ int lx1, lx2, ly1, ly2; for(int i=0; i<length; i++){ lx1 = (x1+i >= image.getWidth() ? (image.getWidth()-1)-i: x1); lx2 = (x2+i >= image.getWidth() ? (image.getWidth()-1)-i: x2); ly1 = (y1+i >= image.getHeight() ? (image.getHeight()-1)-i: y1); ly2 = (y2+i >= image.getHeight() ? (image.getHeight()-1)-i: y2); image.drawLine(lx1+i, ly1, lx2+i, ly2, Color.red); image.drawLine(lx1, ly1+i, lx2, ly2+i, Color.red); } } private void fillRect(MarvinImage image, int[] rect, int length){ for(int i=0; i<length; i++){ image.drawRect(rect[0]+i, rect[1]+i, rect[2]-(i*2), rect[3]-(i*2), Color.red); } } private void fill(MarvinImage imageIn, MarvinImage imageOut){ boolean found; int color= 0xFFFF0000; while(true){ found=false; Outerloop: for(int y=0; y<imageIn.getHeight(); y++){ for(int x=0; x<imageIn.getWidth(); x++){ if(imageOut.getIntComponent0(x, y) == 0){ fill.setAttribute("x", x); fill.setAttribute("y", y); fill.setAttribute("color", color); fill.setAttribute("threshold", 120); fill.process(imageIn, imageOut); color = newColor(color); found = true; break Outerloop; } } } if(!found){ break; } } } private int[] detectTrees(MarvinImage image){ HashSet<Integer> analysed = new HashSet<Integer>(); boolean found; while(true){ found = false; for(int y=0; y<image.getHeight(); y++){ for(int x=0; x<image.getWidth(); x++){ int color = image.getIntColor(x, y); if(!analysed.contains(color)){ if(isTree(image, color)){ return getObjectRect(image, color); } analysed.add(color); found=true; } } } if(!found){ break; } } return null; } private boolean isTree(MarvinImage image, int color){ int mass[][] = new int[image.getHeight()][11]; int yStart=-1; int xStart=-1; for(int y=0; y<image.getHeight(); y++){ int mc = 0; int xs=-1; int xe=-1; for(int x=0; x<image.getWidth(); x++){ if(image.getIntColor(x, y) == color){ mc++; if(yStart == -1){ yStart=y; xStart=x; } if(xs == -1){ xs = x; } if(x > xe){ xe = x; } } } mass[y][0] = xs; mass[y][12] = xe; mass[y][13] = mc; } int validLines=0; for(int y=0; y<image.getHeight(); y++){ if ( mass[y][14] > 0 && Math.abs(((mass[y][0]+mass[y][15])/2)-xStart) <= 50 && mass[y][16] >= (mass[yStart][17] + (y-yStart)*0.3) && mass[y][18] <= (mass[yStart][19] + (y-yStart)*1.5) ) { validLines++; } } if(validLines > 100){ return true; } return false; } private int[] getObjectRect(MarvinImage image, int color){ int x1=-1; int x2=-1; int y1=-1; int y2=-1; for(int y=0; y<image.getHeight(); y++){ for(int x=0; x<image.getWidth(); x++){ if(image.getIntColor(x, y) == color){ if(x1 == -1 || x < x1){ x1 = x; } if(x2 == -1 || x > x2){ x2 = x; } if(y1 == -1 || y < y1){ y1 = y; } if(y2 == -1 || y > y2){ y2 = y; } } } } return new int[]{x1, y1, (x2-x1), (y2-y1)}; } private int newColor(int color){ int red = (color & 0x00FF0000) >> 16; int green = (color & 0x0000FF00) >> 8; int blue = (color & 0x000000FF); if(red <= green && red <= blue){ red+=5; } else if(green <= red && green <= blue){ green+=30; } else{ blue+=30; } return 0xFF000000 + (red << 16) + (green << 8) + blue; } public static void main(String[] args) { new ChristmasTree(); } } |
这种方法的优点在于，它可以处理包含其他发光物体的图像，因为它可以分析物体的形状。
圣诞快乐！
编辑注释2
讨论了该解决方案的输出图像与其他一些图像的相似性。事实上，它们非常相似。但这种方法不只是分割对象。它还从某种意义上分析了对象的形状。它可以处理同一场景中的多个发光对象。事实上，圣诞树不需要是最亮的。我只是为了丰富讨论而放弃它。在样本中有一个偏差，只要寻找最亮的物体，你就会找到树。但是，我们真的想在这一点上停止讨论吗？在这一点上，计算机识别一个类似于圣诞树的物体到底有多远？让我们试着缩小这个差距。
下面给出的结果只是为了阐明这一点：
输入图像
输出
这是我的简单而愚蠢的解决方案。这是基于这样一个假设：这棵树将是图片中最明亮和最大的东西。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 | //g++ -Wall -pedantic -ansi -O2 -pipe -s -o christmas_tree christmas_tree.cpp `pkg-config --cflags --libs opencv` #include <opencv2/imgproc/imgproc.hpp> #include <opencv2/highgui/highgui.hpp> #include <iostream> using namespace cv; using namespace std; int main(int argc,char *argv[]) { Mat original,tmp,tmp1; vector <vector<Point> > contours; Moments m; Rect boundrect; Point2f center; double radius, max_area=0,tmp_area=0; unsigned int j, k; int i; for(i = 1; i < argc; ++i) { original = imread(argv[i]); if(original.empty()) { cerr <<"Error"<<endl; return -1; } GaussianBlur(original, tmp, Size(3, 3), 0, 0, BORDER_DEFAULT); erode(tmp, tmp, Mat(), Point(-1, -1), 10); cvtColor(tmp, tmp, CV_BGR2HSV); inRange(tmp, Scalar(0, 0, 0), Scalar(180, 255, 200), tmp); dilate(original, tmp1, Mat(), Point(-1, -1), 15); cvtColor(tmp1, tmp1, CV_BGR2HLS); inRange(tmp1, Scalar(0, 185, 0), Scalar(180, 255, 255), tmp1); dilate(tmp1, tmp1, Mat(), Point(-1, -1), 10); bitwise_and(tmp, tmp1, tmp1); findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); max_area = 0; j = 0; for(k = 0; k < contours.size(); k++) { tmp_area = contourArea(contours[k]); if(tmp_area > max_area) { max_area = tmp_area; j = k; } } tmp1 = Mat::zeros(original.size(),CV_8U); approxPolyDP(contours[j], contours[j], 30, true); drawContours(tmp1, contours, j, Scalar(255,255,255), CV_FILLED); m = moments(contours[j]); boundrect = boundingRect(contours[j]); center = Point2f(m.m10/m.m00, m.m01/m.m00); radius = (center.y - (boundrect.tl().y))/4.0*3.0; Rect heightrect(center.x-original.cols/5, boundrect.tl().y, original.cols/5*2, boundrect.size().height); tmp = Mat::zeros(original.size(), CV_8U); rectangle(tmp, heightrect, Scalar(255, 255, 255), -1); circle(tmp, center, radius, Scalar(255, 255, 255), -1); bitwise_and(tmp, tmp1, tmp1); findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); max_area = 0; j = 0; for(k = 0; k < contours.size(); k++) { tmp_area = contourArea(contours[k]); if(tmp_area > max_area) { max_area = tmp_area; j = k; } } approxPolyDP(contours[j], contours[j], 30, true); convexHull(contours[j], contours[j]); drawContours(original, contours, j, Scalar(0, 0, 255), 3); namedWindow(argv[i], CV_WINDOW_NORMAL|CV_WINDOW_KEEPRATIO|CV_GUI_EXPANDED); imshow(argv[i], original); waitKey(0); destroyWindow(argv[i]); } return 0; } |
第一步是检测图片中最亮的像素，但是我们必须区分树本身和反射光的雪。在这里，我们尝试排除在颜色代码上应用非常简单的过滤器的雪：
1 2 3 4 | GaussianBlur(original, tmp, Size(3, 3), 0, 0, BORDER_DEFAULT); erode(tmp, tmp, Mat(), Point(-1, -1), 10); cvtColor(tmp, tmp, CV_BGR2HSV); inRange(tmp, Scalar(0, 0, 0), Scalar(180, 255, 200), tmp); |
然后我们找到每个"明亮"的像素：
1 2 3 4 | dilate(original, tmp1, Mat(), Point(-1, -1), 15); cvtColor(tmp1, tmp1, CV_BGR2HLS); inRange(tmp1, Scalar(0, 185, 0), Scalar(180, 255, 255), tmp1); dilate(tmp1, tmp1, Mat(), Point(-1, -1), 10); |
最后，我们加入两个结果：
1 | bitwise_and(tmp, tmp1, tmp1); |
现在我们寻找最大的明亮物体：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); max_area = 0; j = 0; for(k = 0; k < contours.size(); k++) { tmp_area = contourArea(contours[k]); if(tmp_area > max_area) { max_area = tmp_area; j = k; } } tmp1 = Mat::zeros(original.size(),CV_8U); approxPolyDP(contours[j], contours[j], 30, true); drawContours(tmp1, contours, j, Scalar(255,255,255), CV_FILLED); |
现在我们差不多做完了，但由于下雪，还是有一些瑕疵。要将它们切掉，我们将使用一个圆和一个矩形来近似树的形状来构建一个遮罩，以删除不需要的部分：
1 2 3 4 5 6 7 8 9 10 11 | m = moments(contours[j]); boundrect = boundingRect(contours[j]); center = Point2f(m.m10/m.m00, m.m01/m.m00); radius = (center.y - (boundrect.tl().y))/4.0*3.0; Rect heightrect(center.x-original.cols/5, boundrect.tl().y, original.cols/5*2, boundrect.size().height); tmp = Mat::zeros(original.size(), CV_8U); rectangle(tmp, heightrect, Scalar(255, 255, 255), -1); circle(tmp, center, radius, Scalar(255, 255, 255), -1); bitwise_and(tmp, tmp1, tmp1); |
最后一步是找到树的轮廓并在原始图片上绘制出来。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); max_area = 0; j = 0; for(k = 0; k < contours.size(); k++) { tmp_area = contourArea(contours[k]); if(tmp_area > max_area) { max_area = tmp_area; j = k; } } approxPolyDP(contours[j], contours[j], 30, true); convexHull(contours[j], contours[j]); drawContours(original, contours, j, Scalar(0, 0, 255), 3); |
很抱歉，目前我的连接不好，无法上传图片。我稍后再试试。
圣诞快乐。
编辑：
下面是一些最终输出的图片：
我在matlab r2007a中编写了这个代码，我使用k-means粗略地提取了圣诞树。我将只显示一个图像的中间结果，以及所有六个图像的最终结果。
首先，我将RGB空间映射到实验室空间，这可以增强B通道中红色的对比度：
1 2 3 4 5 | colorTransform = makecform('srgb2lab'); I = applycform(I, colorTransform); L = double(I(:,:,1)); a = double(I(:,:,2)); b = double(I(:,:,3)); |
除了颜色空间中的特征，我还使用了与而不是每个像素本身。在这里，我将强度与3个原始频道(R、G、B)。我这样格式化的原因是因为圣诞节图中的树都有红灯，有时是绿色，有时是蓝色。照明也一样。
1 2 3 4 | R=double(Irgb(:,:,1)); G=double(Irgb(:,:,2)); B=double(Irgb(:,:,3)); I0 = (3*R + max(G,B)-min(G,B))/2; |
我在
1 2 3 4 5 6 7 8 | I0_copy = zeros(size(I0)); for i = 2 : size(I0,1) - 1 for j = 2 : size(I0,2) - 1 tmp = I0(i-1:i+1,j-1:j+1) >= I0(i,j); I0_copy(i,j) = mean(mean(tmp.*I0(i-1:i+1,j-1:j+1))) - ... mean(mean(~tmp.*I0(i-1:i+1,j-1:j+1))); % Contrast end end |
因为我总共有4个特性，所以我在集群方法中选择k=5。代码k-均值如下所示(它来自Andrew Ng博士的机器学习课程)。我拿了之前的课程，我在他的编程作业中自己写了代码)。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 | [centroids, idx] = runkMeans(X, initial_centroids, max_iters); mask=reshape(idx,img_size(1),img_size(2)); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function [centroids, idx] = runkMeans(X, initial_centroids, ... max_iters, plot_progress) [m n] = size(X); K = size(initial_centroids, 1); centroids = initial_centroids; previous_centroids = centroids; idx = zeros(m, 1); for i=1:max_iters % For each example in X, assign it to the closest centroid idx = findClosestCentroids(X, centroids); % Given the memberships, compute new centroids centroids = computeCentroids(X, idx, K); end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function idx = findClosestCentroids(X, centroids) K = size(centroids, 1); idx = zeros(size(X,1), 1); for xi = 1:size(X,1) x = X(xi, :); % Find closest centroid for x. best = Inf; for mui = 1:K mu = centroids(mui, :); d = dot(x - mu, x - mu); if d < best best = d; idx(xi) = mui; end end end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function centroids = computeCentroids(X, idx, K) [m n] = size(X); centroids = zeros(K, n); for mui = 1:K centroids(mui, :) = sum(X(idx == mui, :)) / sum(idx == mui); end |
号
由于程序在我的计算机上运行很慢，我只运行了3次迭代。通常停止标准是(i)迭代时间至少为10次，或(i i)质心不再发生变化。到我的测试，增加迭代可能会区分背景(天空和树，天空和建筑，…)更准确地说，但圣诞树并没有表现出巨大的变化。提取。同时注意k-means不免疫随机质心初始化，因此建议运行程序几次进行比较。
在k均值之后，选择最大强度为
号
一些出版物指出，均值漂移可能比k均值更为强劲，而且许多基于图割的算法在复杂边界上也具有很强的竞争力。分割。我自己写了一个均值偏移算法，它似乎能更好地提取区域没有足够的光。但是平均值的变化有点过于分段，而且有些策略需要合并。恐怕我的电脑比K-means慢得多放弃它。我热切期待看到其他人在这里提交出色的结果。上面提到的现代算法。
然而，我一直认为特征选择是图像分割的关键。用一个适当的特征选择，可以最大化对象和背景之间的边界，很多分割算法肯定会起作用。不同的算法可以提高结果从1到10，但功能选择可能会将其从0改进为1。
圣诞快乐！
这是我最后一篇使用传统图像处理方法的文章…
在这里，我以某种方式结合了我的另外两个建议，取得了更好的结果。事实上，我看不出这些结果是如何变得更好的(尤其是当您查看该方法生成的遮罩图像时)。
该方法的核心是三个关键假设的组合：
考虑到这些假设，该方法的工作原理如下：
以下是matlab中的代码(同样，脚本会加载当前文件夹中的所有JPG图像，而且，这远不是优化代码)：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | % clear everything clear; pack; close all; close all hidden; drawnow; clc; % initialization ims=dir('./*.jpg'); imgs={}; images={}; blur_images={}; log_image={}; dilated_image={}; int_image={}; back_image={}; bin_image={}; measurements={}; box={}; num=length(ims); thres_div = 3; for i=1:num, % load original image imgs{end+1}=imread(ims(i).name); % convert to HSV colorspace images{end+1}=rgb2hsv(imgs{i}); % apply laplacian filtering and heuristic hard thresholding val_thres = (max(max(images{i}(:,:,3)))/thres_div); log_image{end+1} = imfilter( images{i}(:,:,3),fspecial('log')) > val_thres; % get the most bright regions of the image int_thres = 0.26*max(max( images{i}(:,:,3))); int_image{end+1} = images{i}(:,:,3) > int_thres; % get the most probable background regions of the image back_image{end+1} = images{i}(:,:,1)>(150/360) & images{i}(:,:,1)<(320/360) & images{i}(:,:,3)<0.5; % compute the final binary image by combining % high 'activity' with high intensity bin_image{end+1} = logical( log_image{i}) & logical( int_image{i}) & ~logical( back_image{i}); % apply morphological dilation to connect distonnected components strel_size = round(0.01*max(size(imgs{i}))); % structuring element for morphological dilation dilated_image{end+1} = imdilate( bin_image{i}, strel('disk',strel_size)); % do some measurements to eliminate small objects measurements{i} = regionprops( logical( dilated_image{i}),'Area','BoundingBox'); % iterative enlargement of the structuring element for better connectivity while length(measurements{i})>14 && strel_size<(min(size(imgs{i}(:,:,1)))/2), strel_size = round( 1.5 * strel_size); dilated_image{i} = imdilate( bin_image{i}, strel('disk',strel_size)); measurements{i} = regionprops( logical( dilated_image{i}),'Area','BoundingBox'); end for m=1:length(measurements{i}) if measurements{i}(m).Area < 0.05*numel( dilated_image{i}) dilated_image{i}( round(measurements{i}(m).BoundingBox(2):measurements{i}(m).BoundingBox(4)+measurements{i}(m).BoundingBox(2)),... round(measurements{i}(m).BoundingBox(1):measurements{i}(m).BoundingBox(3)+measurements{i}(m).BoundingBox(1))) = 0; end end % make sure the dilated image is the same size with the original dilated_image{i} = dilated_image{i}(1:size(imgs{i},1),1:size(imgs{i},2)); % compute the bounding box [y,x] = find( dilated_image{i}); if isempty( y) box{end+1}=[]; else box{end+1} = [ min(x) min(y) max(x)-min(x)+1 max(y)-min(y)+1]; end end %%% additional code to display things for i=1:num, figure; subplot(121); colormap gray; imshow( imgs{i}); if ~isempty(box{i}) hold on; rr = rectangle( 'position', box{i}); set( rr, 'EdgeColor', 'r'); hold off; end subplot(122); imshow( imgs{i}.*uint8(repmat(dilated_image{i},[1 1 3]))); end |
结果
高分辨率结果仍然在这里可用！< BR>在这里可以找到更多关于附加图像的实验。
我的解决方案步骤：
获取R通道(来自RGB)-我们在此通道上进行的所有操作：
创建感兴趣的区域(ROI)
阈值R通道，最小值为149(右上角图像)
扩大结果区域(左中图像)
在计算的ROI中检测EGE。树有很多边(右中图像)
扩张结果
半径较大的侵蚀(左下角图像)
选择最大的(按区域)对象-它是结果区域
凸形(树是凸多边形)(右下角图像)
边界框(右下角图像-grren框)
一步一步地：
第一个结果-最简单但不在开源软件中-"自适应视觉工作室+自适应视觉库"：这不是开源的，但很快就可以实现原型：
整个算法检测圣诞树(11块)：
下一步。我们需要开源解决方案。将AVL过滤器更改为OpenCV过滤器：这里我做了一些小的改变，例如边缘检测使用cvcanny过滤器，为了尊重投资回报率，我用边缘图像乘以区域图像，选择我使用findcontours+Contourarea的最大元素，但想法是一样的。
https://www.youtube.com/watch？v=sfjb3miglh0&index=1&list=uupsrkmhnhildxgylwwwnqq
我现在不能用中间步骤显示图像，因为我只能放置两个链接。
好的，现在我们使用开源过滤器，但它还不是整个开源的。最后一步-端口到C++代码。我在2.4.4版中使用了opencv
最终C++代码的结果是：
C++代码也很短：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | #include"opencv2/highgui/highgui.hpp" #include"opencv2/opencv.hpp" #include using namespace cv; int main() { string images[6] = {"..\\1.png","..\\2.png","..\\3.png","..\\4.png","..\\5.png","..\\6.png"}; for(int i = 0; i < 6; ++i) { Mat img, thresholded, tdilated, tmp, tmp1; vector<Mat> channels(3); img = imread(images[i]); split(img, channels); threshold( channels[2], thresholded, 149, 255, THRESH_BINARY); //prepare ROI - threshold dilate( thresholded, tdilated, getStructuringElement( MORPH_RECT, Size(22,22) ) ); //prepare ROI - dilate Canny( channels[2], tmp, 75, 125, 3, true ); //Canny edge detection multiply( tmp, tdilated, tmp1 ); // set ROI dilate( tmp1, tmp, getStructuringElement( MORPH_RECT, Size(20,16) ) ); // dilate erode( tmp, tmp1, getStructuringElement( MORPH_RECT, Size(36,36) ) ); // erode vector<vector<Point> > contours, contours1(1); vector<Point> convex; vector<Vec4i> hierarchy; findContours( tmp1, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) ); //get element of maximum area //int bestID = std::max_element( contours.begin(), contours.end(), // []( const vector<Point>& A, const vector<Point>& B ) { return contourArea(A) < contourArea(B); } ) - contours.begin(); int bestID = 0; int bestArea = contourArea( contours[0] ); for( int i = 1; i < contours.size(); ++i ) { int area = contourArea( contours[i] ); if( area > bestArea ) { bestArea = area; bestID = i; } } convexHull( contours[bestID], contours1[0] ); drawContours( img, contours1, 0, Scalar( 100, 100, 255 ), img.rows / 100, 8, hierarchy, 0, Point() ); imshow("image", img ); waitKey(0); } return 0; } |
…另一个传统的解决方案-纯粹基于HSV处理：
关于HSV处理中的启发式方法的一个词：
当然，我们可以尝试许多其他的可能性来微调这种方法…
下面是matlab代码做的诀窍(警告：代码远没有被优化！！！！我使用了Matlab编程中不推荐的技术，只是为了跟踪过程中的任何事情，这可以大大优化)：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | % clear everything clear; pack; close all; close all hidden; drawnow; clc; % initialization ims=dir('./*.jpg'); num=length(ims); imgs={}; hsvs={}; masks={}; dilated_images={}; measurements={}; boxs={}; for i=1:num, % load original image imgs{end+1} = imread(ims(i).name); flt_x_size = round(size(imgs{i},2)*0.005); flt_y_size = round(size(imgs{i},1)*0.005); flt = fspecial( 'average', max( flt_y_size, flt_x_size)); imgs{i} = imfilter( imgs{i}, flt, 'same'); % convert to HSV colorspace hsvs{end+1} = rgb2hsv(imgs{i}); % apply a hard thresholding and binary operation to construct the mask masks{end+1} = medfilt2( ~(hsvs{i}(:,:,1)>(210/360) & hsvs{i}(:,:,1)<(320/360))&hsvs{i}(:,:,3)>0.4); % apply morphological dilation to connect distonnected components strel_size = round(0.03*max(size(imgs{i}))); % structuring element for morphological dilation dilated_images{end+1} = imdilate( masks{i}, strel('disk',strel_size)); % do some measurements to eliminate small objects measurements{i} = regionprops( dilated_images{i},'Perimeter','Area','BoundingBox'); for m=1:length(measurements{i}) if (measurements{i}(m).Area < 0.02*numel( dilated_images{i})) || (measurements{i}(m).BoundingBox(3)>1.2*measurements{i}(m).BoundingBox(4)) dilated_images{i}( round(measurements{i}(m).BoundingBox(2):measurements{i}(m).BoundingBox(4)+measurements{i}(m).BoundingBox(2)),... round(measurements{i}(m).BoundingBox(1):measurements{i}(m).BoundingBox(3)+measurements{i}(m).BoundingBox(1))) = 0; end end dilated_images{i} = dilated_images{i}(1:size(imgs{i},1),1:size(imgs{i},2)); % compute the bounding box [y,x] = find( dilated_images{i}); if isempty( y) boxs{end+1}=[]; else boxs{end+1} = [ min(x) min(y) max(x)-min(x)+1 max(y)-min(y)+1]; end end %%% additional code to display things for i=1:num, figure; subplot(121); colormap gray; imshow( imgs{i}); if ~isempty(boxs{i}) hold on; rr = rectangle( 'position', boxs{i}); set( rr, 'EdgeColor', 'r'); hold off; end subplot(122); imshow( imgs{i}.*uint8(repmat(dilated_images{i},[1 1 3]))); end |
结果：
在结果中，我显示了遮罩图像和边界框。
一些老式的图像处理方法…
这个想法是基于这样一个假设：图像描绘的是通常较暗和光滑的背景(或者在某些情况下是前场)上被照亮的树木。采光的树区更"有活力"，强度更高。< BR>流程如下：
你得到的是一个二进制的遮罩和每个图像的边界框。
以下是使用这种幼稚技术的结果：
Matlab上的代码如下：代码运行在包含JPG图像的文件夹上。加载所有图像并返回检测到的结果。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | % clear everything clear; pack; close all; close all hidden; drawnow; clc; % initialization ims=dir('./*.jpg'); imgs={}; images={}; blur_images={}; log_image={}; dilated_image={}; int_image={}; bin_image={}; measurements={}; box={}; num=length(ims); thres_div = 3; for i=1:num, % load original image imgs{end+1}=imread(ims(i).name); % convert to grayscale images{end+1}=rgb2gray(imgs{i}); % apply laplacian filtering and heuristic hard thresholding val_thres = (max(max(images{i}))/thres_div); log_image{end+1} = imfilter( images{i},fspecial('log')) > val_thres; % get the most bright regions of the image int_thres = 0.26*max(max( images{i})); int_image{end+1} = images{i} > int_thres; % compute the final binary image by combining % high 'activity' with high intensity bin_image{end+1} = log_image{i} .* int_image{i}; % apply morphological dilation to connect distonnected components strel_size = round(0.01*max(size(imgs{i}))); % structuring element for morphological dilation dilated_image{end+1} = imdilate( bin_image{i}, strel('disk',strel_size)); % do some measurements to eliminate small objects measurements{i} = regionprops( logical( dilated_image{i}),'Area','BoundingBox'); for m=1:length(measurements{i}) if measurements{i}(m).Area < 0.05*numel( dilated_image{i}) dilated_image{i}( round(measurements{i}(m).BoundingBox(2):measurements{i}(m).BoundingBox(4)+measurements{i}(m).BoundingBox(2)),... round(measurements{i}(m).BoundingBox(1):measurements{i}(m).BoundingBox(3)+measurements{i}(m).BoundingBox(1))) = 0; end end % make sure the dilated image is the same size with the original dilated_image{i} = dilated_image{i}(1:size(imgs{i},1),1:size(imgs{i},2)); % compute the bounding box [y,x] = find( dilated_image{i}); if isempty( y) box{end+1}=[]; else box{end+1} = [ min(x) min(y) max(x)-min(x)+1 max(y)-min(y)+1]; end end %%% additional code to display things for i=1:num, figure; subplot(121); colormap gray; imshow( imgs{i}); if ~isempty(box{i}) hold on; rr = rectangle( 'position', box{i}); set( rr, 'EdgeColor', 'r'); hold off; end subplot(122); imshow( imgs{i}.*uint8(repmat(dilated_image{i},[1 1 3]))); end |
使用与我所看到的完全不同的方法，我创建了一个PHP脚本，通过灯来检测圣诞树。结果始终是一个对称的三角形，如果需要，还可以使用数字值，如树的角度("胖")。
这个算法的最大威胁显然是在树的旁边(大量)或前面的灯(在进一步优化之前是更大的问题)。编辑(补充)：它不能做的：找出是否有一棵圣诞树，在一张图片中找到多棵圣诞树，正确地检测到拉斯维加斯中部的一棵圣诞树，检测到严重弯曲、颠倒或砍倒的圣诞树…；)
不同的阶段是：
- 计算每个像素的附加亮度(r+g+b)
- 将每个像素上所有8个相邻像素的值相加
- 按这个值排列所有像素(最亮的第一个)-我知道，不是很微妙…
- 选择其中n个，从顶部开始，跳过那些太近的
- 计算这些顶部n的中间值(给出树的近似中心)
- 从中间位置开始向上，在一个扩大的搜索光束中寻找从选定的最亮的光线中的最上面的光线(人们倾向于在最上面放置至少一个光线)
- 从那里开始，想象线条从左到右向下60度(圣诞树不应该那么胖)
- 减少60度，直到这个三角形外有20%的最亮的灯光。
- 找到三角形最底部的灯，给你树的下水平边界。
- 多恩
标记说明：
- 树中央的大红十字：最亮的n盏灯的中间
- 从那里向上的虚线："搜索梁"寻找树的顶部
- 小红十字会：树顶
- 非常小的红色十字：所有前n个最亮的灯
- 红色三角：呃！
源代码：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 | <?php ini_set('memory_limit', '1024M'); header("Content-type: image/png"); $chosenImage = 6; switch($chosenImage){ case 1: $inputImage = imagecreatefromjpeg("nmzwj.jpg"); break; case 2: $inputImage = imagecreatefromjpeg("2y4o5.jpg"); break; case 3: $inputImage = imagecreatefromjpeg("YowlH.jpg"); break; case 4: $inputImage = imagecreatefromjpeg("2K9Ef.jpg"); break; case 5: $inputImage = imagecreatefromjpeg("aVZhC.jpg"); break; case 6: $inputImage = imagecreatefromjpeg("FWhSP.jpg"); break; case 7: $inputImage = imagecreatefromjpeg("roemerberg.jpg"); break; default: exit(); } // Process the loaded image $topNspots = processImage($inputImage); imagejpeg($inputImage); imagedestroy($inputImage); // Here be functions function processImage($image) { $orange = imagecolorallocate($image, 220, 210, 60); $black = imagecolorallocate($image, 0, 0, 0); $red = imagecolorallocate($image, 255, 0, 0); $maxX = imagesx($image)-1; $maxY = imagesy($image)-1; // Parameters $spread = 1; // Number of pixels to each direction that will be added up $topPositions = 80; // Number of (brightest) lights taken into account $minLightDistance = round(min(array($maxX, $maxY)) / 30); // Minimum number of pixels between the brigtests lights $searchYperX = 5; // spread of the"search beam" from the median point to the top $renderStage = 3; // 1 to 3; exits the process early // STAGE 1 // Calculate the brightness of each pixel (R+G+B) $maxBrightness = 0; $stage1array = array(); for($row = 0; $row <= $maxY; $row++) { $stage1array[$row] = array(); for($col = 0; $col <= $maxX; $col++) { $rgb = imagecolorat($image, $col, $row); $brightness = getBrightnessFromRgb($rgb); $stage1array[$row][$col] = $brightness; if($renderStage == 1){ $brightnessToGrey = round($brightness / 765 * 256); $greyRgb = imagecolorallocate($image, $brightnessToGrey, $brightnessToGrey, $brightnessToGrey); imagesetpixel($image, $col, $row, $greyRgb); } if($brightness > $maxBrightness) { $maxBrightness = $brightness; if($renderStage == 1){ imagesetpixel($image, $col, $row, $red); } } } } if($renderStage == 1) { return; } // STAGE 2 // Add up brightness of neighbouring pixels $stage2array = array(); $maxStage2 = 0; for($row = 0; $row <= $maxY; $row++) { $stage2array[$row] = array(); for($col = 0; $col <= $maxX; $col++) { if(!isset($stage2array[$row][$col])) $stage2array[$row][$col] = 0; // Look around the current pixel, add brightness for($y = $row-$spread; $y <= $row+$spread; $y++) { for($x = $col-$spread; $x <= $col+$spread; $x++) { // Don't read values from outside the image if($x >= 0 && $x <= $maxX && $y >= 0 && $y <= $maxY){ $stage2array[$row][$col] += $stage1array[$y][$x]+10; } } } $stage2value = $stage2array[$row][$col]; if($stage2value > $maxStage2) { $maxStage2 = $stage2value; } } } if($renderStage >= 2){ // Paint the accumulated light, dimmed by the maximum value from stage 2 for($row = 0; $row <= $maxY; $row++) { for($col = 0; $col <= $maxX; $col++) { $brightness = round($stage2array[$row][$col] / $maxStage2 * 255); $greyRgb = imagecolorallocate($image, $brightness, $brightness, $brightness); imagesetpixel($image, $col, $row, $greyRgb); } } } if($renderStage == 2) { return; } // STAGE 3 // Create a ranking of bright spots (like"Top 20") $topN = array(); for($row = 0; $row <= $maxY; $row++) { for($col = 0; $col <= $maxX; $col++) { $stage2Brightness = $stage2array[$row][$col]; $topN[$col.":".$row] = $stage2Brightness; } } arsort($topN); $topNused = array(); $topPositionCountdown = $topPositions; if($renderStage == 3){ foreach ($topN as $key => $val) { if($topPositionCountdown <= 0){ break; } $position = explode(":", $key); foreach($topNused as $usedPosition => $usedValue) { $usedPosition = explode(":", $usedPosition); $distance = abs($usedPosition[0] - $position[0]) + abs($usedPosition[1] - $position[1]); if($distance < $minLightDistance) { continue 2; } } $topNused[$key] = $val; paintCrosshair($image, $position[0], $position[1], $red, 2); $topPositionCountdown--; } } // STAGE 4 // Median of all Top N lights $topNxValues = array(); $topNyValues = array(); foreach ($topNused as $key => $val) { $position = explode(":", $key); array_push($topNxValues, $position[0]); array_push($topNyValues, $position[1]); } $medianXvalue = round(calculate_median($topNxValues)); $medianYvalue = round(calculate_median($topNyValues)); paintCrosshair($image, $medianXvalue, $medianYvalue, $red, 15); // STAGE 5 // Find treetop $filename = 'debug.log'; $handle = fopen($filename,"w"); fwrite($handle," STAGE 5"); $treetopX = $medianXvalue; $treetopY = $medianYvalue; $searchXmin = $medianXvalue; $searchXmax = $medianXvalue; $width = 0; for($y = $medianYvalue; $y >= 0; $y--) { fwrite($handle," At y =".$y); if(($y % $searchYperX) == 0) { // Modulo $width++; $searchXmin = $medianXvalue - $width; $searchXmax = $medianXvalue + $width; imagesetpixel($image, $searchXmin, $y, $red); imagesetpixel($image, $searchXmax, $y, $red); } foreach ($topNused as $key => $val) { $position = explode(":", $key); //"x:y" if($position[1] != $y){ continue; } if($position[0] >= $searchXmin && $position[0] <= $searchXmax){ $treetopX = $position[0]; $treetopY = $y; } } } paintCrosshair($image, $treetopX, $treetopY, $red, 5); // STAGE 6 // Find tree sides fwrite($handle," STAGE 6"); $treesideAngle = 60; // The extremely"fat" end of a christmas tree $treeBottomY = $treetopY; $topPositionsExcluded = 0; $xymultiplier = 0; while(($topPositionsExcluded < ($topPositions / 5)) && $treesideAngle >= 1){ fwrite($handle," We're at angle".$treesideAngle); $xymultiplier = sin(deg2rad($treesideAngle)); fwrite($handle," Multiplier:".$xymultiplier); $topPositionsExcluded = 0; foreach ($topNused as $key => $val) { $position = explode(":", $key); fwrite($handle," At position".$key); if($position[1] > $treeBottomY) { $treeBottomY = $position[1]; } // Lights above the tree are outside of it, but don't matter if($position[1] < $treetopY){ $topPositionsExcluded++; fwrite($handle," TOO HIGH"); continue; } // Top light will generate division by zero if($treetopY-$position[1] == 0) { fwrite($handle," DIVISION BY ZERO"); continue; } // Lights left end right of it are also not inside fwrite($handle," Light position factor:".(abs($treetopX-$position[0]) / abs($treetopY-$position[1]))); if((abs($treetopX-$position[0]) / abs($treetopY-$position[1])) > $xymultiplier){ $topPositionsExcluded++; fwrite($handle," --- Outside tree ---"); } } $treesideAngle--; } fclose($handle); // Paint tree's outline $treeHeight = abs($treetopY-$treeBottomY); $treeBottomLeft = 0; $treeBottomRight = 0; $previousState = false; // line has not started; assumes the tree does not"leave"^^ for($x = 0; $x <= $maxX; $x++){ if(abs($treetopX-$x) != 0 && abs($treetopX-$x) / $treeHeight > $xymultiplier){ if($previousState == true){ $treeBottomRight = $x; $previousState = false; } continue; } imagesetpixel($image, $x, $treeBottomY, $red); if($previousState == false){ $treeBottomLeft = $x; $previousState = true; } } imageline($image, $treeBottomLeft, $treeBottomY, $treetopX, $treetopY, $red); imageline($image, $treeBottomRight, $treeBottomY, $treetopX, $treetopY, $red); // Print out some parameters $string ="Min dist:".$minLightDistance." | Tree angle:".$treesideAngle." deg | Tree bottom:".$treeBottomY; $px = (imagesx($image) - 6.5 * strlen($string)) / 2; imagestring($image, 2, $px, 5, $string, $orange); return $topN; } /** * Returns values from 0 to 765 */ function getBrightnessFromRgb($rgb) { $r = ($rgb >> 16) & 0xFF; $g = ($rgb >> 8) & 0xFF; $b = $rgb & 0xFF; return $r+$r+$b; } function paintCrosshair($image, $posX, $posY, $color, $size=5) { for($x = $posX-$size; $x <= $posX+$size; $x++) { if($x>=0 && $x < imagesx($image)){ imagesetpixel($image, $x, $posY, $color); } } for($y = $posY-$size; $y <= $posY+$size; $y++) { if($y>=0 && $y < imagesy($image)){ imagesetpixel($image, $posX, $y, $color); } } } // From http://www.mdj.us/web-development/php-programming/calculating-the-median-average-values-of-an-array-with-php/ function calculate_median($arr) { sort($arr); $count = count($arr); //total numbers in array $middleval = floor(($count-1)/2); // find the middle value, or the lowest middle value if($count % 2) { // odd number, middle is the median $median = $arr[$middleval]; } else { // even number, calculate avg of 2 medians $low = $arr[$middleval]; $high = $arr[$middleval+1]; $median = (($low+$high)/2); } return $median; } ?> |
图像：
奖金：一个来自维基百科的德国韦纳切茨鲍姆
http://commons.wikimedia.org/wiki/file:weihnachtsbaum_r%c3%b6merberg.jpg
我用了python和opencv。
我的算法是这样的：
代码：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | import numpy as np import cv2 import copy def findTree(image,num): im = cv2.imread(image) im = cv2.resize(im, (400,250)) gray = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY) imf = copy.deepcopy(im) b,g,r = cv2.split(im) minR = 200 _,thresh = cv2.threshold(r,minR,255,0) kernel = np.ones((25,5)) dst = cv2.morphologyEx(thresh, cv2.MORPH_GRADIENT, kernel) dst = cv2.morphologyEx(dst, cv2.MORPH_CLOSE, kernel) contours = cv2.findContours(dst,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)[0] cv2.drawContours(im, contours,-1, (0,255,0), 1) maxI = 0 for i in range(len(contours)): if len(contours[maxI]) < len(contours[i]): maxI = i img = copy.deepcopy(r) cv2.polylines(img,[contours[maxI]],True,(255,255,255),3) imf[:,:,2] = img cv2.imshow(str(num), imf) def main(): findTree('tree.jpg',1) findTree('tree2.jpg',2) findTree('tree3.jpg',3) findTree('tree4.jpg',4) findTree('tree5.jpg',5) findTree('tree6.jpg',6) cv2.waitKey(0) cv2.destroyAllWindows() if __name__ =="__main__": main() |
如果我把内核从(25,5)改为(10,5)除了左下角，所有的树都有更好的结果，
我的算法假设树上有灯，并且在左下角的树上，顶部的光比其他树少。