GEODESIC-BASED PAVEMENT SHADOWREMOVAL REVISITED

GEODESIC-BASED PA VEMENT SHADOW REMOV AL REVISITED Qin Zou1,2Zhongwen Hu3Long Chen4Qian Wang1,2Qingquan Li?,3 1State Key Laboratory of Software Engineering,Wuhan University,P.R.China

2School of Computer Science,Wuhan University,P.R.China

3Shenzhen Key Laboratory of Spatial Smart Sensing and Service,Shenzhen University,P.R.China 4School of Mobile Information Engineering,Sun Yat-Sen University,P.R.China

qzou@https://www.wendangku.net/doc/447306636.html,,zwhoo@https://www.wendangku.net/doc/447306636.html,,chenl46@https://www.wendangku.net/doc/447306636.html,,qianwang@https://www.wendangku.net/doc/447306636.html,,liqq@https://www.wendangku.net/doc/447306636.html,

ABSTRACT

Shadows often incur uneven illumination to pavement im-ages,which brings great challenges to image-based pavement crack detection.Thus,it is desired to remove pavement shad-ows before detecting pavement cracks.However,due to the large penumbras cast by trees,light poles,etc.,it is dif?cult to locate shadows in a pavement image.In this paper,an au-tomatic pavement shadow removal method is proposed based on geodesic analysis.First,a geodesic shadow model is used to partition a pavement shadow into a number of geodesic re-gions.Then,an optimal background region is selected for reference by statistic analysis.Finally,a texture-balanced il-luminance compensation is applied on all geodesic regions over the image.Experiments demonstrate the effectiveness of the proposed method.

Index Terms—shadow removal,penumbra area,pave-ment shadow,crack detection,texture balance

1.INTRODUCTION

Modern road maintenance has a strong demand for automatic pavement crack detection.In the past ten years,image-based pavement crack detection and classi?cation has been a re-search focus in both academia and industry[1,2,3].How-ever,when pavement images are contaminated by shadows, many complications arise and undermine the performance of crack detection methods.This is because,pavement shad-ows not only destroy the uniform illuminance of pavement images,but also decrease the contrast of pavement cracks in the shadow regions.Thus,a shadow removal procedure is commonly required before detecting the pavement cracks.

In the past two decades,a number of algorithms have been developed to remove shadow from images[4,5,6,7,8,9,10]. Typically,there are two key steps in shadow removal.One This research is supported by the National Natural Science Founda-tion of China(NSFC)under grant No.61301277and No.41371431,the National Basic Research Program of China(973Program)under grant No.2012CB725303,and the Hubei Provincial Natural Science Foundation under grant No.2013CFB299.*Corresponding author:Dr.Q.Q.

Li.

(a)(b)

Fig.1.An example of pavement shadow removal with the proposed method.(a)A pavement image,(b)the result.

is shadow-region location,and the other is illuminance com-pensation.To locate the shadow regions,interactive meth-ods[11,12,13,14,15,16]and automatic methods[17,18, 19,20,21]have been studied.In[11],an image was interac-tively partitioned into the shadow region,non-shadow region and penumbra,which were fed into a Bayesian network as priori probability,to seek an optimal compensation in illumi-nance.In[12],a point was set in advance in the shadow re-gion and the whole shadow region was obtained through seed growing.In[14],a shadow stroke was developed to select the penumbra area with human interaction,similar work was found in[15].In[16],interactive graph cut was utilized to segment the shadow regions from background.While inter-active methods often fall short in ef?ciency,machine-learning based automatic methods have been exploited in[19,17].

Pavement shadows are dif?cult to handle owing to the ex-istence of large penumbra areas,as shown in Fig.1(a).A number of methods have been studied to handle the penum-bra.In[22],the gradient of the shadow boundary was set to zero,interactively.After shadow removal,a2D Poisson algorithm[23]was applied to recover the shadow boundary. This method assumes a recognizable shadow boundary,which would lose texture information of the penumbra.To allevi-ate this problem,image inpainting techniques were employed in[12]to restore the texture in penumbra.However,the re-sulting textures cannot avoid to bias from the original.In[24], the intensity distribution of the shadow was supposed to be a

curved surface,and the texture in the penumbra area was re-stored by illuminance compensation which?attens the curved surface.Similar method was presented in[14].However, these two methods require interactions to locate the penum-bra area,which heavily limits their applications in practice.

In this study,we partition the large penumbra of pave-ment shadow with a geodesic shadow model.Then,we se-lect an optimal reference areas for illuminance compensation by analyzing statistic property of the geodesic regions.In illuminance compensation,textures and intensities are both balanced.Moreover,the proposed method processes every geodesic region,which guarantees an even illuminance over the whole image.

In the rest of this paper,Section2introduces our prior work on geodesic shadow removal.Section3describes the proposed shadow removal method.Section4reports experi-mental results and Section5concludes the paper.

2.GEODESIC SHADOW REMOV AL

The geodesic shadow removal(GSR)algorithm was proposed in our prior work[25].In broad engineering practices,GSR has mostly expressed a great power in handling pavement shadows.However,it also shows weaknesses in some cases. In this section,we?rst brie?y overview GSR,and then exam-ine its weaknesses.

2.1.Geodesic Shadow Removal

Algorithm

(a)original

image(b)morphology

close(c)Gaussian

smooth

(d)geodesic

leveling(e)shadow

regions(f)https://www.wendangku.net/doc/447306636.html,pensate.

Fig.2.An illustration to GSR.

GSR consists of four main steps,as illustrated in Fig.2.

In the following,we introduce each step of them.

i)Morphology close.Note that,cracks often have similar

intensities with shadows,and a morphological close re-

moves the cracks.As a result,in shadow-area estimation,

the cracks will not be counted into the shadows.In Fig.2,

(a)is a pavement image,and(b)shows the result.

ii)Gaussian smooth.As pavement is featured with strong

grain-like textures,a Gaussian smooth operation

will

(a)

(b)(c)

Fig.3.An example to illustrate the weakness of GSR.(a)An

original pavement image.(b)The reference area(in black)

extracted by GSR.(c)Shadow-removal result by GSR.

make shadow-area estimation not be in?uenced by the

textures.Figure2(c)shows the smoothed result.

iii)Geodesic leveling.By using a watersheding strat-

egy,the whole smoothed image is partitioned into a

series of geodesic regions in ascending average il-

luminance,i.e.,{G i|i=1,...,L,...,N}.Suppose

S and B be the shadow area and background(non-

shadow area),then S={G i|i=1,2,...,L},and

B={G i|i=L+1,L+2,...,N}.In GSR,L is em-

pirically set as L=78N.Example results are shown in

Fig.2(d)and(e).

iv)Illuminance compensation.After we get the shadow

from the geodesic levels,i.e.,S i(=G i),an illuminance

compensation operation is applied to each shadow level.

Given I i,j be the intensity of the pixel at(i,j),and I i,j

be the illuminance-compensation result,S and B be the

shadow region and the background(non-shadow region),

then each pixel(i,j)∈S is assigned with a new value

I i,j=α·I i,j+λ.Hereα=D B D

S

,andλ=?I B?α·?I S.

Note that,D S and D B are the standard deviations of

the intensity in S and B,respectively,?I S and?I B are the

average intensities.Figure2(f)shows the illuminance-

compensation result.

2.2.Weaknesses of GSR

The strong points of GSR lie in that,?rst,it locates the

shadow regions using a geodesic leveling strategy which well

handles the large penumbra area of pavement shadows.Sec-

ond,in step iv,by introducing a multiplicative factorαto

illuminance compensation,it can increase the contrast of the

shadow regions to the level of the background region,which

leads to a texture-balanced compensation result.Third,it

utilizes a morphological close step to remove cracks and keep

cracks out from geodesic leveling,which preserves cracks in

the shadow-removal results.

One weak point of GSR is found in the selection of the

shadow regions S and the background B in the step iii.To

be speci?c,?rst,it is not reliable to separate the shadow and

the non-shadow area by consistently setting L=78N.Second,

it does not always work to take the whole background area,

i.e.,the intensity levels from78N to N,as the reference area

for illuminance compensation,especially when there’s over-

illuminance in the identi?ed B.An example is shown in

(a)

(b)

(c)(d)

Fig.4.Statistic values on different geodesic levels.(a)A pavement image.(b)Average intensity values(A).(c)Standard deviation values(D).(d)Intensity span values(H).Note that,the indices of the geodesic levels are displayed on the X axis. Fig.3.It can be seen from Fig.3(b)that,the background

identi?ed by GSR has been suffered from overexposure.It

is improper to take its average intensity and the standard de-

viation as the reference for illuminance compensation.Fig-

ure3(c)show the?nal shadow removal result produced by

GSR,which is apparently visually unsatisfactory.Moreover,

the illuminance in the reference area has been left uneven.

3.IMPROVED GEODESIC SHADOW REMOV AL

We propose to improve GSR on two points.One is to select

an optimal reference area for illuminance compensation,the

other is to apply illuminance compensation to the whole im-

age.In this section,we introduce this improved GSR(IGSR).

3.1.Optimal Reference Area Seclection

Once the geoLevel operation is applied,a number of geodesic

levels can be gained.With these geodesic levels,we can get

some statistic information as follows,

A i=AVE(G i),D i=DEV(G i),H i=SPAN(G i),

where AVE()and DEV()calculate the average intensity

and the standard deviation of pixels in one geodesic level,

and SPAN()calculates the span of intensity values in one

geodesic level,i.e.,(max-min)value.An example is shown

in Fig.4.We?nd that,geodesic levels corresponding to

penumbra area generally have larger H values than non-

shadow regions.Therefore,we roughly estimate the shadow

area and the non-shadow area by applying a bi-partitioning

algorithm OTSU on the H values,

{H S,H B}=OTSU({H i|i=1,2,...,N})(1)

where H S is one class with lower H values,which corre-

sponds to the shadow area,and H B is the other class cor-

responds to the non-shadow area.It is desired to produce a

shadow-free image with higher intensity and stronger texture,

thus we select an optimal reference area using Eq.2,

G o=arg max

G i {(A i?D i)|H i∈H B}.(2)

3.2.Full-level Illuminance Compensation

With an optimal reference area,illuminance compensation

can be conducted on all geodesic levels.Speci?cally,the pro-

posed IGSR can be summarized in the following steps.

?First,a morphological close is applied to the original im-

age,to keep cracks out from shadow-area evaluation.

?Second,a low-pass Gaussian?lter with a radius of25pix-

els is used to smooth off the grain-like textures.

?Third,a geodesic leveling is conducted to partition the

smoothed image into a number of geodesic regions.This

is achieved by a watersheding strategy,where the min-

imum number of pixels in each geodesic region is n g.

?Fourth,an optimal reference area is selected from the gen-

erated geodesic regions by using Eq.2.

?Finally,a texture-balanced illuminance compensation,as

introduced in section2.1,is performed on every geodesic

region.

Note that,IGSR performs illuminance compensate not

only on shadow area but also on non-shadow area.In this way,

it can reduce the illuminance of areas suffering from overex-

posure to the level of the reference area.

4.EXPERIMENTS AND RESULTS

In this section,?rst,experiments are conducted to validate

the proposed IGSR,then the IGSR is compared with GSR,

and?nally the impact of parameter n g on IGSR is examined.

4.1.Effectiveness of IGSR

We test the performance of IGSR by a section-intensity anal-

ysis.Figure5(a)displays an original pavement image with

two section lines Sec.1and Sec.2,in which Sec.1crosses

the shadow area.Figure5(b-d)are shadow-removal results

by IGSR in three different settings.It can be seen from Fig.5

that,all the three results have even illuminance.However,

in Fig.5(f),the?uctuation of intensity in the shadow area

is smaller than that in the non-shadow area,which simply

demonstrates a lower contrast in the shadow area than in the

non-shadow area.It is because that,illuminance compensa-

tion without texture balancing cannot change the contrast in

S e c . 1S e c . 2

P

(a)Origin image (b)IGSR -texture

(c)IGSR-close

(d)full IGSR

050100150200

250

(e)

050100150200

250

(f)

050100150200

250

(g)

050

100

150

200

250

(h)

050100150200

250

(i)050100150200

250

(j)0

050100150200

250(k)0

050

100

150

200

250(l)

https://www.wendangku.net/doc/447306636.html,parison of IGSR results under three different settings.(a)A pavement image with two section lines,i.e.,Sec .1and Sec .2.(b)IGSR-texture:IGSR without texture balancing.(c)IGSR-close:IGSR without morphology close to preserve the cracks.(d)full IGSR.(e)-(h)show the in-tensity on Sec .1in (a)-(d),respectively.(i)-(l)show the intensity on Sec .2in (a)-(d),respectively.

shadow areas.While in Fig.5(g)and (h),similar severity of ?uctuation can be found over the whole section,which sim-ply indicates that,the texture-balance illuminance compensa-tion has enhanced the texture of the shadow area.Note that,Sec .2locates in the non-shadow area,crossing a crack point P .It can be seen from Fig.5(i-l)that,the intensity val-ues of P in (i),(j)and (l)stand nearly the same,exactly 23,25,and 22,which are much lower than that in (k)-a value of 52.It indicates that,

without

a

morphological close,

the

cracks will be counted into the shadow area,who will then be added with a high intensity value in illuminance compen-sation.Consequently,

the contrast of cracks will be

reduced in shadow removal.In

summary,the proposed IGSR,

i.e.,full IGSR,can gain a

balanced texture while removing

the pavement shadows,and the manipulative

factor αis useful

to achieve a texture-balanced illuminance compensation,and the morphology close is a necessary step to preserve the cracks.4.2.IGSR v.s.GSR

Comparisons between IGSR and GSR have been made on a set of pavement images.Two sample images are shown in the left column of Fig.6,which are suffered from overexpo-sure.It can be seen from Fig.6that,shadow-removal results from GSR show an over-bright illuminance,where the illumi-nance is uneven.However,the results from IGSR hold even-illuminance over the whole image.It is because that,an opti-mal reference area has been selected by IGSR for illuminance compensation,and the compensation has been performed on all the geodesic levels in the image.

https://www.wendangku.net/doc/447306636.html,parison of GSR and IGSR.Column 1:two origi-nal images.Column 2:GSR results.Column 3:IGSR results.

(a)n g =2000(b)n g =5000(c)n g =8000(d)n g =11000

(e)n g =2000(f)n g =5000(g)n g =8000(h)n g =11000

Fig.7.The impact of n g on the geodesic leveling and shadow removal.Row 1:geodesic leveling results.Row 2:the corre-sponding shadow-removal results.

4.3.Impact of parameter n g

Experiments are also conducted to analyze the impact of n g on the IGSR result.An illustrative example is shown in Fig.7,where n g is tuned from 2000to 11000,at an interval of 3000.It can be seen from Fig.7that,when n g is tuned from 5000,8000to 11000,the intensity jumps between neighboring geodesic regions are increasingly obvious.This is because the geodesic leveling with a larger n g will partition a wider penumbra annulus into one geodesic region.As the intensity in the penumbra often changes quickly from the shadow cen-ter to the background,the average intensity of one geodesic region will signi?cantly bias from that of its neighboring re-gions.As a result,the illuminance compensation will bring a intensity jump between two neighboring geodesic regions.

5.CONCLUSION

In this work,an improved geodesic-based shadow removal method (IGSR)was proposed to remove pavement shadows.IGRS selected an optimal reference area by analyzing the statistic property of geodesic levels and performed illumi-nance compensation over all geodesic levels.Experiments demonstrated the superior of IGSR over GSR in handling pavement images suffering from overexposure.

6.REFERENCES

[1]L.Ying and E.Salari,“Beamlet transform-based tech-

nique for pavement crack detection and classi?cation,”

Computer-Aided Civil and Infrastructure Engineering, vol.25,pp.572–580,2010.

[2]Q.Li,Q.Zou,D.Zhang,and Q.Mao,“FoSA:F*seed-

growing approach for crack-line detection from pave-ment images,”Image and Vision Computing,vol.29, no.12,pp.861–872,2011.

[3]H.Oliveira and P.L.Correia,“CrackIT-an image pro-

cessing toolbox for crack detection and characteriza-tion,”in IEEE ICIP,2014,pp.798–802.

[4]I.Sato,Y.Sato,and K.Ikeuchi,“Illumination distri-

bution from brightness in shadows:adaptive estimation of illumination distribution with unknown re?ectance properties in shadow regions,”in ICCV,1999.

[5]Y.Weiss,“Deriving intrinsic images from image se-

quences,”in Proc.of the International Conference on Computer Vision(ICCV’01),2001,pp.1–8.

[6]J.J.Yoon,C.Koch,and T.J.Ellis,“Shadow?ash:an

approach for shadow removal in an active illumination environment,”in Proc.of the British Machine Vision Conference(BMVC’02),2002,pp.1–10.

[7]M.F.Tappen,W.T.Freeman,and E.H.Adelson,“Recov-

ering intrinsic images from a single image,”in Proc.of the Neural Information Processing Systems Conference (NIPS’02),2002,pp.1–8.

[8]N.Almoussa,“Variational retinex and shadow re-

moval,”in UCLA Technical Report,2006.

[9]L.Xu,F.H.Qi,R.J.Jiang,Y.F.Hao,and G.R.Wu,

“Shadow detection and removal in real images:A sur-vey,”in SJTU-CVLAB Technical Report,2006. [10]G.D.Finlayson,M.Drew,and C.Lu,“Entropy mini-

mization for shadow removal,”International Journal of Computer Vision,vol.85,pp.35–57,2009.

[11]T.P.Wu and C.K.Tang,“A bayesian approach for

shadow extraction from a single image,”in Proc.

of the International Conference on Computer Vision (ICCV’05),2005,pp.480–487.

[12]Y.Shor and D.Lischinski,“The shadow meets the mask:

pyramid-based shadow removal,”Computer Graphics Forum,vol.27,no.2,pp.577–586,2008.

[13]A.Mohan,J.Tumblin,and P.Choudhury,“Editing

soft shadows in a digital photograph,”IEEE Computer Graphics and Applications,vol.27,no.2,pp.23–31, 2007.[14]F.Liu and M.Gleicher,“Texture-consistent shadow re-

moval,”in Proc.of the10th European Conference on Computer Vision(ECCV’08),2008,pp.437–450. [15]Y.F.Su and H.H.Chen,“A three-stage approach to

shadow?eld estimation from partial boundary informa-tion,”IEEE Transactions on Image Processing,vol.19, no.10,pp.2749–2760,2010.

[16]D.Miyazaki,Y.Matsushita,and K.Ikeuchi,“Interactive

shadow removal from a single image using hierarchical graph cut,”in Proc.of the Asian Conference on Com-puter Vision(ACCV’09),2009,pp.234–245.

[17]J.J Zhu,K.G.G.Samuel,S.Z.Masood,and M.F.Tap-

pen,“Learning to recognize shadows in monochromatic natural images,”in Proc.of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR’10), 2010,pp.223–230.

[18]E.Salvador,A.Cavallaro,and T.Ebrahimi,“Cast

shadow segmentation using invariant color features,”

Computer Vision and Image Understanding,vol.95,no.

2,pp.238–259,2004.

[19]X.Gu,D.Yu,and L.Zhang,“Image shadow removal

using pulse coupled neural network,”IEEE Transac-tions on Neural Networks,vol.16,no.3,pp.692–698, 2005.

[20]J.Yao and Z.Zhang,“Hierarchical shadow detection

for color aerial images,”Computer Vision and Image Understanding,vol.102,pp.60–69,2006.

[21]S.Audet and J.R.Cooperstock,“Shadow removal in

front projection environments using object tracking,”in Proc.of the IEEE CVPR Workshop,2007,pp.1–8. [22]G.D.Finlayson,S.D.Hordley,and M.S.Drew,“Remov-

ing shadows from images,”in Proc.of the7th European Conference on Computer Vision(ECCV’02),2002,pp.

823–836.

[23]P.P′e rez,M.Gangnet,and A.Blake,“Poisson image

editing,”ACM Transactions on Graphics,vol.22,no.3, pp.313–318,2003.

[24]E.Arbel and H.Hel-Or,“Texture-preserving shadow

removal in color images containing curved surfaces,”in Proc.of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR’07),2007,pp.1–8. [25]Q.Zou,Y.Cao,Q.Li,Q.Mao,and S.Wang,“Crack-

Tree:Automatic crack detection from pavement im-ages,”Pattern Recognition Letters,vol.33,no.3,pp.

227–238,2012.