Infrared exploration of the architectural heritage : from passive infrared thermography to hybrid infrared thermography ( HIRT ) approach

Up to now, infrared thermographic approaches have been considered either passive or active. In the latter case, the heat flux is historically attributed to a non-natural heat source. The use of the sun has recently been incorporated into the active approach thanks to multi-temporal inspections. In this paper, an innovative hybrid thermographic (HIRT) approach is illustrated. It combines both the time component and the solar source to obtain quantitative information such as the defect depth. Thermograms were obtained by inspecting the facade of the Santa Maria Collemaggio church (L’Aquila, Italy), whereas quantitative results related to the sub-superficial discontinuities were obtained thanks to the use of advanced techniques. Experimental results linked to passive approach (i.e., the mosaicking procedure of the thermograms) performed by selecting a set of historic churches are also included in order to explain, when and where, the hybrid procedure should be used.


INTRODUCTION
Recently, the use of thermographic quantitative procedures for the inspection of buildings has been generally applied to study thermal properties, such as the transmittance parameter (1)(2)(3)(4)(5).
However, in the materials field, important efforts have been made for the retrieval of quantitative information, such as the depth and the size of the defects (6)(7)(8)(9).
Up to now, the interest to transmit the knowledge obtained from the scientific research of materials to the scientific research of the buildings was focused on a semi-quantitative level (10,11).
Important efforts using the solar radiation as an active approach were made by Lerma et al. (12).The latter work focused on mapping the moisture distribution on building facades at a qualitatively level by using a rapid and simple approach.At the same time, it leaves possibilities for future research focusing on the detection of other kinds of alterations.
The present work tends to shift the focus of attention from a qualitative to a quantitative pointof-view in order to estimate the depth of subsuperficial defects such as splitting, thanks to the use of solar radiation.Indeed, this kind of quantitative information is of utmost importance for the restorer involved in the restoration phase.
The problem to be solved arises when, for example, structures like facades of churches require an homogeneous heating up phase for a long period of time in order to obtain an optimal phase shift of the thermal wave inside the masonry.One of the cases in which obtaining numerical values from a thermographic survey is of great interest, could be the inspection of buildings which were seriously damaged in the past by earthquakes (13)(14)(15).In addition, focusing attention on the civil engineering field, also the facades that are affected by splitting, leaks, buried structures and injections of stabilizing material, are of interest and should be probed by the proposed methodology.
The thermographic inspection of buildings is carried out with a sound knowledge of the analyzed materials (16)(17)(18), as well as on what happened in the past to the facades, structures or frescoes (19)(20)(21).
In this work, the authors propose, for the first time to the best of their knowledge, a combination of the pulsed phase thermography (PPT), the thermographic signal reconstruction (TSR) and the principal component thermography (PCT) techniques (22,23), applied by using solar radiation as thermal stimuli.The combination of these advanced algorithms under the latter condition is called the hybrid approach.Sub-section 3.2.1 provides a brief explanation of the mathematical concepts which these techniques are based on.
The validity was verified working with a fascinating case study.In particular, the study dealt with herein talks about the Santa Maria di Collemaggio church (L'Aquila, Italy), which was built in 1288, and has experienced three catastrophic earthquakesamong many others -dated 1703, 1915 and 2009 (24).The church, partially collapsed in two different points, i.e., the facade and the apse.Instead, the lateral walls gravely suffered the seismic events, therefore, they were consolidated and readapted.The infrared analysis was centered on the facade as the representative zone to be discussed in depth.The pros and cons about the transition from the passive to the active approach, up to the hybrid approach have also been explained, by referring to similar cases of the architectural heritage of the Abruzzo region (Italy).

MATERIALS
Masonry load bearing elements of heavy historic structures frequently exhibit very typical mechanical deterioration phenomena like: a) formation of vertical or sub-vertical, thin but very diffused cracks, and b) local detachment of the outer leaf in multiple leaf walls.Such peculiar crack patterns are often attributable to the prevalent effect of the dead load and to the connected time dependent phenomena.Also wind, temperature variation and even earthquakes can contribute, as well as compressive stresses which develop during the event.
Due to the viscous behavior of the masonry, constant high compression states of stress can cause time dependent phenomena (creep) and consequent gradual degradation of the structure due to damage accumulation.An excessive state of deformation can develop, and an unexpected collapse can eventually occur.Safeguarding the incipient defects such as those treated in the Santa Maria di Collemaggio case, can prevent serious damage like the damage dealt with in the cases of San Pietro a Coppito, San Marciano, Santa Maria Paganica and San Domenico.The latter cases are linked to the dead load effect due to the 2009 earthquake.Indeed, among the most important elements of vulnerability of the historical buildings, the quality of the masonry construction plays an important role (25)(26)(27).
The materials used in the town of L'Aquila are mainly calcareous, usually cavernous and vacuolar, and sometimes compact.In particular, two different typologies are present in the surrounding area (28): the Poggio Picenze limestone, and the improperly named, Travertine.
The first one is soft, whitish and rich in fossils.It is easily workable, although it resists both on the corners under a mechanical load, and in bad weather conditions.Instead, the Travertine of L'Aquila includes limestone which has square edges.In practice, the boulders were taken from the topsoil.
In this town, the use of small pieces of roughedhewn stones to build masonry was widespread.The average thickness of each stone is usually 10-12 cm and they are arranged in horizontal and parallel lines.In this disposition, the masonry is named apparecchio aquilano.Its shape is very similar to the opus reticulatum and to the opus vittatum dating back to Roman times, both for the dimension of the elements and for their arrangement.The length of the pieces is between 10 and 25 cm.The internal nucleus of this type of masonry, whose thickness (>80 cm) is usually constituted by stone flakes mixed with overhead mortar.
In the case of Collemaggio, Travertine was used in order to build a part of the bearing wall, while the white and pink limestone was used for the exterior facade worked in the form of tiles.Therefore, the Travertine fulfills a mechanical function, while the limestone carries out the aesthetic one.Indeed, the size of the tiles creates the effect of a two-toned geometric pattern.
However, the core of the wall is the apparecchio aquilano masonry, that remains visible to the naked eye on the lateral parts of the church.The knowledge of the structure of the wall is important since each material propagates the thermal waves deeper and deeper, in relation to their thermal properties and thicknesses.It is presumable that materials having peculiar thermal diffusivity values like new energy saving materials, would drastically reduce the validity of the method since the penetration of the thermal waves would extensively be weakened.The phase shift and the attenuation of the characteristic wave of the heat flow for transmission modality through an opaque wall are generally interrelated, representing the incrementing functions of the capacity of the heat storage of the wall itself.The higher the value of thermal diffusivity, the shorter the time in which the effect of propagation penetrates deeper into the inspected material.This value is, therefore, very important because it does not only take into account the insulating properties, but also the properties related to the accumulation of heat in an established volume of material.
In addition, multiple leaf walls are directly linked to the defoliation mechanism that is subsequently described.This type of damage could begin inside each leaf composing the wall, afflicting the loss of adherence of the upper layers very quickly.In the cases treated herein, by using the passive approach, two churches lack the external layer in limestone and the internal layer of Travertine, as well as the mortar layer between them, while in the other two, these layers are present.The cross-section is obviously the same in term of layers, unless three layers starting from the outside are added or not.However, the defoliation mechanism could act in each case due to the viscous behaviour previously mentioned (29,30).
The main information related to materials is reported above, while a total cross-section of the Collemaggio facade, which also sums up the San Pietro a Coppito and San Marciano cases, is shown in section 4.However, since the texture (but not the material) of the apparecchio aquilano masonry could be slightly different, it is important to briefly describe all the different types.
In particular, four types of sub-categories of the apparecchio aquilano masonry (A, B, C, D) are typical of the town of L'Aquila and its surroundings (Fig. 1).All these types have been studied by using the infrared thermography technique in the present research work.In particular: the A type is a masonry with regular pieces of sub-parallelepiped-shaped stones having a horizontal sub-parallel recursive structure (Fig. 1a).
In addition, the masonry with small pieces of stones (thickness <20 cm) having a rough-hewn surface and arranged in a horizontal/parallel recursive structure, can be divided into three sub-groups: -1 st , B type: pieces with a rectangular shape in calcareous stone (Fig. 1b); -2 nd , C type: very similar pieces with a square shape in calcareous stone (Fig. 1c); -3 rd , D type: very dissimilar pieces with a square shape in marmoreal stone (Fig. 1d).
Unfortunately, after the 1703 earthquake, the greatest part of the masonry in L'Aquila has been readapted.Indeed, in the current form, they are made by irregular stones that are modeled in irregular lines, without any transversal element of junction and with a great amount of mortar which has poor mechanical properties.In many cases, a brick faced wall is present; it was filled in with stone material of small sizes mixed with sand.In addition, the capstones were not added, therefore, the masonry is vulnerable to the mechanisms of overturning.Most of the junctions between orthogonal masonry are not effective due to the lack of meshing on the planar surface.
Finally, another point of vulnerability is linked to the external layer of the apparecchio aquilano masonry, that is sometimes unconfined (this is not the case of the San Pietro a Coppito, San Marciano and Collemaggio churches) and, usually, more degraded if compared to the remaining internal structure.In general, it easily falls out of the plane of the wall during a quake (Fig. 2).
The latter assumption is confirmed in section 4 through the infrared thermography (IRT) technique.In particular, the case of the church of San Pietro a Coppito corresponds to type A, San Marciano's case is linked to type B, while Santa Maria Paganica and San Domenico churches are linked to type C.
Finally, the rebuilding of the damaged facade of the Collemaggio church in the wake of the 1915 earthquake, followed the approved design of 1919 (31).The external stones were numbered and positioned in the same place again, following one condition: the solid bricks were placed up to +17 m, while the hollow bricks were placed from +17 m to the top of the roof; this choice was taken in order to lighten the upper part, which is more vulnerable to vibrations.The connection to the wall was carried out by using horizontal and vertical anchor bolts applied to the internal structure, which was partially realized in reinforced concrete.The rebuilding of the dichromate stone cladding was performed simultaneously with the wall.The ashlars of the cladding were blocked with iron clamps placed inside the wall up to the Travertine layer.

METHODS
In this section, the paper is divided into two subsections.The first provides the readers with some basic knowledge about the passive IRT approach and how the measurements were carried out, while the second one summarizes the main concepts of the most recent hybrid IRT approach, i.e., the HIRT.
The data analysis procedure, with particular reference to the different result typologies, is described in section 4.After considering the results, the readers can understand how the application of the HIRT technique is essential in detecting the incipient phase of the damage, like in the case of Collemaggio.Instead, for the detection of the weak parts of the structures, the passive approach is the best, since, in such cases, it is important to point out the surface to be consolidated on the x-y plan, while the depth along the z axis is more suitable, also for computational purposes, to be retrieved from other NDT methods.In particular, the case study of the church of Santa Maria di Collemaggio explains the importance of: a) a thermographic diagnostics plan over the years by using the passive approach as a preliminary attempt, b) the active approach in order to obtain the size of a sub-superficial buried structure but, at the same time, the limits of the hybrid approach in order to retrieve the depth of deep features in multi-layered materials, c) the hybrid approach in order to provide the depth of shallow and circumscribed defects, linked to a unique A type B type thermal diffusivity value, which could be quickly repaired by injecting a stabilizer.Thus, in order to safeguard the structure against the propagation of the sub-superficial damage caused by (especially because of the harsh climate in L'Aquila) the cycle of freezing and thawing during the winter season.
In particular, IRT is a process in which an infrared imaging system (an infrared camera) converts the spatial variations in infrared radiance from a surface into a two-dimensional image; in the latter, the variations in radiance are displayed in a range of colours or tones.As a general rule, objects in the image that are lighter in colour are warmer.Whereas darker objects are cooler.

Passive IRT approach
In the passive thermography, abnormal temperature profiles indicate a potential problem, and a key issue is the temperature difference with respect to a reference point, often mentioned as the delta-T (∆T) value or the hot spot.A ∆T value of a few degrees (>5 °C) is generally suspicious, while greater values indicate important evidence of abnormal behavior (9).The infrared camera usually used during a thermographic campaign for building inspections works in the long-wave infrared (LWIR) spectrum, because: a) in this range, the atmosphere, in most cases, is considered transparent since the rate of absorption is limited, b) it is more sensitive to ambient temperatures, c) it is able to detect small temperature differences and, finally, d) it is less affected by the reflection caused by solar rays (32).
In this work, the infrared inspection in a passive modality was carried out in accordance with the 2012 update of the International Standard ISO 6781-1983 that outlines the general procedure for the interpretation of the infrared thermography method (Thermal insulation, Qualitative detection of thermal irregularities in building envelopes -Infrared method).A FLIR S65 HS thermal camera (detector: 320×240 pixels, spectral range: 7.5-13 µm, instant field of view: 1.1 mrad) was used during the thermographic survey.To minimize the effects of solar radiation, the passive approach was used on the building envelope 2 to 3 hours after sundown.In this approach, the test materials are naturally at a different temperature than the surrounding materials; in addition, it is rather qualitative since the goal is simply to locate thermal anomalies.Prior to the inspection of the structures, no precipitation occurred for at least 24 hours (33).

Hybrid IRT approach
During the exporting of the quantitative procedure which combines the use of the PPT and the TSR techniques by the aerospace engineering field (34) and the civil engineering ones (35), some important considerations should be kept in mind.In particular: • Thermal conductivities are generally low for the materials analyzed; • The historic facades or the lateral walls usually have greater dimensions.
These drawbacks could be minimized thanks to the use of telephoto lenses or wide-angle lenses, as well as recording both the heating up and cooling down phases for a long period, i.e., by exploring at least 8 continuous hours into a selected area, where the temperature range is generally high during the summer season.In the present case, the results improved by enhancing the signal-to-noise ratio, and by using the phase or the amplitude for quantitative solutions.Other technical expedients, described in the following, were specifically adapted to reach the best image visualization.
The imaging technique named contrast stretching was used both to expand the dynamic range of an image and to enhance the visibility of its characteristics.It was applied in the present work during the post-processing step, both for the passive and for the hybrid results.The simplest form of contrast stretching is the image normalization implemented in the Matlab ® environment through the imadjust function.
The histogram of a digital image is a discrete function which counts the number of pixels within an assigned level of intensity.In addition, the histogram is a discrete estimation of the probability density function (PDF) of the image being analyzed.The purpose is to enhance the quality of the image having a weak contrast as an initial output.The histeq function available in the Matlab ® environment improves the distribution of the grey tones, in a way that the first histogram becomes similar to an assigned histogram.The equalization process of the final histogram is used to post-process the thermograms.
Two different filters could be applied in order to reduce the influence of noise: the median filter and the Gaussian low pass filter.The first one is a nonlinear filter which classifies the pixels in an area, by replacing the central pixel with the median of the intensity values.For this reason, it is applied in order to remove the unwanted pixels, and it is implemented in the Matlab ® environment by using the medfilt2 function.Instead, the second one is a filter working in the frequencies domain which assumes that the image is limited within a band, and the spatial frequencies above this band are the results of the noise.Because the noise in an image tends to be characterized by the high spatial frequencies, a Gaussian low pass filter can be used in order to reduce this undesirable content.In the Matlab ® environment, it is implemented by using the fspecial and the imfilter functions.
The thermographic signal reconstruction (TSR) technique, that is the core of the HIRT approach, processes each pixel in a temporal sequence, instead of each whole thermogram.Then, the temporal data can be linearized by reaching the logarithmic domain.
Via a natural logarithm (ln), it is possible to obtain a first offset term related to the absorption of energy and to the intrinsic properties of the material, and a straight line having a slope of 1 2 − , as sec- ond term of the Eq.[1].
in which, Q is the input energy, T is the temperature, t is the time, and e is the thermal effusivity of the material.
Eq. [1] implies that, apart from the thermal properties of the material being inspected, the logarithmic response of the decay will be a straight line having a slope of 1 2 − for an ideal material, and above a defect-free (or sound) area.In real conditions, the exact behavior of the ln[T(t)](ln(t)) temperature evolution, will be different due to variable parameters.However, the simple form of the logarithmic evolution of the temperature (T), i.e., ln[T(t)], is useful to be fitted through the least squares method for a n order polynomial, for each (i, j) pixel within the interested field of view (36) [2]: In practice, it is clear that the entire time series at a spatial location can be represented using the coefficients a 1 , a 2 … a n .
It was verified that, the fifth and/or the sixth polynomial order act as a low-pass filter for a smoothing of the data without reconstructing the noise; indeed, greater values of n provide oscillating values, while lower values of n prevent a sufficient smoothing of the function.By this technique, an interesting advantage can be obtained, such as an important reduction of the noise thanks to the combination of the data; in addition, the next image processing (i.e., through the PPT technique) can be analytically and quickly implemented because only the polynomial coefficients must be inserted into the pre-determined analytical solution.
Eq. ( 2) allows the use of the derivatives to enhance the contrast of the defects.In addition, only the polynomial coefficients are involved in the data processing.The raw thermogram corresponds to the superficial temperature of the object to be measured, while the first derivative provides information about the speed of the cooling down phase of the superficial temperature, and the second derivative is inherent to the rate of changing of the cooling velocity (t = instant time).
Both the concept of the square pulsed thermography (SPT) and the HIRT concept in the frequency domain come from the PPT concept.The HIRT concept is an adaptation of the SPT technique applied in civil engineering.Unfortunately, the quantitative approaches through the PPT technique in order to estimate the depth of the defects for the analysis of thin structures, cannot be implemented for the data recorded during a very long heating up time, and for the observation of the subsequent cooling down phase.Indeed, the previous explanation is linked to a Dirac pulse; therefore, it cannot be applied for large and thick structures (37).The quantitative approach in the frequency domain used after a very long time of heating is based on the fact that a long signal, in the time domain, has a clearly defined peak in the frequencies spectrum of the same signal.The long square pulse in the time domain, as well as the hybrid approach, lead to very well defined amplitude data in the frequency domain.According to the authors, the latter strategy can be used effectively to make an estimate of the depth of the defects.
Therefore, it is basically a quantitative approach in the frequency domain, not only based on the phase, but also on the amplitude data.Indeed, a characteristic frequency named f ch [Hz] corresponds to the frequency of the first extreme (maximum or minimum) of the phase-or the amplitude-contrast, and not to the blind frequency of the phase (f b ), as in the usual PPT technique that routinely works after a Dirac pulse.By considering a corrective factor k c , the z depth of the defect [m] can be retrieved through Eq. [3]: in which, α is the thermal diffusivity parameter [ ] m s 2 .The results are independent from the heating time, above all the frequency resolution; hence, the k c value can be approximately set at 1, although it might seem that for the amplitude data, the percentage in the miscalculation of the estimation of the depth should be inversely proportional to the time of heating (35).Eq. ( 3) fits very well with the hybrid approach.Although solar heating is not a square pulse, it can be approximated to an oscillating function.
For the sake of clarity, in the next subsection, the main concepts both of the PPT and the PCT techniques are illustrated since they will be used in section 4.

I. Pulsed phase thermography (PPT)
The technical background of the PPT technique is that the heating delivered to the inspected object may be regarded as a superimposition of a number of harmonic signals of different frequencies.The frequency components of this stimulation signal can be processed independently through Fourier Transform (FT).Ideally, the delta-function-shaped pulse contains an infinite number of frequencies with equal amplitude.The technique is based on extracting the phase information for each wave.A 3D array of data collected is processed point by point, and the phase image is then constructed (38).

II. Principal component thermography (PCT)
PCT uses singular value decomposition (SVD) to extract the spatial and temporal information from a thermogram matrix.Unlike FT, which relies on a sinusoidal basis function to decompose signals in the temperature-time space to a phase-frequency space, SVD is an eigenvectorbased transform that forms an orthonormal space.Assuming data is represented as an M×N matrix A (M>N), then the SVD can be applied in the following way [4]: with, R being a diagonal N×N matrix (with singular values of A present in the diagonal), U is a M×N matrix, V T is the transposal of principal components (PC s ) that represent time variations (39).

RESULTS AND DISCUSSION
The purpose of this section is to correlate the thermal discontinuities, i.e., the critical areas emerged after the thermographic inspection conducted by using the passive approach, with the damages due to the 2009 quake, at a qualitative or quantitative level, by following the guidelines described in the previous sections.As explained in section 2, the reported cases are obviously linked to the apparecchio aquilano masonry, while the thermographic surveys has been conducted before the quake.At that time, the authors had not considered the hybrid method yet.In case the method was already on the authors' minds, it is possible to understand by observing Fig. 3 how the selection of the defect-free area was a very difficult request to be fulfilled due to the large sub-superficial damage detected in the long-wave spectrum; in addition, in the churches of Santa Maria Paganica and San Domenico, the method was not applicable since the respective facades have many stones located outside the main flat surface, and this point invalidates the inverse retrieval of the defect depth.
In the San Pietro a Coppito church (built in the second half of the sixteenth century), the most important collapse mechanism is inherent to the facade's overturning which caused the materials to fall down from the top-right (Fig. 3a).In particular, the external tiles, the Travertine layer and the apparecchio aquilano masonry were damaged.
In this figure, an image fusion between the thermograms captured before the 2009 earthquake (precisely in 2006) and a photograph recorded after the quake, are shown.The mechanical load which happened in 2009 highlights the weak area of the facade.As explained in section 3, it is much more evident thanks to a re-adaptation, made through the Matlab ® computer program, performed on the color tones of the affected part.Readers should refer to the irregular dotted line in order to understand the connection between the defoliation damage and the thermal imprint.
In the church of Santa Maria Paganica, seriously damaged and then re-built after the 1703 quake, it is possible to notice, without any image processing, the sound correspondence between the thermal anomalies below the truss and what happened just after the quake.In this case, an image fusion was not carried out, since the photograph was not captured from the same position in which the thermograms were recorded, due to the fact that the building located in front of the church was classified as unfit for use.The collapsed part that has a suitable connection with the high temperature mapped just below the ceiling is surrounded by means of a red dotted oval.In this case, it was interesting to consider the influence of the shadow cast (11) on the facade in order to verify the correlation between the higher temperature revealed and the damage which arose after the quake (Fig. 4).
It is possible to notice how the part of the masonry just below the ceiling is always covered with a shadow cast effect.In a normal case, a lower temperature with respect to the surrounding area should be expected.Therefore, it is possible to assume that the collapsed part was jeopardized by another type of defect.Conversely, if the hybrid approach had been applied, it would have failed since the ∆T between the heating up phase and the cooling down phase could not have been enough to propagate the thermal waves by conduction up to the defected area positioned inside the structure.Readers should keep this point in mind, when the case study of Collemaggio church will be explained.Instead, the shaded areas that appeared on the right (10:15 a.m.) and on the left (03:30 p.m. and 04:45 p.m.) parts of the facade (the border line is highlighted with a yellow dotted line) were due to the buildings surrounding the church.Supposing there is a sub-superficial defect positioned on the right side, then applying the hybrid method could be useful thanks to the fact that the shadow cast, that is linked to the cooling down phase, disappears slowly after 10:15 a.m., while the heating up phase becomes predominant.
In terms of damage, a similar case happened in the San Domenico church (Fig. 3c) where the triangular zone just below the ceiling suffered a   Finally, the San Marciano church is very interesting for two reasons.The first one is linked to the combination between the damage which arose after the 2009 quake on the decorative moulding, and the thermal imprint already detected before it (a red dotted oval circumscribes the weaker zone in the upper part); the superposition of the images describes the good use of the IRT method as a preventive approach.The second one is the confirmation of the historical data that describes a restoration previously carried out on the upper facade around 1300.Indeed, this is evident by observing the thermogram shown in Fig. 3d.A horizontal red dotted line was added as a reference.Although not reported in this context, it is also very interesting to note how the first thermal imprint bisects the rose window, while the second discontinuity is just above the first one.No evidence regarding the different materials used during the restoration process can be seen by the naked eye.The new hybrid method subsequently introduced loses lots of its sense in this case, considering the large number of hot spots that appear (see the yellow color) on the thermogram; it is possible to assume that each point of interest is linked to a variation in the defect depth.In addition, the defectfree area cannot be chosen in the blue part, i.e., above the red dotted line, because the reintegration of the tiles using a different type of mortar and having a lower thermal diffusivity value with respect to the zone included between the dotted lines, should be highly probable.
At this point, in order to validate the in situ quantitative method described in sub-section 3.2, it is necessary to work with a church where the stratigraphy, as well as its historical evolution, are known.The church of Santa Maria di Collemaggio (40) has been chosen in order to fulfill this purpose.The main information related to its stratigraphy is reported in the following.
In the present work, the thermograms were gathered by using a frame rate (∆t) set at 0.017 1/s, which was useful both to avoid a data redundancy and to properly link this value with the ∆f value.Indeed, the latter value pinpoints the maximum depths to be reached.Preliminarily, the cold image was subtracted from the whole thermographic sequence in order to reduce the noise impact on the final result.In the HIRT approach, the cold image coincides with the latest image captured just before sunrise.After the implementation of the fast Fourier transform (FFT), the phases and the amplitudes were calculated by obtaining the respective images.
Subsequently, the phase contrast was set via the absolute thermal contrast technique (9), by choosing 5 points forming a defect-free area and then calculating their average value.The median filter and/ or the Gaussian filter were applied through a 3×3 mask.In particular, during the application of the latter filter, a 0.8•σ value was used.The TSR technique has been applied by using the 3 rd polynomial order.However, all images were normalized in order to enhance their contrast.
In the graphs, the color blue was attributed to map the evolution of the defects, while the color green indicates the defect-free area.The inspected area was the upper left hand corner of the facade This area was chosen in order to verify the historical data reported in (31) and previously mentioned.In addition, this part corresponds to the defoliation mechanism which occurred after the 1915 quake (40).
The thermal sequence consists of 848 thermograms.However, only the cooling down phase was analyzed (i.e., 592 thermograms).The external layer is made up of white and pink limestone.The average dimensions of the stones are 15-20 cm × 10-15 cm, while their thickness is between 17-20 cm (41,42).
Starting from the passive approach, a detached tile can be observed in Fig. 5b.No sign of deterioration appears by observing the photograph reported in Fig. 5a, captured at exactly the same moment of the thermogram.The anomalous thermal behavior of the inspected tile is better visualized in the magnification reported in the upper part of Fig. 5c.However, two years before (2010) the historical moment in which the hybrid approach was thought up by the authors (2012), the same tile had already appeared in a detached state.An arrow links the same zone gathered over the years, while a blue cross and a green one identify the defective and defect-free areas, respectively.In Tab. 1, the most important information inherent to the thermographic inspection is reported.At the initial stage of this test, the ambient temperature was 17.5 °C, the wind speed in the range of 0-1 Km/h (Beaufort's scale = 0), the relative humidity equal to 61%, the distance between the lens of the thermal camera and the center of the inspected part was set at 65 m, while a telephoto was applied, bearing in mind this distance.
Historical and technical information of the rebuilding procedure and the thermographic survey is summarized in Fig. 6.
In particular, the facade is oriented towards the North (Fig. 6a), the thermal camera's position in the garden is referred both to the facade and to the solar cycle in the sky (Fig. 6b), while the rays of the sun on the part of interest of the facade, as well as the time (2:30 p.m.) when the thermal camera started recording the thermograms -less than the cold image -is depicted in Fig. 6c.By observing the latter figure, it is possible to notice how the surrounding area (trees, buildings, and so on) has no influence in terms of shadows cast on the facade.
The open view in front of the facade helped to obtain the data also because the position where the thermal camera was installed did not have any constrictions.
In addition, no clouds were present for the whole day of recording.In the past, the authors also focused their attention on the detection of the border line of the remaining erect part concerning the collapsed upper left hand corner (Fig. 6d).The detected border, marked by an irregular dotted line (Fig. 6e), is well-connected both to the real damage (Fig. 6d) and then with the schematization shown in Fig. 6f.It is interesting to note how a higher temperature characterizes the decorative moulding along an ideal horizontal axis and, in particular, the upper left hand corner.The high temperature representing this cornice was also confirmed by the thermogram shown in Fig. 5b.Bearing in mind this result, although the cold image was recorded just before sunrise, the heating up phase started at 2:30 p.m. (Fig. 6c) in order to limit the action of the shadow cast on the facade coming from the upper decorative moulding.
In this work the authors -exploiting the solar path in the sky -focused their attention both on the detection of the sub-superficial concrete beams installed after the 1915 quake (Fig. 6f), and, as subsequently explained, on the characterization of the areas rebuilt by using different materials (31).
By observing the cross-section of the upper wall of the Collemaggio church (Fig. 6g), the thermal waves must propagate their action by conduction up to the boundary line separating the Travertine layer from the mortar layer.The Travertine layer plays an important role also from a thermophysical point Front side 592 60 2.72•10 of view, since it acts as a thermal barrier.The same thing happens on the back of the facade that, as visible in Fig. 6b and in Fig. 6g, is partially uncovered with an external layer of ceramic tiles.The back layer performs its physical action from sunrise up to around noon, in order to limit an inverse direction of the heat flux during the inspection of the front side.
The aim to retrieve the sub-superficial beams was solved by using the PCT technique (Fig. 6h) (43); surprisingly, the beams appeared by looking at the empirical orthogonal function (EOF 103 ) no. 103.In Fig. 6f, the grey area represents the defoliation damage caused by the quake, although the respective positions of the sub-superficial beams are also marked.
In the most recent publications inherent to this matter (44,45), the authors show the best visualizations before the EOF 14 .This point is also very interesting considering that EOF 1 represents the most important variability of the new orthogonal components, EOF 2 contains the second most important variability, and so on.It is also very surprising to notice the ability of this technique to retrieve, with a good contrast, only one of two sub-superficial reinforced concrete beams.This result was enhanced thanks to the use of the top-hat filter.The latter can be considered a conditional gradient that mostly highlights sharp gradients on mountain peaks.
Readers can compare Fig. 6f with Fig. 6h in order to check this result.In particular, the concrete beam that supports the decorative moulding, is only slightly detectable.Despite the precaution previously mentioned about the accurate choice of the initial time of recording, probably the screen action caused by the protruding edge of the cornice on the upper row of tiles of the facade continues to exhibit its thermal barrier effect.
Because the position of the sub-superficial beams is different with respect to the position of the detached tile to be inspected by the hybrid method, i.e., no mutual influence regarding the respective thermal properties should be considered (46), therefore, the realization of a quantitative procedure in order to obtain the depth of the defect positioned beneath this tile is feasible.
In Fig. 7a, the curves of the temperature of the whole thermographic sequence are shown, while the TSR (3 rd order) graphical result inherent to the cooling down phase is reported in Fig. 7b.
Obviously, the defect-free area was captured on the nearest pink stone (Fig. 5c), taking into consideration the defective area (ε = 0.96).Both the phase curve and the absolute phase contrast one related to these areas, are depicted in Figs.7c,d, respectively.The definition of the absolute phase contrast is: ∆ Ø = Ø defective -Ø defective-free [5] where, φ defective is the phase for a defective area, and φ defect-free is the phase value for a defined sound area (47).It is well known that Eq. ( 5) comes from the absolute thermal contrast concept (by changing φ with T) previously mentioned, and introduced for the first time in (9).The absolute phase contrast definition ∆φ was used in combination with the f ch to solve the inverse problem deriving from the PPT technique.
Fig. 8 summarizes what happened in the case of Collemaggio.It is possible to understand how the splitting defect concerns entire thickness of the mortar layer up to the f ch value.The tile marked by an arrow in Fig. 5b, is therefore glued to its layer thanks to the mortar inserted along the z axis, i.e., between the nearby tiles.On the other hand, it appears to be detached since 2010 and probably even before; the variation of the ambient parameters, like T and the relative humidity (%), may have intensified the growth of this invisible damage to the naked eye.
Instead, in Fig. 9, phasegrams and amplitudegrams selected at different frequencies are shown.It is possible to notice a stronger contrast of the defective area in the amplitudegrams than in the phasegrams.
These results come from the same set of data used to obtain the graphs shown in Fig. 7 and, accordingly, the quantitative result described subsequently.A discontinuity, similar to a stripe, appears in the upper zone of Fig. 9.It is delimited by an irregular dotted line for comparison purposes with Fig. 5a.At the same time, it is possible to see how the detached tile is detectable in the amplitudegrams (Figs.9e,f) without using any Matlab ® functions (see the sub-section 3.2) already applied in Fig. 5c in order to enhance its contrast.
At this point, the historical data related to the different types of materials that divide what is present above and below +17 m from the ground-level of the Collemaggio church was confirmed and, at the same time, the depth of the defect, i.e., a splitting, can be estimated.Eq. ( 3) validates the assumption of the position of the defect, also explained in Fig. 8. Indeed, the splitting arose between the pink tile and the mortar which was used as an adhesive support.In this case, the calculation of the combined diffusivity (α c ) (48) is not necessary, because the only type of material above the defect is calcareous (49).The f ch value obtained by Eq. ( 3) corresponds to the minimum estimable frequency in the contrast phase curve, i.e., it is congruent with ∆f.It indicates the maximum probing depth in relation to the experimental parameters (Tab. 1) used in the thermographic survey.Finally, as can be seen in Tab. 2, the estimated depth value, i.e., z = 17.8 cm, is completely within the expected range.

CONCLUSIONS
This work clarifies the methodological approach to be reserved for the thermographic inspection of historic buildings.According to the authors, a preliminary measurement survey working with the passive approach should be the first step to take in order to verify the state of the conservation of the building's facade which was inspected.After this verification, the conservation scientist could plan a more in-depth analysis by using, e.g., the innovative hybrid approach proposed herein.This approach could be useful for the estimation of the depth of small defects which appeared inside vertical known structures.Detached tiles, bricks, stones, or buried structures having a defined size like beams could also be the focus of the nondestructive inspection (50,51).At this point, more research is needed in order to provide quantitative information along the z direction concerning the rising damp effect (52), that is a typical damage not confined in a single layer.In practice, the hybrid approach could be classified as an extension of the multi-temporal approach (12) recently introduced in the scientific panorama.Its validity is undeniable and confirmed herein thanks to the PPT and PCT processing applied to the raw thermal data.
It is important to underline that the thickness of facades of historic structures is not constant in every probed transverse section.Therefore, the possibility to test at a quantitative level large subsuperficial damage such as the initial stage of the defoliation mechanism described both in Fig. 3 and in Fig. 6e, partially loses its interest.In fact, although the thermographic technique is telemetric, the computational procedure is not fully automated since it needs a preliminary selection of defect-free point (which form an area) linked to the experience of the conservation scientist.In addition, a comparative visual inspection of the raw thermograms recorded in passive modality is required.The use of the ground penetrating radar (13) method could be more effective (although it works in contact with the surface of interest, therefore, it requires a forklift truck in the case-studies similar to those described herein) to safeguard the defoliation mechanism.The result from the passive thermography is greatly influenced by the historical moment of the day in which the thermogram is captured by the thermal camera.The recording of many thermograms during a sunny day that is preceded by sunnier ones, and at a time of the day when the sun directly irradiates the inspected surface (Fig. 6b,c) tends to minimize the operator's discretional contribution during the interpretation of the defects present in a thermogram.It also minimizes the miscalculation during a measurement focused on the defect depth retrieval.The influence of the shadow cast, e.g., was recorded by a camera set up in front of the facade of the Santa Maria Paganica church (Fig. 4).
The hybrid approach was validated on the lateral side of the same church, where the apparecchio aquilano masonry is visible to the naked eye, i.e., it is not covered by additional layers.In this case, too, the methodology followed the diagram shown in Fig. 10, summarizing the main steps in order to make the entire process self-explanatory.
The quantitative inspection was focused on the detection of sub-superficial buried structures having a square shape, located at z~12.9 cm beneath the first layer composed by stones.The estimated depth is in line with the maximum and minimum dimensions of the stones used in order to build the typical masonry discussed herein.The latter result will be shown in the future as the second part of the present research, since the miscalculation in the choice of the f b value instead of the f ch value could be emphasized.
Finally, the interesting result shown in the present work comes from an accurate inspection that, in addition to the process schematized in Fig. 10, also took into consideration: (a) the surface color conducted by a visual inspection, as it can mask defects (53), (b) the emissivity value although its contribution is minimized by working with the PPT and PCT techniques (54), and (c) the meteorological conditions such as air temperature, precipitation, wind speed and cloud cover (55).

Figure 1 .
Figure 1.Different typologies of the apparecchio aquilano masonries: a) A type, b) B type, c) C type, d) D type, and e) the typology constructed in the Collemaggio Church's case.The similarity with the D type is striking.

Figure 3 .
Figure 3. Damaged areas detected by IRT method before the 2009 earthquake, superimposed on the defoliation mechanism captured after the quake.The most interesting parts are indicated by lines and ovals and explained subsequently in the text.The Churches are oriented with respect to a N-S-E-W coordinate system, and the sun path in the sky is combined with a timetable.In particular: (a) San Pietro a Coppito church (orientation of the facade: Southwest) and the related thermogram (image fusion), (b) Santa Maria Paganica Church (orientation of the facade: West -Northwest) and the related thermogram (image fusion), (c) San Domenico church (orientation of the facade: Southwest) and the related thermogram (the inspected facade is pointed out in red), and (d) San Marciano Church (orientation of the facade: West) and the related thermogram (image fusion).Temperatures are expressed in °C.
defoliation mechanism.Indeed, by comparing the photograph taken before the earthquake of 2009 with the thermogram obtained after the quake, a strong thermal imprint can be detected.The consecutive damage already mentioned after the image processing performed via the Matlab ® computer program, shows the potential of the appropriate function described in sub-section 3.2.Readers should refer to the lines forming a triangle in Fig.3c.

Figure 4 .
Figure 4.The shadow cast effect on the facade of Santa Maria Paganica Church.It is possible to notice how the facade is completely obscured at 08:30 a.m. and completely illuminated by the sun at 12:30 p.m.

Figure 5 .Figure 6 .
Figure 5. (a) Photograph of the upper left hand corner of Collemaggio's facade (taken at 2:30 p.m.); the horizontal dotted line marks the limit between the solid bricks and the hollow ones, (b) thermogram with the cold image subtracted; it was obtained in 2012 by means of the passive approach (the arrow marks the detached wall tile), and (c) the enlarged area marked in (b) by a brown dotted square; it contains both the defect-free area and the defective one that are marked by green and blue crosses, respectively.The detached wall tile was already inspected in 2010; the sub-superficial damage is evident without any image processing (see the bottom figure).An arrow links the wall tile collected over the years via the IRT technique.

Figure 7 .
Figure 7. (a) Evolution of the temperatures (heating up and cooling down phases) for the defective and defect-free areas marked in Fig. 5c, (b) cooling down phase of the temperatures reconstructed by means of TSR (3 rd order), (c) phase curves, and (d) phase contrast curve.

Figure 8 .
Figure 8.General schematization applied for the detection of the f ch value in reference to ∆ φ .

Figure 10 .
Figure 10.Diagram showing all the main steps relative to the HIRT approach.

Table 1 .
Experimental parameters inherent to the thermographic test named front side of the Collemaggio church

Table 2 .
(35) results pertaining to the test named front side, by using the method introduced for a square pulse (SP)(35)