A different way to look at the intercomparison of datasets - illustrated with SCIAMACHY v5.02 versus lidar ozone profiles

Abstract : In traditional validations of atmospheric profiles, the intercomparison of two datasets is usually carried out in predefined groups of observational characteristic like longitude, stellar magnitude or solar zenith angle. Here we present an alternative method in which we trained a self organizing map (SOM) with a full time series of relative difference profiles of SCIAMACHY limb v5.02 and ozone profiles from seven NDACC lidar. For each individual observation, a set of observations characteristics from the SCIAMACHY and lidar data was mapped to the trained SOM. These maps were studied to see if the variation for a given characteristic corresponds to the variation seen in the SOM map. For the studied datasets, altitude-dependent relations for the global dataset were found between the difference profiles and studied variables. From the lowest altitude studied (18 km) ascending, the most influencing factors were found to be longitude, followed by solar zenith angle and latitude, sensor age and again solar zenith angle together with the day of the year at the highest altitudes studied here (up to 45 km). Clustering into three classes showed that there are also some local dependencies, with for instance one cluster having a much stronger correlation with the sensor age (days since launch) between 36 and 42 km. The validation approach based on using SOM proved to be a powerful tool for the exploration of differences between datasets without being limited to a-priori defined data subsets.