Re: [ferret_users] grid compressed after subtract area average

On Wed, Jan 22, 2020 at 2:15 PM Ansley C. Manke <ansley.b.manke@xxxxxxxx> wrote:

Hi,

Once the data has been averaged, applying @DIN in the same directions does not make sense, as there is nothing to integrate. I'm not sure how adj_EB_land_CNRM_AM5 is computed but if its grid in the X and Y directions shows as compressed, then there is no dependence on those directions and pyferret should refuse to even compute a @DIN transformation. To show a simple example,

yes? use coads_climatologyyes? let sstave = sst[x=@ave,y=@ave]

yes? show grid sstave    GRID GSQ1 name       axis              # pts   start                end                 subset COADSX    LONGITUDE          180mr   21E                  19E(379)            compressed COADSY    LATITUDE            90 r   89S                  89N                 compressed normal    Z TIME      TIME                12mr   16-JAN 06:00         16-DEC 01:20        full

yes? list sstave[x=@din,y=@din] **ERROR: illegal use of transformation: SSTAVE[X=20E:20E(380)@IN4,D=1] applied to point

So, I am not sure I understand what you are doing. How is adj_EB_land_CNRM_AM5 defined?

Sometimes if the order of operations that PyFerret uses is not what you need, a method can be to write intermediate results to a file and then read that back in to, say, subtract from another _expression_.

-Ansley

On 1/21/2020 12:05 PM, Xiaoyu Bai wrote:

Hi Ansley,

What I am trying to do is to subtract global average (@AVE, which is only ONE value for the whole globe) from model output (which has value at every grid). This calculation may be weird to pyferret but I have to use this method to adjust the model's energy budget.

yes? sh grid adj_EB_land_CNRM_AM5 (adjusted value), AHT_land_CNRM_AM5 (energy budget)
GRID (G033)
name axis # pts start end subset
LON LONGITUDE 256mr 0E 1.406W(358.59) compressed
LAT LATITUDE 128 i 88.928S 88.928N compressed
PLEV Z (Pa) 17 i- 1000 100000 compressed
MONTH_GREGORIAN TIME 12mi 16-JAN 12:00 15-DEC 17:49 full
GRID (G033)
name axis # pts start end subset
LON LONGITUDE 256mr 0E 1.406W(358.59) full
LAT LATITUDE 128 i 88.928S 88.928N full
PLEV Z (Pa) 17 i- 1000 100000 compressed
MONTH_GREGORIAN TIME 12mi 16-JAN 12:00 15-DEC 17:49 full

The incorrect grid affects my further calculation. I need to apply @DIN to my adj_EB_land_CNRM_AM5. Since the grid is compressed, pyferret doesn't apply cosine correction anymore. My proof for this is the value for AHT_land_CNRM_AM5 is around -100 to 50. The @AVE value is 4.021. So the adj_EB_land_CNRM_AM5 value is roughly around -104 to 46. If I apply @DIN for both latitude and longitude to AHT_land_CNRM_AM5, the outcome is 2.051E+15 W/m2 which makes sense. If I apply @DIN to adj_EB_land_CNRM_AM5, the outcome is -61.33 W/m2 which doesn't make sense.

list/L=7 AHT_land_CNRM_AM5[X=@DIN,Y=@DIN], adj_EB_land_CNRM_AM5[X=@DIN,Y=@Din]
LONGITUDE: 0.7W(-0.7) to 0.7W(359.3) (XY integ.)
LATITUDE: 90S to 90N (XY integ.)
TIME: 15-JUL 17:49
Column 1: AHT_LAND_CNRM_AM5 is SWABS_LAND_CNRM_AM5-OLR_LAND_CNRM_AM5+SHF_LAND_CNRM_AM5-STORAGE_LAND_CNRM_AM5
Column 2: ADJ_EB_LAND_CNRM_AM5 is AHT_LAND_CNRM_AM5-EB_LAND_CNRM_AM5
AHT_LAND_ ADJ_EB_LAND_CNRM_AM5
I / *: 2.051E+15 -61.33

So I tried with a single month to use AHT_land_CNRM_AM5 just minus the ave value which is 4.021.

yes? let test=AHT_land_CNRM_AM5-4.021
yes? list/L=7 AHT_land_CNRM_AM5[X=@DIN,Y=@DIN], adj_EB_land_CNRM_AM5[X=@DIN,Y=@DIN], test[X=@DIN,Y=@DIN]
LONGITUDE: 0.7W(-0.7) to 0.7W(359.3) (XY integ.)
LATITUDE: 90S to 90N (XY integ.)
TIME: 15-JUL 17:49
Column 1: AHT_LAND_CNRM_AM5 is SWABS_LAND_CNRM_AM5-OLR_LAND_CNRM_AM5+SHF_LAND_CNRM_AM5-STORAGE_LAND_CNRM_AM5
Column 2: ADJ_EB_LAND_CNRM_AM5 is AHT_LAND_CNRM_AM5-EB_LAND_CNRM_AM5
Column 3: TEST is AHT_LAND_CNRM_AM5-4.021
AHT_LAND_ ADJ_EB_ TEST
I / *: 2.051E+15 -61.33 1.507E+11

These results are indicating that pyferret was not applying cosine correction to compressed latitude and longitude (which makes sense to ferret since it's compressed). But I have 10 models with 12 months' data, it's hard for me to subtract the value one by one. I am wondering whether there are some methods that I can transfer the @AVE value into a numeric or I can regrid adj_EB_land_CNRM_AM5 to be like AHT_land_CNRM_AM5 (I already tried with let/like=AHT_land_CNRM_AM5 adj_EB_land_CNRM_AM5=AHT_land_CNRM_AM5-EB_land_CNRM_AM5(this the @AVE variable) and set variable/name=tt adj_EB_land_CNRM_AM5 \ let adj_EB_land_CNRM_AM5=tt[g=AHT_land_CNRM_AM5,gt=@ASN] but neither of them worked). I tend to use transfering @AVE to a numeric method now. Any suggestions?

Thank you very much.

Xiaoyu

On Wed, Dec 4, 2019 at 1:04 PM Ansley C. Manke <ansley.b.manke@xxxxxxxx> wrote:

Hi,

Maybe this is just some detail in the syntax. The coordinate axes for the variable "temp" are not actually compressed, it's just that SHOW GRID is incorrectly saying they are. Are you seeing an error message? If this doesn't resolve things, please send more about the commands you are using and the results.

Your original example was with levitus_climatology, so let's go back to that.

yes? use levitus_climatology
yes? let test=TEMP-TEMP[X=@AVE,Y=@AVE]

yes? list test[x=@din,y=@din]
             VARIABLE : TEMP-TEMP[X=@AVE,Y=@AVE]
             FILENAME : levitus_climatology.cdf
             FILEPATH : /home/users/tmap/ferret/linux/fer_dsets/data/
             SUBSET   : 20 points (DEPTH (m))
             LONGITUDE: 20E to 20E(380) (XY integ.)
             LATITUDE : 90S to 90N (XY integ.)
0       / 1: -538.5
10      / 2: -633.9
20      / 3: -530.8
30      / 4: -575.1
50      / 5: -335.4
75      / 6: -440.2
100     / 7: -459.0
150     / 8: -405.4
200     / 9: -287.2
300     / 10: -305.7
400     / 11: -234.4
600     / 12: -118.7
800     / 13: -113.0
1000    / 14: -73.9
1200    / 15: -52.0
1500    / 16: -35.0
2000    / 17:   -6.6
3000    / 18: -13.9
4000    / 19:    7.7
5000    / 20:    0.5

On 12/3/2019 6:02 PM, Xiaoyu Bai wrote:

I picked the wrong word. I do want to remove the mean over lat and Lon. After that I want to apply @DIN. The compressed coordinates blocked @DIN to work. I tried with TEMP-Temp[x=@ave, y=@ave] and TEMP-number. They gave me very different results.

Thank you 😊

Xiaoyu

在 2019年12月3日，下午8:13，Ansley C. Manke <ansley.b.manke@xxxxxxxx> 写道：

Hi Xiaoyu,

First, what you have done with TEMP-TEMP[X=@AVE,Y=@AVE] is to remove the mean over longitude and latitude. That will not be less noisy. If you want to work with a less-noisy version of TEMP, you might use one of the smoothing transformations such as the boxcar smoother, @SBX. Here is a list of all the transformations:

https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/XPRESSIONS#Chapter3-Transformations, and further down that page are detailed descriptions of the different transformations.

Now, it looks as if you've discovered an error in the SHOW GRID output. It should not be listing the x and y axes as "compressed". So we'll look into that.

yes? use coads_climatology yes? let sst_demean = sst - sst[x=@ave,y=@ave] yes? show grid sst_demean     GRID GSQ1 name       axis              # pts   start                end                 subset COADSX    LONGITUDE          180mr   21E                  19E(379)            compressed COADSY    LATITUDE            90 r   89S                  89N                 compressed normal    Z TIME      TIME                12mr   16-JAN 06:00         16-DEC 01:20        full

However sst_demean does still vary in longitude and latitude. See how STAT shows 180 points in X and 90 points in Y, and a plot at one time and latitude varies in X.

yes? stat/brief sst_demean Total # of data points: 194400 (180*90*1*12*1*1) # flagged as bad data: 89622 Minimum value: -23.622 Maximum value: 12.722 Mean    value: -2.1127 (unweighted average)yes? plot/L=1/y=0 sst_demean

First, what you have done with TEMP-TEMP[X=@AVE,Y=@AVE] is merely to remove the mean. If you want a less-noisy version of TEMP, you might look at the smoothing transformations. Here is a list of all the transformations:

https://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/variables-xpressions/XPRESSIONS#Chapter3-Transformations

On 12/3/2019 4:38 PM, Xiaoyu Bai wrote:

Dear ferreters,

I noticed that when I subtract area average from a variable, it will make the variable's Lat and Lon become compressed instead of full. For example:

use levi_climatology

let test=TEMP-TEMP[X=@AVE,Y=@AVE]

sh grid TEMP,test
GRID GMS1
name axis # pts start end subset
XAXLEVITR LONGITUDE 360mr 20.5E 19.5E(379.5) full
YAXLEVITR LATITUDE 180 r 89.5S 89.5N full
ZAXLEVITR DEPTH (m) 20 i- 0 5000 full
normal T
GRID GMS1
name axis # pts start end subset
XAXLEVITR LONGITUDE 360mr 20.5E 19.5E(379.5) compressed
YAXLEVITR LATITUDE 180 r 89.5S 89.5N compressed
ZAXLEVITR DEPTH (m) 20 i- 0 5000 full
normal T

What I want is to get a less noisy variable, then apply @DIN to it. But this compressed grid was blocking me to use @DIN.

So I am wondering how to get my coordinate to full again.

Thank you in advance.

--

Xiaoyu

--

Xiaoyu