Re: [ferret_users] memory issues while computing EOFs

["Ansley C. Manke" <ansley.b.manke@xxxxxxxx>]

Hi,

The EOF functions are inverting a large matrix. It must solve the entire problem even to output the first few EOF spatial fields.

The matrix is a square matrix whose size is the product of the x and y sizes, so for your grid, that is a matrix of 499545 x 499545.

Search for "EOF memory" in the Users List Archives for discussion of this. There have been many discussions, for instance:

https://www.pmel.noaa.gov/maillists/tmap/ferret_users/fu_2012/msg00240.html
https://www.pmel.noaa.gov/maillists/tmap/ferret_users/fu_2003/msg00696.html

I would also suggest using the EOFSVD functions.  They are faster, but do not require any less working memory .

-Ansley

On 5/14/2017 1:01 PM, afwande juliet wrote:

Dear ALL

I am using CHIRPS data to compute EOFs and only I need upto 6 eofs . I am using this link http://ferret.pmel.noaa.gov/static/Demos/ef_eof_demo/ef_eof_demo.html

However, i continue to get memory Issue even after reducing the eofs to only 6


yes? use chirps_1988_2012_monthly_pr_rg_anom.nc
yes? sh d
     currently SET data sets:
    1> ./chirps_1988_2012_monthly_pr_rg_anom.nc  (default)
 name     title                             I         J         K         L         M         N
 PR       PRE*MASK                         1:653     1:765     ...       1:300     ...       ...
 
yes? SET REGION/X=28E:42E/Y=11S:5N
yes? LET eof_xyfcn = eof_space(pr, 0.5)
yes? SAVE/CLOBBER/FILE=sst_clim_eof_space.nc eof_xyfcn[l=1:6]
 **ERROR: request exceeds memory setting: 55 Mwords were requested.
           *** NOTE: You can use SET MEMORY/SIZE=xxx to increase memory.
           *** NOTE: The "Memory use" section of the FERRET Users Guide has further tips.
yes? set memory/size=100
 Cached data cleared from memory
yes? SAVE/CLOBBER/FILE=sst_clim_eof_space.nc eof_xyfcn[l=1:6]
 **ERROR: request exceeds memory setting: 16273 Mwords were requested.
           *** NOTE: You can use SET MEMORY/SIZE=xxx to increase memory.
           *** NOTE: The "Memory use" section of the FERRET Users Guide has further tips.
yes? set memory/size=10000
 Cached data cleared from memory
Unable to resize Ferret's memory cache to 10000.000000 Mdoubles
Ferret's memory cache remains at 100.000000 Mdoubles

Does someone has an Idea how to go about this issue