Hi Saurabh,
As Karl said there's no allocation of memory/clearing of cache. What you need to do is issue the CAN MEM or possibly CAN MEM/ALL commands. You only need to issue the SET MEM command once (this has always been the case)
set mem/size=3000
repeat/range=1:3600:100/name=ii (repeat/range=1:450:15/name=jj save/append/file=mss19932012.nc/i=`ii`:`ii+99`/j=`jj`:`jj+14` mss;can mem/all);can mem/all)
I presume your files are being chunked 450 in longitude and 15 in latitude otherwise you are accessing/writing non contiguous records. This may also be hurting you.
Russ
On 4/1/19 8:20 am, saurabh rathore wrote:
Dear Karl,G'day with new year wishes (apologies for late wishes)
Thank you for your email but I think I am not able to understand the difference between previous version of ferret and the latest one. Previously when we were using e.g. repeat/l=1:300:1 (save/append/file=var.nc var;set mem/size=xxxx), so at every time step saving the ram was clear. This means let say during processing first time step my whole 16 gb of ram was occupied but after finishing the saving of first time step data my whole was available to process next time step of data. But in latest version, even after issuing set mem/size=xxxx is not helping ram of my machine to clear the chache data of previous time step.
hence I am facing problem to save a large model output and this is making my machine really super slow. So I need any idea how to overcome this problem. I am attaching the command that I am using to save my chunk of data from a model output of 1/10 degree resolution.repeat/range=1:3600:100/name=ii (repeat/range=1:450:15/name=jj save/append/file=mss19932012.nc/i=`ii`:`ii+99`/j=`jj`:`jj+14` mss;set mem/size=3000);set mem/size=3000) and this is not working with memory command.
I hope I am able to clearly state my problem.
Cheers, Saurabh
On Fri, Jan 4, 2019 at 1:51 AM Karl Smith - NOAA Affiliate <karl.smith@xxxxxxxx> wrote:
Hi Saurabh,
The command 'SET MEM /SIZE=xxx' is correct and works.
In versions 7.2 and later, this command only sets a maximum on the amount of memory (in units of 8*MB; in other words, units of a million double-precision floating point values) that Ferret/PyFerret can use; no memory is actually allocated at that time (as was the case in previous versions of Ferret/PyFerret). We are working on allowing users to specify the units for the number in this command so it is clearer how much memory one is allowing.
Karl
PS, Many users on this list, including Ansley, are US federal employees and cannot even check email during the US government shutdown. (I am a cooperative institute employee.)
On Wed, Jan 2, 2019 at 9:05 PM saurabh rathore <rohitsrb2020@xxxxxxxxx> wrote:
Dear Ferreters,I am using a high resolution model output and I am trying to save my out come with new ferret version 7.44. So just want to know is command set mem/size = xxxx working in this new ferret version or not ? if not then what is the new command to issue memory to make life easy.
cheers, saurabh
--
REGARDS
Saurabh RathoreResearch Scholar (PhD.)Centre For Oceans, Rivers, Atmosphere & Land Science TechnologyIndian Institute Of Technology, Kharagpurcontact :- 91- 8345984434
--
Karl M. Smith, Ph.D.
JISAO Univ. Wash. and PMEL NOAA
"The contents of this message are mine personally and do
not necessarily reflect any position of the Government
or the National Oceanic and Atmospheric Administration."
--
REGARDS
Saurabh RathoreResearch Scholar (PhD.)Centre For Oceans, Rivers, Atmosphere & Land Science TechnologyIndian Institute Of Technology, Kharagpurcontact :- 91- 8345984434