Rosetta ignoring memory usage limits while not idle

Message boards : Number crunching : Rosetta ignoring memory usage limits while not idle

To post messages, you must log in.

AuthorMessage
Oran Lord

Send message
Joined: 5 Jun 09
Posts: 5
Credit: 128,821
RAC: 0
Message 68043 - Posted: 12 Oct 2010, 1:42:48 UTC
Last modified: 12 Oct 2010, 1:43:09 UTC

The rosetta program is using more than the maximum allowed memory as set by the BOINC client while the computer is in use. I have my client set to allow programs to use 1% of RAM at most while computer is in use, and 80% while computer is not in use. This memory issue happens every time this program runs now.

I've caught milkyway doing the same thing before, but it doesn't do it anymore, and neither do any of the other BOINC projects as far as I have seen. The screenshot shows the memory usage while the computer is not idle, but sometimes it leaves much more in memory than shown.

Isn't there any way for the program to save the RAM to the HDD while the computer is in use, and load it back into RAM when it becomes idle again?

ID: 68043 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Oran Lord

Send message
Joined: 5 Jun 09
Posts: 5
Credit: 128,821
RAC: 0
Message 68045 - Posted: 12 Oct 2010, 3:45:53 UTC

The first screenshot is only showing what's in RAM, and doesn't include VM. Here's a better idea of how much memory it's actually trying to use.

ID: 68045 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,174,569
RAC: 3,096
Message 68048 - Posted: 12 Oct 2010, 10:02:55 UTC - in response to Message 68045.  

The first screenshot is only showing what's in RAM, and doesn't include VM. Here's a better idea of how much memory it's actually trying to use.



You are using version 6.10.18 of Boinc, try upgrading to the latest certified version, 6.10.58, and see if it is better.
ID: 68048 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 68051 - Posted: 12 Oct 2010, 16:38:02 UTC

When limiting BOINC to 1% of memory when the machine is in use, you might as well set things to NOT run when your machine is in use. That would likely avoid suspending one task for exceeding the memory limits, only to start another and hit the limit in the first few seconds of processing. Having BOINC NOT run when the machine is in use would just suspend the active tasks and not try to start anything more. Thus saving space in your page file.

When CPU % is zero for a task, it's memory is available to be swapped out. And that is probably what has occurred. However, you've told BOINC to keep applications "in memory" (the swap file really) when preempted, and so they are still active tasks, ready to run when the machine goes idle again. So they still show in the list.

Looking at your screenshot, the work set of the tasks is very small, and basically appears to represent memory that has not been required for the applications you are running on your machine. Note that you still have memory free. Windows isn't going to force a task OUT of memory unless there is some other good use for that memory.

In short, if you are not seeing impaired performance on your machine when you are using it, I wouldn't worry too much about the details of how BOINC is making that happen for you.

Isn't there any way for the program to save the RAM to the HDD while the computer is in use, and load it back into RAM when it becomes idle again?


That is exactly what Windows will do for you when memory is needed for other things on the machine. That is what the "page file" is.
Rosetta Moderator: Mod.Sense
ID: 68051 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Oran Lord

Send message
Joined: 5 Jun 09
Posts: 5
Credit: 128,821
RAC: 0
Message 68059 - Posted: 13 Oct 2010, 3:07:28 UTC - in response to Message 68051.  

When limiting BOINC to 1% of memory when the machine is in use, you might as well set things to NOT run when your machine is in use. That would likely avoid suspending one task for exceeding the memory limits, only to start another and hit the limit in the first few seconds of processing. Having BOINC NOT run when the machine is in use would just suspend the active tasks and not try to start anything more. Thus saving space in your page file.

When CPU % is zero for a task, it's memory is available to be swapped out. And that is probably what has occurred. However, you've told BOINC to keep applications "in memory" (the swap file really) when preempted, and so they are still active tasks, ready to run when the machine goes idle again. So they still show in the list.

Looking at your screenshot, the work set of the tasks is very small, and basically appears to represent memory that has not been required for the applications you are running on your machine. Note that you still have memory free. Windows isn't going to force a task OUT of memory unless there is some other good use for that memory.

In short, if you are not seeing impaired performance on your machine when you are using it, I wouldn't worry too much about the details of how BOINC is making that happen for you.

Isn't there any way for the program to save the RAM to the HDD while the computer is in use, and load it back into RAM when it becomes idle again?


That is exactly what Windows will do for you when memory is needed for other things on the machine. That is what the "page file" is.

I've set BOINC to not do computation unless the computer has been idle for 5 minutes.

I really don't want Windows managing the memory, because I don't trust when it believes it is in use or not. When playing games, I notice much more choppiness when the rosetta program is in the process list than when it is not.

My main problem is that all other BOINC projects are invisible when I'm using my computer, and they continue to compute normally after it's been idle for 5 minutes, but rosetta seems to want to keep a hold of my RAM and HDD while I'm trying to use my computer. While I'm playing a game on my system, you can understand how any loss of performance would be perceived as significant, even if it's just for a few seconds while Windows does some swapping.

I will try updating the BOINC client to see if there's any change.
ID: 68059 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Michael Gould

Send message
Joined: 3 Feb 10
Posts: 39
Credit: 15,436,476
RAC: 4,358
Message 68060 - Posted: 13 Oct 2010, 5:00:33 UTC - in response to Message 68059.  



...I really don't want Windows managing the memory, because I don't trust when it believes it is in use or not. When playing games, I notice much more choppiness when the rosetta program is in the process list than when it is not.


I'm not a Windows user, so I'm not sure I quite understand the process, but couldn't you just quit the BOINC Manager while you are doing something that it seems to affect negatively, like the games? Then just restart it when you are done. Or does it have to be installed as a screensaver on Windows?

I know on my Mac, I have to quit the BOINC Manager if I am watching an MLB.com baseball gamecast, for instance. Helping mankind is all well and good, but when the Phillies are playing mankind just has to wait!
ID: 68060 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,174,569
RAC: 3,096
Message 68063 - Posted: 13 Oct 2010, 11:04:01 UTC - in response to Message 68059.  

I really don't want Windows managing the memory, because I don't trust when it believes it is in use or not. When playing games, I notice much more choppiness when the rosetta program is in the process list than when it is not.


Windows7, which you use, did a bit of a switch up when it comes to memory management, MS now believes that free memory is wasted memory, so it uses almost all available memory as L2 cache memory trying to make the pc faster. The old days of running memory managers to free up more memory is out the window now as it is not applicable anymore.

My main problem is that all other BOINC projects are invisible when I'm using my computer, and they continue to compute normally after it's been idle for 5 minutes, but rosetta seems to want to keep a hold of my RAM and HDD while I'm trying to use my computer. While I'm playing a game on my system, you can understand how any loss of performance would be perceived as significant, even if it's just for a few seconds while Windows does some swapping.

I will try updating the BOINC client to see if there's any change.


You can also do a right click on the Boinc Manager and click 'snooze', this stops Boinc from crunching for 2 hours and then auto it restarts itself after that. It does NOT release the memory when you click snooze but if you really want the memory back maybe it would help if you upgraded the memory to 6gb or even 8gb of memory. I see you are also crunching for MW, do you stop it when you game? If not alot of your performance loss is with it and the 'snoozing' will help quite a bit.
ID: 68063 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,174,569
RAC: 3,096
Message 68064 - Posted: 13 Oct 2010, 11:05:45 UTC - in response to Message 68060.  



...I really don't want Windows managing the memory, because I don't trust when it believes it is in use or not. When playing games, I notice much more choppiness when the rosetta program is in the process list than when it is not.


I'm not a Windows user, so I'm not sure I quite understand the process, but couldn't you just quit the BOINC Manager while you are doing something that it seems to affect negatively, like the games? Then just restart it when you are done. Or does it have to be installed as a screensaver on Windows?

I know on my Mac, I have to quit the BOINC Manager if I am watching an MLB.com baseball gamecast, for instance. Helping mankind is all well and good, but when the Phillies are playing mankind just has to wait!


No most of us do not use any screen saver as it wastes precious crunching time, pushing he power button on the monitor also gives us the power to think we are being 'green' too!
ID: 68064 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 68070 - Posted: 13 Oct 2010, 13:49:15 UTC

Now that BOINC will not try to run at all when the machine is in use, that should be the identical result to snoozing BOINC without having to remember to do anything. It is still possible that as a game ramps up and loads various routines as you explore it, that things have to be pushed out to the page file, but there really isn't much you can do about that. The only option there would be to completely end BOINC, or NOT keep the tasks in memory when they are suspended. But this can end up throwing away a fair amount of work that hasn't been checkpointed yet.

I think Oran Lord will find the games run better with these settings. Another idea would be, just once to see, to exit BOINC, which removes everything from the task list and loses any work that has not been checkpointed, and see if the games are STILL choppy. That would at least be an indication that BOINC and Rosetta are not the only cause of erratic response time.
Rosetta Moderator: Mod.Sense
ID: 68070 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Oran Lord

Send message
Joined: 5 Jun 09
Posts: 5
Credit: 128,821
RAC: 0
Message 68094 - Posted: 15 Oct 2010, 1:39:56 UTC

My BOINC client is running under the separate user account "boinc_master", as set up by the BOINC installer. So closing the BOINC Manager does nothing, since the tasks continue to run under a separate account. This also allows you to log out of your account, and the boinc_master user can continue doing tasks.

I'm sure that rosetta is the culprit for the slowness because nothing else is running, and it occurs when the rosetta processes are left suspended in memory and doesn't occur when they are not running. I don't want to keep tasks in memory when they are suspended, as I've specified in my preferences on the BOINC client. All the other projects are obeying this preference, and I have seen them completely kill any running processes when I start using the computer. Rosetta seems to be the only one that will stay in memory after the computer is in use. Redoing work is fine with me, since I only use my computer for a few hours a day, giving it plenty of time to make up the work it discarded.

The bottom line being that if there is a preference saying "Leave applications in memory while suspended", and I have it unchecked, then when I am using the computer the applications should not be consuming any resources.
ID: 68094 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 68102 - Posted: 15 Oct 2010, 15:37:20 UTC

...if there is a preference saying "Leave applications in memory while suspended", and I have it unchecked, then when I am using the computer the applications should not be consuming any resources.


...well, two things actually. If work is suspended when the machine is in use, and no check on the leave applications in memory while suspended. You need the first to actually get the tasks suspended, then the second to remove them from memory.

The BOINC client is responsible for such system management functions. Are you running the BOINC version currently recommended by Berkeley?
Rosetta Moderator: Mod.Sense
ID: 68102 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Oran Lord

Send message
Joined: 5 Jun 09
Posts: 5
Credit: 128,821
RAC: 0
Message 68106 - Posted: 16 Oct 2010, 2:24:10 UTC - in response to Message 68102.  

The BOINC client is responsible for such system management functions. Are you running the BOINC version currently recommended by Berkeley?

I have updated to BOINC version 6.10.58.
ID: 68106 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Michael Gould

Send message
Joined: 3 Feb 10
Posts: 39
Credit: 15,436,476
RAC: 4,358
Message 68108 - Posted: 16 Oct 2010, 4:23:20 UTC - in response to Message 68064.  



No most of us do not use any screen saver as it wastes precious crunching time, pushing he power button on the monitor also gives us the power to think we are being 'green' too!


Okay, I worded my question poorly. I didn't mean the graphics, what I meant to ask is if BOINC manager could be installed as a normal app which one could start and stop manually, or if it had to be run as a "Background app," starting and stopping automatically, which sounds like what Oran is describing.

It seems to me Oran's problem would be solved if he could install BOINC manager as an app which he could simply start up when he was leaving the computer, and shut down while he was using it, if necessary. On the Mac, you can install it either way.
ID: 68108 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Adam Gajdacs (Mr. Fusion)

Send message
Joined: 26 Nov 05
Posts: 13
Credit: 2,813,842
RAC: 2,236
Message 68109 - Posted: 16 Oct 2010, 7:41:25 UTC - in response to Message 68108.  

Okay, I worded my question poorly. I didn't mean the graphics, what I meant to ask is if BOINC manager could be installed as a normal app which one could start and stop manually, or if it had to be run as a "Background app," starting and stopping automatically, which sounds like what Oran is describing.


There should be two ways to install BOINC on Windows too, either as a service (which runs in the background all the time regardless of the currently logged in OS user, if any), and as a regular application that you need to start manually, but also can shut down completely at any time to reliably free up any and all used resources if need be.
It's been some time since the last time I actually installed a new BOINC client, but I suppose these two installation options still should exist.

I prefer the second option myself, and had been using BOINC in that mode from the start.
ID: 68109 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Rosetta ignoring memory usage limits while not idle



©2024 University of Washington
https://www.bakerlab.org