WU Download Scheduling

Message boards : Number crunching : WU Download Scheduling

To post messages, you must log in.

AuthorMessage
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76434 - Posted: 16 Feb 2014, 13:17:08 UTC

Hey all, I've currently got my 2 laptops set to connect to the network only between 3a and 5a, but they pretty much always wait until 4:30a to request work for the whole day, which for them takes longer than 30 minutes to download.

Is there any way I can tell them to request work earlier in the day so they don't run out of network time?
ID: 76434 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 76437 - Posted: 17 Feb 2014, 5:46:49 UTC

If you are running out of work, with a small network window like that, I'd suggest configuring BOINC Manager to keep more days of work on hand setting a larger value in the additional work buffer.
Rosetta Moderator: Mod.Sense
ID: 76437 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76439 - Posted: 17 Feb 2014, 12:44:35 UTC - in response to Message 76437.  

If you are running out of work, with a small network window like that, I'd suggest configuring BOINC Manager to keep more days of work on hand setting a larger value in the additional work buffer.


I've currently set #days to 1.0 and cached (additional) to 2.0, and when it can finish the downloads, that's more than enough work for a single day; my problem is that within the 3-5a download window, it wais until 4:30 to download, then 5a comes before the downloads finish.

I used to have the window set at 3-6a, and they would wait until 5:30a to start downloading, so it seems like a 30-minutes-before-window-closed rule is coded somewhere in a config file
ID: 76439 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,560,400
RAC: 7,883
Message 76440 - Posted: 17 Feb 2014, 20:14:58 UTC
Last modified: 17 Feb 2014, 20:21:39 UTC

Hi mgruben

I think you can use a scheduled task to run a batch file containing the following at 3:01am:

"c:program filesboincboinccmd.exe update"

Oops - just realised you're running Linux... not sure how to schedule jobs in that OS.
ID: 76440 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76441 - Posted: 17 Feb 2014, 21:12:11 UTC
Last modified: 17 Feb 2014, 21:13:20 UTC

From the wiki,
for url in $(boinccmd --get_project_status | sed -n 's/s*master URL: //p'); do
  boinccmd --project ${url} update;
done

It would be nice if it didn't wait until it had 30 minutes left, but the above at least seems to be a quick fix
ID: 76441 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 76442 - Posted: 17 Feb 2014, 21:48:37 UTC

Agreed, but that is more of a BOINC Manager question than Rosetta.

I don't think it is waiting, it's just fairly oblivious to the allowed network window. So it just waits until it would normally try to use the network, and if it is available (i.e. within your configured time constraints), then it uses it. If not, it waits. If it gets started on downloads and the network window ends, then it should pick up where it left off the next night (assuming you allow the same time window each night??)
Rosetta Moderator: Mod.Sense
ID: 76442 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76443 - Posted: 18 Feb 2014, 2:48:11 UTC - in response to Message 76442.  
Last modified: 18 Feb 2014, 2:50:06 UTC

Agreed, but that is more of a BOINC Manager question than Rosetta.
Conceded.
If it gets started on downloads and the network window ends, then it should pick up where it left off the next night (assuming you allow the same time window each night??)
It does, but it only finishes night 1's leftover downloads at the start of night 2. It then waits until T-0:30 to request night 2's downloads, which it can't finish during night 2 and which it finishes at the start of night 3, ad infinitum.
ID: 76443 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76450 - Posted: 19 Feb 2014, 10:03:10 UTC - in response to Message 76443.  
Last modified: 19 Feb 2014, 10:09:13 UTC

Until I find the boinc-specific setting I'm looking for, I just set up a systemd job (running Arch linux; we shun cron):

/etc/systemd/system/boinc-scheduler-update.service
[Unit]
Description=For Rosetta@Home, forces a scheduler request.

[Service]
Type=oneshot
ExecStart=/usr/bin/boinccmd --passwd [from /var/lib/boinc/gui_auth_rpc.cfg] --project https://boinc.bakerlab.org/rosetta/ update

/etc/systemd/system/boinc-scheduler-update.timer
[Unit]
Description=the timer for daily BOINC project updates (currently pegged to the start of the network availability window at slightly after 3am)

[Timer]
OnCalendar=*-*-* 03:01:00
Unit=boinc-scheduler-update.service

[Install]
WantedBy=basic.target

Then of course "sudo systemctl enable boinc-scheduler-update.timer && sudo systemctl start boinc-scheduler-update.timer"
ID: 76450 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 76465 - Posted: 20 Feb 2014, 15:04:54 UTC

I'd suggest adding another day to your work buffer (which is done in the network preferences).
Rosetta Moderator: Mod.Sense
ID: 76465 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76466 - Posted: 20 Feb 2014, 15:15:00 UTC - in response to Message 76465.  
Last modified: 20 Feb 2014, 15:18:59 UTC

I'd suggest adding another day to your work buffer (which is done in the network preferences).

My initial qualm was admittedly one of efficiency aesthetics (that, within a sufficiently-long connectivity window, new work isn't fetched until there is insufficient time remaining to complete its download).

While your suggestion would probably accomplish the same ends (namely, having enough work on the rig so that it never idles), it differs with my update philosophy, and still leaves me with the question of how the client decides when to initiate a project update (which is consistently 30.0 minutes prior to the close of the network window; an odd artifact suggesting to me that this project-update behavior is changeable).

Edit:
For anyone hoping to use a similar setup, I actually don't recommend initiating a project update request at the absolute beginning of the network connectivity window; some projects may throw you a "Not requesting tasks: too many uploads in progress" message back. Better to wait until you think all (or most) of your uploads will finish.
Accordingly, I've changed "03:01:00" in boinc-scheduler-update.timer below to "04:00:00", which gives me enough time within my 3:00am to 5:00am network window for the majority of my uploads to finish.
ID: 76466 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 76482 - Posted: 21 Feb 2014, 22:06:57 UTC

Sounds like a good question for the BOINC message boards.

When you get work loaded, does it process in something pretty close to the initially estimated time? If you recently changed your Rosetta runtime preference that might throw off the BOINC Manager's estimates and cause it not to realize it is going to need work.
Rosetta Moderator: Mod.Sense
ID: 76482 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76483 - Posted: 21 Feb 2014, 22:19:37 UTC - in response to Message 76482.  

Sounds like a good question for the BOINC message boards.
Agreed
When you get work loaded, does it process in something pretty close to the initially estimated time? If you recently changed your Rosetta runtime preference that might throw off the BOINC Manager's estimates and cause it not to realize it is going to need work.
BOINC's estimates of time-to-completion seem to be fairly accurate, and I have been fiddling with the 'Computer is connected to the internet every … days' and 'Maintain enough work for an additional … days' options since this thread began, but consistently BOINC doesn't initiate a scheduler request until 4:30a (hence my systemd script which functionally solves my wait-until-too-late-in-the-network-availability-window problem).
ID: 76483 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mgruben

Send message
Joined: 5 Oct 13
Posts: 14
Credit: 3,692,308
RAC: 0
Message 76493 - Posted: 24 Feb 2014, 15:03:32 UTC - in response to Message 76483.  

Wouldn't you know, the one setting I hadn't tried was just what I was looking for.

That is, setting
Computer is connected to the Internet about every [2.0] days
Maintain enough work for an additional [0.0] days

Sends scheduler requests at (well, really near) the beginning of the network availability window.
It's exactly the setup I was looking for, and was the one I hadn't yet tried.
Also, this setup renders obsolete the systemd setup mentioned below, which I turned off (disable; stop)
ID: 76493 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : WU Download Scheduling



©2024 University of Washington
https://www.bakerlab.org