Message boards : Number crunching : WUs with different deadlines?
Author | Message |
---|---|
scole of TSBT Send message Joined: 2 May 14 Posts: 3 Credit: 17,378,638 RAC: 0 |
Why do some WUs give 14 days to return and some only 5? |
David E K Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Jul 05 Posts: 1018 Credit: 4,334,829 RAC: 0 |
Why do some WUs give 14 days to return and some only 5? The jobs with a 5 day deadline are Robetta jobs which are structure prediction jobs for academics. |
Murasaki Send message Joined: 20 Apr 06 Posts: 303 Credit: 511,418 RAC: 0 |
I'm getting some robetta jobs with a 2 day deadline, which I have never seen before. They pop into my work queue and immediately jump to high priority. I have my tasks set to 6 hours, so it shouldn't be a problem normally. However, would someone with a 1 day runtime also get a 2 day deadline? If so, it would be cutting things a little fine if their system is juggling with some other tasks as well. |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
I'm getting some robetta jobs with a 2 day deadline, which I have never seen before. They pop into my work queue and immediately jump to high priority. Yes. I run 24hr WUs, and once tasks pop in and run at BOINC Manager's high priority, it prevents downloads of new tasks, so these short deadlines can become disruptive. It depends how many CPUs you have, and how many short task deadlines you get as to how much it interferes with BOINC's normal schedule planning. And I could easily end up with more work than I could crunch in 48 hours. I try to take on extra tasks on Fridays and crunchy over the weekend, but when Friday came, I had tasks running at high priority and so no additional R@h work was able to download. Rosetta Moderator: Mod.Sense |
Timo Send message Joined: 9 Jan 12 Posts: 185 Credit: 45,649,459 RAC: 0 |
As far as I recall, the 'high priority' is not so much something requested by the project, as it is applied by the BOINC client when it detects that the remaining time to complete a task is close to causing the task to return back past it's deadline. I work with database queries every day at work, and if I had to wait for a couple days for a query to finish I would pull my hair out. Thus, after reviewing the many 'queries' (prediction jobs) in the Robetta Queue that are actually quite simple/short in nature - indeed, many of them go from 'Queued' to 'Completed' in less than 24 hours - I understand why the target runtimes were reduced. Any machine that takes >24 hours to return results is basically holding up the pack (though it also looks like these jobs will often complete anyways with someone else's cycles and thus, the people running 24 hour target runtimes are burning cycles for nothing on these shorter predictions). This is - I believe - why David had been toying with restricting some RB jobs to machines that have a fast average turn around time. I'm happy to say that most all of my machines are < 0.5 days TAT from the time a WU is received to the time it's reported back. Getting there has required going the other way of my strategy last year though - reducing the caching settings to next to nothing (min '0.05'; max '0.10' of work cached) - and reducing my target runtime (now at 8 hours). I suspect this is all at least partly due to the huge expansion in the base of active hosts: **38 cores crunching for R@H on behalf of cancercomputer.org - a non-profit supporting High Performance Computing in Cancer Research |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
Timo, thanks for crunching... and with renewable energy, win/win! You have crossed a few terms, so I wanted to clarify my prior comment and ask you to clarify as well. What I've seen to date is reduced task "deadlines", not reduced "runtimes". In other words, I may have a 24hr runtime preference, but the deadlines have historically been running 14 days from date a task is issued to my machine. The task deadline is what has been, in some cases, reduced to 2 or 5 days. So, a person running with 24hr target runtimes is not burning cycles for nothing, they are simply running a lot of models... at "high priority" against the tasks issued. I'm not following how you say the jobs often complete on another machine. For any one specific WU, it will not be issued to another machine until it passes it's deadline without being reported back. But I love the idea of finding a way to issue short deadline tasks only to "reliable hosts" which have historically rapid turnarounds. If that is what DK was heading for all along, I missed that detail in his other descriptions of what was done. As you say, BOINC Manager decides when to denote that tasks are running "high priority", and as you say, this is basically when the BOINC Manager concludes that the task is in risk of not completing before the deadline if the manager doesn't keep it running (even though it might normally cycle through and run tasks from another project). If the underlying tasks change on the client, the "high priority" status can change as well. If one completes earlier than estimated being a good example. Rosetta Moderator: Mod.Sense |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
I think I get it now... you are referring to tasks from the robetta queue, not R@h work units. And if the robetta job has already been flagged as complete, then there still may be R@h hosts crunching on related work. On R@h, many tasks are created for any prediction job. The more tasks that report data back, the more models are attempted and potentially finding better results. I wonder how robetta jobs translate into R@h tasks. Rosetta Moderator: Mod.Sense |
David E K Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Jul 05 Posts: 1018 Credit: 4,334,829 RAC: 0 |
The robetta jobs that complete within 24 hours are most likely run locally on our cluster. These are easier targets that have high identity to known structures so they do not require much sampling and are thus run on our local computers. The 2 day deadline jobs are for CAMEO which is a weekly benchmark/evaluation. CAMEO targets are submitted to our server on Friday and are expected to complete by late Monday/early Tuesday before the new PDB structures are released. CAMEO is like a continuous CASP for automated structure prediction servers. |
Message boards :
Number crunching :
WUs with different deadlines?
©2024 University of Washington
https://www.bakerlab.org