I do have a host here which does not have discrete GPU, it only uses integrated graphics. It has been running Rosetta without errors.

---

This is a report on my troubleshooting activities. Last night on the problem machine I ran 8 Rosetta WU's with the following concurrent changes to the system:
1) Folding@Home was stopped on CPU and GPU's,
2) All applications except BOINC/Rosetta were terminated,
3) All nonessential processes were terminated with Task Manager,
4) CPU was not overclocked,
5) screen saver and desktop image were not used, only blue background used.

The results were the same. All WU's failed due to client error, invalid, and application version not reported (showing only three dashes).

A.M. reported via e-mail using an EVGA 550Ti GPU, previously with driver 285.62 and now with 295.73. It would be useful to hear from William Blakemore, Alpha Laser, Sky King, and Digital Savior if they are running EVGA's, how many, what model, and if they have been running Folding@Home. Here is a summary of the machines from this thread that appear to be dealing with the same problem:

a) my problem machine: i7-3930K, 3.2GHz, Family 6, model 45, stepping 7
b) William Blakemore: i7-2700K, 3.5GHz, Family 6, model 42, stepping 7
c) Alpha Laser: i7-Q740 CPU, 1.7GHz, Family 6, model 30, stepping 5
d) Sky King: i7-920 CPU, 2.7GHz, Family 6, model 26, stepping 4
e) Digital Savior: i7-2600K, 3.4GHz, Family 6, model 42, stepping 7
f) A.M.: i7-2600K, 3.4GHz, Family 6, model 42, stepping 7

I think all of the stepping 7 processors are the latest CPU version. Suspicious, but I found other i7-3930K's stepping 7 without our issue (computer ID's 1520085 & 1524601).

I suspect there are plenty more machines out there with this problem, but either the users haven't noticed it yet, or they haven't bothered to report it.

For tonight's continued troubleshooting I'm considering uninstalling & physically removing the EVGA GPU's and installing a low end GPU of another type. Other suggestions/approaches would be appreciated if anyone has any ideas.

Yes, I'm running a single EVGA board -- GTX 560 ti, with 2GB RAM onboard. My CPU is also overclocked to 4.00 GHz.

I don't crunch for Folding@Home. My other projects are Seti@Home and WCG. No corresponding problems at either place.

Unfortunately, I don't have any progress to report. We're still not sure why these runs look like they're successfully completing, but then resulting in errors while reporting back to the server.

If you're up to it, something might reveal itself if people encountering these errors turned on the extra debugging information in the cc_config.xml file
The relevant portion to focus on is the post-result reporting. Log flags like file_xfer_debug, http_debug, http_xfer_debug, network_status_debug, proxy_debug are ones to try. If anything looks off/wonky in clients who have these issues, then that's a lead we can follow up on in debugging. (That's not to say that the error will necessarily show itself on the client end - but it may be worth a shot.)

Edit: Others on the forum have indicated that the issue may be GPU related. (Rosetta@home doesn't use GPUs, but that's not to say a GPU-related setting might be involved.) You can also try (temporarily) turning off GPU crunching, to see that fixes things - even if you turn GPUs back on for non-Rosetta projects, posting that the issue was fixed on your machine with a GPU setting change will help us in tracking down the issue.

At the risk of making a silly observation ... more than anything else, this feels like a client/server communications problem. I'm not sure how useful it is to explore hardware issues or solutions.

If I were to make a SWAG, I'd be looking for a change made to the BOINC manager software. Perhaps they made a change to accommodate newer NVidia GPU processing that your server people didn't pick up because they weren't running GPU WU's. I know that the Seti folks had to upgrade their NVidia GPU apps to handle newer hardware.

Troubleshooting Update:

Before reaching home last night I had decided the problem was most likely a software problem of some sort. Therefore I did not uninstall the EVGA graphics cards. Instead I formatted the C: drive and reinstalled Windows. Here are my installation steps:
1) installed Win7, 64-bit which already included SP1.
2) turned off Windows update to keep Windows at stock configuration.
3) installed the motherboard drivers & utilities from the ASUS CD/DVD
4) did NOT install the EVGA driver/utilities (used only the default video driver)
5) did NOT install Folding@Home or any other s/w to run on the GPU's
6) installed BOINC and Rosetta@Home, then ran for about 7 hours.

Results: Success! Rosetta reported valid WU's and successfully report the 'application version' as 3.24.

This morning at approximately 6 AM (local/Seattle Time) or 13:00 UTC, I installed the EVGA graphics driver version 285.62 that has been running successfully with R@H on my "old machine" since November. I did NOT install any of the EVGA utilities, but only the driver. All tasks that have completed since that time failed due to client error, invalid, 'application version' not reported.

Testing will continue tonight, and I have slowed down the processing so there will be some tasks left in the queue to play with when I get home. I'm thinking the next step is to uninstall the current EVGA driver, see what happens for a few hours, then install the most recent EVGA driver. Based on driver version information reported by others here, I expect the new driver to cause failure also. 'Thank you' to William Blakemore for the interesting comments. Anyone have other ideas for troubleshooting?

To Rocco Moretti - can you tell by looking at the uploaded results of a failed task if the data computed/returned is good, but it's only failing because of some more trivial absence of information like the missing application version. Is the upload being prematurely terminated? What exactly is causing the task to be marked as invalid? Thanks.

Troubleshooting Update:

Before reaching home last night I had decided the problem was most likely a software problem of some sort. Therefore I did not uninstall the EVGA graphics cards. Instead I formatted the C: drive and reinstalled Windows. Here are my installation steps:
1) installed Win7, 64-bit which already included SP1.
2) turned off Windows update to keep Windows at stock configuration.
3) installed the motherboard drivers & utilities from the ASUS CD/DVD
4) did NOT install the EVGA driver/utilities (used only the default video driver)
5) did NOT install Folding@Home or any other s/w to run on the GPU's
6) installed BOINC and Rosetta@Home, then ran for about 7 hours.

Results: Success! Rosetta reported valid WU's and successfully report the 'application version' as 3.24.

This morning at approximately 6 AM (local/Seattle Time) or 13:00 UTC, I installed the EVGA graphics driver version 285.62 that has been running successfully with R@H on my "old machine" since November. I did NOT install any of the EVGA utilities, but only the driver. All tasks that have completed since that time failed due to client error, invalid, 'application version' not reported.

Testing will continue tonight, and I have slowed down the processing so there will be some tasks left in the queue to play with when I get home. I'm thinking the next step is to uninstall the current EVGA driver, see what happens for a few hours, then install the most recent EVGA driver. Based on driver version information reported by others here, I expect the new driver to cause failure also. 'Thank you' to William Blakemore for the interesting comments. Anyone have other ideas for troubleshooting?

Just for grins ... you might try reinstalling ALL the suspect hardware drivers (and all the bells and whistles), with the exception of any GPU apps from any BOINC project. If everything still runs, that's telling us that something about the GPU app software and/or interface is somehow being inadvertently corrupted by the GPU app process.

Troubleshooting Progress Report:

At the end of the previous troubleshooting session I had shown that BOINC/Rosetta would work fine until the EVGA GPU driver was installed. Last night I continue testing by uninstalling the EVGA driver and going back to the Windows generic display adapter driver. Once again, with the Windows generic driver installed, BOINC/Rosetta began working correctly. I tested yet a second EVGA driver several hours later. Per suggestion I also installed all other useful applications from the EVGA disk along with all Window 7(64bit) updates. Again BOINC/Rosetta became unable to deliver valid tasks without client errors.

Is it time to get the BOINC people involved? I'm fresh out of idea here. The "Help Wanted" sign is hanging in the window!

Troubleshooting Progress Report:

At the end of the previous troubleshooting session I had shown that BOINC/Rosetta would work fine until the EVGA GPU driver was installed. Last night I continue testing by uninstalling the EVGA driver and going back to the Windows generic display adapter driver. Once again, with the Windows generic driver installed, BOINC/Rosetta began working correctly. I tested yet a second EVGA driver several hours later. Per suggestion I also installed all other useful applications from the EVGA disk along with all Window 7(64bit) updates. Again BOINC/Rosetta became unable to deliver valid tasks without client errors.

Is it time to get the BOINC people involved? I'm fresh out of idea here. The "Help Wanted" sign is hanging in the window!

IIRC, the EVGA display driver is really from the NVidia website. What you're really saying is that something about that driver (which includes the CUDA driver) is breaking the upload. The obvious question is, if the CUDA driver is flawed, why isn't it breaking any of the other CPU apps at other projects?

Hi

for information I have the "no finished file error" since at least 15/03 (my results are all erroring, I hadn't noticed) on an iMac 27 (2010 / i7 / 16 GB) with Mac OS X 10.6.8


I have reseted the project yesterday, no success.

////
Ven 23 mar 09:41:17 2012 | rosetta@home | Task rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0 exited with zero status but no 'finished' file
Ven 23 mar 09:41:17 2012 | rosetta@home | If this happens repeatedly you may need to reset the project.
Ven 23 mar 09:41:17 2012 | rosetta@home | Restarting task rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0 using minirosetta version 324 in slot 1
Ven 23 mar 09:41:18 2012 | rosetta@home | Task rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0 exited with zero status but no 'finished' file
Ven 23 mar 09:41:18 2012 | rosetta@home | If this happens repeatedly you may need to reset the project.
Ven 23 mar 09:41:18 2012 | rosetta@home | Restarting task rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0 using minirosetta version 324 in slot 1
Ven 23 mar 09:41:19 2012 | rosetta@home | Computation for task rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0 finished
Ven 23 mar 09:41:19 2012 | rosetta@home | Output file rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0_0 for task rb_03_22_29991_60688__t000__SAVE_ALL_OUT_IGNORE_THE_REST_45429_2203_0 absent
Ven 23 mar 09:41:21 2012 | rosetta@home | Scheduler request completed: got 0 new tasks
Ven 23 mar 09:41:21 2012 | rosetta@home | No work sent
Ven 23 mar 09:41:21 2012 | rosetta@home | (reached daily quota of 8 results)


Boinc :
--------
Mer 21 mar 17:27:44 2012 | | Starting BOINC client version 7.0.20 for x86_64-apple-darwin
Mer 21 mar 17:27:44 2012 | | log flags: file_xfer, sched_ops, task
Mer 21 mar 17:27:44 2012 | | Libraries: libcurl/7.21.7 OpenSSL/0.9.7l zlib/1.2.3 c-ares/1.7.4
Mer 21 mar 17:27:44 2012 | | Running as a daemon
Mer 21 mar 17:27:44 2012 | | Data directory: /Library/Application Support/BOINC Data
Mer 21 mar 17:27:44 2012 | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz [x86 Family 6 Model 30 Stepping 5]
Mer 21 mar 17:27:44 2012 | | Processor features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 POPCNT
Mer 21 mar 17:27:44 2012 | | OS: Mac OS X 10.6.8 (Darwin 10.8.0)
Mer 21 mar 17:27:44 2012 | | Memory: 16.00 GB physical, 829.07 GB virtual
Mer 21 mar 17:27:44 2012 | | Disk: 1.82 TB total, 828.83 GB free
Mer 21 mar 17:27:44 2012 | | Local time is UTC +1 hours
Mer 21 mar 17:27:44 2012 | | VirtualBox version: 4.1.10
Mer 21 mar 17:27:44 2012 | | WARNING: get_ati_mem_size_from_opengl failed to create PixelFormat
Mer 21 mar 17:27:44 2012 | | OpenCL: ATI GPU 0: Radeon HD 4850 (driver version 1.0, device version OpenCL 1.0, 512MB, 512MB available)
Mer 21 mar 17:27:44 2012 | | Config: report completed tasks immediately
Mer 21 mar 17:27:44 2012 | | Config: GUI RPC allowed from any host

---

for information I have the "no finished file error" since at least 15/03 (my results are all erroring, I hadn't noticed) on an iMac 27 (2010 / i7 / 16 GB) with Mac OS X 10.6.8

Hi Jerome - I looked at one of your failed WU's, and it's a different problem than we are working on here. Up above in this same thread there is a mention of the same mechanism of failure that you are experiencing:

From the stderr reports for the task, you're getting a "Process creation failed: errno=13" according to this post, errno=13 is a permissions error. Others have also experienced this with the update to 3.24. Best I can tell, this is an issue with Boinc 7.0, rather than with Rosetta@home itself. (Boinc 7 is currently development code, and not recommended for general use - at least not with Rosetta@home)

I see you are using BOINC 7 and have errno=13. You might want to go back to the previous version of BOINC. The latest is not necessarily the greatest!

Ok thanks for the info, yes I know about potential instability, I'm trying version 7 since we are trying to see about GPU support implementation under Mac OS X, GPU started to be recognized for us by boinc with v7 but no GPU project is working with my iMac so far, it seems I'd have to upgrade to Lion, but I don't want to for the moment. Some of my Alliance Francophone fellows have better results (with Lion) but afaik it's not working 100% yet.

For information, I have quite a high number of projects running on my machine and it's only happening with Rosetta for the moment.

I'll see what I do regarding my version, thanks.

---

Troubleshooting Report:

A recent version of the GPU driver was uninstalled and the machine reverted back to the Windows default driver. Rosetta once again started working correctly. Next, a much older July 2011 version of the NVIDIA driver was installed. Rosetta reported client errors again. One thing of note is that if the driver is switched while a Rosetta WU is paused/suspended, the final results (success or failure) is only influenced by the driver loaded at the time the WU completes and uploads.

The current recommended version of BOINC is 6.12.34; it has been in place since July 2011. A glaring characteristic of our problem is that it was first reported three days after Rosetta 3.22 was released. With older tasks quickly falling off the bottom of contributer's task lists, it's a little late to do a detailed analysis of what happened when 3.22 replaced the previous version of Rosetta. Spring break is almost over at the University of Washington, so hopefully next week we can get some support. In the mean time, does anyone know if there is a way to set up a test running a pre-Rosetta 3.22 task? If the task will complete normally with an NVIDIA driver installed in the GPU, some change that went into Rosetta 3.22 would likely be the source of the problem.

Markus Elfring's suggestion above is noteworthy, and I read his additional comments on the other thread.

In the mean time, does anyone know if there is a way to set up a test running a pre-Rosetta 3.22 task? If the task will complete normally with an NVIDIA driver installed in the GPU, some change that went into Rosetta 3.22 would likely be the source of the problem.

The problem already existed with with version Rosetta version 3.19. Howver I seemed to be the only one to report this problem.

I started this thread (https://boinc.bakerlab.org/rosetta/forum_thread.php?id=5875) back then.

Regards,

Rayburner

---

In the mean time, does anyone know if there is a way to set up a test running a pre-Rosetta 3.22 task? If the task will complete normally with an NVIDIA driver installed in the GPU, some change that went into Rosetta 3.22 would likely be the source of the problem.

The problem already existed with with version Rosetta version 3.19. Howver I seemed to be the only one to report this problem.

I started this thread (https://boinc.bakerlab.org/rosetta/forum_thread.php?id=5875) back then.

Yes, that definitely sounds like what the rest of us are experiencing - are you also running NVidia-based GPU apps for other projects?

In the mean time, does anyone know if there is a way to set up a test running a pre-Rosetta 3.22 task? If the task will complete normally with an NVIDIA driver installed in the GPU, some change that went into Rosetta 3.22 would likely be the source of the problem.

The problem already existed with with version Rosetta version 3.19. Howver I seemed to be the only one to report this problem.

I started this thread (https://boinc.bakerlab.org/rosetta/forum_thread.php?id=5875) back then.

Yes, that definitely sounds like what the rest of us are experiencing - are you also running NVidia-based GPU apps for other projects?

Yes, I do. GPUGrid and PrimeGrid

---

OK, to sum up ... it looks like we have a bug, first reported in version 3.19, that's causing all Rosetta WU's to error out, for any users running NVidia GPU apps elsewhere.

Speaking for only myself, while I enjoy getting credit like everyone else, my primary reason for being here is the science. I don't see any benefit in running WU's that just error out, to no one's benefit, so I stopped crunching here until the problem is resolved (and no, turning off my GPU is NOT something I'm inclined to do). If others feel the same way I do, then Rosetta is losing some pretty capable machines.

Umm ... support people?

OK, to sum up ... it looks like we have a bug, first reported in version 3.19, that's causing all Rosetta WU's to error out, for any users running NVidia GPU apps elsewhere.

Speaking for only myself, while I enjoy getting credit like everyone else, my primary reason for being here is the science. I don't see any benefit in running WU's that just error out, to no one's benefit, so I stopped crunching here until the problem is resolved (and no, turning off my GPU is NOT something I'm inclined to do). If others feel the same way I do, then Rosetta is losing some pretty capable machines.

Umm ... support people?

I moved my pc's, all of them, to other projects a couple of weeks ago!! Poem seems to be enjoying the benefit of my @34 cpu cores right now!!

Umm ... support people?

I wish I had more success to report. We're looking into it, and are in contact with the BOINC people about it. Unfortunately, we don't have a solution for the problem yet. Frankly speaking, it's embarrassing on our end that it's taking this long to solve this problem.

I want to extend a heartfelt thanks to everyone on the forums who is helping with troubleshooting the issue, especially those (like In Memory of Kimsey M Fowler Sr) who have gone above and beyond in diagnosing things. Thanks to all of your efforts, I think we can be relatively confident that the issue is directly related to NVidia GPU drivers on Windows 7.

I realize it sounds trite, but I'll say it anyway: Thanks again for your patience. We really do appreciate you volunteering your computers, and for putting up with us in our off moments.

I wish I had more success to report. We're looking into it, and are in contact with the BOINC people about it. Unfortunately, we don't have a solution for the problem yet. Frankly speaking, it's embarrassing on our end that it's taking this long to solve this problem.

I want to extend a heartfelt thanks to everyone on the forums who is helping with troubleshooting the issue, especially those (like In Memory of Kimsey M Fowler Sr) who have gone above and beyond in diagnosing things. Thanks to all of your efforts, I think we can be relatively confident that the issue is directly related to NVidia GPU drivers on Windows 7.

I realize it sounds trite, but I'll say it anyway: Thanks again for your patience. We really do appreciate you volunteering your computers, and for putting up with us in our off moments.

I'm in the computer business myself and I do understand that some bugs are devilishly hard to find. Perhaps instead of talking to the BOINC people, you'd have better luck talking to other projects that have simultaneously working CPU and GPU apps. Possibly, they could point out something that you're missing.

But damn ... since 3.19???

---

Umm ... support people?

I wish I had more success to report. We're looking into it, and are in contact with the BOINC people about it. Unfortunately, we don't have a solution for the problem yet. Frankly speaking, it's embarrassing on our end that it's taking this long to solve this problem.

I want to extend a heartfelt thanks to everyone on the forums who is helping with troubleshooting the issue, especially those (like In Memory of Kimsey M Fowler Sr) who have gone above and beyond in diagnosing things. Thanks to all of your efforts, I think we can be relatively confident that the issue is directly related to NVidia GPU drivers on Windows 7.

I realize it sounds trite, but I'll say it anyway: Thanks again for your patience. We really do appreciate you volunteering your computers, and for putting up with us in our off moments.

I believe you are barking up the wrong tree but since you are the one doing the barking feel free. I have ONE Nvidia gpu but over 8 AMD gpu's and I do not believe my pc with the Nvidia gpu in it was crunching for Rosetta at the time of my departure, yet ALL but one of my pc's had the same problems that others are seeing! The one pc that did not have any problems was a pc that did not have a gpu in it that crunched! I believe your problem is in how you are handling, or maybe NOT handling, the gpu drivers, both Nvidia AND AMD, and how they relate to Boinc in general. BUT I am NOT a programmer, I am just a guy with some disposable income and also a guy that 'does' pc's as a hobby.