CASP16

Message boards : Rosetta@home Science : CASP16

To post messages, you must log in.

AuthorMessage
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109015 - Posted: 18 Mar 2024, 8:35:25 UTC
Last modified: 18 Mar 2024, 8:36:03 UTC

CASP (Critical Assessment of Structure Prediction) experiments are held every two years. Recent rounds have seen dramatic increases in modeling accuracy, resulting from the introduction of deep learning methods: In 2018, for the first time, the folds of most proteins were correctly computed [1]; in 2020, the accuracy of many computed protein structures rivaled that of the corresponding experimental ones [2]; in 2022, there was an enormous increase in the accuracy of protein complexes [3].

We have seen the beginning of what deep learning methods may achieve in structural biology. In addition to further increases in the accuracy of protein complexes, methods are being developed for RNA structures, organic ligand-protein complexes, and for moving beyond single macromolecular structures to compute conformational ensembles. Accurate computational methods together with experimental data also offer the prospect of probing previously inaccessible biological systems. CASP has expanded its scope to provide critical assessment in all these areas.

CASP is only possible with the generous participation of the experimental structural biology community in providing suitable targets: A total of over 1100 targets have been obtained over the previous CASP rounds. We are now requesting targets for the 2024 CASP16 experiment.

The timeline for the 2024 CASP requires that targets are submitted starting now and until July 1. We would like to hear from you as soon as possible if you may have something suitable or have suggestions about other target sources. In order to maintain rigor, the experimental data for a target must not be publicly available until after computed structures have been collected. For assessment, CASP requires the experimental data by August 15, but the data can remain confidential after that. Target providers are invited to contribute to papers [11-15] for a special CASP issue of the journal Proteins.

ID: 109015 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109017 - Posted: 18 Mar 2024, 8:52:19 UTC

Timetable

April 2, 2024 - Start of the registration for CASP16 prediction experiment.
April 16, 2024 - Start of the testing of server connectivity ("dry run" for server predictors).
May 1, 2024 - Release of the first CASP16 modeling targets.
June/July 2024 - Early bird registration for the December CASP16 conference.
July 31, 2024 - Last date for releasing targets.
August 31, 2024 - End of the modeling season.
Early September 2024 - Collection of abstracts describing the methods used in CASP16.
August-October 2024 - Evaluation of predictions.
November 2024 - Invitations to groups with the most accurate models and the most interesting methods to give talks at the CASP16 conference.
November 2024 - Program of the conference finalized.
December 2024 - CASP16 Conference (tentatively Nov 30 - Dec 3).

ID: 109017 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109018 - Posted: 18 Mar 2024, 8:57:03 UTC
Last modified: 18 Mar 2024, 8:59:30 UTC

Organizing committee
John Moult, CASP chair and founder; IBBR, University of Maryland, USA
Krzysztof Fidelis, founder, University of California, Davis, USA
Andriy Kryshtafovych, University of California, Davis, USA
Torsten Schwede, University of Basel, Switzerland
Maya Topf, Centre for Structural Systems Biology, Hamburg, Germany

Scientific advisory board
Minkyung Baek, Seoul University, South Korea
David Baker, University of Washington, USA
Charlotte Dean, University of Oxford, UK
Nick Grishin, University of Texas, USA
Andrzej Joachimiak, Argonne National Lab, USA
David Jones, University College, London, UK
John Jumper, Google Deepmind, London, UK

ID: 109018 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109113 - Posted: 14 Apr 2024, 19:24:25 UTC
Last modified: 14 Apr 2024, 19:25:41 UTC

CASP16 registration is in full swing, with 83 groups already registered. The first CASP16 target will be released on Wednesday, May 1 at 9 am Pacific Time.

We will start a 'dry run' on checking format compliance and server connectivity on Tuesday, April 16. If you plan to enroll a server in CASP16, please do so as soon as possible so that we can test your server in the dry run and ensure flawless data transfer during the CASP16 prediction season. If you registered a human-expert group, you could also test the format of your model during the dry run by submitting a test prediction when the appropriate server test target is released.

The dry run will be conducted in four stages.

In the first stage (Tuesday-Thursday), we will test servers registered for protein and RNA tertiary structure prediction. We will release two test targets from the previous CASP - T1104 (protein) and R1117 (RNA) - and send their sequences to the registered servers. We expect the servers to return predictions in the requested TS format (https://predictioncenter.org/casp16/index.cgi?page=format#TS). Please check the Server Queries link from the main CASP16 web page (which will be available on Tuesday) to see the status of queries to your server. In case you notice any problems, please check your registration settings and contact us if in doubt. If you do not see your prediction(s) as accepted in our server status page (link Server Prediction Status) or through the Model Viewer /My CASP16 Profile pages, then either your prediction did not reach us, or was rejected due to an unregistered submission email or format issues.

On Thursday, we will start the second stage with testing automated predictors of protein and RNA oligomers. We will send you a query for a protein homodimer (T1124), protein heterodimer (H1106), and an RNA homodimer (R1107).

We will continue the dry run on Monday, April 22, with testing EMA servers on the released oligomeric targets.

And finally, starting Wednesday, April 24 we will be testing format for RNA oligomers (R1107) and protein-RNA(DNA) complexes (M1189). We would assume that your server can do RNA oligomers if it is registered for RNA prediction, and can do protein-RNA modeling if it is registered for both categories. Please write to us if this is not the case.

ID: 109113 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109151 - Posted: 23 Apr 2024, 7:02:32 UTC

Dear CASPers,

0. A reminder that CASP15 season will start in less than 2 weeks. We have already 125 groups registered. Last week we started testing servers for the connectivity and format compliance. The first two stages of the testing are currently running (tertiary structure and assembly for proteins and RNA). Next week we will continue with the EMA servers and servers for hybrid targets (e.g., protein-RNA complexes).

1. Two-stage oligomeric prediction.
This prediction season we are introducing a two-stage oligomeric prediction. First, we will release a target without stoichiometry information, and then with it. This will be a pilot experiment on a real-life modeling without knowing the protein aggregation state in advance. We will allow five models in each stage, but as it is common in CASP, the assessment will concentrate on the top-ranked prediction (Model 1). This experiment will be conducted on selected targets where we have enough prediction time. The whole prediction time for this kind of target tandems will be four weeks, and we would like to hear predictors' opinion on how to split this time between the two stages (default: two weeks each). Please discuss this and other CASP-related issues on the CASP16 Discord channels: https://discord.gg/mhnQCTVE.

2. ColabFold models and MSAs.
In CASP15, AlphaFold2 models generated by Arne Elofsson /Claudio Mirabello and ColabFold served as a very useful starting modeling point for predictors and the reference point for assessors. For CASP16, CASP organizers approached ColabFold group, and they graciously agreed to make their models and MSAs of CASP16 protein targets publicly available after closing the server prediction window (typically 3 days after the target release). The data will be provided for targets with known stoichiometry at http://casp16.colabfold.com .

3. Model 6 (not Tesla).
It has been suggested that it might help the assessors to untangle MSA effects from method features if there was one additional model from each participating group, generated using a reference MSA. Exploiting the availability of ColabFold MSA, we ask human-expert groups with MSA-based approaches to generate a special 'Model 6' based on this reference MSA. This model should be submitted alongside the five regular models per target, and it will receive a special treatment in the assessment.

4. Collecting MSAs.
In CASP16 we will be asking predictors with MSA-based methods to submit MSAs corresponding to their CASP16 models through a separate gateway. These will be used by assessors for additional tests /analyses. So please have your MSAs handy.

5. Re-release of difficult targets for SRA-based modeling (time allowing).
In late June - early July, when big portion of CASP modeling will be done, we will ask Martin Steinegger's group to generate MSAs for selected targets on the huge Sequence Read Archive (https://cshperspectives.cshlp.org/content/early/2024/02/05/cshperspect.a041465.abstract). Once generated, we will ask predictors to build models using those MSAs similarly to what we asked for the ColabFold MSAs (item 3 above).

6. MassiveFold for the EMA stage_2 experiment.
Björn Wallner's massive sampling approach proved to be one of the most successful methods in CASP15. Guillaume Brysbaert from the University of Lille implemented this approach in MassiveFold (https://github.com/GBLille/MassiveFold), however it remains out of reach for many groups due to high GPU requirements. For CASP16, Guillaume offered to generate 6000-9000 predictions per target (except for bigger ones) and make these available to predictors. We plan to use these models in the second stage of the EMA experiment by asking accuracy estimators to pick the best 5 models from the dataset and send their IDs as the stage_2 EMA predictions. More details to follow.

7. Protein-RNA-DNA complexes.
We have been collecting potential targets for the future round of CASP for a couple of months now, and already have a good starting set of targets. Among these, we already have two protein-RNA-DNA complexes, and two more have been promised. We have announced before that we may have this type of complexes in CASP16, and now we know that we will. With this, we ask predictors who have methods that are able to handle protein-RNA-DNA complexes to be ready for this kind of targets.

8. Ligand targets.
In addition to CASP targets with incidental ligands, we have secured four sets of ligand targets from pharma companies. Since each of these pharma 'super-targets' includes multiple ligand-receptor systems (e.g., one of the sets contains 200+ ligands for the same receptor), we will allot much more time for their prediction than we do for regular targets. We plan on releasing these four ligand super-targets in the first week of May, and these will be available for prediction until some time in July. Please check our format page for the latest LG format details: https://predictioncenter.org/casp16/index.cgi?page=format#LG

ID: 109151 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rilian
Avatar

Send message
Joined: 16 Jun 07
Posts: 13
Credit: 1,551,915
RAC: 9,955
Message 109156 - Posted: 23 Apr 2024, 16:04:44 UTC - in response to Message 109151.  

Dear CASPers,

0. A reminder that CASP15 season will start in less than 2 weeks. We have already 125 groups registered.


I tried but could not find the participants list. Is Rosetta@home participating? If so, on what dates should we put all efforts to the project? Would be fantastic if you provide your suggestions on this matter as you follow the process. Thank you!
i crunch for Ukraine. Join our team forums about Rosetta@home
ID: 109156 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109166 - Posted: 24 Apr 2024, 12:45:01 UTC - in response to Message 109156.  
Last modified: 24 Apr 2024, 12:47:06 UTC

I tried but could not find the participants list. Is Rosetta@home participating? If so, on what dates should we put all efforts to the project? Would be fantastic if you provide your suggestions on this matter as you follow the process. Thank you!


The list will be public at the beginning of competition.

If you see the list of organizators there is David Baker, the dad of Rosetta@home/IPD.
Usually IPD partecipate with 2 teams (automatic and manual simulations)
Sometimes, in the past, they distributed some works on R@H (with specific names, like "_casp_")
But it's years they do not

Obviously they, during the competition, use R@H protocols (and others)
ID: 109166 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rilian
Avatar

Send message
Joined: 16 Jun 07
Posts: 13
Credit: 1,551,915
RAC: 9,955
Message 109213 - Posted: 1 May 2024, 21:48:16 UTC
Last modified: 1 May 2024, 21:48:26 UTC

There are already some results uploaded and we can see 40 participant names https://predictioncenter.org/casp16/servers_distribution.cgi?server=-1&target=51&register=Submit&action=process&view=table

I do not know under which name Rosetta team will be participating yet
i crunch for Ukraine. Join our team forums about Rosetta@home
ID: 109213 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1873
Credit: 8,259,674
RAC: 8,187
Message 109219 - Posted: 2 May 2024, 15:19:36 UTC - in response to Message 109213.  

I do not know under which name Rosetta team will be participating yet


Maybe this edition they decide to not partecipate.

But seems strange
ID: 109219 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Rosetta@home Science : CASP16



©2024 University of Washington
https://www.bakerlab.org