Inquiry on Applying Causality Lab's Work to Protein Causal Inference #6
Description
Dear Causality Lab Team,
My name is Xavier, and I am a master student working on causal inference for proteins. We have been deeply inspired by your excellent work presented at NeurIPS 2023, particularly the causal interpretation of Transformer self-attention and the CLEANN algorithm .
Our project aims to build causal graphs from pre-trained protein language models (like ESM2) using protein sequences and structures to identify key functional residues. Our pipeline primarily involves:
Graph Construction: Building a causal DAG from ESM2's attention matrices, integrated with 3D structural priors.
Causal Inference: Estimating causal effects on the DAG to identify residues with crucial causal effects on protein function.
Validation: Evaluating the results using DMS (Deep Mutational Scanning) data and other benchmarks.
We note that your work provides a solid theoretical foundation for the idea that "the self-attention matrix can be viewed as an approximation of the total effect matrix in a Structural Causal Model (SCM)" . This strongly reinforces our research direction.
Therefore, we would be incredibly grateful to hear your insights:
What is your perspective on the potential and value of applying your framework (especially CLEANN and ABCD) to protein language models (like the ESM family)?
From your vantage point, what are the most promising research directions for building upon your work? For instance, developing protein-specific causal explanation methods or exploring more complex causal structures (e.g., protein-protein interactions).
We firmly believe that marrying cutting-edge causal inference with protein science holds great promise. Thank you very much for your time and consideration. We look forward to the possibility of learning from you.
Best regards,
Xavier
Zhengzhou University
xaviersivan22@gmail.com