Discovering Equations from Graph Neural Networks using Symbolic Regression
Sebastian Höfler
WW8, FAU
08. July 2022, 15:00
WW8, Zoom, Fürth
Symbolic regression algorithms that scale exponentially with the number of input variables and operators [21] are not suited for problems in quantum chemistry. Deep learning models, which are able to work effectively with these large and complex datasets, lack interpretability. This thesis builds on the work of Cranmer [3] and tries to extract symbolic representations from a heavily regularized trained deep learning framework. The aim is to obtain an interpretable analytic approximation for the energy U0 of a molecule by using symbolic regression on the internal message functions. We use different graph neural network architectures and apply a L1 – regularization parameter to reduce the number of of messages components in the network. The components are then approximated with symbolic regression. The hypothesis that applying a strong L1 -regularization to the network will result in information being contained within a few message components, was observed not to be true in this case. Symbolic regression performed on the message components was unable to find equations consistent with the underlying governing equations like (e. g. Kohn-Sham). Further research should focus on determining different ways to reduce the message components and examine if PySR is suitable for the complexity of this problem.
[3] Miles D. Cranmer et al. “Discovering Symbolic Models from Deep Learning
with Inductive Biases”. In: CoRR abs/2006.11287 (2020). arXiv: 2006.11287.
url: https://arxiv.org/abs/2006.11287.
[21] Michael Schmidt and Hod Lipson. “Distilling Free-Form Natural Laws from
Experimental Data”. In: Science 324.5923 (2009), pp. 81–85. doi: 10.1126/
science.1165893. eprint: https://www.science.org/doi/pdf/10.1126/
science.1165893. url: https://www.science.org/doi/abs/10.1126/
science.1165893.