The Journal
COMMUNICATION scitation.org/journal/jcp
of Chemical Physics
Δ-machine learning for potential energy surfaces:
A PIP approach to bring a DFT-based PES
to CCSD(T) level of theory
Cite as: J. Chem. Phys. 154, 051102 (2021); doi: 10.1063/5.0038301
Submitted: 20 November 2020 • Accepted: 14 January 2021 •
Published Online: 4 February 2021
Apurba Nandi,1,a) Chen Qu,2 Paul L. Houston,3,b) Riccardo Conte,4,c) and Joel M. Bowman1,d)
AFFILIATIONS
1
Department of Chemistry and Cherry L. Emerson Center for Scientific Computation, Emory University, Atlanta,
Georgia 30322, USA
2
Department of Chemistry and Biochemistry, University of Maryland, College Park, Maryland 20742, USA
3
Department of Chemistry and Chemical Biology, Cornell University, Ithaca, New York 14853, USA
and Department of Chemistry and Biochemistry, Georgia Institute of Technology, Atlanta, Georgia 30332, USA
4
Dipartimento di Chimica, Università Degli Studi di Milano, Via Golgi 19, 20133 Milano, Italy
Note: This paper is part of the JCP Special Topic on Quantum Dynamics with Ab Initio Potentials.
a)
Electronic mail:
b)
Electronic mail:
c)
Electronic mail:
d)
Author to whom correspondence should be addressed:
ABSTRACT
“Δ-machine learning” refers to a machine learning approach to bring a property such as a potential energy surface (PES) based on low-
level (LL) density functional theory (DFT) energies and gradients close to a coupled cluster (CC) level of accuracy. Here, we present such
an approach that uses the permutationally invariant polynomial (PIP) method to fit high-dimensional PESs. The approach is represented
by a simple equation, in obvious notation V LL→CC = V LL + ΔV CC–LL , and demonstrated for CH4 , H3 O+ , and trans and cis-N-methyl
acetamide (NMA), CH3 CONHCH3 . For these molecules, the LL PES, V LL , is a PIP fit to DFT/B3LYP/6-31+G(d) energies and gradients
and ΔV CC–LL is a precise PIP fit obtained using a low-order PIP basis set and based on a relatively small number of CCSD(T) energies.
For CH4 , these are new calculations adopting an aug-cc-pVDZ basis, for H3 O+ , previous CCSD(T)-F12/aug-cc-pVQZ energies are used,
while for NMA, new CCSD(T)-F12/aug-cc-pVDZ calculations are performed. With as few as 200 CCSD(T) energies, the new PESs are in
excellent agreement with benchmark CCSD(T) results for the small molecules, and for 12-atom NMA, training is done with 4696 CCSD(T)
energies.
Published under license by AIP Publishing. https://doi.org/10.1063/5.0038301., s
I. INTRODUCTION comparison with the experiment robust. Thus, it has only been
applied to triatomic molecules, and it is limited to these and possi-
Correcting ab initio-based potential energy surfaces (PESs) bly tetratomics. Another approach is to modify an existing poten-
has been a long-standing goal of computational chemistry. Sev- tial using scaling methods that go under the heading of “morph-
eral approaches dating from 30 years ago have been suggested. ing.”4–6 An impressive example is a PES for HCN/HNC reported
In one, a correction potential is added to an existing PES, and by Tennyson and co-workers7 who morphed a CCSD(T)-based
parameters of the correction potential are optimized by matching PES.8
ro-vibrational energies to the experiment.1–3 This approach relies More recent approaches using machine learning (ML) aim to
on being able to calculate exact rovibrational energies to make the bring a PES based on a low level of electronic theory to a higher
J. Chem. Phys. 154, 051102 (2021); doi: 10.1063/5.0038301 154, 051102-1
Published under license by AIP Publishing
, The Journal
COMMUNICATION scitation.org/journal/jcp
of Chemical Physics
level. As the field moves to consideration of larger molecules and Unlike H3 O+ and CH4 , for NMA, there is no previous
clusters where high-level methods are prohibitively expensive, the CCSD(T)-based PES, and so the present CCSD(T)-corrected one is,
motivation for doing this is obvious. There are two classes of such we believe, the most accurate one available.
approaches, one is “Δ-machine learning” (Δ-ML) and the other
is “transfer learning.”9 Δ-ML, which is of direct relevance to the
present paper, seeks to add a correction to a property obtained using II. COMPUTATIONAL DETAILS
an efficient and thus perforce low-level ab initio theory.10–15 This In order to develop a corrected PES, we need to generate a
approach includes an interesting, recent variant based on a “Pople” dataset of high and low-level energies for training and testing. In
style composite approach.11 In this sense, the approach is related, in this study, we need both DFT and CCSD(T) datasets. Training is
spirit at least, to the correction potential approach mentioned above, done for the correction PES ΔV CC–LL , and testing is done for the
when the property is the PES. However, it is applicable to much corrected V LL→CC . Do note that this two-step “training and testing”
larger molecules. is on different datasets. Our objective is to see the impact of the train-
The transfer learning approach has been developed exten- ing dataset size on the fidelity of the corrected PES V LL→CC for CH4
sively in the context of neural networks,9 and so much of the and H3 O+ .
work in that field has been brought into chemistry.12–16 The idea For H3 O+ , CCSD(T) energies are available from our previously
of transfer learning comes from the fact that knowledge gained reported PES, which is a fit to 32 142 CCSD(T)/aug-cc-pVQZ ener-
from solving one problem can often be used to solve another gies.28 From this large dataset, we select 1000 configurations with
related problem. Therefore, a model learned for one task, e.g., energies in the range 0–24 000 cm−1 for new DFT calculations of
a ML-PES fit to low-level electronic energies/gradients, can be energies and gradients. These are done at the efficient B3LYP/6-
reused as the starting point of the model for a different task, e.g., 311+G(d,p) level of theory, using the Molpro quantum chemistry
an ML-PES with the accuracy of a high-level electronic structure package.29 Histograms of the distributions of DFT energies are given
theory. in supplementary material. Note that these DFT configurations span
Most work using transfer learning or Δ-ML has been on devel- the same large range of configurations as the much larger CCSD(T)
oping general transferable force fields with application mainly in ones but have less dense sampling.
the area of thermochemistry and molecular dynamics simulations For CH4 , we take the DFT datasets from our recently reported
at room temperature and somewhat higher. Käser et al. used trans- work where the total of 9000 energies and their corresponding gra-
fer learning to improve neural network PESs for malonaldehyde, dients were generated from ab initio molecular dynamics (AIMD)
acetoacetaldehyde, and acetylacetone.15
Here, we report a Δ-ML approach for PESs using the permu-
tationally invariant polynomial (PIP) approach. The PIP approach
has been applied to many PESs for molecules, including chem-
ical reactions, dating back roughly 15 years. For reviews, see
Refs. 17–19. Recent extensions of the PIP software to incorpo-
rate electronic gradients20,21 have extended the PIP approach to
amino acids (glycine)22 and molecules with 12 atoms and 15
atoms, e.g., N-methyl acetamide,21,23,24 tropolone,25 and acetylace-
tone,26 respectively. As is widely appreciated in the field, incor-
porating gradients into fitting requires efficient, low-level elec-
tronic structure methods, such as density functional theory or
MP2, as these provide analytical gradients.27 These levels of the-
ory were used for the PES fits of the three molecules mentioned
above.
Our approach is given by the simple equation
VLL→CC ≙ VLL + ΔVCC−LL , (1)
where V LL→CC is the corrected PES, V LL is a PES fit to low-level
density functional theory (DFT) electronic data, and ΔV CC–LL is the
correction PES based on high-level coupled cluster energies. The
assumption underlying the hoped-for small number of high-level
energies is that the difference ΔV CC–LL is not as strongly varying as
V LL with respect to the nuclear configuration.
We demonstrate the efficacy and high-fidelity of this approach
for two small molecules, H3 O+ and CH4 , and for 12-atom N-methyl
acetamide (NMA). In all cases, V LL is a PIP fit to DFT energies and FIG. 1. Plot of ΔV CC–LL (relative to the reference value, i.e., −12 110 cm−1 ) vs DFT
gradients, and ΔV CC–LL is a PIP fit to a much smaller database of energy relative to the H3 O+ minimum value with the indicated number of training
differences between CCSD(T) and DFT energies. datasets.
J. Chem. Phys. 154, 051102 (2021); doi: 10.1063/5.0038301 154, 051102-2
Published under license by AIP Publishing
COMMUNICATION scitation.org/journal/jcp
of Chemical Physics
Δ-machine learning for potential energy surfaces:
A PIP approach to bring a DFT-based PES
to CCSD(T) level of theory
Cite as: J. Chem. Phys. 154, 051102 (2021); doi: 10.1063/5.0038301
Submitted: 20 November 2020 • Accepted: 14 January 2021 •
Published Online: 4 February 2021
Apurba Nandi,1,a) Chen Qu,2 Paul L. Houston,3,b) Riccardo Conte,4,c) and Joel M. Bowman1,d)
AFFILIATIONS
1
Department of Chemistry and Cherry L. Emerson Center for Scientific Computation, Emory University, Atlanta,
Georgia 30322, USA
2
Department of Chemistry and Biochemistry, University of Maryland, College Park, Maryland 20742, USA
3
Department of Chemistry and Chemical Biology, Cornell University, Ithaca, New York 14853, USA
and Department of Chemistry and Biochemistry, Georgia Institute of Technology, Atlanta, Georgia 30332, USA
4
Dipartimento di Chimica, Università Degli Studi di Milano, Via Golgi 19, 20133 Milano, Italy
Note: This paper is part of the JCP Special Topic on Quantum Dynamics with Ab Initio Potentials.
a)
Electronic mail:
b)
Electronic mail:
c)
Electronic mail:
d)
Author to whom correspondence should be addressed:
ABSTRACT
“Δ-machine learning” refers to a machine learning approach to bring a property such as a potential energy surface (PES) based on low-
level (LL) density functional theory (DFT) energies and gradients close to a coupled cluster (CC) level of accuracy. Here, we present such
an approach that uses the permutationally invariant polynomial (PIP) method to fit high-dimensional PESs. The approach is represented
by a simple equation, in obvious notation V LL→CC = V LL + ΔV CC–LL , and demonstrated for CH4 , H3 O+ , and trans and cis-N-methyl
acetamide (NMA), CH3 CONHCH3 . For these molecules, the LL PES, V LL , is a PIP fit to DFT/B3LYP/6-31+G(d) energies and gradients
and ΔV CC–LL is a precise PIP fit obtained using a low-order PIP basis set and based on a relatively small number of CCSD(T) energies.
For CH4 , these are new calculations adopting an aug-cc-pVDZ basis, for H3 O+ , previous CCSD(T)-F12/aug-cc-pVQZ energies are used,
while for NMA, new CCSD(T)-F12/aug-cc-pVDZ calculations are performed. With as few as 200 CCSD(T) energies, the new PESs are in
excellent agreement with benchmark CCSD(T) results for the small molecules, and for 12-atom NMA, training is done with 4696 CCSD(T)
energies.
Published under license by AIP Publishing. https://doi.org/10.1063/5.0038301., s
I. INTRODUCTION comparison with the experiment robust. Thus, it has only been
applied to triatomic molecules, and it is limited to these and possi-
Correcting ab initio-based potential energy surfaces (PESs) bly tetratomics. Another approach is to modify an existing poten-
has been a long-standing goal of computational chemistry. Sev- tial using scaling methods that go under the heading of “morph-
eral approaches dating from 30 years ago have been suggested. ing.”4–6 An impressive example is a PES for HCN/HNC reported
In one, a correction potential is added to an existing PES, and by Tennyson and co-workers7 who morphed a CCSD(T)-based
parameters of the correction potential are optimized by matching PES.8
ro-vibrational energies to the experiment.1–3 This approach relies More recent approaches using machine learning (ML) aim to
on being able to calculate exact rovibrational energies to make the bring a PES based on a low level of electronic theory to a higher
J. Chem. Phys. 154, 051102 (2021); doi: 10.1063/5.0038301 154, 051102-1
Published under license by AIP Publishing
, The Journal
COMMUNICATION scitation.org/journal/jcp
of Chemical Physics
level. As the field moves to consideration of larger molecules and Unlike H3 O+ and CH4 , for NMA, there is no previous
clusters where high-level methods are prohibitively expensive, the CCSD(T)-based PES, and so the present CCSD(T)-corrected one is,
motivation for doing this is obvious. There are two classes of such we believe, the most accurate one available.
approaches, one is “Δ-machine learning” (Δ-ML) and the other
is “transfer learning.”9 Δ-ML, which is of direct relevance to the
present paper, seeks to add a correction to a property obtained using II. COMPUTATIONAL DETAILS
an efficient and thus perforce low-level ab initio theory.10–15 This In order to develop a corrected PES, we need to generate a
approach includes an interesting, recent variant based on a “Pople” dataset of high and low-level energies for training and testing. In
style composite approach.11 In this sense, the approach is related, in this study, we need both DFT and CCSD(T) datasets. Training is
spirit at least, to the correction potential approach mentioned above, done for the correction PES ΔV CC–LL , and testing is done for the
when the property is the PES. However, it is applicable to much corrected V LL→CC . Do note that this two-step “training and testing”
larger molecules. is on different datasets. Our objective is to see the impact of the train-
The transfer learning approach has been developed exten- ing dataset size on the fidelity of the corrected PES V LL→CC for CH4
sively in the context of neural networks,9 and so much of the and H3 O+ .
work in that field has been brought into chemistry.12–16 The idea For H3 O+ , CCSD(T) energies are available from our previously
of transfer learning comes from the fact that knowledge gained reported PES, which is a fit to 32 142 CCSD(T)/aug-cc-pVQZ ener-
from solving one problem can often be used to solve another gies.28 From this large dataset, we select 1000 configurations with
related problem. Therefore, a model learned for one task, e.g., energies in the range 0–24 000 cm−1 for new DFT calculations of
a ML-PES fit to low-level electronic energies/gradients, can be energies and gradients. These are done at the efficient B3LYP/6-
reused as the starting point of the model for a different task, e.g., 311+G(d,p) level of theory, using the Molpro quantum chemistry
an ML-PES with the accuracy of a high-level electronic structure package.29 Histograms of the distributions of DFT energies are given
theory. in supplementary material. Note that these DFT configurations span
Most work using transfer learning or Δ-ML has been on devel- the same large range of configurations as the much larger CCSD(T)
oping general transferable force fields with application mainly in ones but have less dense sampling.
the area of thermochemistry and molecular dynamics simulations For CH4 , we take the DFT datasets from our recently reported
at room temperature and somewhat higher. Käser et al. used trans- work where the total of 9000 energies and their corresponding gra-
fer learning to improve neural network PESs for malonaldehyde, dients were generated from ab initio molecular dynamics (AIMD)
acetoacetaldehyde, and acetylacetone.15
Here, we report a Δ-ML approach for PESs using the permu-
tationally invariant polynomial (PIP) approach. The PIP approach
has been applied to many PESs for molecules, including chem-
ical reactions, dating back roughly 15 years. For reviews, see
Refs. 17–19. Recent extensions of the PIP software to incorpo-
rate electronic gradients20,21 have extended the PIP approach to
amino acids (glycine)22 and molecules with 12 atoms and 15
atoms, e.g., N-methyl acetamide,21,23,24 tropolone,25 and acetylace-
tone,26 respectively. As is widely appreciated in the field, incor-
porating gradients into fitting requires efficient, low-level elec-
tronic structure methods, such as density functional theory or
MP2, as these provide analytical gradients.27 These levels of the-
ory were used for the PES fits of the three molecules mentioned
above.
Our approach is given by the simple equation
VLL→CC ≙ VLL + ΔVCC−LL , (1)
where V LL→CC is the corrected PES, V LL is a PES fit to low-level
density functional theory (DFT) electronic data, and ΔV CC–LL is the
correction PES based on high-level coupled cluster energies. The
assumption underlying the hoped-for small number of high-level
energies is that the difference ΔV CC–LL is not as strongly varying as
V LL with respect to the nuclear configuration.
We demonstrate the efficacy and high-fidelity of this approach
for two small molecules, H3 O+ and CH4 , and for 12-atom N-methyl
acetamide (NMA). In all cases, V LL is a PIP fit to DFT energies and FIG. 1. Plot of ΔV CC–LL (relative to the reference value, i.e., −12 110 cm−1 ) vs DFT
gradients, and ΔV CC–LL is a PIP fit to a much smaller database of energy relative to the H3 O+ minimum value with the indicated number of training
differences between CCSD(T) and DFT energies. datasets.
J. Chem. Phys. 154, 051102 (2021); doi: 10.1063/5.0038301 154, 051102-2
Published under license by AIP Publishing
