ERROR: java.lang.NullPointerException (path reaction has no structure)

[rajeshdparmar]

One of the job (out of 6 submitted jobs [3 jobs PM3 ON and 3 jobs PM3 off]) has given me following error. Condition file and error message is as follows: (Note: It is a case of PM3 OFF but pdep is ON)

Error Message

Edge species SPC(34021) has a low maximum flux ratio (5.7e-06) and will be pruned to reduce the edge size to the maximum (15000).
Edge species SPC(43569) has a low maximum flux ratio (5.7e-06) and will be pruned to reduce the edge size to the maximum (15000).
Edge species SPC(43574) has a low maximum flux ratio (5.7e-06) and will be pruned to reduce the edge size to the maximum (15000).
Pruning...
Removed path reverse SPC(35105)=SPC(35107) from 0 dictionaries
Removed path reverse SPC(15029)=SPC(35107) from 0 dictionaries
Removed path reverse SPC(15029)=SPC(35117) from 0 dictionaries
Removed path reverse SPC(15029)=SPC(35097) from 0 dictionaries
Removed path reverse SPC(15029)=SPC(35101) from 0 dictionaries
Removed path reverse SPC(33335)=SPC(36791) from 0 dictionaries
Removed path reverse SPC(37616)=SPC(47936) from 0 dictionaries
Removed path reverse SPC(37616)=SPC(47946) from 0 dictionaries
Removed path reverse SPC(37616)=SPC(47965) from 0 dictionaries
ERROR: java.lang.NullPointerException
    at jing.rxn.Reaction.getReactants(Reaction.java:1091)
    at jing.rxnSys.ReactionModelGenerator.reactionPrunableQ(ReactionModelGenerator.java:4223)
    at jing.rxnSys.ReactionModelGenerator.pruneReactionModel(ReactionModelGenerator.java:4111)
    at jing.rxnSys.ReactionModelGenerator.modelGeneration(ReactionModelGenerator.java:1458)
    at RMG.main(RMG.java:96)

Exception in thread "main" java.lang.NullPointerException
        at jing.rxnSys.Logger.log(Logger.java:160)
        at jing.rxnSys.Logger.critical(Logger.java:204)
        at RMG.main(RMG.java:106)

condition file

Database: RMG_database

MaxCarbonNumberPerSpecies: 20
//MaxOxygenNumberPerSpecies:
MaxRadicalNumberPerSpecies: 4
MaxCycleNumberPerSpecies: 100
//MaxSulfurNumberPerSpecies:
//MaxSiliconNumberPerSpecies:
//MaxHeavyAtomPerSpecies:

PrimaryThermoLibrary:
Name: RMG_Default
Location: primaryThermoLibrary
Name: DFT_QCI        // used from Dr. Richard West condition file
Location: DFT_QCI_thermo
Name: GRI-Mech3.0
Location: GRI-Mech3.0
END

// 1. Default - used to calculate species thermo properties from Benson's additivity rule
// 2. Rwest -  
// 3. DFT_QCI - unpublished high level quantem chemistry calcualtions from Green group.
// 4. GRI-Mech3.0 - Methane combustion optimized mechanism

PrimaryTransportLibrary:
Name: GRIMech3.0
Location: GRI-Mech3.0
END
// This is what RMG comes with


// Forbidden structures are used from Dr. Richard West's condition file 
ForbiddenStructures:
// In addition to PM3, B3LYP also seems to want to break this up into CO2 + CO2 + CO.
// No record currently in SciFinder, the closest analog is a C3O5 with
// slightly different structure discussed in doi:10.1016/S0166-1280(00)00601-1
// It caused Gaussian03 to seg-fault 35 times in a row and then freeze, so I'm forbidding
// it so that I can move on.
O=C=C1OOC(=O)O1
1  C 0 {2,S} {3,D} {5,S}
2  O 0 {1,S} {6,S}
3  O 0 {1,D}
4  C 0 {5,S} {6,S} {7,D}
5  O 0 {4,S} {1,S}
6  O 0 {4,S} {2,S}
7  C 0 {8,D} {4,D}
8  O 0 {7,D}

//   O   this structure kills the QM calcs
//  / \  and it is nowhere to be found
//  C=C  in SciFinder, so I am
//  | |  assuming it doesn't exist
//  O-O  and I am forbidding it.
//  also, Greg reports that it's a saddle point at the PM3 level
o1oc2Oc12
1  C 0 {2,S} {3,S} {4,D}
2  O 0 {1,S} {5,S}
3  O 0 {1,S} {4,S}
4  C 0 {5,S} {1,D} {3,S}
5  O 0 {4,S} {2,S}

END

ReadRestart: no
WriteRestart: yes

TemperatureModel: Constant (K) 673.15 873.15 1073.15 1273.15 
PressureModel: Constant (atm) 1.35 5.0

InChIGeneration: off

// Quantum calculations for cyclic species. obtained from Dr. Richard West (MIT)
// ThermoMethod: QM both
// QMForCyclicsOnly: on
// MaxRadNumForQM: 0 // with more than this number of radicals, HBI method is used.

InitialStatus:

(1) TD (mol/m3) 0.6033
1 C 0 {3,S}
2 C 0 {4,S}
3 C 0 {1,S} {5,S}
4 C 0 {2,S} {6,S}
5 C 0 {3,S} {7,S}
6 C 0 {4,S} {8,S}
7 C 0 {5,S} {9,S}
8 C 0 {6,S} {10,S}
9 C 0 {7,S} {11,S}
10 C 0 {8,S} {12,S}
11 C 0 {9,S} {13,S}
12 C 0 {10,S} {14,S}
13 C 0 {11,S} {14,S}
14 C 0 {12,S} {13,S} 

(2) H2O (mol/m3) 12.6698
1 O 0 

(3) O2 (mol/m3) 1.6893
1 O 1 {2,S}
2 O 1 {1,S}

END

InertGas:
N2 (mol/m3) 10.6185
Ne (mol/m3) 0
He (mol/m3) 0
Ar (mol/m3) 0
END
// All other inert gases are kept zero as per the manual


// Obtained from Dr. Richard West (MIT)
SpectroscopicDataEstimator: FrequencyGroups
PressureDependence: ReservoirState  // ModifiedStrongCollision
PDepKineticsModel: Chebyshev 4 2
TRange: (K) 673.15 1473.15 6
PRange: (atm) 1 5 3
// Specify whether fame calculation should be re-run in the event a
// pdep rate exceeds the high-p-limit rate by > 2 (default is off)
DecreaseGrainSize: yes

FinishController:
(1) Goal ReactionTime: 0.6 (sec)
(2) Error Tolerance: 0.1

DynamicSimulator: DASSL
TimeStep: AUTOPRUNE
TerminationTolerance: 1.0E20
PruningTolerance: 1.0E-18
MinSpeciesForPruning: 1000
MaxEdgeSpeciesAfterPruning: 15000
Atol: 1e-20
Rtol: 1e-8

PrimaryKineticLibrary:
END
// Set to default

ReactionLibrary:
END
// set to default

// Seed Mechanism is obtained from Dr. Richard West (MIT)
SeedMechanism:
Name: GRI-Mech3.0
Location: GRI-Mech3.0
GenerateReactions: yes
END

ChemkinUnits:
Verbose: on
A: moles
Ea: kcal/mol

[rwest]

It looks to me that there must have been no structure in the reaction that was being checked by reactionPrunableQ. This occured when iterating over PathReactions from a PDepNetwork, when iterating over all Networks, in order to find which PathReactions to prune. Why could a PathReaction not have a structure? And why did nine path reactions in earlier PDepNetworks have reverse structures not in any ReactionTemplate dictionaries?

@rajeshdparmar, do you still have the log file and pruning folder? If so, please could you point us to the right folder on the server? (And if not, please run it again and keep the results.)

[rajeshdparmar]

Please find the log file in the following directory on the server
rajesh@pharos:~/Rajesh/New_jobs/MultiT_pdep_Prun_15000_single_concentration

[rwest]

Thanks Rajesh - I forgot to look in your New_jobs folder.
The reaction that gave the warning Removed path reverse SPC(35105)=SPC(35107) from 0 dictionaries is an intra_H_migration reaction, and it was created in both directions:

RMG.log:Created new intra_H_migration reaction: C14H23J(35105) --> C14H23J(35107)
RMG.log:Created new intra_H_migration reaction: C14H23J(35107) --> C14H23J(35105)

I guess something strange happened like we removed one direction, and its reverse, then tried to remove the other direction, and it's reverse. I'm not sure what happens with reverse reactions of families that are their own reverse. Perhaps @jwallen or @mrharper have a clue.

[rajeshdparmar]

Other job also failed due to similar error. Please find the folder for the job as follows:
rajesh@pharos:~/Rajesh/New_jobs/MultiT_pdep_Prun_15000$

[rwest]

The MultiT_pdep_Prun_15000 job ended:

Pruning...
Removed path reverse SPC(8815)=SPC(9089) from 0 dictionaries
Removed path reverse SPC(17408)=SPC(17818) from 0 dictionaries
Removed path reverse SPC(13368)=SPC(21650) from 0 dictionaries
Removed path reverse SPC(13368)=SPC(21647) from 0 dictionaries
ERROR: java.lang.NullPointerException
    at jing.rxn.Reaction.getReactants(Reaction.java:1091)
    at jing.rxnSys.ReactionModelGenerator.reactionPrunableQ(ReactionModelGenerator.java:4223)
    at jing.rxnSys.ReactionModelGenerator.pruneReactionModel(ReactionModelGenerator.java:4111)
    at jing.rxnSys.ReactionModelGenerator.modelGeneration(ReactionModelGenerator.java:1458)
    at RMG.main(RMG.java:96)

Looking at the last of these, (13368) has not been selected for pruning but (21647) has.
Again, the Removed path reverse <reaction> from 0 dictionaries all come from intra_H_migration, thus both forward and reverse are in the same reaction template dictionary. How many copies of each we have in various lists, in either direction, I'm not sure.

[rwest]

The reactions A->B and B->A are both created as intra_H_migration reactions. B->A is in a pdep network for B and A->B is in a separate pdep network for A.

First up for pruning is A->B.
ReactionModelGenerator.java lines 4156-4161 remove both A->B and its reverse from the intra_H_migration dictionary,
then lines 4162 and 4163 call .prune() on the forward and reverse reactions; this sets the structures equal to null.

Then it is B->A's turn for pruning.
Somehow in line 4147 the reaction.getStructure() returns a valid forward structure (the reverse of A->B, which got wiped, was a separate instance of the same B->A reaction - we never checked the global list of reactions for duplicates of this type of duplicate) . The reverse structure is found/generated from this, then both are removed from the intra_H_migration dictionary. The forward is found and removed (how? shouldn't it have been removed earlier?), but the reverse structure (A->B) is already missing because it was removed above. This gives a "Removed path reverse %s from 0 dictionaries" warning.

or, to get the NullPointerException...

When trying to decide if B->A can be pruned, we check reactionPrunableQ and find that it has no structure, and hence die. This would be because the .prune() call was on the same instance of the reaction as the one we've already pruned. (But why is the same reaction in two pdep networks?)

[mrharper]

Going back to @rwest question on the following:

RMG.log:Created new intra_H_migration reaction: C14H23J(35105) --> C14H23J(35107)
RMG.log:Created new intra_H_migration reaction: C14H23J(35107) --> C14H23J(35105)

These lines appear consecutively in the RMG.log file. The reason this appears in both directions is because the first structure is in the "reverse" direction (based on thermodynamics, since this reaction template is its own reverse). (Looking at the makeTemplateReaction() function in TemplateReaction class) The first reaction is made (and we print "Created new ..."). Since the structure as written is in the reverse direction:

• generateReverseForBackwardReaction is called
• calculateForwardRateConstant (line 330) is called
• makeTemplateReaction (line 1089) is called
• neither this structure(nor its reverse) is in the template library, so we make another reaction (and print "Created new ...")
• this structure is in the forward direction, we so add it (and its reverse) to the template library

Long story short, I do not think we have duplicate instances of any intra_H_migration reactions in the template library. We only get both directions printed to screen because the reverse structure was made first. If the forward was made first, we would only get one "Created new reaction" printed. MRH confirmed this by starting with only 1-butyl in a RMG simulation (only one print statement) and only 2-butyl in a RMG simulation (two print statements).

[rajeshdparmar]

Third job also filed due to similar error. Please find the folder for the job in:
rajesh@pharos:~/Rajesh/New_jobs/MultiT_pdep_Prun_15000_Reduced$

Pruning...
Removed path reverse SPC(45087)=SPC(77812) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(52449) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(77813) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(77801) from 0 dictionaries
Removed path reverse SPC(48364)=SPC(52432) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(45096) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(52447) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(52445) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(77799) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(77818) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(45088) from 0 dictionaries
Removed path reverse SPC(48364)=SPC(52442) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(45092) from 0 dictionaries
Removed path reverse SPC(48364)=SPC(52447) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(52450) from 0 dictionaries
Removed path reverse SPC(48364)=SPC(52458) from 0 dictionaries
Removed path reverse SPC(48364)=SPC(52441) from 0 dictionaries
Removed path reverse SPC(45087)=SPC(77802) from 0 dictionaries
ERROR: java.lang.NullPointerException
    at jing.rxn.Reaction.getReactants(Reaction.java:1091)
    at jing.rxnSys.ReactionModelGenerator.reactionPrunableQ(ReactionModelGenerator.java:4223)
    at jing.rxnSys.ReactionModelGenerator.pruneReactionModel(ReactionModelGenerator.java:4111)
    at jing.rxnSys.ReactionModelGenerator.modelGeneration(ReactionModelGenerator.java:1458)
    at RMG.main(RMG.java:96)

Exception in thread "main" java.lang.NullPointerException
        at jing.rxnSys.Logger.log(Logger.java:160)
        at jing.rxnSys.Logger.critical(Logger.java:204)
        at RMG.main(RMG.java:106)

[rwest]

Ok, so according to @mrharper's last comment, my assumption at the top of my previous comment is slightly wrong. Perhaps we make the same path reaction twice in different PDepNetworks? the second or third time we make it, we don't print the "Created new reaction" line.

[rwest]

Another example from @rajeshdparmar (listed as issue #188)
Folder ~/Rajesh/New_jobs/MultiT_PM3_pdep_Prun_15000

Error

Pruning...
ERROR: java.lang.NullPointerException
    at jing.rxn.Reaction.getReactants(Reaction.java:1091)
    at jing.rxnSys.ReactionModelGenerator.reactionPrunableQ(ReactionModelGenerator.java:4223)
    at jing.rxnSys.ReactionModelGenerator.pruneReactionModel(ReactionModelGenerator.java:4111)
    at jing.rxnSys.ReactionModelGenerator.modelGeneration(ReactionModelGenerator.java:1458)
    at RMG.main(RMG.java:96)

Exception in thread "main" java.lang.NullPointerException
        at jing.rxnSys.Logger.log(Logger.java:160)
        at jing.rxnSys.Logger.critical(Logger.java:204)
        at RMG.main(RMG.java:106)

[rajeshdparmar]

another job also failed
folder: ~/Rajesh/New_jobs/MultiT_PM3_pdep_Prun_15000_single_concentration$

Error

Pruning...
Removed path reverse SPC(6922)=SPC(41457) from 0 dictionaries
Removed path reverse SPC(6922)=SPC(41459) from 0 dictionaries
Removed path reverse SPC(6922)=SPC(41461) from 0 dictionaries
ERROR: java.lang.NullPointerException
    at jing.rxn.Reaction.getReactants(Reaction.java:1091)
    at jing.rxnSys.ReactionModelGenerator.reactionPrunableQ(ReactionModelGenerator.java:4223)
    at jing.rxnSys.ReactionModelGenerator.pruneReactionModel(ReactionModelGenerator.java:4111)
    at jing.rxnSys.ReactionModelGenerator.modelGeneration(ReactionModelGenerator.java:1458)
    at RMG.main(RMG.java:96)

Exception in thread "main" java.lang.NullPointerException
        at jing.rxnSys.Logger.log(Logger.java:160)
        at jing.rxnSys.Logger.critical(Logger.java:204)
        at RMG.main(RMG.java:106)

[rwest]

We seem to be stuck. Perhaps we could just wrap this in a try/catch block and carry on? @gmagoon, do you have any thoughts?

[gmagoon]

I haven't been following this too closely, but it sounds like maybe some additional debugging lines would be helpful?

[gmagoon]

I've added a number of debugging lines in ed86222 . If no-one sees any issue with these debugging lines or has anything else to add, I suggest that @rajeshdparmar try to rerun the problematic case(s) with this version.