Tutorial

Running Experiments

  1. Activate Environment

First, activate the project environment:

conda activate opengt

  1. Run a Single Experiment

Use main.py along with a configuration file to launch an experiment. Configuration files are stored in: configs/<ModelName>/<DatasetName-ModelName>.yaml

Example: Run DIFFormer on Cora (3 runs)

python main.py --cfg configs/DIFFormer/cora-DIFFormer.yaml --repeat 3

Results will be saved automatically to: results/DIFFormer/cora-DIFFormer/

  1. Hyperparameter Tuning

To perform grid search on hyperparameters, use the provided run_batch.sh script and corresponding grid files under the grids/ directory.

bash run_batch.sh

Please note that the grid file and the configuration file names should be modified in the script.

Each line of the grid file should be written in the following format:

<Config parameter name> <Display name> <List of possible values>

For example:

gt.layers nlayer [1,2,3,4]
gt.aggregate agg ['add', 'cat']
gt.dropout dropout [0.2, 0.5, 0.8]

This will explore all parameter combinations defined in the relevant grid configuration.

  1. Aggregating Results

To summarize and aggregate results from multiple runs:

python agg_test.py

This script collects results across seeds and outputs averaged performance metrics with standard deviations in to a .csv file.