Command line usage

Model formats

Onnx

ReX natively understands onnx files. Train or download a model (e.g. Resnet50) and, from this directory, run:

ReX tests/test_data/dog.jpg --model resnet50-v1-7.onnx -vv --output dog_exp.jpg

Pytorch

ReX also works with PyTorch, but you will need to write some custom code to provide ReX with the prediction function and model shape, as well as preprocess the input data. See the sample scripts in scripts/.

ReX tests/test_data/dog.jpg --script scripts/pytorch.py -vv --output dog_exp.jpg

Saving output in a database

To store all output in a sqlite database, use:

ReX <path_to_image> --model <path_to_model> -db <name_of_db_and_extension>

ReX will create the db if it does not already exist. It will append to any db with the given name, so be careful not to use the same database if you are restarting an experiment. ReX comes with a number of database functions which allow you to load it as a pandas dataframe for analysis.

Config

ReX looks for the config file rex.toml in the current working directory and then $HOME/.config/rex.toml on unix-like systems.

If you want to use a custom location, use:

ReX <path_to_image> --model <path_to_model> --config <path_to_config>

An example config file is included in the repo as example.rex.toml. Rename this to rex.toml if you wish to use it. ReX will ignore the file example.rex.toml is you do not rename it.

Overriding the config

Some options from the config file can be overridden at the command line when calling ReX. In particular, you can change the number of iterations of the algorithm:

ReX <path_to_image> --model <path_to_model> --iters 5

Preprocessing

Input data should be transformed in the same way the model’s training data was transformed before training. For PyTorch models, you should specify the preprocessing steps in the custom script. See the sample scripts in scripts/ for examples of using models provided by the torchvision and timm packages. You can also write a script to load an onnx model as well if you have custom preprocessing to do.

Otherwise, ReX tries to make reasonable guesses for image preprocessing. This includes resizing the image to match that needed for the model, converting it to a PyTorch tensor, and normalising the data. This can be controlled in rex.toml. In the event the model input is multi-channel and the image is greyscale, then ReX will convert the image to pseudo-RGB. If you want more control over the conversion, you can do the conversion yourself and pass in the converted image.

Occlusion Strategy

ReX supports a number of occlusion strategies, which can be set in the config file or overridden at the command line.
The default occlusion value is 0, which could mean a number of things, depending on the input data type, the most common being a black square.

There are the following occlusion strategies:

  • <value>, which any integer or float value that is within the range of the input data type using the flag --mask_value <value>.
    For example, for an image, this could be 0, 255, or 1.0.

  • min, which will use the minimum of the input data using the flag --mask_value min.
    For example, for an image data with minimum value 0.01 in the image, this could be 0.

  • mean, which will use the mean of the input data using the mask_value mean.

  • spectral, which is a special occlusion strategy that uses linear interpolation for spectral data with some additional noise that can be added.
    This is useful for spectral data and can be set using the --mask_value spectral flag and be in the mode --mode spectral.

  • none, which will use the tt.nan value for the occlusion. The flag --mask_value none will set the occlusion to tt.nan.
    This is useful for models that can handle missing data, such as some neural networks.

  • random, which will use a random value from the input data using the flag --mask_value random.

  • linear, which will choose a value incrementing from the minimum to the maximum of the input data using the flag --mask_value linear.

  • context, A context data is the same shape as the input, which will be used to superimposed with the mask. This will use the context data to fill in the occluded area.
    The context data’s path needs to be provided using the --context flag and provide noise value using the --noise flag. The context data will be preprocessed the same way as the input. Example usage:

 ReX tests/test_data/dog_hide.jpg --script scripts/pytorch.py -v --context tests/test_data/park.jpg --noise 5 --output exp_context_dog.png