umami.plotting_tools package#

Submodules#

umami.plotting_tools.eval_plotting_functions module#

Collection of plotting function for ftag performance plots.

umami.plotting_tools.eval_plotting_functions.plot_confusion_matrix(df_results: dict, tagger_name: str, class_labels: list, plot_name: str, colorbar: bool = True, show_absolute: bool = False, show_normed: bool = True, transparent: bool = False, dpi: int = 400) → None#

Plotting the confusion matrix for a given tagger.

Parameters:

df_results (dict) – Loaded pandas dataframe from the evaluation file
tagger_name (str) – Name of the tagger in the evaluation file
class_labels (list) – List of class labels used
plot_name (str) – Full path + name of the plot with the extension
colorbar (bool, optional) – Decide, if colourbar is shown or not, by default True
show_absolute (bool, optional) – Show the absolute, by default False
show_normed (bool, optional) – Show the output normed, by default True
transparent (bool, optional) – Decide, if the background is transparent or not, by default False
dpi (int, optional) – Sets a DPI value for the plot that is produced (mainly for png), by default 400

umami.plotting_tools.eval_plotting_functions.plot_fraction_contour(df_results_list: list, tagger_list: list, label_list: list, rejections_to_plot: list, plot_name: str, rejections_to_fix_list: list, marked_points_list: list | None = None, colour_list: list | None = None, linestyle_list: list | None = None, **kwargs)#

Plot contour plots for the given taggers for two rejections. The rejections are calulated with different fractions. If more than two rejections are available, the others need to be set to a fixed value.

Parameters:

df_results_list (list) – List of dicts with the results of evaluate_model.py inside for the different models which are to be plotted.
tagger_list (list) – List of the models/taggers that are to be plotted.
label_list (list) – List with the labels for the given taggers.
rejections_to_plot (list) – List with two elements. The elements are the names for the two rejections that are plotted against each other. For example, [“cjets”, “ujets”].
plot_name (str) – Path, Name and format of the resulting plot file.
rejections_to_fix_list (list) – List of dicts with the extra rejections. If more than two rejections are available, you need to fix the other rejections to a specific value. The dict entry key is the name of the rejection, for example “bbjets”, and the entry is the value that it is set to, for example 0.2.
marked_points_list (list, optional) – List with marker dicts for each model provided. The dict contains the information needed to plot the marker, like the fraction values, which colour is used etc, by default None
colour_list (list, optional) – List with colours for the given taggers, by default None
linestyle_list (list, optional) – List with linestyles for the given taggers, by default None
**kwargs (kwargs) – kwargs for fraction_scan_plot function

Raises:

IndexError – If the given number of tagger names, labels and data dicts are not the same.
ValueError – If the given marker values summed (together with the fixed rejection values) is not equal to one (or not on the plotted curve).

umami.plotting_tools.eval_plotting_functions.plot_prob(df_list: list, model_labels: list, tagger_list: list, class_labels_list: list, flavour: str, plot_name: str, **kwargs) → None#

Plot the probability output for the given flavour for one/multiple models with a ratio plot in a subpanel (only if multiple models are provided).

Parameters:

df_list (list) – List of pandas DataFrames with the probability values inside
model_labels (list) – List of labels for the given models
tagger_list (list) – List of tagger names
class_labels_list (list) – List with the class_labels used in the different taggers
flavour (str) – Probability of this flavour is plotted.
plot_name (str) – Path, Name and format of the resulting plot file.
**kwargs (kwargs) – kwargs for HistogramPlot function

umami.plotting_tools.eval_plotting_functions.plot_roc(df_results_list: list, tagger_list: list, rej_class_list: list, labels: list, plot_name: str, main_class: str = 'bjets', df_eff_key: str = 'effs', draw_errors: bool = True, labelpad: int | None = None, working_points: list | None = None, same_height_working_points: bool = True, linestyles: list | None = None, colours: list | None = None, n_test: list | None = None, reference_ratio: list | None = None, **kwargs)#

Plotting the rejection curve for a given background class for the given models/taggers against the main_class efficiency. A ratio plot (first given model is the reference) is plotted in a subpanel underneath.

Parameters:

df_results_list (list) – List of dicts with the results of evaluate_model.py inside for the different models which are to be plotted.
tagger_list (list) – List of the models/taggers that are to be plotted.
rej_class_list (list) – List of the class rejection which is to be plotted for each model.
labels (list) – List of labels for the given models.
main_class (str) – The main discriminant class. For b-tagging obviously “bjets”
plot_name (str) – Path, Name and format of the resulting plot file.
df_eff_key (str, optional) – Dict key under which the efficiencies of the main class are saved, by default “effs”
draw_errors (bool, optional) – Binominal errors on the lines, by default True
labelpad (int, optional) – Spacing in points from the axes bounding box including ticks and tick labels, by default None
working_points (list, optional) – List of working points which are to be plotted as vertical lines in the plot, by default None
same_height_working_points (bool, optional) – Decide, if all working points lines have the same height or not, by default True
linestyles (list, optional) – List of linestyles to use for the given models, by default None
colours (list, optional) – List of linecolors to use for the given models, by default None
n_test (list, optional) – A list of the same length as rej_class_list, with the number of events used to calculate the background efficiencies. We need this To calculate the binomial errors on the background rejection, using the formula given by http://home.fnal.gov/~paterno/images/effic.pdf, by default 0
reference_ratio (list, optional) – List of bools indicating which roc used as reference for ratio calculation, by default None
**kwargs (kwargs) – kwargs passed to RocPlot

Raises:

ValueError – if n_test not int, float of given for each roc
ValueError – if lists don’t have the same length

umami.plotting_tools.eval_plotting_functions.plot_saliency(maps_dict: dict, plot_name: str, target_eff: float, jet_flavour: str, PassBool: bool, nFixedTrks: int, cmap: str = 'PiYG', **kwargs)#

Plot the saliency map given in maps_dict.

Parameters:

maps_dict (dict) – Dict with the saliency values inside
plot_name (str) – Path, Name and format of the resulting plot file.
target_eff (float, optional) – Working point to use, by default 0.77
jet_flavour (str, optional) – Class which is to be plotted, by default “bjets”
PassBool (bool, optional) – Decide, if the jets need to pass (True) or fail (False) the working point cut, by default True
nFixedTrks (int, optional) – Decide, how many tracks the jets need to have, by default 8
cmap (str, optional) – Colour map to use for the saliency map. By defaulkt PiYG
**kwargs – Keyword arguments handed to the plotting API

Raises:

ValueError – If the number of variables in the given dict is not the same as the number of variables used to calculate the saliency map.

umami.plotting_tools.eval_plotting_functions.plot_score(df_list: list, model_labels: list, tagger_list: list, class_labels_list: list, main_class: str, plot_name: str, working_points: list | None = None, same_height_working_points: bool = False, **kwargs) → None#

Plot the tagging discriminant scores for the evalutated jets.

Parameters:

df_list (list) – List with the pandas DataFrames inside.
model_labels (list) – List of labels for the given models
tagger_list (list) – List of tagger names of the used taggers
class_labels_list (list) – A list of the class_labels which are used.
main_class (str) – The main discriminant class. For b-tagging obviously “bjets”
plot_name (str) – Path, Name and format of the resulting plot file.
working_points (list, optional) – List of workings points which are plotted as vertical lines. By default, the working points for the first provided model is calculated and plotted, not for the rest! By default None
same_height_working_points (bool, optional) – Decide, if all working points lines have the same height or not, by default True
**kwargs (kwargs) – kwargs for HistogramPlot function

umami.plotting_tools.eval_plotting_functions.plot_var_vs_eff(df_list: list, tagger_list: list, model_labels: list, plot_name: str, class_labels: list, main_class: str, flavour: str, variable: str | None = None, working_point: float = 0.77, disc_cut: float | None = None, fixed_eff_bin: bool = False, bin_edges: list | None = None, working_point_line: bool = False, grid: bool = False, colours: list | None = None, alpha: float = 0.8, linewidth: float = 1.6, **kwargs) → None#

For a given list of models, plot the b-eff, l and c-rej as a function of jet pt.

Parameters:

df_list (list) – List of dicts with the results of evaluate_model.py inside for the different models which are to be plotted.
tagger_list (list) – List of the models/taggers that are to be plotted.
model_labels (list) – List of the labels which are to be used in the plot.
plot_name (str) – Path, Name and format of the resulting plot file.
class_labels (list) – A list of the class_labels which are used.
main_class (str) – The main discriminant class (= signal class). For b-tagging obviously “bjets”
flavour (str) – Class that is to be plotted. For all non-signal classes, this will be the rejection and for the signal class, this will be the efficiency.
variable (str, optional) – The variable against which the efficiency/rejection is plotted. By default None, which will use the pT as variable in GeV.
working_point (float, optional) – Working point which is to be used, by default 0.77.
disc_cut (float, optional) – Set a discriminant cut value for all taggers/models.
fixed_eff_bin (bool, optional) – Calculate the WP cut on the discriminant per bin, by default None.
bin_edges (list, optional) – As the name says, the edges of the bins used. Will be set automatically, if None. By default None.
working_point_line (bool, optional) – Print a WP line in the upper plot, by default False.
grid (bool, optional) – Use a grid in the plots, by default False
colours (list, optional) – Custom colour list for the different models, by default None
alpha (float, optional) – Value for visibility of the plot lines, by default 0.8
linewidth (float, optional) – Define the linewidth of the plotted lines, by default 1.6
**kwargs (kwargs) – kwargs for VarVsEffPlot function

Raises:

ValueError – If deprecated options are given.

umami.plotting_tools.preprocessing_plotting_functions module#

Plotting functions for the preprocessing.

umami.plotting_tools.preprocessing_plotting_functions.plot_resampling_variables(concat_samples: dict, var_positions: list, variable_names: list, sample_categories: list, output_dir: str, bins_dict: dict, sample_id_position: int = 3, fileformat: str = 'pdf', **kwargs) → None#

Plot the variables which are used for resampling before the resampling starts.

Parameters:

concat_samples (dict) – Dict with the format given in the Undersampling class by the class object concat_samples.
var_positions (list) – The position where the variables are stored in the sub-dict jets.
variable_names (list) – The name of the 2 variables which will be plotted.
sample_categories (list) – List with the names of the sample categories (e.g. [“ttbar”, “zprime”]).
output_dir (str) – Name of the output directory where the plots will be saved.
bins_dict (dict) – Dict with the binning for the resampling variables. First key must be the variable name with a tuple of 3 int which gives the lower limit, upper limit and the number of bins to use.
sample_id_position (int, optional) – Position in the numpy.ndarray of the concat_samples where the sample id is stored. By default 3
fileformat (str, optional) – Format of the plot file, by default “pdf”.
**kwargs (kwargs) – kwargs from plot_object

Raises:

ValueError – If unsupported binning is provided.

umami.plotting_tools.preprocessing_plotting_functions.plot_unique_jet_appearence(sample: str, class_labels: list, output_dir: str, n_jets: int, fileformat: str = 'pdf', **kwargs) → None#

Plot a histogram which shows how often the jets are duplicated. This is done per class.

Parameters:

sample (str) – Path to the resampled file (after resampling before writing)
class_labels (list) – List with the class labels used (flavours)
output_dir (str) – Path where the plot wil be stored. The filename is always Duplicated_jet_multiplicity + the format of the file.
n_jets (int) – Number of jets used for the histogram
fileformat (str, optional) – Fileformat used for the plot, by default “pdf”
**kwargs (kwargs) – kwargs from plot_object

umami.plotting_tools.preprocessing_plotting_functions.plot_variable(df_in, labels: ndarray, variable: str, variable_index: int, var_type: str, class_labels: list, output_dir: str, fileformat: str = 'pdf', **kwargs) → None#

Plot a given variable.

Parameters:

df_in (pd.DataFrame or np.ndarray) – DataFrame (for jets) or ndarray (for tracks) with the jets/tracks inside.
labels (np.ndarray) – One hot encoded array with the truth values.
variable (str) – Name of the variable which is to be plotted.
variable_index (int) – Index of the variable in the final training set. This is used to identify the variables in the final training set.
var_type (str) – Type of the variable that is used. Either jets or tracks.
class_labels (list) – List with the flavours used (ORDER IMPORTANT).
output_dir (str) – Directory where the plot is saved.
fileformat (str, optional) – Fileformat of the plots, by default “pdf”
**kwargs (kwargs) – kwargs from plot_object

Raises:

TypeError – If the given variable type is not supported.

umami.plotting_tools.preprocessing_plotting_functions.preprocessing_plots(sample: str, var_dict: dict, class_labels: list, plots_dir: str, use_random_jets: bool = False, jet_collection: str = 'jets', track_collection_list: list | None = None, n_jets: int = 30000.0, seed: int = 42, **kwargs)#

Plotting the different track and jet variables after the preprocessing steps.

Parameters:

sample (str) – Path to output file of the preprocessing step.
var_dict (dict) – Loaded variable dict.
class_labels (list) – List with the flavours used (ORDER IMPORTANT).
plots_dir (str) – Path to folder where the plots are saved.
use_random_jets (bool, optional) – Decide if random jets are drawn from the sample to ensure correct mixing. Otherwise the first n_jets are used for plotting, by default False
jet_collection (str, optional) – Name of the jet collection, by default “jets”
track_collection_list (list, optional) – List of str of the track collections which are to be plotted, by default None
n_jets (int, optional) – Number of jets to plot, by default int(3e4)
seed (int, optional) – Random seed for the selection of the jets, by default 42
**kwargs (kwargs) – kwargs from plot_object

Raises:

TypeError – If the provided track collection list is neither a string or a list.

umami.plotting_tools.train_plotting_functions module#

Plotting functions for NN training.

umami.plotting_tools.train_plotting_functions.get_comp_tagger_rej_dict(file: str, unique_identifier: str, tagger_comp_var: list, recommended_frac_dict: dict, n_jets: int, working_point: float, class_labels: list, main_class: str, cut_vars_dict: dict | None = None)#

Load the comparison tagger probability variables from the validation file and calculate the rejections.

Parameters:

file (str) – Filename of the validation file.
unique_identifier (str) – Unique identifier of the used dataset (e.g. ttbar_r21)
tagger_comp_var (list) – List of the comparison tagger probability variable names.
recommended_frac_dict (dict) – Dict with the fractions.
n_jets (int) – Number of jets to use for calculation of the comparison tagger rejections
working_point (float) – Working point at which the rejections should be evaluated.
class_labels (list) – List with the used class_labels.
main_class (str) – The main discriminant class. For b-tagging obviously “bjets”
cut_vars_dict (dict) – Dict with the cut variables and the values for selecting jets.

Returns:

Dict with the rejections for against the main class for all given flavours.

Return type:

dict

Raises:

ValueError – If none of the given tagger comp vars (the probability variables) are given in the dataset.
ValueError – If the given tagger comp vars don’t match with the given class labels.

umami.plotting_tools.train_plotting_functions.plot_accuracies(df_results: dict, plot_name: str, val_files: dict | None = None, plot_datatype: str = 'pdf', **kwargs)#

Plot the training and validation accuracies per epoch.

Parameters:

df_results (dict) – Dict with the epochs and accuracies.
plot_name (str) – Path where the plots is saved.
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
plot_datatype (str, optional) – Datatype of the plot., by default “pdf”
**kwargs (kwargs) – kwargs for PlotBase function

umami.plotting_tools.train_plotting_functions.plot_accuracies_umami(df_results: dict, plot_name: str, val_files: dict | None = None, plot_datatype: str = 'pdf', **kwargs)#

Plot the training and validation accuracies per epoch for Umami model (with DIPS and Umami accuracies).

Parameters:

df_results (dict) – Dict with the epochs and accuracies.
plot_name (str) – Path where the plots is saved.
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
plot_datatype (str, optional) – Datatype of the plot., by default “pdf”
**kwargs (kwargs) – kwargs for PlotBase function

umami.plotting_tools.train_plotting_functions.plot_disc_cut_per_epoch(df_results: dict, plot_name: str, frac_class: str, val_files: dict | None = None, target_beff: float = 0.77, frac: float = 0.018, plot_datatype: str = 'pdf', **kwargs)#

Plot the discriminant cut value for a specific working point over all epochs.

Parameters:

df_results (dict) – Dict with the epochs and disc cuts.
plot_name (str) – Path where the plots is saved + plot name.
frac_class (str) – Define which fraction is shown in ATLAS Tag.
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
target_beff (float, optional) – Working Point to use, by default 0.77
frac (float, optional) – Fraction value for ATLAS Tag, by default 0.018
plot_datatype (str, optional) – Datatype of the plot, by default “pdf”
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.plot_disc_cut_per_epoch_umami(df_results: dict, plot_name: str, val_files: dict | None = None, target_beff: float = 0.77, plot_datatype: str = 'pdf', **kwargs)#

Plot the discriminant cut value for a specific working point over all epochs. DIPS and Umami are both shown.

Parameters:

df_results (dict) – Dict with the epochs and disc cuts.
plot_name (str) – Path where the plots is saved + plot name.
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
target_beff (float, optional) – Working Point to use, by default 0.77
plot_datatype (str, optional) – Datatype of the plot, by default “pdf”
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.plot_losses(df_results: dict, plot_name: str, val_files: dict | None = None, plot_datatype: str = 'pdf', **kwargs)#

Plot the training loss and the validation losses per epoch.

Parameters:

df_results (dict) – Dict with the epochs and losses.
plot_name (str) – Path where the plots is saved.
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
plot_datatype (str, optional) – Datatype of the plot., by default “pdf”
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.plot_losses_umami(df_results: dict, plot_name: str, val_files: dict | None = None, plot_datatype: str = 'pdf', **kwargs)#

Plot the training loss and the validation losses per epoch for Umami model (with DIPS and Umami losses).

Parameters:

df_results (dict) – Dict with the epochs and losses.
plot_name (str) – Path where the plots is saved.
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
plot_datatype (str, optional) – Datatype of the plot., by default “pdf”
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.plot_rej_per_epoch(df_results: dict, tagger_label: str, comp_tagger_rej_dict: dict, unique_identifier: str, plot_name: str, class_labels: list, main_class: str, label_extension: str, rej_string: str, taggers_from_file: dict | None = None, trained_taggers: dict | None = None, target_beff: float = 0.77, plot_datatype: str = 'pdf', **kwargs)#

Plotting the Rejections per Epoch for the trained tagger and the provided comparison taggers in separate plots. One per rejection.

Parameters:

df_results (dict) – Dict with the rejections of the trained tagger.
tagger_label (str) – Name of trained tagger.
comp_tagger_rej_dict (dict) – Dict with the rejections of the comp taggers.
unique_identifier (str) – Unique identifier of the used dataset (e.g. ttbar_r21).
plot_name (str) – Path where the plots is saved.
class_labels (list) – A list of the class_labels which are used
main_class (str) – The main discriminant class. For b-tagging obviously “bjets”
label_extension (str) – Extension of the legend label giving the process type.
rej_string (str) – String that is added after the class for the key.
taggers_from_file (dict) – Dict with the comparison taggers as keys and their labels for the plots as values, by default None
trained_taggers (list, optional) – List of dicts with needed info about local available taggers, by default None
target_beff (float, optional) – Target Working point., by default 0.77
plot_datatype (str, optional) – Datatype of the plot., by default “pdf”
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.plot_rej_per_epoch_comp(df_results: dict, tagger_label: str, comp_tagger_rej_dict: dict, unique_identifier: str, plot_name: str, class_labels: list, main_class: str, label_extension: str, rej_string: str, taggers_from_file: dict | None = None, trained_taggers: dict | None = None, target_beff: float = 0.77, plot_datatype: str = 'pdf', leg_fontsize: int = 10, **kwargs)#

Plotting the Rejections per Epoch for the trained tagger and the provided comparison taggers.

This is only available for two rejections at once due to limiting of axes.

Parameters:

df_results (dict) – Dict with the rejections of the trained tagger.
tagger_label (str) – Name of trained tagger.
comp_tagger_rej_dict (dict) – Dict with the rejections of the comp taggers.
unique_identifier (str) – Unique identifier of the used dataset (e.g. ttbar_r21).
plot_name (str) – Path where the plots is saved.
class_labels (list) – A list of the class_labels which are used
main_class (str) – The main discriminant class. For b-tagging obviously “bjets”
label_extension (str) – Extension of the legend label giving the process type.
rej_string (str) – String that is added after the class for the key.
taggers_from_file (dict) – Dict with the comparison taggers as keys and their labels for the plots as values, by default None
trained_taggers (list, optional) – List of dicts with needed info about local available taggers, by default None
target_beff (float, optional) – Target Working point., by default 0.77
plot_datatype (str, optional) – Datatype of the plot., by default “pdf”
leg_fontsize (int, optional) – Fontsize of the legend., by default 10
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.plot_validation_files(metric_identifier: str, df_results: dict, plot_object: Line2DPlot, label_prefix: str = '', label_suffix: str = '', val_files: dict | None = None, **kwargs)#

Helper function which loops over the validation files and adds the line object for the chosen metric for each epoch. Meant to be called in other plotting functions.

The fuction loops over the validation files and plots a line with the label f”{label_prefix}{validation_file_label}{label_suffix}” for each file.

Parameters:

metric_identifier (str) – Identifier for the metric you want to plot, e.g. “val_loss” or “disc_cut”.
df_results (dict) – Dict which contains the results of the training.
plot_object (puma.Line2DPlot) – Plot object you want to plot the curves on.
label_prefix (str, optional) – This string is put at the beginning of the plot label for each line. For accuracy for example choose “validation accuracy - “, by default “”
label_suffix (str, optional) – This string is put at the end of the plot label for each line, by default “”
val_files (dict, optional) – Dict that contains the configuration of all the validation files listed in the train config. If None, nothing happens and a warning is printed to the logs, by default None
**kwargs – Keyword arguments handed to the plotting API

umami.plotting_tools.train_plotting_functions.run_validation_check(train_config: object, tagger: str, tagger_comp_vars: dict | None = None, train_metrics_file_name: str | None = None, val_metrics_file_name: str | None = None, working_point: float | None = None)#

Loading the validation metrics from the trained model and calculate the metrics for the comparison taggers and plot them.

Parameters:

train_config (object) – Loaded train_config object.
tagger (str) – String name of the tagger used.
tagger_comp_vars (dict, optional) – Dict of the tagger probability variables as lists, by default None
train_metrics_file_name (str, optional) – Path to the json file with the per epoch train metrics, by default None
val_metrics_file_name (str, optional) – Path to the json file with the per epoch validation metrics, by default None
working_point (float, optional) – Working point to evaluate, by default None

Raises:

ValueError – When no training metrics json could be found.

umami.plotting_tools.utils module#

Helper functions for plotting

umami.plotting_tools.utils.translate_binning(binning, variable_name: str | None = None)#

Helper function to translate binning used in some configs to an integer that represents the number of bins or an array representing the bin edges

Parameters:

binning (int, list or None) – Binning
variable_name (str, optional) – Name of the variable. If provided, and the name contains “number”, the binning will be created such that integer numbers are at the center of the bins, by default None

Returns:

Number of bins or array of bin edges

Return type:

int or np.ndarray

Raises:

ValueError – If unsupported type is provided

umami.plotting_tools.utils.translate_kwargs(kwargs)#

Maintaining backwards compatibility for the kwargs and the new plot_base syntax.

Parameters:: kwargs (dict) – dictionary with kwargs
Returns:: kwargs compatible with new naming.
Return type:: dict

Module contents#

Plotting functions for umami