1.Introduction

In this example we will show you how to setup and get started with the zgli.py module. To get started you will need to download the following example files.

2.Set up

Before starting it is important to note that for accessibility purposes, we will use google colab in this example, but this example can be done using other tools such as jupyter notebook. Also, a working version of this notebook is available on github here. Now, with the files downloaded, let's create a notebook. Start by creating a new notebook in google colab, and upload the previously downloaded files into the session storage. When all of this is done you should end up with something like this:
collab_screenshot

3.Imports

With everything set up, we can begin writing our code. Start by importing all the libraries we will be needing. 
>>> !pip install zgli                                       # Install zgli module
>>> from zgli.folder import Folder                          # Module for distance computation
>>> from sklearn.cluster import AgglomerativeClustering     # Module for clustering
>>> from scipy.cluster.hierarchy import dendrogram          # Module to plot results
>>> from matplotlib import pyplot as plt                    # Module to plot results
>>> import numpy as np                                      # Numpy for operations

4.Distance Matrix

Now, lets compute the distances between files using the zgli module. 
>>> data_path = '/content/10-mammals/'                    # Define data path to the folther containing the files
>>> compressor = 'bzlib'                                  # Define the compressor to be used
>>> output_path = '/content'                              # Define output path for distance_matrix.txt
>>> folder = Folder(data_path)                            # Initialize Folder class
>>> dm = folder.distance_matrix(compressor, output_path)  # Compute distances

0_blueWhale.txt 0.0 0.854465 0.95903 0.955691 0.960157 0.96105 0.960009 0.949342 0.957282 0.958167
0_finWhale.txt 0.854465 0.0 0.958172 0.953301 0.956238 0.954923 0.953671 0.948465 0.960088 0.966924
1_cat.txt 0.95903 0.958172 0.0 0.961604 0.959459 0.942299 0.94423 0.952166 0.965251 0.958172
2_chimpanzee.txt 0.955691 0.953301 0.961604 0.0 0.8649 0.959818 0.960252 0.949175 0.960252 0.958732
2_human.txt 0.960157 0.956238 0.959459 0.8649 0.0 0.964729 0.962769 0.95123 0.967124 0.960157
3_graySeal.txt 0.96105 0.954923 0.942299 0.959818 0.964729 0.0 0.77382 0.949891 0.964551 0.966302
3_harborSeal.txt 0.960009 0.953671 0.94423 0.960252 0.962769 0.77382 0.0 0.947334 0.967002 0.96132
4_horse.txt 0.949342 0.948465 0.952166 0.949175 0.95123 0.949891 0.947334 0.0 0.960965 0.955044
5_mouse.txt 0.957282 0.960088 0.965251 0.960252 0.967124 0.964551 0.967002 0.960965 0.0 0.941648
5_rat.txt 0.958167 0.966924 0.958172 0.958732 0.960157 0.966302 0.96132 0.955044 0.941648 0.0

5. Cluster

With a distance matrix computed, we can now cluster our files. In this example we will be using the AgglomerativeClustering technique from sklearn. 
>>> # Create and fit model
>>> model = AgglomerativeClustering(affinity='precomputed', n_clusters=6, linkage='complete', compute_distances=True).fit(dm)
>>> m_labels = ['2_blueWhale', '1_graySeal', '1_harborSeal', '0_rat', '0_mouse',
            '3_chimpanzee', '4_cat', '3_human', '2_finWhale', '4_horse']
>>> print(model.labels_)

[0 0 3 3 2 1 2 1 4 5]

6.Plot Results

Now that we have a model we can plot our results! To plot our results we use the plot function available here. 
>>> # Define plot function
>>> def plot_dendrogram(model, l, **kwargs):

    # Create the counts of samples under each node
    counts = np.zeros(model.children_.shape[0])
    n_samples = len(model.labels_)
    for i, merge in enumerate(model.children_):
        current_count = 0
        for child_idx in merge:
            if child_idx < n_samples:
                current_count += 1  # leaf node
            else:
                current_count += counts[child_idx - n_samples]
        counts[i] = current_count

    # Create linkage matrix
    linkage_matrix = np.column_stack(
        [model.children_, model.distances_, counts]
    ).astype(float)

    # Plot the corresponding dendrogram
    dendrogram(linkage_matrix, labels = l, leaf_rotation=90)

# Create and fit model
>>> model = AgglomerativeClustering(affinity='precomputed', n_clusters=6, linkage='complete', compute_distances=True).fit(dm)

# Plot dendogram
>>> plt.title("Hierarchical Clustering Dendrogram")
>>> plot_dendrogram(model, m_labels)
>>> >>> plt.xlabel("File_names")
>>> plt.show()
Jump to