How to modify data granularity in Python for Graphing data in Matplotlib or another application

This tutorial video covers how to modify your data granularity. Granularity in graphs refers to the level of detail they contain. Usually, the higher the granularity, the better the quality of the data is. That said, once you pass a certain point, say 10,000 plots on a single graph, the eye can not tell the difference and processing is wasted. Instead, developers should program into their applications granularity adjustments depending on the window that a user is looking.

This will show you how to modify how many plots are actually shown on the graph, by averaging them in groups and only plotting up those averages. This will allow you to plot high granularity data sets without waiting for an hour for them to actually generate! They are not moving averages, so there is not lag in the line either, and all data is calculated backwards.

[GM-cmdq]

thanks for putting it in hd really helps the screen

[wilsonfam]

Since the raw data is already in a numpy object it is more efficient to iterate over it using numpy.

Numpy has an average function that can average the rows of a two dimensional array. np.average(data, axis=1)

So, to turn your data into a two dimensional array use the numpy function reshape.  Reshape takes a tuple describing the desired shape.  We don't know how many rows we will need so we can use the argument -1 for the number of rows desired.  x.reshape((-1, divby)) - Except this won't work if the number of elements in x is not evenly divisible by divby.  In your video you truncate the first elements of the array until the array is evenly divisible by divby.  This can be done using numpy sub scripting; data = x[len(x)%divby:].

Putting it all together in one line, in one well documented function yields:

def chngGran(x, divby):
    """Computes windowed averages of x where the window size is divby.

    Assumes x is numpy objects"""
    return np.average(x[len(x) % divby:].reshape(-1, divby), axis=1)

Now changeGranularity can simply call chngGran twice.

def changeGranularity(x, y, divby):
    "assumes x & y are numpy objects"
    return chngGran(x, divby), chngGran(y, divby)

[GM-cmdq]

nice video how do you start making videos