At the start of this week I tried to clean and analyze more recent Tstat logs from January and July of this year. To my surprise, these files were formatted differently from my previous files which meant I had to clean and compile them differently as well. This posed many unexpected problems that took much longer to solve than I first imagined. Also planned for the week was to add a way to identify the "normal" cluster to the algorithm and perform quantitative analysis on the similarities of the calculated sequences with the log(throughput). All at the same time, this content needed to be put into my poster for the intern poster session by noon on Friday. After cleaning, I tried to add the cluster identification step to the algorithm. I was running behind, so without finishing it I moved on to calculating quantitative similarity metrics. I didn't want to consider the similarity of the sequences when throughput is high (because our values turn out to be low) so I took the ...
This week has been very busy. The results section of my paper was due so the majority of my time was spent typing and editing and re-editing that. There were also some final decisions made about future steps. Instead of the NA cost strategy we've been following, we've decided to employ a delete strategy. In this strategy (which only works because there are only missing values at the end of the sequences) we will delete all missing values and be left with sequences of different lengths. Then these mismatching sequences will be normalized and compared. This method gives faster and more accurate results than simply picking a substitution cost. Other than that change in beginning procedure, the rest of our methods have not changed. In addition to this decision, we have also decided on an ending age of 39 for our sequences. The new variables are finally fixed and we have a final dataset. We are ready to receive our final clustering results and have ensured they will be accurate wi...
In this week much effort was put into being able to compare average window throughput and degree of change. One challenge to overcome was how to scale the throughput data. To best understand the distribution of the average throughput's for every window, we made a histogram using the unscaled throughputs. It was evident that the magnitudes of the very high throughput dwarfed the small throughput by at least 10^2. Two methods were considered to scale the throughput; apply the reciprocal operation, (^-1), or apply a logarithmic function. The logarithmic function most evenly represents small throughput and high throughput. After choosing our scaling method we graphed the log(throughput) with the degree of change over time. From these figures, it seems there is no discernible relationship between degree of change and log(throughput). This was a big step back as Alina and I had to rethink an approach to this project.
Comments
Post a Comment