Week 4 Review: 6/25 - 6/29

By
The beginning of this week's work was dedicated to solving some problems that came from reproducing the second experiment. With Alina's help we were able to get a much better match to the graphics from the paper, though there were still some dissimilarities. Some data was still missing and the clustering didn't always match the paper. The clustering difference can be explained by the possibility of using different parameters.

Next, we worked on representing data as averages in different sized time windows. We tried examples of hour long time windows with five minute throughput averages, day long time windows with hourly throughput averages, and a window of the entire cleaned data set with daily throughput averages. This technique will become extremely useful in the future of this project.

After my weekly meeting with my mentors on Thursday, I began working more directly with my project. Part of the new plan was to use PCA to determine the data's most prominent features. We do this by running PCA on a set of data and look at the first component's explained variance and subcomponents. A setback during this process was that the subcomponents of the first component weren't labeled, so we couldn't directly match subcomponent values with their feature name. At the tail end of Friday, I was able to solve this issue and now the code runs smoothly and quickly. My next steps are to try different scaling methods before PCA to see if it affects the results of finding the most prominent features. Then I can cluster data in time windows.
2

Comments

Post a Comment

Popular posts from this blog

Week 10 Review: 9/6 - 9/10

By
Image
I spent all my time this week preparing for the end of the internship. I submitted a poster for admittance to the Association for Computing Machinery poster competition to be held at the Super Computing Conference in Dallas, Texas this year. Beyond that, I spent a lot of time organizing files and finishing mandatory assignments. On Monday I submitted a minute long video to a small competition for giving elevator pitches and got an honorable mention award on Thursday, which was a tie between 5th and 6th place out of 10. Not bad for one hour of recording at midnight. Thursday was the poster session for all the interns and it went fairly well. The expense of organizing the event probably wasn't worth the amount of visitors that came to participate, though. I will say that I was one of the few interns that presented their poster to Michael Stewart Witherell, the director of the lab! Now that was cool! The research poster presented at the final poster session There was a lot plan
Read more

Week 9 Review: 7/30 - 8/3

By
At the start of this week I tried to clean and analyze more recent Tstat logs from January and July of this year. To my surprise, these files were formatted differently from my previous files which meant I had to clean and compile them differently as well. This posed many unexpected problems that took much longer to solve than I first imagined. Also planned for the week was to add a way to identify the "normal" cluster to the algorithm and perform quantitative analysis on the similarities of the calculated sequences with the log(throughput). All at the same time, this content needed to be put into my poster for the intern poster session by noon on Friday. After cleaning, I tried to add the cluster identification step to the algorithm. I was running behind, so without finishing it I moved on to calculating quantitative similarity metrics. I didn't want to consider the similarity of the sequences when throughput is high (because our values turn out to be low) so I took the
Read more