Week 2 Review: 6/11 - 6/15

By
Another week gone by and a little bit more done. I have learned a lot more about the origin of my project data and a more descriptive version of what my project goal is. I am working on network data flow prediction by multivariate clustering in time windows. The data is coming from TStat which monitors over a hundred variables of TCP (Transmission Control Protocols). Basically, I'm getting a ton of measurements for different TCP variables and will be looking for the variables that are most closely related to the behavior of network traffic in different time intervals. Well, nobody said this was going to be easy.

I have a research proposal due in a week that I got a pretty good start on. It encompasses the main idea, goal, and contribution of the project. I have also set up a slide show presentation for Monday to introduce my project officially to other members of the SDM (Scientific Data Management) group.

I learned a bit more about other tools I may be using such as t-SNE and DBSCAN. I used t-SNE, a dimensionality reduction tool, on the same data set from last week. The result is a bit obscure.
 
There was a bit too much going on to be useful and it also took forever to run. So I took a 10% sample of the previous data and ran it again with a few different parameters.






 Pretty cool I'd say. Though, it may have less relevance now than I thought it would have. Regardless, this means I can start doing more hardcore work on my actual data.
2

Comments

Post a Comment

Popular posts from this blog

Week 10 Review: 9/6 - 9/10

By
Image
I spent all my time this week preparing for the end of the internship. I submitted a poster for admittance to the Association for Computing Machinery poster competition to be held at the Super Computing Conference in Dallas, Texas this year. Beyond that, I spent a lot of time organizing files and finishing mandatory assignments. On Monday I submitted a minute long video to a small competition for giving elevator pitches and got an honorable mention award on Thursday, which was a tie between 5th and 6th place out of 10. Not bad for one hour of recording at midnight. Thursday was the poster session for all the interns and it went fairly well. The expense of organizing the event probably wasn't worth the amount of visitors that came to participate, though. I will say that I was one of the few interns that presented their poster to Michael Stewart Witherell, the director of the lab! Now that was cool! The research poster presented at the final poster session There was a lot plan
Read more

Week 9 Review: 7/30 - 8/3

By
At the start of this week I tried to clean and analyze more recent Tstat logs from January and July of this year. To my surprise, these files were formatted differently from my previous files which meant I had to clean and compile them differently as well. This posed many unexpected problems that took much longer to solve than I first imagined. Also planned for the week was to add a way to identify the "normal" cluster to the algorithm and perform quantitative analysis on the similarities of the calculated sequences with the log(throughput). All at the same time, this content needed to be put into my poster for the intern poster session by noon on Friday. After cleaning, I tried to add the cluster identification step to the algorithm. I was running behind, so without finishing it I moved on to calculating quantitative similarity metrics. I didn't want to consider the similarity of the sequences when throughput is high (because our values turn out to be low) so I took the
Read more