Research Update 4 ~Alex

By
After the craziness of week 3, week 4 has seemed relatively calm. At the beginning of the week I was simply finishing up what my breakthrough last week enabled me to. The new variables are fully created and we are in the process of comparing them to the variables from the survey. We are also starting to focus on how we will be able to visualize our data. Because of the longitudinal and high dimensional nature of our dataset, it is almost impossible to visualize without manipulation. To reduce the number of dimensions, we decide to employ t-sne (a dimensional reduction algorithm). T-sne can take hundreds of thousands of dimensions down to 2 which is much easier to visualize. T-sne can be a little touchy though, so right now we are carefully tuning the parameters, especially perplexity, a number that tells the algorithm whether to prioritize the global or the local aspects of the data (basically a guess on how many neighbors each point has). A graph with the data sets we're focused on is pictured below with varying perplexities.



 Another way we are utilizing t-sne is to determine the optimal NA cost for our dataset. Those plots have yet to work. They all  look clusterless, instead of the slowly forming clusters as NA cost increases that we would expect. The debugging of this program has consumed most of the end of the week.

I know this update is a little sparse, but debugging has taken most of the time we were given, so there aren't a lot of new results. Also, with the holiday quickly approaching, we are attending more meetings than normal before people leave for vacation. Hopefully more exciting news next week!
2

Comments

Post a Comment

Popular posts from this blog

Week 10 Review: 9/6 - 9/10

By
Image
I spent all my time this week preparing for the end of the internship. I submitted a poster for admittance to the Association for Computing Machinery poster competition to be held at the Super Computing Conference in Dallas, Texas this year. Beyond that, I spent a lot of time organizing files and finishing mandatory assignments. On Monday I submitted a minute long video to a small competition for giving elevator pitches and got an honorable mention award on Thursday, which was a tie between 5th and 6th place out of 10. Not bad for one hour of recording at midnight. Thursday was the poster session for all the interns and it went fairly well. The expense of organizing the event probably wasn't worth the amount of visitors that came to participate, though. I will say that I was one of the few interns that presented their poster to Michael Stewart Witherell, the director of the lab! Now that was cool! The research poster presented at the final poster session There was a lot plan
Read more

Week 9 Review: 7/30 - 8/3

By
At the start of this week I tried to clean and analyze more recent Tstat logs from January and July of this year. To my surprise, these files were formatted differently from my previous files which meant I had to clean and compile them differently as well. This posed many unexpected problems that took much longer to solve than I first imagined. Also planned for the week was to add a way to identify the "normal" cluster to the algorithm and perform quantitative analysis on the similarities of the calculated sequences with the log(throughput). All at the same time, this content needed to be put into my poster for the intern poster session by noon on Friday. After cleaning, I tried to add the cluster identification step to the algorithm. I was running behind, so without finishing it I moved on to calculating quantitative similarity metrics. I didn't want to consider the similarity of the sequences when throughput is high (because our values turn out to be low) so I took the
Read more