Simple and Elegant

I was recently reminded of the power of graphical analysis to illustrate trends and represent data in a way that makes the data easy to understand and almost obvious to anyone looking at the graphic. A coworker of mine was working on a rather pesky case for a customer who had uploaded an 80MB text dump of his logs. The problem was of an unknown origin with no error messages, but happened at a known interval of time. How do you begin to analyze this data? Look for trends of course. ;)

What he did, using good ol' sed and awk, was process the log and generate trending data which he then graphed in Excel. He ended up with very simple graphs showing the statistical information he wanted.

This reminded me of the works of Edward R. Tufte. Of course, the type of information and graphics generated by my coworker weren't near those expressed by Tufte, but it reminded me how a simple and elegant expression of large quantities of data can be very powerful in our comprehension of that data.

Data representation was an issue I was particularly concerned with when I did application development and interface design. At a previous employer I was part of the ground station software team. It was our responsibility to create interfaces which would expose the myriad of telemetry data to the satellite operations team. We needed to represent that data in a clear and concise way to prevent confusion or misinterpretation on the part of the operator. If we didn't, we could increase the potential for a very expensive mistake to happen. We had a tight feedback loop with the ops team, and any changes we made to the interface had to be vetted and approved. In the end, the interface was very utilitarian, but got the job done.

In the Cloud

I've started playing with two cloud based applications. The first is Dropbox. I like to think of it as my USB stick in the sky. It provides online storage that can be accessed from any machine which has the client installed and keeps the data synced. It also provides rollback to previous file versions via deltas and easy sharing of files via URL.

The second application I've been using is Evernote. Evernote is a note storing application that lets you upload notes in a a variety of different formats from text to images. One of the features I find really amazing is the ability for Evernote to detect text in uploaded images and allow you to search on it. From their site:

"Evernote allows you to easily capture information in any environment using whatever device or platform you find most convenient, and makes this information accessible and searchable at any time, from anywhere."

To get a better look at both these great tools, check out the following videos.

Dropbox




Evernote