Building ‘Meteorites: Earth Impact’

It all started with this tweet. Time was scarce indeed and I had not created an interactive visualization till now. I was supposed to have been doing some d3 this holidays but I never got round to it.

The dataset provided for the contest has 45k+ rows and that was slightly intimidating for a beginner like me.

So I initially decided to take a small set of the data and use it. Like the top 10 found and fell meteorites size wise. Using Tilemill to geotag these was simple anyway.MeteoritesAs you can clearly see above, the size of the red and yellow dots represent the found and fallen meteorites respectively. The relative size of theirs is because of their actual size varies that much with the biggest weighing in at 60 tons. Problem with this graphic was that the size was too big for you to pinpoint the targeted coordinate. Certain impact points hid/overlapped with others. No additional information about the meteorites could be presented.

This led me to the second iteration. Importing a slightly better map into Illustrator(via Export as PDF from Tilemill, SVG didn’t work for me), I played around with labelling the meteorites and this resulted with the map below.

Meteorites-vizThings seemed a little clearer in the above version, but there were still some problems. The scale was skewed a lot if I considered only these values. The yellow dots were nearly invisible and the red ones overshadowed everything. About that time I came across this visualization. That prompted me to play with the impact sizes a bit more. Iterations were made to get the size right. It was a slow process because most of the dots overlapped and made sections obscure. Finally I came up with this before I slept just past midnight.

reddotsPlaying with the opacity above gave me an idea about the denser areas. At around this time, when I was discussing these visualizations with @Rasagy, @Hashnuke asked if I wanted the reverse geotagged locations so that I could perhaps map the Countries in a sort of choropleth. He said he would write a script and run it using a reverse geotag service like Google. So we set Sunday as the day to do this.

I continued to work on Tilemill and decided to export the map and host it on Mapbox. Decided to learn Wax so that I could build an interactive visualization. At that moment, most of the visualization that you see at the end was forming in my head. With 2 days left to go for the submissions at Visualizing.org. I decided to give it a go.

Playing around with Wax led me to a bug when the map was fullscreen and led me to file my first issue at Github. After that I got stuck while trying to figure out pivot tables in MS Excel. @Sevenaces helped me realise my stupid mistake and I got the data I need to plot the Column charts.

Choosing a charting library was the next thing. I wanted something simple that did not need too much work, offered interactivity out of the box, etc. I went through my bookmarks to check dviz. I had been meaning to use this sometime soon and decided to for this project. The other option I had was dviz but the simplest examples looked like a lot of work. That tipped the scales in dviz’s favour. dviz is the simplest charting solution for people who don’t know javascript and wont be bothered to learn javascript. So the column charts worked wonderfully. Stacked column charts were my first option but that showed me the fact the the fallen meteorites were much lesser and were easily hidden by the found meteorite data. Hence I decided to separate the two charts and show the data separately. Poking around Google’s Visualization api, I figured how to customize the dviz charts some more. I used Flatuicolors for the colours. That done, I turned to Foundation 3 to build something simple.

Next, Akash walked me through setting up a github account, hosting a .io repo there. I installed Github for Windows and everything was simple and intuitive. Git incidentally was something I was meaning to learn from a long time, this project gave me and opportunity to do that. The prototype visualization was up and online on Saturday night but it was quite a long way from finishing.

There were obviously problems with the data set. The section at 0,0 seemed to be awfully dense for a point in the middle of the ocean. This led me to review the dataset. I found that more than 10k rows didnt have coordinate data and some of them had 0,0 instead. I decided to clean these rows out of the geotagging. They were bad locations and did not contribute of anything. The dataset was now slightly above 32k rows. More Tilemill followed. Tilemill kept hanging every now and then and I had to close it every few minutes. Frustrating indeed. The huge dataset could be a possible reason. Figuring out the legend and tooltip design took me some more time. Finally the map was done. More hangups followed and I was finally able to export and host the final map on Mapbox.

The next problem came due to the 32k row .csv file. The big file was throwing errors. We then split the dataset into 3 sections and Akash ran the script on Geonames via nitrous.io. He should really write a post on how he did all that. Here’s the scripts and the processed data. There were about 40 bad locations in the dataset which were removed.

The output of the reverse geocoding was the country code. I wanted the country names. This is how I learnt about vlookup in MS Excel. I also learnt how to fill all the blanks in a table and how to divide a column by a number. These are not as straightforward as you think. Excel hung up on me as well. Lot of times. Remember making everything a table helps a lot when doing Excel operations. I used the country name list from here(It’s missing SS=South Sudan). Finally everything seemed ready. Now all I needed was a good scatterplot example to borrow 😉

A quick search of mbostock’s d3 gallery and I located a scatterplot that I could use. It was simple to understand. I promptly hacked the example to meet my demands. I learnt a bit of d3 along the way.
With the final changes all done, I was done with ‘Meteorites: Earth Impact’. In 3 days I learnt such a lot. It was indeed a wicked journey.

Without anymore delay, Ladies and Gentlemen I present to you ‘Meteorites: Earth Impact’.

Update: The visualization got ‘Staff Picked’ on Visually. #proud

A brief history of space exploration

A brief history of space exploration visualized by me

I started doing some research on humans and their interaction with Space throughout the ages early in April as part of a freelancing project. Unfortunately we never got it, and the research had to wait till now to become tangible. I had never made a timeline based visualization prior to this and that is why I wanted to give it a try.

The timeline initially included the discoveries of space made by astronomers throughout the ages as well as the space exploration timeline but I ended up ditching the space discovery timeline as I found it too big for this project. Research and early sketches shaped the way this timeline looks like. My limited knowledge of Illustrator made this possible to complete it digitally.

Shown below is the current version of the infographic(Click and zoom to view). I would be glad if I can get some feedback on this. Space-v3.6Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Thanks to Nievu, Rasagy, Hafsah, Parag and Bijoy for their inputs on its initial stages.
I uploaded it to Visually.

Paldi Diaries: First Impressions

One of the specialties of NID is that there is a concept of an ‘open’ elective which allows 4th semester UG and 2nd semester PG students to indulge themselves with a course module in any of the 3 campuses. I had heard so much about the main heritage campus at Paldi that I made up my mind to visit the place for my open elective. I chose ‘Shadows of Age’ because it had a foreign faculty, talked about yoga, free dance and meditation and also the end product of a shadow play seemed interesting.
We arrived at Paldi and spent time exploring the campus. We see how chilled out the place is. We see some great work displayed. We see no obvious rules being followed. There is heritage on display. Its just a free zone. It acts as a cocoon from the outside world. In this part of the world, life seems fun, lively and filled with interesting things. This is both good and bad but it was obvious that the more I saw the more I fell in love with it. I would not mind spending more time here; reading books at the kmc, playing Frisbee on the lawn, hanging at the BMW, watching movies or eating at the mess. I am sure that I shall miss this place after these 2 weeks. But I am glad that I got to experience it. The main difference here is that I see people working with fervour which is what I miss seeing in Bengaluru. But then I wonder if I would want to work that much myself.

P.S. Rough post written via the tablet.

8bit Tribute

Sometime in November 2011, after some contemplation i decided to buy http://dsouza.im, then i after much dawdling i finally uploaded version 1 somewhere in february. My main intention of buying the specific domain is for using domain name hacks making my personal site having a url: http://kenneth.dsouza.im Very existential.

So version 1 was just something i pulled online without giving much thought. When i got shortlisted to NID second round, i decided to code a portfolio using deck.js and uploaded it.

Essentially, i havent yet done anything much on the domain that i would be proud of i.e., until now. Work has been hectic and in between 12 hour days and work on Saturdays i somehow got a Sunday for myself and spent it reading manga. Sunday evening was when i decided to code something new onto the domain.

I like pixel art. I use Gmail’s High Score theme. I grew up playing cartridge video games. The love for the 8 bit is ingrained in me. So here’s my personal tribute to video games.

First, i stole background images from Gmail for use. Please be excusing my plagiarism. So i coded up a design that had been growing in my head.

Then i needed a pixel font. So i went ahead and tried out using @font-face for the first time.

Finally. A video game tribute would be incomplete without having a Konami code* reference. So a quick google found me a library which was widely used to recreate konami code.

So now with that out-of-the-way, i finally release http://kenneth.dsouza.im on this blog. Please head down there and then enter the Konami code for the pixel version that i am proud of.

*Konami Code: ↑ ↑ ↓ ↓ ← → ← → B A Start(Enter)

Fonts used: My Logo: Pacifico, a Google Web Font. The Text is Silkscreen via fontsquirrel.

Objectified

Image

I just finished watching ‘Objectified‘, a film by Gary Hustwit. Its a documentary which takes us through the various facets of design; the conception, the sustainability and the innovation. Objectified is the second part of a three-film “design trilogy” by Gary Hustwit, who is currently working on the final film: ‘Urbanized’.

Watching it made me realize some things about design. About how design is everywhere and how good design is invisible. Whenever an individual designs he/she brings to it their own being. I know i am not happy when i am doing something eventhough it looks okay. I always tend to get my geeky nature involved into things i create like those quizzes.

“At the end of the day, when you are looking around the objects in your house and you are deciding ‘what here really has value to me. They are going to be the things that have some meaning in your life.
The Hurricane is coming, you have 20 minutes to get your stuff and go. You are not going to be saying. “Well that got an amazing writeup on the design blog”. You will pick the most meaningful objects to you because those are the true objects that truly reflect the true story of who you are and what your personal narrative is, the story that you are telling to yourself and no one else. Coz thats the only audience that matters.”

Thanks to Citruz for the recommend.