The iPad Wall consists of 12 iPads in a 4×3 configuration housed in a custom designed wooden frame which is only 23mm thick. All the iPads are powered from a single power source again housed inside the frame which charges the iPads when needed. The wall, in its current configuration, displays data from City-Dashboard and allows users to interact with the data. By touching some of the iPad’s, the view flips to a historic graph of data over the last 24 hours. We can show individual videos on any of the iPads and we usually show the live stream of BBC News 24 on iPad number 11 which allows us to keep up with all the latest news happening around the world.

When we have visitors in the office we can grab our mobile phones and open up the iPad Video Wall mobile application and play one of the six visualization videos we have created for the wall. These videos play over all 12 iPads creating a virtual video wall that gives the illusion that you are looking through a window. The effect is fantastic when you view it in person as you can see in the video below.

I envision that a wall like this can be used in decision making, giving an insight into what is happening in real-time. Unlike traditional walls, which are heavy bulky, and fixed to walls, the iPad Video Wall is very portable and can be moved around the office where-ever it is needed. It also opens up new interaction designs for the traditional video wall. By having 12 touch screens we can come up with new methods to interact with real-time data. The wall was funded through a generous grant by UCL Enterprise and forms an output to the Talisman project

In the following days I’ll be posting up a few blog entries on how I built the wall and how the software works so that you can get a real behind the scenes look at the current state of the wall.

I’m @frogo on Twitter. If you get a chance then you should follow me.

Telefonia plans to set up a group called “Telefonica Digital Insights” to explore this data and link the data together to create anonymous profiles based on demographics and location. Using the data they will be able to sell fine grain locational data (up to hundredths of meters) to retailers, for example, of certain demographics groups frequenting the area. This data combined with footfall data that retailers already have would allow them to see where demographics go when they are outside. The one controversial decision is that there will be no opt-out of this service. If your’e one of the 23 million UK subscribers then you’re already collecting the data for them. But they are not the only company to use this sort of data to understand the demographics using their services.

Verizon, a US telecommunications provider, are also selling anonymised personal data. One astute user noticed a clause in the terms and conditions which allows then to sell the data derived and collected through normal use of service but unlike Telefonica, customers have 30 days to opt-out of the service.

It remains unclear what the data will show and how these companies will start to make value from the analysis of data but it is clear that our normal day to day activities can be linked and we can be used as sensors. The original story can be read on BBC News.

Back in 1955 the Government tasked a team of people, including Roger Coleman, to calculate the distances between nodes on the British Rail system using the first computer created for a commercial company called LEO owned by the Lyons company. They had to compute distances over 12 million nodes which represent the connection between the 5,550 stations in the United Kingdom using a computer that had the equivalent of a 500Hz CPU and 2 Kilobytes of RAM. To give you an idea of the size of the RAM on this mainframe, 2K is about the equivalent to storing 2048 characters in memory and that’s not including the space you would need to store the program while running. The problems don’t stop there. This before graph theory and the shortest path algorithms exsisted and pre-dates some of the standard, accepted algorithmic solutions by 5 years

John bridges some parallels between today’s big data challenges and the challenges these pioneers had to deal with from renting out time from the Lyons company (renting out time on Amazon’s EC2), challenges with storing output (reams of punchcards vs. hard drive space) and memory contraints (2K was the limit due to there being not enough mercury in the earth to build larger medium)

The talk is a very entertaining and provides insight into what appears to be a very old problem in big data processing. A very interesting watch indeed.

Mira, the 3rd largest super computer in the world, is gearing up to simulate the first 13 billion years of our universe from the very start of the big bang. Physicists from the Argonne National Laboratory will use the cluster to simulate the trillions of interactions between particles, hopefully creating giant masses of planets, stars and galaxies. The simulation will hopefully give scientists an understanding of how these elemental interactions create a time lapse of expanding bodies in the universe.

To give a sense of the power of IBM’s Mira, the whole task is scheduled to complete in just 2 weeks.  With the advances in hardware, distributed computing and CPU design, engineers can cram more cores on a single chip and if Moore’s Law holds then clusters by the end of the decade will be 1000 times more powerful than Mira.

If you want to read more about the experiment then the full article is over at the Alantic.

After a discussion with a few of my colleagues about new exhibition pieces for upcoming events we thought it would great to have a video wall to showcase some of our visualisations created at CASA.  Using 8 iPads and a custom application to connect them together via a centralised server, we now have a basic video wall.

The iPads connect to a server (my iMac) which in turn notifies them which video segment they have to play, then once connected the server will tell each device to start buffering the video. After a few seconds all iPads are told to play simultaneously and the result is what you see in the video above, a silly movie of myself.

The next steps for this project is to add some interactive screens so users can interact with the wall and select content that should be displayed on the wall. Also I’m also looking into different ways to charge multiple iPads while retaining the ability to sync them all with 1 cable. I’m really happy with the results, especially being a proof of concept, so once I’ve ironed out the few synchronisation bugs and added more functionality wall then maybe we will build a full size wall in the near future.

There were some little gems of information I found in my data. For example, I have a near 50/50 split of male to female friends and around 36% of them are married. My network graph shows all the distinct groups of my friends including UCL friends, Glasgow University friends and my disconnect groups are my family. The biggest insight into my data was my own personal use of facebook mainly involves posting lots of photographs and not that many textual update posts.

It’s really great to see that Wolfram Alpha is joining data like your birthday and historical weather reports or moon phases to give even more insight into your personal data. I think that the most impressive feature is that the speed that the analytics are created. Some of these quires are computationally expensive yet Wolfram Alpha seems to correlate the data extremely quickly and have the interactive graphs ready to view in less than a minute.

If you’re curious about your own facebook analytics then head over to the site and give it a try.

One central server manages the swarm and asks each collector every 5 seconds to send back statistics giving us a live view of each server just incase a machine stalls. This is a big step towards a completed system allowing users to initiate collectors on services such as Amazon’s EC2 to takle large scale data capture and still get responsive, live statistics of what each individual machine is collecting.

We have already incorporated this data back into City DashBoard which you can read more about at Oliver O’Brien’s blog

Update: I’ve now enabled the relevant web sockets ports on our servers so you can see the stats from the live collectors here

During the “Next Dimension of Maps” event Google announced that the StreetView cars have driven over 5 million unique miles of roads and collected 20 petabytes of data on their travels. Let’s stop and think about how much data that is. 20 petabytes is equivalent to 20,480 terabytes (TB), if you buy a standard PC today it usually comes with a 1TB hard drive. It’s like you have sat in 1 seat of Pittodrie Stadium in Aberdeen and Google has bought out the rest of the stadium. Although putting size aside it wasn’t really clear to me what they were doing with all that data until I read a recent article that made me really start to think.

Pretty lonely up there in Pittodrie Stadium

As Google drives around the world taking images of the streets they collect Lidar data, a high detail 360-degree laser scan of the surrounding area. This gives an accurate point cloud of buildings, signs, surfaces etc. around the car. You may have seen the rectangle box in StreetView showing you the front of buildings. So why are they collecting this data? Google has never said.

Google's self-driving car which I saw in April 2012

Well another little project that hit the headlines was Google’s automatous, self-driving cars. These cars also use Lidar to detect the surrounding traffic, people crossing the street etc., to give the car’s software an accurate image of the cars current location. This is where it gets interesting. Have Google used the StreetView data they collected to train the self-driving cars software?

What the self-driving can see.

It makes perfect sense! Drive around all the roads in the world and gather the data to train the AI in the car to all the possibilities of junctions, road types, crossings, signs, international differences, if you can see it on the road – chances are a StreetView car has saw it as well. The big question is what came first? Was it Google’s intension to use StreetView data to train a car? Or did some sharp engineer say, “I know what to do with this data” in his 20% time? We will never know the answer but we can all learn a lesson from Google.

When you plan a project to collect data then collect as much data as possible. Don’t throw away data that you collect because you think it’s not valuable now. Chances are that somewhere down the line the data you collect may be useful to someone somewhere.

Well the researchers at Facebook’s Data Team have released some fantastic visualisations of global checkins collected by their 900 million users. By joining this locational data with other information such as age, political affiliation or even distances between checkins, the researchers can start to understand our travel patterns or even the distrubtion of certain demographic groups.

At CASA we have looked at the geo-located tweets of London from Twitter data and also James Cheshire, over at spatialanalysis.co.uk, and I have looked at the language distribution of London based on Twitter data. This certainly is a hot topic at the moment and proves just what can be done with this type of crowd sourced data.

You can read more about the process of making these image and even have a look at higher quality versions of the maps over at the Facebook’s Data Team blog.

We recently held a one day conference here at CASA called Smart Cities. For the conference we built various exhibition pieces and my contribution to the conference was the London Data Table, a projection table the shape of Greater London. The table had various visualisation projected onto the surface; from live aircraft positions, live traffic and bike hire usage to movies of public transport over 24 hours. We got some great feedback from the attendees and I thought I would share my documentation of the whole process of build the table, from start to finish.

Step 1: The Planning

The biggest issue in this project was size. How big do we make the table? Can we find a projector with a short enough throw to project to the table? How were we going to mount the projector. I downloaded the outline of London from OS OpenData (http://os.openstreetmap.org/data/) and created a pdf of the London outline, this would serve as our master vector throughout the process. I’d be lying if I said we knew all the answers the questions at the start of the project but with a few calculations and a lot of paper we mocked up the outline on paper (to scale) and stuck it to the wall of the office.

Planning the project this way allowed us to see that a normal projector would be impractical (due to the distance required to project an image of that size) and that we would have to buy a short throw projector, not ideal. Time was against us so we ordered the projector, took the leap and started to build the table.

Step 2: Building the Table
I wanted this table to become a piece of furniture! There was no point putting in all this effort to build a table which would only last the length of the conference and then started to sag, as MDF eventually does. So after some advice from the staff in the Bartlett workshop I decided to order a 10ft x 5ft (3m x 1.5m) piece of birch plywood. Birch plywood has excellent strength with a beautiful finish but is some what more expensive that MDF.

Here at the Bartlett @ UCL we have an excellent workshop that has a wealth of tools including laser cutters, a full wood and metal workshop, 3D prototyping and a CNC milling machine. This would cut the outline from our sheet. Armed with an Adobe Illustrator file of Greater London, I exported to DXF (a CAD standard file format) and loaded it into the milling machine. Watch the video to see what happened next.

Within 10 minutes we had the outline of London cut out. A quick chisel to remove the frame and we were ready to go into the preparation stage.

Step 3: Preparation
Out of the whole project this was the most boring part. Sanding the table down for preparation then painting a layer, then sanding again. Repeat this 4 times until I got an even finish over the whole surface and the table top was complete. I used a standard white emulation, the same paint you would use for walls with 10% water added to ease application. After a whole day of painting and drying and this was what the table looked like from above.

Step 4: Table Legs and Projector
I had the idea straight from the start that I was going to use a set of legs from Ikea to quickly make a table. A trip to Ikea later we had a set of T-Legs from a Galant desk. Screwed them on and we have an instant table.

Step 5: Align the projector
This was the most time consuming part of the project, aligning the projected image to the table. We mounted the projector on a basketball stand, which was a great idea from George MacKerron, which was custom mount screwed into the projector mounts made of the same birch as the table top and filled the base with water for stability. We created another vector to align the river Thames to the table and adjusted the height of the stand to fit. The results were fantastic!

So there you go, that’s how you build a table the shape of London and create an exciting new exhibit to show off the vast number of open data visualisations we create at CASA. In the next few posts, I’ll go into the in’s and out of the software and how I created the base software and the Aircraft Visualisation that we displayed on the table.