With the rise of data, the power of processing has risen significantly. From a time of initial multi-tonne computers capable of lesser computations than your washing machine, to super computers and power packed cloud applications, the journey has been wonderful.
Dell recently published an interesting article which suggests at least the contrast in terms of storage capacity.
http://techpageone.dell.com/history-of-data-storage/. An interactive setting to let you browse through the wonderful technological advances in the field of data storage.
Another not so graphically appealing but a nice infographic on the growth of Computation is found at this link: http://www.rsvlts.com/2013/12/06/a-visual-history-of-computers-infographic/
The links only provide the context of what is coming ahead. Most of statistical and analytical concepts have been around for a long long time. These concepts h
ave been previously proven on a far smaller data set by hard working mathematicians who worked manually to establish the credibility of their hypothesis. In fact most proofs were done by Mathematical Induction to avoid computations. Now almost all of the theorems and techniques have been proven on a much larger scale.
The most famous saying in the world of Statistics, "Correlation does not imply causation". Well it is confusing is it not. To explain this I will take a very simple example. When the streets are wet, you will see a lot more people carrying an umbrella. Now the question is, what is causing people to carry umbrella. Are they afraid of the wet roads. The correlation suggests that one happens when the other does. Does it imply that wet roads lead to people carrying umbrella or vice versa. From our prior knowledge we know that is not the case. There is a common cause to both; the falling rain. Hence the proof of Correlation does not imply causation.
However with the current amount of data and computation power, do we need to establish causation. Consider this, if you are selling coca cola and cranberries and for some reason they are selling together and in larger quantities, you would want to stock up on both. Now it may be the case that there is a group nearby preaching about the benefits of taking Coca Cola with Cranberies ( How great examples I come up with !), and you are unaware of it. But you find that the sales are going up. What will you want? You would definitely want more stock of the same. You might want to know about the cause but you would definitely not worry about it as long as the sales are going up. Now there is a correlation between Coca Cola and Cranberry sales, which did not exist before. This correlation does not imply causation but it does imply that people are buying both with decent linkage and you need to stock up more.
Such patterns as taken in the example above are often hidden deep within the data. Without large computational powers, the techniques are useless and not feasible. However, with the enhanced computation power, machine learning provides an excellent opportunity to find such patterns. Even though, you are not aware of the cause, just the awareness about the pattern helps you make better decisions. Sometimes some correlations are because of causation as well. If you find that, jackpot, otherwise also you get lot more information than before.
It is the computation power that can simulate millions of future scenario with thousands of different variables to suggest the best alternate. Sometimes elegant methods work best, at other times; brute force takes you through. The net result remain that with increased power, comes more and more empowerment. Geniuses have been known to exist throughout humanity. Some of them thought about methods in their pre computer era which can only be utilized now. I am avoiding listing algorithms in a trial to remain closer to the lay man of the field.
Everyone knows of Leonardo Da Vinci. It is known that he made sketches of what could have evolved into a helicopter. However, the real helicopters came ages after his death. The technological advances have made his dreams into reality. There have been many such geniuses who dreamed of a lot of things and tried to provide a solution. Those solutions are now being emulated, simulated and implemented across the world to help people have a better life.
Who would have thought years back, that google will tell you what to do. Google Now makes your life easy because your phone has more computational power than a desktop around 5 years back.Google has the data and definitely the computational means to provide you guidelines in advance. You see your current weather before leaving home, it even tells you if the traffic is bad in your current route, etc etc. How is it able to do this for so many people remotely. The answer is simple, it collects a lot of data about your usage and also possesses the computational power to make sense of that data.
What companies need to realize that this alone is not sufficient. Increase in computational power definitely has had far reaching consequences on the world but if everything could be shown with numbers alone, the artists of the world will feel betrayed.
Dell recently published an interesting article which suggests at least the contrast in terms of storage capacity.http://techpageone.dell.com/history-of-data-storage/. An interactive setting to let you browse through the wonderful technological advances in the field of data storage.
Another not so graphically appealing but a nice infographic on the growth of Computation is found at this link: http://www.rsvlts.com/2013/12/06/a-visual-history-of-computers-infographic/
The links only provide the context of what is coming ahead. Most of statistical and analytical concepts have been around for a long long time. These concepts h
ave been previously proven on a far smaller data set by hard working mathematicians who worked manually to establish the credibility of their hypothesis. In fact most proofs were done by Mathematical Induction to avoid computations. Now almost all of the theorems and techniques have been proven on a much larger scale.
The most famous saying in the world of Statistics, "Correlation does not imply causation". Well it is confusing is it not. To explain this I will take a very simple example. When the streets are wet, you will see a lot more people carrying an umbrella. Now the question is, what is causing people to carry umbrella. Are they afraid of the wet roads. The correlation suggests that one happens when the other does. Does it imply that wet roads lead to people carrying umbrella or vice versa. From our prior knowledge we know that is not the case. There is a common cause to both; the falling rain. Hence the proof of Correlation does not imply causation.However with the current amount of data and computation power, do we need to establish causation. Consider this, if you are selling coca cola and cranberries and for some reason they are selling together and in larger quantities, you would want to stock up on both. Now it may be the case that there is a group nearby preaching about the benefits of taking Coca Cola with Cranberies ( How great examples I come up with !), and you are unaware of it. But you find that the sales are going up. What will you want? You would definitely want more stock of the same. You might want to know about the cause but you would definitely not worry about it as long as the sales are going up. Now there is a correlation between Coca Cola and Cranberry sales, which did not exist before. This correlation does not imply causation but it does imply that people are buying both with decent linkage and you need to stock up more.
Such patterns as taken in the example above are often hidden deep within the data. Without large computational powers, the techniques are useless and not feasible. However, with the enhanced computation power, machine learning provides an excellent opportunity to find such patterns. Even though, you are not aware of the cause, just the awareness about the pattern helps you make better decisions. Sometimes some correlations are because of causation as well. If you find that, jackpot, otherwise also you get lot more information than before.
It is the computation power that can simulate millions of future scenario with thousands of different variables to suggest the best alternate. Sometimes elegant methods work best, at other times; brute force takes you through. The net result remain that with increased power, comes more and more empowerment. Geniuses have been known to exist throughout humanity. Some of them thought about methods in their pre computer era which can only be utilized now. I am avoiding listing algorithms in a trial to remain closer to the lay man of the field.
Everyone knows of Leonardo Da Vinci. It is known that he made sketches of what could have evolved into a helicopter. However, the real helicopters came ages after his death. The technological advances have made his dreams into reality. There have been many such geniuses who dreamed of a lot of things and tried to provide a solution. Those solutions are now being emulated, simulated and implemented across the world to help people have a better life.
Who would have thought years back, that google will tell you what to do. Google Now makes your life easy because your phone has more computational power than a desktop around 5 years back.Google has the data and definitely the computational means to provide you guidelines in advance. You see your current weather before leaving home, it even tells you if the traffic is bad in your current route, etc etc. How is it able to do this for so many people remotely. The answer is simple, it collects a lot of data about your usage and also possesses the computational power to make sense of that data.
What companies need to realize that this alone is not sufficient. Increase in computational power definitely has had far reaching consequences on the world but if everything could be shown with numbers alone, the artists of the world will feel betrayed.
