Can you work with Big Data in Excel? From the barrage of recent news, white papers, and sales calls about Big Data, you would think not. A common theme is that “spreadsheets can’t handle Big Data and advanced analytics,” and that companies need to “move up” to new tools, that the vendors with the white papers offer -- implicitly, the benefits outweigh the expense and steep learning curve.
Over the past decade, thousands of organizations have made their data available to the public, either by posting an extract online or through an API. These data sets range from government data to restaurant reviews to metadata on songs. One of the more famous data sets released recently was the New York City Taxi data, which was posted by the NYC Taxi and Limousine Commission in 2013 as a result of a Freedom of Information Law (FOIL) request.
Trevor Hardy of the Canadian Football League -- Canada's counterpart to the NFL -- faced the task of scheduling 18 games for nine clubs throughout the 20-week season, balancing competing priorities for revenue, TV ratings, or player time -- while meeting a wide range of constraints, from times of day for fans in four time zones to traditional Labor Day rivalry match-ups. Here's how he used optimization in Microsoft Excel to create significantly better schedules than the league had been able to create by hand.
Choosing vendors to serve a business or government organization is essential. Several vendors compete for contracts during the bidding process, and it is up to the administrator to make a wise and impartial decision. With many variables to weigh in the decision, optimization software can solve the relationships between the variables and find the optimal solution. This method is both optimal and impartial, and it is faster than human analysis.
“Big data” is more than a buzz-word. But if you have a lot of data that you aren’t using effectively, “big data” is not the solution – spend your energy instead learning, and getting value from the data you have.
Our take: One of the ways Apache Spark is making data analysis easier is in Frontline’s Analytic Solver Platform. Its built-in link to Apache Spark clusters makes big data analysis a point-and-click operation -- no R, Java or Python programming required.
Data science is an extraordinarily helpful tool to monitor company operations and trends in demand to make important business decisions, but can it be used to measure happiness? According to this post from Forbes, the people of the Kingdom of Bhutan think so. By hosting an upcoming workshop with data scientists, Buddhist leaders and others in power in various fields of importance, the government of Bhutan hopes be able to measure what they call Gross National Happiness, or GNH.
