When one starts a new project that involves analyzing his company's data especially when the data is stored across functional areas, that person is in trouble. The data lake model helps in this case. To get access to data doesn't require an integration effort, because data is already there in the lake and one can apply MapReduce and other algorithms to use it. In the lake some data are unstructured or not structured by us for a given project. To construct a data lake one needs to learn some of the Hadoop stack such as Sqoop, Oozie and Flume. Next a data scientist should be found who understands Hadoop as well as business and the company’s business data in particular. Then one should start with basic cases and use simple and familiar tools like Tableau to make nice charts, graphics, and reports demonstrating that he can do something useful with the data. Next security up front should be considered, as well as who can access what data. Use of core Hadoop platform is beneficial. Apart from this one should keep in mind that lake security may have business unit implications and one should not have a lot of mini lakes i.e. data ponds that are separate and not equal. Read more at:http://www.infoworld.com/d/application-development/how-create-data-lake-fun-and-profit-246874?page=0,0

Nowadays a new set of tools is developed to help business users visualize data and interact with it in real time. One may think of mobile reporting as a good device for business but it is not good enough indeed and we need analytics tools too. Mobile analytics can be quite different from traditional business intelligence, in fact, and may even change the way you work with data comfortably behind the desk. Three basic aspects of mobile analytics are- For any application, the touch interface makes the difference between mobile and desktop environments. It encourages exploration. It is possible to make interactive areas large enough to touch through a mobile device. Mobile analytics also demands responsive interface design and the analysis must work across many devices. A responsive interface supports many form factors and one’s BI platform if designed for mobile ought to handle these scenarios generically. Mobile analytics also helps in speedy discovery and facilitates an interactive experience which is touch-enabled. These practices not only enable, but encourage exploration.

Read more at:http://www.infoworld.com/t/business-intelligence/getting-your-hands-data-mobile-analytics-246331?page=0,0

One can use the big data, available in hand, in a right or a wrong way. Here is the list of top 10 worst big data practices which one should try to avoid. First, though MongoDB has an aggregation platform, it is not good as an analytical system and thus should not be used as big data platform. Second, RDBMS schema is used as files by many which should be avoided too. Third, creating a series of data points. Fourth, failing to develop use cases. Fifth, over-dependence on Hive should be reduced as the whole point of big data is to expand beyond what one could do with one technology. Sixth, it's not right to treat HBase like an RDBMS. Seventh, trying to install Hadoop and all its moving parts on 100 nodes by hands is also a worst practice. Eighth, one should also avoid RAID/LVM/SAN/VM-ing one's data nodes. Ninth, instead of treating HDFS as just a file system one needs to think about how one is going to secure all of this and for whom. Finally, everyone is free but each one should have a plan. Read more at:http://analytics.theiegroup.com/article/53c925453723a81857000073/The-10-Worst-Big-Data-Practices-

Massive amount of climate data waits to be  interpreted by the press, public agencies, and general public and the challenge lies in finding analytics software that is easy to use, and produces understandable results, and can handle the volumes of big data available. However, to get a handle on the sort of governmental data on climate and resources that often resides in generic comma-delimited files, Circle of Blue, who specializes in reports on the global competition for water, food, energy in a changing climate, uses QlikView Business Discovery Platform. Such visual analysis software helps users to view, explore, and interpret the data with little technical training. Circle of Blue hopes to make the public more informed about the vulnerability of water supplies in the era of climate change by merging technology with on-the-ground reporting and online networks. With the use of Qlik View platform, it becomes easier for them to deliver data to a wide range of people. QlikView dashboards work with the large data sets to produce sophisticated, engaging, and state-of-the-art graphics. The data can be scaled to compare local information with national and global trends and that information, in turn, can help in formming public policy discussions.

Read more at:http://tdwi.org/Articles/2014/07/08/Massive-Climate-Data-Awaits-Analysis.aspx?Page=1

A Survey, based on the responses from 111 data scientists in US, found that Hadoop is too slow according to 76% of data scientists as they believe that the open source software framework requires too much effort to program and isn't fast enough to keep up with big data demands. On the other hand almost 91% of the survey respondents claim that they are performing complex analysis of data on the basis of which 39% of overall respondents say that their job is getting tougher. However, Big Data is becoming highly important for all enterprises. According to a research commissioned by Dell and conducted by Competitive Edge Research, a big section of midmarket companies with 2,000 to 5,000 employees are embracing the rise of big data and almost 80% percent of the midmarket thinks they need to better analyze their data, as they believe big data initiatives provide a significant boost to company decision making. Read more at:http://analytics.theiegroup.com/article/53baa9d23723a81e1300007b/Survey-Finds-Hadoop-Is-Too-Slow-Big-Data-Is-Too-Complex

Today, with the rise of mobile devices and simple health trackers, people will soon be able to analyze their own health data themselves. The proliferation of mobile devices has helped liberate the insights from that huge amount of data organizations are collecting. For years, big data and analytics has been the solitary domain of the enterprise and today there is no shortage of people in analytics space, from traditional enterprise players such as Oracle, IBM, SAP Business Objects, to relative newcomers such as Roambi, Tableau, and Pentaho. While businesses are analyzing big data to make decisions, individuals will soon be able to analyze big data to improve their own lives. Consumer can also choose which fitness band to use to check calories, number of steps, activity level, heart rate, sleep patterns, and so on. With this type of data collection, real time biometrics could help in reaching out alerts to doctor so that it can save lives. New innovations will allow individuals to compare their health metrics to others in similar demographics. Thus, analytics along with the interconnection between mobile device, wearable devices and appliances, we will soon have access to greater insights to improving our health.

Read more at:http://analytics.theiegroup.com/article/53a7f76e3723a85c3a0000a1/The-Health-Tech-Revolution-Will-Turn-All-Of-Us-Into-Big-Data-Wonks

In spite of being important, big data analytics is yet to be deployed successfully by most of the organizations. Many companies are struggling with how to maximize big data, and properly incorporate the results into something substantial. Results of the survey showed that the investment in analytics was growing rapidly. 64.4 percent of those surveyed said that their firm is investing more in analytics. However, just 12.6 percent of respondents said their company has completed several big data projects. One reason that prevents organizations from moving forward despite understanding the benefits of big data analytics, is the shortage of expertise in the field and with such lack of big data skills organizations are reluctant to take the plunge. It is also a major concern to keep sensitive information from the gathered big data, secured. On the basis of Big data analytics businesses should conduct their own research and see what options best fit their needs. However, technological innovation should be pursued to make big data analytics accessible to ordinary business users as without such innovation business could be left behind. Read more at:http://analytics.theiegroup.com/article/53a04cf93723a81d72000021/Is-Big-Data-Just-A-Big-Problem

The impact of big data in health care is tremendous and it has potential impact on every person as well. It helps in the advancement of disease diagnosis and treatment. Big Data is able to determine whether men need to undergo prostate cancer surgery or not, also can assess the risk of heart disease later in life, based on our health status as teenagers. Genomics, the genetic information, aims to discover the basis of heritable traits and understand how genes work to prevent disease and we may soon be able to see Web-based patient profiles that aggregate genomic data with other types of Big Data and produce "risk map" mobile apps that people can download to a smartphone. If it is about the hospital treatment, then also comes the importance of big data which requires the integration of information including admissions, records, nursing, diagnostic imaging, rehabilitation and home care. Researchers around the world are investigating ways to access, analyze and apply Big Data in healthcare. Corporations are looking for ways to use it to support their product development. Moreover, regardless of whether it's how patients are treated in the hospital or how they keep themselves healthy at home, they are learning about, interacting with and embracing Big Data .

Read more at:http://analytics.theiegroup.com/article/53a31b043723a81ea9000096/Making-It-Personal-Why-Everyone-Should-Care-About-Big-Data

With the advent of web and social media the speed of the evolution of language has increased dramatically. There are many contributing factors to language that affect the changes. Big data takes linguistics to the next level and the technology like Hadoop helps in assisting interested parties in gaining deeper and clearer insights into linguistics. The reasons why Linguistics should be understand are that- Firstly, to benefit from the insights into linguistics provided by big data whether it may be vocabulary or grammar or something else. Secondly, today's technology continues to develop and improve, the use of voice commands for phones, TV's and game systems is going to increase and it's more important that developers understand the language people will be speaking to their devices in order to ensure the responsiveness. Big data will greatly enhance their ability to provide such speech oriented aspects. Thirdly, in case of learning a language and the way it is learned, understanding of linguistics matters a lot. Finally, to understand the past and looking to the future, it is again important to understand linguistics. With big data technology, the huge amount of data and information can be gathered and used to provide better insights into the past and future of language. Read more at:http://analytics.theiegroup.com/article/53bd6b6d3723a864d8000023/The-Impact-Of-Big-Data-On-Linguistics

Despite having the technological advancements which makes the process of job finding easier and user friendly, the whole thing is not so easy and in many cases it's more complicated. Here comes the role of Big Data which is now making its way into the field of recruitment and helps recruiters to find best people for the right positions. Some companies receive thousands of resumes for a single post and with such a huge number of resumes, companies are engaging in people analytics, applying big data analytics practices to a field of prospective job seekers. Many businesses build their own resumes of candidates by identifying details from people's social media profiles on Facebook, LinkedIn, Twitter, and other sites. From these profiles, companies use big data to identify patterns of behavior, interests, skills and attitudes that they are qualifying factors for current and future job openings. So, job seekers need to manage their profiles to get a job. But, as a negative impact, there are concerns over relying on it too much and as a matter of fact a heavy use of big data also takes factors like race, gender, and religion out of the equation. However, despite the drawbacks big data is of vital importance and job seekers should put themselves in a position to take advantage of big data and utilize it to get noticed.

Read more at:http://analytics.theiegroup.com/article/53b522e83723a80d7e000065/The-Modern-Job-Hunt-How-to-Beat-the-Big-Data-System

Insurers are today trawling big data about customers’ finances and other sources. The development shows how access to big data is revolutionizing the insurance industry and companies are accessing a wide range of information on everything from financial probity to shopping habits of the customers to determine the risk premium for individual customer. One official of AA insurance company said that the winners in the insurance market will be the ones that have got the data insights that others don’t have. It could be supermarkets, banks or social media companies. However, ultimately the big data trend means some policyholders will pay more than others not for the climate change only but also for insurers to know more about the risks posed by particular properties. But, as with the sensitive data some companies also go far beyond what customers would expect them to do with it, customers should be given the option to opt in to such analysis than allowing them to opt out. Read more at:http://analytics.theiegroup.com/article/53c7a5c03723a8014f0000cd/Insurers-Trawl-Big-Data-For-Clues

Today,banks are using analytics to control frauds in electronic payments. Private banks such as HDFC Bank have implemented analytics software. The thing is that the out-of-the way transaction would decline if one fails to respond to a phone call or message immediately after the transaction. There are two kinds of fraud detection in payments- One is during the transaction, and the other is using analytics to identify suspicious transactions based on past behaviour. Banks are now specializing in the analytics part. According to an official of an analytics software company that provides banks with software to detect frauds, the software can be used to personalize services, like ATMs, for customers depending on their preferences. Read more at:http://timesofindia.indiatimes.com/business/india-business/Banks-use-analytics-to-check-fraud-in-online-payments/articleshow/38347410.cms

The desire to be mobile and make a mark is not new to us. Today we use GPS for wherever we go and communicate on a variety of devices. Big data is the phenomenon which has helped in generating and sharing information. The difference between data in ancient times and now is that before, only humans created and collected data whereas now with the rise of sensors and other technology that creates and collects data. However, the big thing about big data is the self-organization i.e. without human intervention and awareness, data is organizing itself.  But, this leads to a big question that-Are we playing with fire? With big data revolutionizing, there comes a new responsibility, because the purpose of managing data is not to predict the future but to shape it which is a huge responsibility. However, revolution hasn't stopped. Changes took place slowly in the evolutionary manner. Using technology that provides insight into data, today's business leaders have a unique opportunity to make thoughtful decisions that will have long-lasting impacts. But along with all this, disruptive changes are happening in every industry around the world which increasingly making us concern about whether today's leaders rise to the challenge of shaping the future in a responsible way or not. Read more at:http://analytics.theiegroup.com/article/53c3f6413723a87216000156/Shaping-The-Future-With-Big-Data-Are-We-Playing-With-Fire

Importance of Big data and Talent Analytics is a much discussed topic for the last couple of years. One just has to know how to use talent analytics as it can do a whole lot of difficult task like quantifying different behaviors, skills, intelligence, and mindsets of a HR. Talent analytics uses data in management decisions like talent acquisition, retention, placement, promotion, compensation, and succession planning. By analyzing the skills and attributes of high performers in the present, it enables organizations to build a template for future hires. Advanced software algorithms can identify talent and match it to an organization's needs. Some intangible aspects like social skills, flexibility, emotional intelligence, initiative, attitudes are now measurable- thanks to talent analytics. Along with these, new mobile apps also make talent searches a matter of anytime and anywhere. However, as it's still growing, Gartner predicts that the market for Big Data and analytics will generate $3.7 trillion in products and services and generate 4.4 million new jobs by 2015. Read more at:http://analytics.theiegroup.com/article/53b2b9c13723a81923000046/Talent-Analytics-A-Crystal-Ball-For-Your-Workforce

Today big data is able to change the way real-estate professionals, buyers, sellers and even banks think about transactions involving property. Companies, promoting services that plug consumers into big data real estate information which gives a better education and insight.There are five key ways through which big data is changing the game of real estate business. They are- 1. Big data helps to democratize data for the real-estate customer. Companies such as Zillow combine big data with real estate and offer services like- Mining census information, the results of consumer surveys, listings of homes for sale and rent etc. 2. Big data is not only providing new information to consumers but also new ways of looking at developments and community planning. From that gathered information, real-estate developers can learn what kinds of spaces work best in terms of tenant health, energy efficiency and other points. 3. Institutions like banks are also able to plug into big data resources. 4. As an expert real estate advisor knows recent sales, incentives, and inside secrets to getting the best deal, buyers should have to be careful and should start property searching with the help of big data sites. 5. Finally, Big data let the professionals know what visitors are doing when looking for real estate online and they adjust their paid and organic efforts based on this data daily. Read more at:http://analytics.theiegroup.com/article/53be79af3723a84f1000003f/5-Ways-Big-Data-Is-Changing-Real-Estate

Analytics for Small Medium Businesses (SMB) today, is a much discussed topic. SMBs face the challenges of effectively using analytical tools to gain precious business insights from data generated. Today markets are able to provide solutions to SMBs which were costly before and this was made possible by the advent of Qlikview, Tableau etc. in Analytics sector. SMBs are realizing that analytics can help them understand customer preferences, expand their market share, cut down cost, increase efficiency and give them a competitive advantage even against the big players. Moreover, with the advent of new technologies like cloud, social media and open source platforms like Pentaho and Hadoop, the requirement for big infrastructural set-up and capital cost have been reduced considerably. The success of Analytics tools depends to a large extent on collecting and managing data and in such case ERP and CRM tools are a must for success. For successful implementation of analytics tool, SMBs need to assess the external market as well as their internal systems and processes. However, SMBs will soon be able to adapt their systems to bring in the external big data from social media like any other big enterprises and hence make their analytics more robust. Read more at:http://www.informationweek.in/informationweek/perspective/296888/value-analytics-businesses

Today, a smartphone app would tell one what to eat, how much to eat, when to visit a doctor etc. based on analysis of medical research, medical history of that person, family medical records etc. Quality of our health will be increasingly improved by the quality of data and the ability to bring it all together. The growth of big data in the health industry will only take place once privacy concerns are addressed because health data, unlike the marketing data, is lot more personal. Big data analysis is already being used to make diagnoses in some hospitals. In Canada, Toronto Hospital uses big data to detect blood infections in premature babies. It could save the American health care system $300 billion per year and the European public sector €250 billion, according to a 2011 report. Doctors today are using Watson, IBM's supercomputer, to keep up with health research and to leverage the latest breakthroughs. Big data analytics also could be used to follow epidemic outbreaks. For example, Big Data enabled doctors and scientists to learn so much about the Severe Acute Respiratory Syndrome (SARS), and how quickly it spread, within weeks of the World Health Organization's initial warnings. In such case social networks and mobile data are used to ensure the delivery of real-time information. Read more at:http://analytics.theiegroup.com/article/53b6b4c43723a83b82000035/Big-Data-Could-Help-Your-Health-If-You-Let-It

Generally people think that the role of a data scientist is just to examine the relationships between diverse sets of data as well as the disparate systems, processes and locations which store them. But the role is actually mature across certain sectors like retail. With the help of this, Amazon, for e.g., is able to analyze the behavior across multiple accounts, and knows exactly when and why to push a certain product to a customer. But the case is somewhat different in financial services where the role is not properly organized. Though Big Data analytics is used across the retail banking industry from fraud and sanctions management to improving account management processes, analysis of Big Data provides the potential for banks to create new income streams and the sector as a whole is benefitted when it comes to deriving value from vast quantities of information. Thus financial services, in spite of having people with good skills to do modeling and statistical analysis, need people who are able to spot key trends and focuses on looking for the relationships between data across disparate sources. Once these two skills are combined, the financial sector will start to see the rise of data scientists in it like other industries. Read more at:http://www.banktech.com/business-intelligence/piecing-together-the-data-scientist-puzz/240168604

Manufacturing industry, in the past 20 years, have been able to reduce the quantity of wastage and the variability in the production process and improve their product quality after implementing Lean and Six Sigma programs. However, extreme variations are found in certain processing environments. Thus manufacturers need a better approach that would remove such flaws and advanced analytics helps in this way.  In manufacturing, managers use advanced analytics to identify patterns of data, relationships among discrete process steps and inputs and then optimize the factors that greatly affect the yields. Advanced analytics also helps to increase yield. Manufacturers that want to use advanced analytics to improve yield, consider how much data the company has at its disposal, as their first step. Some companies have too little data to be statistically meaningful and the challenge for these companies lies in taking a long-term focus and investing in systems and practices to collect more data. Advanced analytics and big data forms a critical tool to realize improvements in yield. Process complexity, process variability, and capacity restraints are present in such manufacturing environment. Read more at:  http://www.mckinsey.com/insights/operations/how_big_data_can_improve_manufacturing

Is our privacy at risk?- This is the most discussed question in today's world of big data where companies like Google is able to capture data from homes and offices as well as video footage for storage anywhere in cloud, provided by Amazon Web Services. According to Danielle Hughes of Divine Capital, computers are learning to interact with one another and this is raising concerns. Though, it is true that people are living in the post-privacy world today as younger ones have no issues in sharing their personal information in social media. But, Hughes thinks, this is the beginning and in the future machines will start to teach other machines and tell back the information analytically to big companies. She also concludes that it will not lack investments in future, as for example IBM is already projecting $20 billion in revenue from big data in 2015. Read more at:http://analytics.theiegroup.com/article/53ac20613723a8031500002b/Our-Connected-World-Big-Data-Big-Profits-And-No-Privacy