/home/leansigm/public_html/components/com_easyblog/services

SigmaWay Blog

SigmaWay Blog tries to aggregate original and third party content for the site users. It caters to articles on Process Improvement, Lean Six Sigma, Analytics, Market Intelligence, Training ,IT Services and industries which SigmaWay caters to

Hadoop Adoption Ahead

The mission of Matt Morgan, the vice-president of global product marketing of Hortonworks is to establish Hadoop as the foundational technology of modern enterprise data architecture. Hortonworks Data Platform (HDP 2.3) is the only enterprise Hadoop-based platform that is made up of 100% Apache open source components. Enhanced security and data governance have been added to HDP 2.3 including new encryption of data, and the extension of the data governance initiative with Apache Atlas. But many doubt that skill shortage is one of the barrier to Hadoop adoption. Read more about this article at:  http://www.cmswire.com/big-data/is-hortonworks-paving-the-way-for-pervasive-hadoop-adoption/

 

Rate this blog entry:
5174 Hits
0 Comments

Big Data In A Future IT Landscape

For a future IT landscape dominated by Big Data technologies, it is crucial to use technologies and tools for Big Data. Hadoop and MongoDB are designed to perform in the Cloud which gives firms ability to scale computing and storage resources. Also, data professionals have to undergo proper training to acquire skills. Data stored in cloud should be according to the company's security policies and compliance laws. Mastering the art of messages, promotions, and marketing on a micro-level, will lead to more customer-centricity, and deeper customer engagement, will ultimately result in better ROI. Read more about this article at: http://www.marketingprofs.com/articles/2015/27754/seven-ways-to-get-ready-for-big-data-of-the-future

Rate this blog entry:
4399 Hits
0 Comments

Public Transport Improved By Big Data And IoT

Transport for London (TfL) data, collected through ticketing systems, censored vehicles, traffic signals, survey groups etc. is provided through open API's for 3rd party app developers. This data is then used to produce maps showing when and where people are traveling, and allowing analysis at the level of individual journeys by using Big Data. The key priority to initiate this data was to provide travel information which gives the routes customers use and to send travel updates to them. Thus Bernard Marr from Forbes in his article showed how big data played a big part in re-energizing London's transport network. Read more about this article at: http://www.content-loop.com/big-data-internet-things-improve-public-transport-london/

Rate this blog entry:
5247 Hits
0 Comments

IoT can help in tracking operations: A Study

IoT technology is not only for retailers or e-commerce, but it can be also used by critical industries like oil and production for tracking events. Let's discuss how IoT can help these industries? Traditional method of communicating and controlling operations in large industries is SCADA (supervisory control and data acquisition). SCADA works by getting information from the central controller. But IoT sensitivity detectors can track information from both mobile devices and other communications. These devices not only send key messages directly to the sites, but also provide better and simpler communications along with sending information into headquarters. It provides lower cost and improved communications. To know more about, how IoT can help critical industries in better operation, please follow this link:  http://recode.net/2015/05/28/the-iot-and-the-needs-of-business/

Rate this blog entry:
4996 Hits
0 Comments

How can Banks utilize Big Data?

Proficiency in Big Data provides a competitive advantage to banks. Banks too often depended on traditional technologies such as aggregation and normalization of data which resulted in several weaknesses like lack of flexibility in responding to upstream and downstream data changes. Data lineage may be lost after aggregation and summarization and data governance is likely weakened when several constituents retain responsibility for an extended, multi-stage data flow. These weaknesses are detrimental to the success of big data initiatives. So a new approach is required.  Big data represents a new way that banks can interact with and leverage their data. As a result, banks need to shift the paradigm for designing, developing, deploying, and maintaining big data solutions with new approaches to data storage (e.g., NoSQL databases)  and maturity of distributed-computation software frameworks (e.g., Hadoop). The approach to Big Data implementation also needs to change through rapid, iterative, and incremental deployment of solutions in a way that aligns well to the speed at which the underlying data are measured, understood, and parsed. This will take banks to an acceptable level of competency and capability. Read more at:

http://www.informationweek.in/informationweek/news-analysis/297426/mean-banks

Rate this blog entry:
5689 Hits
0 Comments

Big Data Platform in the Cloud for SMBs

Nowadays we are witnessing an enormous data explosion which is set to continue and even accelerate. The volume of data is growing at a very high velocity and is rapidly becoming more varied, complex and less structured. As a result, the word Big Data has grown strong on the mind of every business leader who wants to extract critical insights and business benefits from data. Many organizations are planning to implement Big Data related initiatives or have got them already. However most organizations lack an articulated strategy for Big Data execution. Thus there is a strategy gap between high potential and risk about investing in Big Data initiatives. Although, the essential mix of technologies may deliver on the promise of Big Data, what leaders must choose and incorporate for interlocking the set of available data sources and technology is a specific business goal which makes the initiative unique. In response, Big Data providers have prepared a strong ground through the use of cloud computing at the core which address these issues. Organizations can analyze the feasibility and cost of investing. Termed as "Big-data-as-a-service" (BDaaS), it basically refer to services that provide analysis of enormous or complex data sets, typically over the cloud platform as a managed service. The adaptation of Big Data on above grounds precedes with Hadoop which was a major stepping stone, but it still has its own limitations, specifically for Small and Medium Businesses (SMBs) that do not have the resources to create a Hadoop infrastructure in house. Thus, on a conclusive note, It's difficult to predict which Big Data solution businesses will freeze on, but having a majority of Big Data service providers now providing a version of their platform in the cloud, it will emerge a safe bet for SMBs to venture, in wherein cloud will play a major role in building Big Data an integrated part of their business strategy. Read more at:

http://www.informationweek.in/informationweek/perspective/297257/cloud-leveling-playing-field-smbs

Rate this blog entry:
5508 Hits
0 Comments

Creating data lake to make profit

When one starts a new project that involves analyzing his company's data especially when the data is stored across functional areas, that person is in trouble. The data lake model helps in this case. To get access to data doesn't require an integration effort, because data is already there in the lake and one can apply MapReduce and other algorithms to use it. In the lake some data are unstructured or not structured by us for a given project. To construct a data lake one needs to learn some of the Hadoop stack such as Sqoop, Oozie and Flume. Next a data scientist should be found who understands Hadoop as well as business and the company’s business data in particular. Then one should start with basic cases and use simple and familiar tools like Tableau to make nice charts, graphics, and reports demonstrating that he can do something useful with the data. Next security up front should be considered, as well as who can access what data. Use of core Hadoop platform is beneficial. Apart from this one should keep in mind that lake security may have business unit implications and one should not have a lot of mini lakes i.e. data ponds that are separate and not equal. Read more at:http://www.infoworld.com/d/application-development/how-create-data-lake-fun-and-profit-246874?page=0,0

Rate this blog entry:
5593 Hits
0 Comments

Top ten worst Big Data practices

One can use the big data, available in hand, in a right or a wrong way. Here is the list of top 10 worst big data practices which one should try to avoid. First, though MongoDB has an aggregation platform, it is not good as an analytical system and thus should not be used as big data platform. Second, RDBMS schema is used as files by many which should be avoided too. Third, creating a series of data points. Fourth, failing to develop use cases. Fifth, over-dependence on Hive should be reduced as the whole point of big data is to expand beyond what one could do with one technology. Sixth, it's not right to treat HBase like an RDBMS. Seventh, trying to install Hadoop and all its moving parts on 100 nodes by hands is also a worst practice. Eighth, one should also avoid RAID/LVM/SAN/VM-ing one's data nodes. Ninth, instead of treating HDFS as just a file system one needs to think about how one is going to secure all of this and for whom. Finally, everyone is free but each one should have a plan. Read more at:http://analytics.theiegroup.com/article/53c925453723a81857000073/The-10-Worst-Big-Data-Practices-

Rate this blog entry:
6049 Hits
0 Comments

Survey claims Big Data is too complex and Hadoop is too slow

A Survey, based on the responses from 111 data scientists in US, found that Hadoop is too slow according to 76% of data scientists as they believe that the open source software framework requires too much effort to program and isn't fast enough to keep up with big data demands. On the other hand almost 91% of the survey respondents claim that they are performing complex analysis of data on the basis of which 39% of overall respondents say that their job is getting tougher. However, Big Data is becoming highly important for all enterprises. According to a research commissioned by Dell and conducted by Competitive Edge Research, a big section of midmarket companies with 2,000 to 5,000 employees are embracing the rise of big data and almost 80% percent of the midmarket thinks they need to better analyze their data, as they believe big data initiatives provide a significant boost to company decision making. Read more at:http://analytics.theiegroup.com/article/53baa9d23723a81e1300007b/Survey-Finds-Hadoop-Is-Too-Slow-Big-Data-Is-Too-Complex

Rate this blog entry:
5484 Hits
0 Comments

Big data in understanding Linguistics

With the advent of web and social media the speed of the evolution of language has increased dramatically. There are many contributing factors to language that affect the changes. Big data takes linguistics to the next level and the technology like Hadoop helps in assisting interested parties in gaining deeper and clearer insights into linguistics. The reasons why Linguistics should be understand are that- Firstly, to benefit from the insights into linguistics provided by big data whether it may be vocabulary or grammar or something else. Secondly, today's technology continues to develop and improve, the use of voice commands for phones, TV's and game systems is going to increase and it's more important that developers understand the language people will be speaking to their devices in order to ensure the responsiveness. Big data will greatly enhance their ability to provide such speech oriented aspects. Thirdly, in case of learning a language and the way it is learned, understanding of linguistics matters a lot. Finally, to understand the past and looking to the future, it is again important to understand linguistics. With big data technology, the huge amount of data and information can be gathered and used to provide better insights into the past and future of language. Read more at:http://analytics.theiegroup.com/article/53bd6b6d3723a864d8000023/The-Impact-Of-Big-Data-On-Linguistics

Rate this blog entry:
5063 Hits
0 Comments

Importance of Analytics for SMBs

Analytics for Small Medium Businesses (SMB) today, is a much discussed topic. SMBs face the challenges of effectively using analytical tools to gain precious business insights from data generated. Today markets are able to provide solutions to SMBs which were costly before and this was made possible by the advent of Qlikview, Tableau etc. in Analytics sector. SMBs are realizing that analytics can help them understand customer preferences, expand their market share, cut down cost, increase efficiency and give them a competitive advantage even against the big players. Moreover, with the advent of new technologies like cloud, social media and open source platforms like Pentaho and Hadoop, the requirement for big infrastructural set-up and capital cost have been reduced considerably. The success of Analytics tools depends to a large extent on collecting and managing data and in such case ERP and CRM tools are a must for success. For successful implementation of analytics tool, SMBs need to assess the external market as well as their internal systems and processes. However, SMBs will soon be able to adapt their systems to bring in the external big data from social media like any other big enterprises and hence make their analytics more robust. Read more at:http://www.informationweek.in/informationweek/perspective/296888/value-analytics-businesses

Rate this blog entry:
5275 Hits
0 Comments

Big data needs big and object-based storage

Big Data is about large volumes of unstructured data along with rapid analysis with insights being noted within seconds. Big Data allows narrower customer segments and help in tailoring precise products and services which will then allow for companies to develop the next-generation products and services. The fact is that Big Data requires more capacity, highly efficient accessibility. It would require scale-out or clustered storage systems - such as scale-out NAS (Network Attached Storage) which can scale out to meet capacity and uses systems which are distributed across many storage devices and can handle billions of files without degradation of performance. Big Data using Hadoop stack has been gaining acceptance widely. Also, organizations which create and store more transactional data in digital form can collect more accurate and detailed performance information on everything. RAID-based storage systems have huge storage capacity but not necessarily what Big Data requires and RAID based systems cannot protect data from loss. Thus, most IT organizations incur additional costs which use RAID for Big Data storage as they need to copy it two or three times to protect it from loss. Read more at:http://www.informationweek.in/informationweek/perspective/296730/environments-object-storage

Rate this blog entry:
5019 Hits
0 Comments
Featured

Rapid Miner & Hadoop: Turning Big Data into Action!

rh_1

Rapid Miner had an existing partnership with Radoop - an analytics company that optimizes the big data platform known as Hadoop. Now, after successfully acquiring Radoop, Rapid Miner will be able to provide access to many other Hadoop features to its customers which will in turn build a larger presence in the Hadoop ecosystem for RapidMiner. The acquisition also brings partnerships with Hadoop platforms Cloudera and Hortonworks, and adds 20 new clients to RapidMiner’s customer base. The powerful combination of RapidMiner and Radoop will allow applications of advanced analytics to big data. Apart from providing scripting and advanced predictive analytics for experts, it will also help non-technical people to access, analyze, and visualize big data.

To read more, Visit the following link:

http://betaboston.com/news/2014/06/17/rapidminer-acquires-big-data-analytics-company-radoop/

 

 

 

Rate this blog entry:
14132 Hits
0 Comments

Docker ported into Hadoop as benchmarks show SCREAMING FAST performance

Docker is an open source Linux containerization technology which lets an admin run multiple apps with all their dependencies in secure sandboxes on the same underlying Linux OS. It makes an attractive alternative to typical virtualization. Hadoop community is working on patches that will bring Docker into the data management system. To know more, go through the article by Jack Clark, World's only Distributed Systems Reporter.

http://www.theregister.co.uk/2014/05/02/docker_hadoop/

Rate this blog entry:
7743 Hits
0 Comments

Red Hat forges Hortonworks engineering pact, ties storage into OpenStack

Red Hat outlined engineering partnership with Hortonworks to collaborate on enabling more storage file systems. The integration allows Hadoop to run directly on a POSIX(Portable Operating System Interface)-compliant storage node. The two companies will create test suites to validate compatibility between Hadoop and alternative file systems, which will be contributed to the open source community. To know more on this, go through the article by Larry Dignan, Editor in Chief of ZDNet and SmartPlanet as well as Editorial Director of ZDNet's sister site TechRepublic.

http://www.zdnet.com/red-hat-forges-hortonworks-engineering-pact-ties-storage-into-openstack-7000016801/

Rate this blog entry:
6424 Hits
0 Comments

Red Hat throws its hat into the Big Data ring

Red Hat has gathered a selection of open source software to create a Big Data development and deployment environment. It announced a software stack which includes Red Hat Enterprise Linux, Red Hat Storage, a Hadoop plug-in allowing Hadoop to process data stored using Red Hat Storage, Red Hat Enterprise Virtualization etc. The company has thrown its hat into the ring to compete with and cooperate with many in the Hadoop community. To know more on this, go through the article by Daniel Kusnetzky, a reformed software engineer and product manager.

http://www.zdnet.com/red-hat-throws-its-hat-into-the-big-data-ring-7000011680/

Rate this blog entry:
6340 Hits
0 Comments

Hadoop's rise: Why you don't need petabytes for a big data opening

According to Forrester principal analyst Mike Gualtieri, "Hadoop is not big data. It's a big-data technology. You can break down the siloes but Hadoop is also a framework for processing the data". So, proper handling of big data is very essential for any company. Hadoop can help in explaining corporate data by managing it across clusters of commodity hardware. Research shows that many of the big companies using Hadoop. To know more on the advantages of Hadoop, follow the article by Toby Wolpe, senior reporter at ZDNet in London:

http://www.zdnet.com/hadoops-rise-why-you-dont-need-petabytes-for-a-big-data-opening-7000028282/

Rate this blog entry:
7121 Hits
0 Comments

Red Hat and Hortonworks unveil Hadoop big data collaboration

The two open-source software companies, Red Hat and Horton Networks are aiming at speeding up enterprise big data Apache Hadoop projects. The two firms unveiled the beta of Hortonworks Data Platform (HDP) combined with Red Hat Storage, which is designed to give companies a unified storage pool for all Hadoop workloads. They also announced the integration of HDP with Red Hat JBoss Data Visualization, enabling Hadoop to work with existing data sources including warehouses, SQL and NoSQL databases, on premise and cloud apps and flat and XML files. To know more on this aspect, go through the article by Toby Wolpe, senior reporter at ZDNet:

http://www.zdnet.com/red-hat-and-hortonworks-unveil-hadoop-big-data-collaboration-7000026154/

Rate this blog entry:
8856 Hits
0 Comments

Google’s Cloud Platform Gets Improved Hadoop Support

Long ago, Google has integrated Hadoop on its cloud platform to provide customers a framework for storing and processing large amount of data. Earlier, the only way to get in and out of Hadoop on Cloud Platform was through Google’s Cloud Storage service. In order to run Hadoop on data within its “BigQuery SQL” and “Cloud Datastore NoSQL” databases, recently Google has launched connectors for both of these two products, complementing the existing Cloud Storage implementation.

To read more, visit the following link:

 

http://techcrunch.com/2014/04/16/googles-cloud-platform-gets-improved-hadoop-support-with-bigquery-and-cloud-datastore-connectors/

Rate this blog entry:
15373 Hits
0 Comments
Sign up for our newsletter

Follow us