​Big data: The obstacles we’re seeing and how to overcome them

Consider the pillars of people, place and technology, and how each is impacted or influenced by the other

Picture: Ron Mader, Flickr

Picture: Ron Mader, Flickr

The digital universe is doubling every two years, according to EMC. That means, by 2020, about 1.7 MB of new information will be created every second, for every human being on the planet. In business, this previously unimaginable proliferation of data can be overwhelming and seemingly impossible to navigate. But it also poses the question - how can we ignore something that promises to solve every conceivable business problem?

Seductive, yes. Impossible, no.

We are rapidly approaching a digital inflection point, and big data analytics is no longer just a “good idea.” Turning mountains of data into insight, efficiently and cost effectively, perhaps in real time even, is a business imperative that is critical to staying competitive. Or for some, staying in business.

What follows are the obstacles we’re seeing, and how to overcome them.

The talk

Mostly, the conversations we’re hearing on big data today tend to focus on its technology challenges. The biggest hurdle that enterprise customers face is that they don’t know which technology to act on, or to bet on, for their big data strategy. This is further muddled now that all of the players - Microsoft, IBM, Oracle, and so on – all have offerings on the table.

The truth

Big data in itself, however, is fundamentally not a technology issue, and the first step in the right direction is in recognising that point. The arguably more difficult element of solving the big data puzzle is actually cultural. Big data’s pervasiveness – the way it can disrupt people and processes across every part of an organisation – means any big data initiative requires careful and significant thought.

The way

The change that’s more essential to tapping into big data’s potential is one of perspective.

Big data offers a new way to look at the world through a completely different lens. For this reason, big data initiatives require a comprehensive, holistic approach to the ‘problem’, with a balanced focus on the pillars of people, processes and technology.

This includes considerations around:

  • How people work, think and communicate,
  • Their existing expertise and skills,
  • The culture of the organisation and its receptiveness to change,
  • Current processes and the adjustments required, however big or small,
  • The technology needs and necessities of the company.

The method

The more data of different forms that is introduced into the organisation, the more pressing the need for a sophisticated and scalable approach to managing that big data environment.

When implementing a big data initiative, there’s often a temptation to expedite projects by skipping elements and taking shortcuts. But like most things, a push for immediate results is likely to end in disappointment and unmet expectations.

[Related: What big data can reveal about your staff]

‘Going rogue’, as I call it, is rarely an effective approach to big data and may even undermine the effort entirely. Rogue projects tend to be focused on isolated technology deployment, lack collaboration, cost too much, and end up nearly impossible to sustain.

The fact big data projects are by nature, highly iterative, does not eliminate the need for a plan. Rather, the plan needs to be flexible and adaptable to consistent, dynamic change. Importantly, establishing open channels of communication and interaction across relevant stakeholders is critical to making sure that everyone remains aligned and collaborative on the plan and its staging.

According to Paul Gibbons, author of The Science of Successful Organisational Change, “The human side of analytics is the biggest challenge to implementing big data.” Why? Because most enterprise cultures are hierarchical, and this corporate style may present as an obstacle to achieving the continuous improvement and innovation that’s necessary to survive in today’s dynamic digital world.

Instead, organisations should cultivate a learning culture that values problem identification, knowledge transfer, and reflective post-audits across the full team.

The existing franchise

Big Data projects also need to be an integral part of your existing business and IT infrastructure. So, it’s critical to involve internal process experts from the beginning. Inspecting and adapting processes and data flow with the right people in the organisation at regular intervals is key to growing and continuously improving big data initiatives by reducing cycle times, modelling change, increasing analytics speed and getting the right information to decision makers faster.

Focusing on small, discreet projects is a good place to start. Identifying and prioritising those that solve concrete problems helps to establish realistic expectations and demonstrate value for those that may be resistant to change.

[Related: Is big data and artificial intelligence a (r)evolution in outsourcing?]

No matter what big data technologies are used, they require the same commitment as your other information systems. Big data is not an island, and big data projects have a significant impact on existing IT infrastructure and operations.

To conclude, a successful Big Data adoption program should consider the pillars of people, place and technology, and how each is impacted or influenced by the other. To get to the stage where your business is benefiting from clear, data-driven insights, requires an iterative process, incorporating learnings along the way. When it comes to the enabling technology, it’s vital that big data platforms integrate with existing enterprise applications in order to optimise the cost of underlying infrastructure, and ensure that your foray into big data is a fruitful one.

Gavin Selkirk is VP APAC for BMC Software

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags businessbig data

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Gavin Selkirk

PC World
Show Comments



Sansai 6-Outlet Power Board + 4-Port USB Charging Station

Learn more >



Back To Business Guide

Click for more ›

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Louise Coady

Brother MFC-L9570CDW Multifunction Printer

The printer was convenient, produced clear and vibrant images and was very easy to use

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?