​Big data: The obstacles we’re seeing and how to overcome them

Consider the pillars of people, place and technology, and how each is impacted or influenced by the other

Picture: Ron Mader, Flickr

Picture: Ron Mader, Flickr

The digital universe is doubling every two years, according to EMC. That means, by 2020, about 1.7 MB of new information will be created every second, for every human being on the planet. In business, this previously unimaginable proliferation of data can be overwhelming and seemingly impossible to navigate. But it also poses the question - how can we ignore something that promises to solve every conceivable business problem?

Seductive, yes. Impossible, no.

We are rapidly approaching a digital inflection point, and big data analytics is no longer just a “good idea.” Turning mountains of data into insight, efficiently and cost effectively, perhaps in real time even, is a business imperative that is critical to staying competitive. Or for some, staying in business.

What follows are the obstacles we’re seeing, and how to overcome them.

The talk

Mostly, the conversations we’re hearing on big data today tend to focus on its technology challenges. The biggest hurdle that enterprise customers face is that they don’t know which technology to act on, or to bet on, for their big data strategy. This is further muddled now that all of the players - Microsoft, IBM, Oracle, and so on – all have offerings on the table.

The truth

Big data in itself, however, is fundamentally not a technology issue, and the first step in the right direction is in recognising that point. The arguably more difficult element of solving the big data puzzle is actually cultural. Big data’s pervasiveness – the way it can disrupt people and processes across every part of an organisation – means any big data initiative requires careful and significant thought.

The way

The change that’s more essential to tapping into big data’s potential is one of perspective.

Big data offers a new way to look at the world through a completely different lens. For this reason, big data initiatives require a comprehensive, holistic approach to the ‘problem’, with a balanced focus on the pillars of people, processes and technology.

This includes considerations around:

  • How people work, think and communicate,
  • Their existing expertise and skills,
  • The culture of the organisation and its receptiveness to change,
  • Current processes and the adjustments required, however big or small,
  • The technology needs and necessities of the company.

The method

The more data of different forms that is introduced into the organisation, the more pressing the need for a sophisticated and scalable approach to managing that big data environment.

When implementing a big data initiative, there’s often a temptation to expedite projects by skipping elements and taking shortcuts. But like most things, a push for immediate results is likely to end in disappointment and unmet expectations.

[Related: What big data can reveal about your staff]

‘Going rogue’, as I call it, is rarely an effective approach to big data and may even undermine the effort entirely. Rogue projects tend to be focused on isolated technology deployment, lack collaboration, cost too much, and end up nearly impossible to sustain.

The fact big data projects are by nature, highly iterative, does not eliminate the need for a plan. Rather, the plan needs to be flexible and adaptable to consistent, dynamic change. Importantly, establishing open channels of communication and interaction across relevant stakeholders is critical to making sure that everyone remains aligned and collaborative on the plan and its staging.

According to Paul Gibbons, author of The Science of Successful Organisational Change, “The human side of analytics is the biggest challenge to implementing big data.” Why? Because most enterprise cultures are hierarchical, and this corporate style may present as an obstacle to achieving the continuous improvement and innovation that’s necessary to survive in today’s dynamic digital world.

Instead, organisations should cultivate a learning culture that values problem identification, knowledge transfer, and reflective post-audits across the full team.

The existing franchise

Big Data projects also need to be an integral part of your existing business and IT infrastructure. So, it’s critical to involve internal process experts from the beginning. Inspecting and adapting processes and data flow with the right people in the organisation at regular intervals is key to growing and continuously improving big data initiatives by reducing cycle times, modelling change, increasing analytics speed and getting the right information to decision makers faster.

Focusing on small, discreet projects is a good place to start. Identifying and prioritising those that solve concrete problems helps to establish realistic expectations and demonstrate value for those that may be resistant to change.

[Related: Is big data and artificial intelligence a (r)evolution in outsourcing?]

No matter what big data technologies are used, they require the same commitment as your other information systems. Big data is not an island, and big data projects have a significant impact on existing IT infrastructure and operations.

To conclude, a successful Big Data adoption program should consider the pillars of people, place and technology, and how each is impacted or influenced by the other. To get to the stage where your business is benefiting from clear, data-driven insights, requires an iterative process, incorporating learnings along the way. When it comes to the enabling technology, it’s vital that big data platforms integrate with existing enterprise applications in order to optimise the cost of underlying infrastructure, and ensure that your foray into big data is a fruitful one.

Gavin Selkirk is VP APAC for BMC Software

Join the PC World newsletter!

Error: Please check your email address.

Tags businessbig data

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Gavin Selkirk

PC World
Show Comments

Most Popular Reviews

Latest News Articles


PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?