oil gas data high performance data tools in the prodcution of industrial powers

High-Performance Data Tools in the Production of Industrial PowerOil, Gas, and Data Daniel CowlesISBN: 978-1-491-92289-7... n Learn business applications of data technologies nDevelop ne

High-Performance Data Tools in the Production of Industrial Power

Oil, Gas, and Data

Daniel CowlesISBN: 978-1-491-92289-7

Make Data Work

strataconf.com

Presented by O’Reilly and Cloudera, Strata + Hadoop World is where cutting-edge data science and new business fundamentals intersect— and merge.

n Learn business applications of data technologies

nDevelop new skills through trainings and in-depth tutorials

nConnect with an international community of thousands who work with data

Job # 15420

Daniel Cowles

Oil, Gas, and Data

High-Performance Data Tools in the

Production of Industrial Power

[LSI]

Oil, Gas, and Data

by Daniel Cowles

Copyright © 2015 O’Reilly Media, Inc All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://safaribooksonline.com) For more information, contact our corporate/institutional sales department:

800-998-9938 or corporate@oreilly.com.

Editor: Tim McGovern

Production Editor: Kara Ebrahim

Interior Designer: David Futato

Cover Designer: Ellie Volckhausen April 2015: First Edition

Revision History for the First Edition

2015-04-10: First Release

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc Oil, Gas, and

Data, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limi‐ tation responsibility for damages resulting from the use of or reliance on this work Use of the information and instructions contained in this work is at your own risk If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsi‐ bility to ensure that your use thereof complies with such licenses and/or rights.

Table of Contents

Oil, Gas, and Data 1

Introduction 1

Overview 3

Upstream 5

Well Optimization and Mature Wells 6

Remote Sensors and Network Attached Devices/I of T 7

Security 8

Health, Safety, and Environment 11

High-Performance Computing and Beyond 11

More Cloud and Mobile 12

Midstream and Downstream 13

Emerging Tech 13

Summary 18

Innovation in Tough Economic Times 18

Post-Mortem 19

iii

1 Karren, Charles “Insight Report: Data Centre Developments Get up Close and Per‐ sonal.” OFFCOM News CTLD Publishing Ltd Web 20 Mar 2015.

2 Boman, Karen “What Upstream Oil, Gas Can Learn About Big Data from Social Media.” Rigzone News Dice Holdings, Inc., 10 Dec 2014 Web 20 Mar 2015.

Oil, Gas, and Data

Introduction

When you hear “innovation in oil and gas,” your first thoughtsmight go to hardware—bigger, faster, deeper drilling; more powerfulpumping equipment; and bigger transport—or to the “shale revolu‐tion”—unconventional wells, hydraulic fracturing, horizontal drill‐ing, and other enhanced oil recovery (EOR) techniques But, justlike any other industry where optimization is important—and due

to large capital investment and high cost of error, it’s perhaps evenmore important in oil and gas than in most other industries—thepotential benefits of predictive analytics, data science, and machinelearning, along with rapid increases in computer processing powerand speed, greater and cheaper storage, and advances in digitalimaging and processing, have driven innovation and created a richand disruptive movement among oil and gas companies and theirsuppliers

The truth is, the oil and gas industry has been dealing with largeamounts of data longer than most, some even calling it the “originalbig data industry.”1,2 Large increases in the quantity, resolution, andfrequency of seismic data, and advances in “Internet-of-Things"-likenetwork-attached sensors, devices, and appliances, are being com‐bined with large amounts of historical data—both digital and

1

3 “WORLD ENERGY INVESTMENT OUTLOOK 2014 FACTSHEET - OVERVIEW.”

International Energy Agency OECD/IEA, 1 Jan 2014 Web 20 Mar 2015.

physical—to create one of the most complex data science problemsout there, and a new industry is developing to help solve it

In oil and gas more than in almost any other industry, efficiency andaccuracy is highly valued, and small improvements in efficiency andproductivity can make a significant economic difference When atypical well can cost upwards of ten million dollars—and oftenmuch more—the cost of error is great, and managing cost versusbenefit can mean the difference between profitability and loss Andnot unlike a tech startup, where a meaningful investment upfront isrequired before knowing how much the return will be—if any—youmay have to dig many holes to find a successful well Obviously, themore certainty you can have, the better, and incrementally increas‐ing certainty is a place where data science and predictive analyticspromise to help The payoff from analytics isn’t limited to explora‐tion: once a well has been successfully drilled, production efficiencyand optimization remains important in the lifetime ultimate recov‐ery of a well

In addition, given crude price fluctuations and many other unpre‐dictable outside variables, capital project planning itself is rife withuncertainty, and large-scale projects often face significant overages

“In 2011, upstream offshore oil and gas projects…around 28% had acost blowout of more than 50% and the root cause of that is…theygot the numbers wrong,” says Dominic Thasarathar, who watchesthe energy sector for the Thought Leadership team at Autodesk

“Their costs have gone up, they’re dealing in everything from fron‐tier environments to difficulties raising finance.” According to theInternational Energy Agency, capital investments in energy projectshave more than doubled since 2000, and are expected to grow by $2trillion annually by 2035, so accurately predicting cost versus benefit

is extremely important.3 “Where we see big data fitting in,” contin‐ues Thasarathar, “is…if you look at the performance for those bigprojects, it’s pretty much a horror story in terms of how it’s droppedoff over the last 15–20 years, and the root cause of that is, there’s somuch that project teams need to understand and assimilate in terms

of information to make the right decision.”

But exploration and production aren’t the only areas that can benefitfrom innovative data and data science driven solutions From

2 | Oil, Gas, and Data

health, safety, and environmental, to cyber security, to transporta‐tion and manufacturing—opportunities to create greater efficienciesexist throughout the entire hydrocarbon production and deliverycycle.

Overview

The oil and gas industry is traditionally broken down into three

broader categories: upstream, which includes exploration, discovery, and both land and sea drilling and production; midstream, which

includes transportation, wholesale markets, and manufacturing and

refinement of crude; and downstream, which is primarily concerned

with the delivery of refined products to the consumer The majority

of big and fast data related innovation is found upstream, in the dis‐covery and exploration phase, where risk and uncertainty are high,conditions can be—to put it mildly—challenging, and where failure

is very expensive

The industry is a mature and unique one, built on experience andhard-won knowledge, and employing the world’s leading geologicalscientists and engineers They’re very good at what they do, andthey’ve been doing it for a long time, but there is an imperative toadd more big data and data science skills like machine learning andpredictive analytics into the mix, skills that oil companies haven’ttraditionally and broadly had in-house According to Boaz Nur, for‐mer VP of Energy at data science startup Kaggle, energy analyststhink big data and analytics are the next frontier in oil and gas, butthey’re only now in the early adoption phase “They [oil and gascompanies] don’t shy away from technology, they’re just careful,”Nur says “A lot of snake oil has been sold to the oil and gas compa‐nies over the years They’ve also historically done a pretty good job

of producing oil They’re [already] doing OK; what we’re proposingwill help them take it up to the next level.” Adds Nur: “They’re cau‐tious but they’re optimistic.”

Halliburton is using big data and data science techniques to try tosolve a variety of problems in the E & P (exploration and produc‐tion) upstream phase “We are looking at trying to optimize seismicspace, trying to optimize drilling space, well planning,” says Dr.Satyam Priyadarshy, Halliburton’s recently hired Chief Data Scien‐tist Priyadarshy is bringing some big data techniques to the space:

“For example, we are looking at how to optimize in the seismic

Overview | 3

world through distributed computing [techniques] because it takes along time to process the data.” But Priyadarshy says that it’s a mis‐take to think that data science methods and techniques are new tooil and gas “They’ve actually been using machine learning for manyyears,” he says “People have been using neural networks, fuzzy logic,SVM, SVRs—pretty much any algorithm you want to talk about inmachine learning, they have been using it But, they have been usingthese in limited cases, to limited value, and the goal is now for peo‐ple like us (data scientists) to build this into a more valuable prod‐uct.” He says that the oil and gas industry is unique in terms of thecomplexity of the data and models, and that turnkey solutions fromother traditional big data industries can’t be easily applied here “It’s

a complex challenge It’s not the same as the other big data players,”says Priyadarshy, who has worked widely on big data projects in thenews, media, Internet, and insurance spaces “The complexity in theoil and gas industry outweighs any other.”

Because of that complexity, Priyadarshy stresses the need fordomain area expertise when dealing with petrotechnical data, and

he has his own definition of the skills a data scientist should have forthe space “You need a person who has domain expertise, a personwho is a computer scientist, and a business person—these threeactually form a real domain data scientist” for the oil and gas space.Another complication is legacy and historical data: some is digital,but much is still found in binder and paper form From a predictivemodeling standpoint, there’s value to be had, but dealing with oldsystems and documents, often at isolated physical properties, or—asoften happens in the industry—inherited through acquisitions andneglected, makes integrating these pieces into your modelchallenging

Remote standalone locations and physical records and manuals alsohamper efforts to digitally connect a company’s systems and assets—the much discussed “digital oilfield” idea, where systems are integra‐ted and automated to tune and optimize operations across thebreadth of the production cycle “The move to digital operations isincreasing steadily, but there’s an awful lot of legacy out there, thingsgoing back decades, where the drawings were done with, literally,pen and paper,” says Neale Stidolph, Head of Information Manage‐ment at Lockheed Martin and based in Aberdeen, where he primar‐ily deals with North Sea oil fields, including many older legacy wells

“A large part of the industry is very much tied to documents and

4 | Oil, Gas, and Data

4 “Marine Seismic Imaging.” BP.COM British Petroleum Web 20 Mar 2015.

records So, there’s still a need to maintain vast physical archives…ofboxes full of old information And there’s a need to analyze and stripthat to get more value.” And since many of the physical sitesinvolved are isolated, supplying their own power, without moderncommunication networks, there are additional barriers to fully digi‐tizing operations “One of the factors the rigs have to cope with iswhat they call a black start,” says Stidolph “If your rig goes down, itmeans you’ve lost everything: you’ve lost all power generation, allconnectivity, all systems of every type You need a flashlight and youneed a manual to be able to see how to get this thing operationalagain.” Many of these rigs are in hazardous and remote environ‐ments, so off-the-shelf connectivity solutions aren’t typically suffi‐cient

But, challenges and cultural resistance aside, big data methods arechanging how the industry does business, and these changes willultimately result in a changed oil and gas industry

Upstream

As previously mentioned, oil and gas has long been familiar withlarge and diverse datasets, and improvements in technology andmethodology are driving an exponential increase in the amount ofdata being collected

In the exploration space, for example, due to advances in seismicacquisition methodology, storage capabilities, and processing power,data gathered via offshore seismic acquisition has gotten both bigger

—due to increased resolution—and faster, due to increase in fre‐quency and rate of acquisition The result is 4D data (x/y/z space,and time) at a far higher resolution, providing far better under‐standing of subsurface deposits and reservoirs than previously pos‐sible Wide azimuth towed streamer acquisition (WATS)—seismicexploration using multiple ships deploying a miles-wide array ofacoustic equipment—allows companies like Chevron and BP to cre‐ate high-resolution topographic maps under the earth and beneathsalt canopies, and locate new oil fields that may not have been foundotherwise.4 Time-lapse seismic data acquisition also allows them tosee how reservoirs are behaving as oil begins to flow, allowing them

to optimize production once it begins As the world’s energydemands continue to grow, and exploration efforts move farther

Upstream | 5

5 “Maximizing the Value of Mature Fields.” Halliburton.com Halliburton, 1 May 2012 Web 20 Mar 2015.

offshore and into deeper waters, the ability to accurately visualizedeep, complex, subsurface topography is essential Recent deepwaterdiscoveries in the Gulf of Mexico have been greatly aided by newseismic techniques, and there is a direct relationship betweenimprovements in data storage and data processing, and improve‐ments in seismically generated image resolution, which in turnresults in new and better understood hydrocarbon discoveries Andthere is still room for improvement in seismic acquisition image res‐olution: “Even at very high resolution, the images we can maketoday still have gaps bigger than the size of a conference room,” saysBP’s John Etgen

Well Optimization and Mature Wells

Although a lot of recent press and activity focus on the “shale boom”and other unconventional extraction techniques, according to Halli‐burton, 70% of the world’s oil and gas comes from mature wells.5 Amature well is usually defined as one where peak production levelshave been reached, and extraction rates are on the decline, or whenthe majority of the relatively “easy to get” hydrocarbons that the wellwill ultimately deliver have been extracted Typically in wells theearly oil and gas is easier and cheaper to extract, and the industryhasn’t been enthusiastic about optimizing extraction, holding a com‐mon belief that there is an “economic limit” where it costs more toget the resources out of the ground than they’re worth on the mar‐ket However, modern EOR techniques have become more efficient,and sensor data and predictive models play a part in that Thesewells have a wide range of factors that make them more complicated

—poor flow, poor rock formations, bore cracks, complex geologicalconditions—but they still have a lot to offer in terms of hydrocar‐bons In an industry where small margins mean large sums ofmoney, getting the most from mature and end-of-life wells at thelowest cost is another area where improved use of data can have asignificant impact on results Again according to Halliburton, a 1%increase in production from the mature fields currently active wouldadd two years to the world’s oil and gas supply

6 | Oil, Gas, and Data

Remote Sensors and Network Attached

Devices/I of T

There is already a lot of application and ongoing interest innetwork-attached devices, appliances, and Internet of Things-likeconnected devices in the oil and gas space Halliburton’s Priyadarshyprefers the term “emerging technology devices,” to the “Internet ofThings” label, which causes some confusion and resistance in theindustry In any event, remoteness, geographic breadth of facilitiesand pipelines, hazardous environments, and inaccessibility of manyaspects of the oil and gas production cycle make it highly disposed

to automation and remote monitoring and optimization Remotelymonitored and controlled devices can help lower cost, effort, anderror in resource tracking, and can decrease workforce overhead,improve logistics, and drive well and operations automation andoptimization It’s a big piece of the “digital oilfield” concept, and onethat the industry has already embraced

Sensors of all kinds are already used throughout the detection, pro‐duction, and manufacturing cycle to better understand and monitorprocesses and gather data Sensors can capture fluid pressure, veloc‐ity and flow, temperature, radiation levels (gamma ray energy is auseful indication in hydrocarbon discovery), relative orientation andposition, as well as chemical and biological make-up of physicalmaterials Trending toward cheaper, smaller, and connected arrays,newer microsensors can communicate with each other and withexternal networks

From exploration to the gas pump, there are opportunities to usenetworked devices Offshore, submersible devices that gather infor‐mation can be remotely controlled and are safer alternatives tohuman-piloted crafts Pumps can be remotely monitored and adjus‐ted, and can be far more economical than manual maintenance.Midstream in the transportation phase, networked devices can helptrack resources through the many and various stages and handoffsthat happen throughout the crude transport process Pipelines andremote equipment can be monitored and even maintained remotely.Biomonitoring workers could increase safety Gartner has predicted

Remote Sensors and Network Attached Devices/I of T | 7