National Virtual Core Library (NVCL) - explained for Melbourne Science Hackfest
If you were interested in the ERML dataset being provided by AuScope for Melbourne Science Hackfest, you can find more information on that at ERMLGovHackOerview
This page is for participants in Melbourne Science Hackfest
who are interested in creating a mashup of the NVCL data. This page provides a quick introduction to the NVCL data, services and its use. If you are looking for the full overview then you can find it at WebHome
What is NVCL?
The NVCL is best thought of as a library or catalog of borehole (alternatively drillhole) observations. So what is a borehole? Without turning this into a lesson in geoscience, a borehole is simply a narrow hole drilled deep into the ground to allow geoscientists to see what minerals and rocks are there. Unlike your average home drill which would pulverise the rock into dust, these 'borehole drills' extract the drilled material mostly
intact. This extracted rock comes out as a long tube of rock known as a borehole core. The borehole core is what geoscientists are really interested in as they provide a narrow snapshot into what lies underneath the ground.
Borehole cores in trays (courtesy of the Geological Survey of Western Australia)
Now these borehole cores (long segments of rock) are typically broken up and assembled into smaller trays before being warehoused somewhere. Originally a geologist would examine the resulting rock core by hand in order to report on what minerals are available at what depths. These days the borehole core is run through a machine known as a hyperspectral logger (HyLogger) which estimates the quantities of minerals along the length of the borehole.
Detected Mineral Quantity vs Borehole Depth (For illustrative purposes only)
All of this analytical data along with information about the drilling process itself is now completely digital and available through web services, this is the National Virtual Core Library.
So what is available?
Quite a bit actually, It wasn't mentioned before but most of the state/territories are participating in the NVCL program and have public facing web services. This data is exposed through three different endpoints for each provider:
- A Web Feature Service For the borehole and observational metadata (where is the borehole, who drilled it)
- A NVCL Data service For requesting high resolution imagery and plots of the HyLogger data
- A NVCL Download service For requesting the underlying data be bundled up in a variety of formats (This is a queing service, it does not do realtime requests).
As of February 2016 - these are the known NVCL providers and their service endpoints:
- http://nvclwebservices.vm.csiro.au/geoserverBH/wfsData service
- http://nvclwebservices.vm.csiro.au/NVCLDataServices/Download service
Geoscience Western AustraliaWFS
- http://geossdi.dmp.wa.gov.au/services/wfs Data service
- http://geossdi.dmp.wa.gov.au/NVCLDataServices/Download service
Mineral Resources Tasmania:WFS
- http://www.mrt.tas.gov.au/web-services/wfsData service
- http://www.mrt.tas.gov.au/NVCLDataServices/Download Service
Northern Territory Geological SurveyWFS
- http://geology.data.nt.gov.au/geoserver/wfsData service
- http://geology.data.nt.gov.au/NVCLDataServices/Download service
Primary Industries and Resources South AustraliaWFS
- https://sarigdata.pir.sa.gov.au/nvcl/geoserver/wfs Data service
Download service - https://sarigdata.pir.sa.gov.au/nvcl/NVCLDownloadServices/
Department of Primary Industries New South WalesWFS
- http://auscope.dpi.nsw.gov.au/geoserver/wfs Data service
- http://auscope.dpi.nsw.gov.au/NVCLDataServices/ Download service
Department of Economic Development, Jobs, Transport and Resources VictoriaWFS
- http://geology.data.vic.gov.au/nvcl/wfsData service
- http://geology.data.vic.gov.au/NVCLDataServices/Download service
Department of Natural Resources and Mines Queensland
- http://geology.information.qld.gov.au/geoserver/wfsData service
- http://geology.information.qld.gov.au/NVCLDataServices/Download service
How do I access the data?
Well that depends on what you are after, up until now I've been talking about borehole cores, metadata, observational data etc and you've probably assumed they are all related. Well, you are correct, there is a series of relationships between the various instances of data that will require you to navigate the various services. Hopefully I can illustrate these relationships with the following map:
As a consumer of NVCL you'll typically want to start by querying for Boreholes from the Web Feature Service (WFS), from there you can always provide 'drill down' (pun intended) functionality for more information about the data for that borehole. To get you started, here is a basic WFS query that will request the first 5 boreholes from a WFS as XML:
Let's decipher the WFS parameters:
| This should always be 'WFS'
| The version of WFS you wish to query (leave this as 1.1.0)
| WFS services provide a variety of request types, for simplicities sake let's keep it as 'GetFeature'. Read the wfs spec for more information on the others.
| A WFS is a collection of feature types (think data types) which can be queried individually. In our case the borehole's have the type name 'gsml:Borehole'
| We can limit the size of the response to X features, bigger requests take longer for the server to generate and for you to parse. I'd suggest keeping the request size under 200.
Now WFS is a well defined standard with lots of nuances and particulars that are too varied to cover here so instead we'll focus on the basics. Essentially a provider of a WFS will serve one or more different feature types which can be best thought of as Objects if you are a software developer. That is, each feature type will have a number of attributes and possible 'child' features forming a complex schema. When you make a request for a feature type to a WFS, the service responds with XML encoded instances of that feature type. Eg - If I have a feature type 'City' and query a WFS for all 'City' feature types, I'll get a number of 'City' features (Objects) with names like 'Perth', 'Melbourne', 'Sydney' etc.
If you plan on representing the boreholes spatially (i.e. on a map) then you will be interested in the borehole spatial attribute which can be extracted using the following XPath
#This is relative to the gsml:Borehole element
#For extracting the point coordinates
#For extracting the spatial reference system
Now because this page is supposed to be a quick introduction to NVCL (and by extension WFS) I'm going to gloss over the parts that involve asking the WFS to filter your boreholes by some attribute. Instead I'm just going to say that you can also make HTTP POST requests to a WFS which can contain filters, if you were to POST the following XML query to any of the above WFS endpoints (sans URL parameters) you would get a response of all boreholes within the specified bounding box (roughly Western Australia) that have a drilling method of 'diamond core'.
POST this to http://geossdi.dmp.wa.gov.au/services/wfs
<?xml version="1.0" encoding="UTF-8"?>
<wfs:GetFeature service="WFS" version="1.1.0" maxFeatures="20" xmlns:wfs="http://www.opengis.net/wfs" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml" xmlns:gsml="urn:cgi:xmlns:CGI:GeoSciML:2.0" xmlns:sa="http://www.opengis.net/sampling/1.0" xmlns:om="http://www.opengis.net/om/1.0" xmlns:nvcl="http://www.auscope.org/nvcl">
<wfs:Query typeName="gsml:Borehole" srsName="EPSG:4326">
<gml:lowerCorner>110 -13 </gml:lowerCorner>
<gml:upperCorner>130 -36 </gml:upperCorner>
Once you have a borehole you are interested in the next step is requesting the list of datasets to go with that borehole, to do this you must extract the borehole name from the gsml:Borehole feature type (the WFS response). The borehole name is identified by the gml:name element with the attribute codeSpace="http://www.csiro.au". If you are familiar with XPath then you will be able to extract the correct value with the following XPath expression:
If you are unfamiliar with XPath, then the following XML snippet should highlight the part you need to parse:
Once you have the borehole name you'll need to query the data service for the list of datasets using the getDatasetCollection
This will return a simple XML response that looks something like this (please note that you may receive an empty collection or a collection with multiple datasets). You will be reusing the 'DatasetID' element in future queries so make sure you extract it!
Requesting Logged Elements
So now that you have a borehole and dataset that you wish to query, the next step is identifying 'logged elements'. The data service classifies all high resolution imagery and detected analyte (eg Iron, Gold) levels as logged elements. In order to request any of this information you need the ID of the element to request, to discover these ID's you can request all logged elements for a dataset with the following getLogCollection
For high resolution imagery logged elements:
For analyte logged elements:
The above requests will both generate a series of XML Log elements containing the logged element ID (important), a descriptive name of the logged element and an optional sample count. The sample count will identify how many images/samples are associated with a particular logged element.
Using logged elements for imagery
The response from the above getLogCollection request (with mosaicsvc=yes) will contain a number of logged elements relating to high resolution images.You can differentiate between the logged elements based on their logName as the following table will demonstrate:
There are two ways you can request the logged images:
- Query the images directly. Use the logId from the getLogCollection response, the sampleId is 0 based and references to the image index to request
- Query the service to generate a HTML page which can be rendered to show a range of images:
Using logged elements for analytes
The logged elements for analytes can be plotted into a graph (raster image) of the data using a plotScalar
This request will plot the variance of the detected analyte as a function of the length of the borehole.
|| The logId as extracted from the getLogCollection response
|| Where to begin the plot of the graph (in metres) as relative to the top of the borehole.
|| Where to end the plot of the graph (in metres) as relative to the top of the borehole
|| The interval of the sampling (in metres)
|| The width of the output image in pixels
|| The height of the output image in pixels
What type of graph should be returned. It should be an integer in the range of 1 to 3,
1 = Stacked Bar Chart
2 = Scattered Chart
3 = Line Chart
How is it currently being used?
The AuScope Portal
is a web based GIS portal for displaying spatial data sets like NVCL. It supports the full spectrum of NVCL services, to see it in action please visit the portal
and click on any of the blue markers.
AuScope Portal with NVCL layer
If you were at all interested in how the portal works then feel free to download the source and play around, info on getting started is available on the AuScope wiki
, alternatively have a chat to one of the AuScope data mentors.
- 24 May 2012
-- Updates from JesseRobertson
- 07 Jul 2014, -- VincentFazio
- 18 Feb 2016