A major benefit of high-performance computing is the vast improvement it offers scientists and researchers in Australia when it comes to exploring large data problems.
High-performance computing and grid computing have become key components in exploration and mining applications in the past few years; from running numerical simulations to creating a virtual rock laboratory or providing a mechanism to discover and process geodetic information collected by GPS stations.
Go deeper with GlobalData
However, scientists have been restricted in their abilities to access high-performance computing for these and other uses due to the fragmented nature of the infrastructure.
CSIRO Exploration and Mining’s Computational Geoscience Group project leader Ryan Fraser says Australia’s high-performance centres host a range of different supercomputers from different manufacturers, with differing configurations and architectures.
“Each supercomputer is different so users who want to use more than one supercomputer successfully need to know the difference between them all,” he says.
“Each machine requires different usernames and passwords and they are faced with having to know their different configurations. Also, software is typically installed in different locations on each machine and the scripting languages to utilise each machine can vary.”
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataCSIRO has found an answer to this dilemma via the AuScope Grid, a project to create an e-Research Infrastructure to federate and make nationally distributed datasets and high-performance computing resources interoperable, to develop tools to manipulate large data volumes and to establish an appropriate governance framework to ensure sustainability.
AuScope Grid’s premise is to comprise distributed data storage hardware, high-bandwidth network links, data management protocols, middleware and software, and eventually link the major geoscience and geospatial data stores of the government agencies with high-performance resources and high-bandwidth networks of the academic community.
AuScope has created a grid computing platform which standardises access to high-performance computers regardless of the machine type, manufacturer and location. “Such seamless access is important if our scientists and researchers are to be able to answer the larger scientific questions they have embarked upon,” Fraser says.
Fraser says that grid computing was ‘middleware infrastructure’, the essential catalyst that sits between a high-performance computer and a workflow client, such as one of the examples below.
“What grid computing does is simplify the access to high-performance computing for the user and developer of workflows by providing seamless and interoperable access to an array of high-performance computing resources. It makes an array of heterogeneous supercomputers look the same on the outside,” he says.
“These workflows were possible before the deployment of grid computing infrastructure but some required ‘an order of magnitude longer to produce the desired outcome (differences of up to six months have been recorded) and typically required the user to have a degree in Computer Science’.
CSIRO’s Fraser highlights three workflows that AuScope has helped to establish.
Desktop modelling toolkit
The Desktop Modelling Toolkit, or DMT, is a software suite with a familiar user interface which scientists and researchers can use to develop complex geoscientific models and process them either locally or for larger problems using high-performance computing resources via the AuScope Grid.
In essence, it abstracts the process of creating and running numerical simulations by providing a conceptual view. Geoscientific computational software is extremely complex and typically requires users to know how to program computing code – a limiting factor for many scientists who simply want to answer questions.
The DMT has permitted many scientists to perform large geoscientific simulations using high-performance computers, something that was previously limited to only a few with programming capabilities.
DMT is a development by CSIRO Exploration & and now a part of CSIRO’s Minerals Down Under National Research Flagship, in collaboration with AuScope.
Virtual rock laboratory
The Virtual Rock Laboratory is a web-based intuitive user interface developed to allow the creation of a virtual laboratory to analyse rocks, specifically the mechanics of rocks.
The interface provides scientists access to the Discrete Element Method code ESyS-Particle, developed by the Earth Systems Science Computational Centre (ESSCC) at University of Queensland.
ESyS-Particle is a powerful numerical software tool for rock mechanics and specifically used to examine rock breakage. However, as with the DMT, users of this tool must know how to program software code. AuScope Grid and ESSCC collaborated to develop a web portal to provide users with an environment in which to create models using pre-constructed components which they can customise through the interface.
Once users have constructed their model they can submit it to the grid for processing. This portal provides users with an intuitive web environment for processing large rock crushing simulations without the cost of using real laboratory equipment. It also offers the opportunity to conduct multiple experiments simultaneously.
Geodesy workflow
Geodesy Workflow is a recent activity and another effort to both exploit Australia’s high-performance computing resources and to simplify the access to geoscientific software.
Geodesy offers a mechanism to discover and process geodetic information collected by the many GPS stations deployed in Australia by AuScope and Geoscience Australia. Scientists use this data to conduct time series analyses and comparative assessments on the stations to determine size, shape and movement (among other things) of the continent.
The workflow is intended to ease the use of the geodetic simulation codes for both researchers and students and to provide a library of results and studies for general reference.
The Geodesy Workflow is a web portal that combines geodetic datasets provided by Geoscience Australia via a web map interface and incorporates grid technologies to process the data on high-performance computers.
The users can use the interface to discover the data and choose which datasets they want to process using several geodetic simulation software packages. Once they have chosen the datasets of interest, they use the web portal to submit the data for processing on one or many high-performance computing resources around Australia via the AuScope Grid.
The portal also allows users to monitor the status of their jobs, download and move files/datasets, register input and result datasets to a catalogue registry for later use and discovery. Finally, it also incorporates a post-processing portal which allows users to visualise the outcomes of their processing job on a web map.
They can view the effects of their simulation on the various GPS stations in Australia and interpret the effects on the Australian continent.
The Geodesy Workflow is a project within AuScope Grid and a collaboration between AuScope, Curtin University, University of Tasmania, Australian National University and Geoscience Australia.
This article first appeared in CSIRO’s magazine earthmatters.
