Borehole Pump Installation Practice Part 3 - Root Cause Analysis and Remedial Strategies
/In Part 3 of this series of articles aimed at ensuring a long and cost-effective borehole pump installation, we look at the need for Root Cause Analysis (RCA) and remedial strategies to prevent any further premature failures.
Part 2: Preventing Overheating in Submersible Electric Motors
by John Tonkin
Who else loves Air Crash Investigators? I have to admit that I'm a big fan. In the show, the investigators analyse the causes of aeroplane crashes by piecing together all the bits of evidence they can find. This evidence includes (of course) the Black Box (which, by the way is actually orange), pieces of the plane's wreckage, historical documentation, and sometimes a visit to the site where the crash occurred.
I think besides my fascination with all things aviation, the reason I enjoy the show is that it has many parallels to the process of investigating a failed pump installation - albeit on a less grand, and generally less serious scale!
Learning from failure
“Those who don’t learn from the past are doomed to repeat it”. Wise words attributed to George Santayana.
The best way to prevent a re-occurrence of history (dead installation in our case) is to:
- gather relevant data
- analyse the data in a Root Cause Analysis (RCA)
- come up with remedial actions that will stop the same thing from happening again.
Straight forward, yes. But too often, in the rush to get the repair done, or because of a lack of skills, this elementary process is not followed. History, all too often, comes back to give us expensive reminders.
Gathering evidence
The key to any investigative work is good, accurate and relevant information. What evidence can you collect at a pump installation site to help you to find out what happened to cause the failure? Well, the sources of evidence are similar to those used by our friends the Air Crash Investigators: the "Black Box", the wreckage, historical documentation, and a site visit!
The "Black Box"
All commercial aircraft are fitted with a data recorder that stores information on control settings, performance and other relevant information. The frenetic pace of development in electronics has allowed many pump manufacturers to offer protection relays that also store real time system information and transmit the data over the web.
The information that you can get from a data recorder is investigator's gold! Here's just one of many examples: I identified the source of a fault (phase 2 on the high voltage side of a transformer) in a borehole installation on a farm in a remote part of the Karoo - all while comfortably seated in my office in Johannesburg! All I needed was the information from the data recorder in the installation.
Data retrieved from a borehole installation in a housing complex in Cape Town. Everything was going well until event number 101 when things started to go wrong. Cause? Loose connection.
The wreckage
Snapped shaft of a 300kW pump (R500 000). Unless the faulty valve arrangements are corrected, failures will re-occur.
Examining what’s left of the pump, motor, cabling, electrical components and other equipment is always something of a sad but very necessary process.
Premature failure means cost and ultimately, “what goes around comes around”. If it was a farmer or a miner or even a home user who suffered the loss, there will be costs to be absorbed.
Don’t throw the parts in the scrap bin before they have been looked at. If they are examined by someone with experience and skill, they will tell virtually all there is to know: Abrasion? Cavitation? Corrosion? Bad motor/pump setting? “Dry” running? Hammer? Electrical problems? And yes, it could be a faulty part or a slip up by someone in the factory. Whatever the root cause, it will leave distinctive symptoms.
The most important part is to find the root cause of the failure before buying replacement components, which will certainly fail if the root cause is not addressed.
Historical documentation and events
While it's interesting to watch the air crash investigators uncover all the "high-tech" evidence in data recorders and by analysing parts of the wreckage, they often find some of their most pivotal evidence from other sources, like a repair report or during an interview with one of the surviving passengers.
Back to our pump installation investigation, we can chat to people who worked on, or who simply used the installation prior to its failure. Did they notice anything unusual? Did they change anything on the installation? Were there any significant events that happened prior to the failure, like extreme weather?
It will often be necessary to access documentation like repair reports, job cards, or other historical documentation to get a few more pieces of the puzzle.
Site visits
No protection, no locks, incorrect size, attached at the same height as an inquisitive three-year-old, totally illegal. Replacement of the failed pump will not sort out the failure
Travel to site always comes at a cost - vehicle costs, air tickets, accommodation costs - to mention a few. But sometimes it is the only way to gather the evidence you need to get to the root cause of the problem.
A skilled visitor will be able to identify key pieces of evidence at the "crash scene", analyse them on site, and then identify the root cause of the problem.
A site visit has the additional benefit of a training opportunity. While on site, you can pass on valuable advice and insight to operators and owners, which is likely to help prevent a repeat failure.
Analysis
Many times, a skilled person can simply look at a few pieces of evidence and then correctly identify the root cause of the problem immediately.
In other cases, the answer is not quite as obvious and some more thorough analysis is required. Pumps, motors and other components may have to be sent back to the manufacturers to be stripped and assessed. You may need further evidence, such as water or soil samples, or additional documentation to get to the root cause of the problem.
Time well spent
A root cause analysis could take up a lot of time, but in my experience, it is time well spent. It is always going to be tempting to simply replace the failing component and move on, but this very rarely solves the problem, and just leads to further wastage of time and money.
While problem installations in the borehole industry are relatively few and far between, it is vital that when premature failures do occur, steps must be taken to ensure that the first is also the last.
In addition, the skills and experience gained from a failure must become the foundation of a knowledge base that can be used to stop similar failures in future.
