Showing posts with label Dreamliner. Show all posts
Showing posts with label Dreamliner. Show all posts

Thursday, December 18, 2014

Key Manufacturing Related Items from NTSB 787 Battery Fire Report

Key Manufacturing Related Items from NTSB 787 Battery Fire Report:

I am having trouble opening any of the NTSB sites, so I do not have a link to the report at the moment. I am linking to a previous post I made about the Dreamliner Battery

Inline image 2

The NTSB report on the Boeing 787 battery fires is worth looking through.  I picked out some items to highlight related to what  I am interested in, manufacturing and quality.
  1. Tiered Supply Chain Management
  2. Failure to Identify and Control Failure Modes
  3. Weakness on End Of Line Tests and Inspections

1. Tiered Supply Chain Management:
NTSB noted that the battery was contracted by Boeing to Thales, who subcontracted the battery to GS Yuasa, with Boeing's approval. 
Boeing did not conduct any audits of GS Yuasa before the incident and relied on Thales to audit its subtier suppliers.85 After the incident, Boeing sent an audit team to Thales and GS Yuasa (and KAI) to review the management of subtier suppliers, quality of manufacturing and business processes, and adherence to Boeing standards. The audit found 17 items of noncompliance with Boeing requirements. Most of the noncompliance items at GS Yuasa involved adherence to written procedures and communication with Thales and Boeing regarding authorization for proposed procedural and testing changes for the battery.
85 Boeing had a source inspector at GS Yuasa, but the inspector was contractually limited to determining whether specific inspection and checklist items, as detailed in agreements among Boeing, Thales, and GS Yuasa, met minimum quality standards. Any issues that the inspector found had to be routed to a US Boeing representative to coordinate through Thales.

2. Failure to Identify and Control Failure Modes:
The NTSB auditors recognized "perturbations" of the electric foil as a nonconformance. In the report, a perturbation is defined as a change in the electrode foil nominal form due to compressive buckling.
Inline image 1
The NTSB, having identified perturbations as a defect, conveyed:
some of the cell manufacturing processes were not consistent with industry practices.117 For example, production of wound prismatic (rectangular-shaped) battery cells is typically performed on a flattened elliptical or rectangular mandrel, which reduces the chance for perturbations and folds during winding, but GS Yuasa used a cylindrical mandrel during the winding process.

Debris/FOD (foreign object debris):
Two of the welding operations—ultrasonic welding of the current collectors to the winding and tungsten inert gas welding of the cell header to the cell case—occur in an area where internal cell components are also assembled. Welding can generate FOD in the form of weld spatter and small metallic particles that become airborne. No physical shielding was used at the tungsten inert gas weld station to isolate this FOD-generating process from adjacent FOD-sensitive processes, such as those involving internal components of nearby open cells. Even though the ultrasonic welding machine incorporated a vacuum system to mitigate the amount of FOD generated, this process was observed generating airborne FOD.

Creases
​ and Wrinkles​

​NTSB Concludes:

Industry research on lithium-ion battery hazards showed that, about the time that the LVP65 cells were being manufactured, cell failures in various industries had been caused by, among other things, internal faults resulting from manufacturing defects (Mikolajczak and others 2011). However, GS Yuasa did not establish its cell manufacturing process to minimize the potential for manufacturing defects or develop formal inspection criteria of the cells that would reliably identify any defects that were introduced during the process.





3. Weakness in EOL (end of line) Tests - CT scan and Visual Inspections


At
​ 
the end of the cell production process the assembly is subjected to a 
x-ray computed tomography (CT) scan
​ to confirm that the
 current collectors were properly welded to the winding edges and that 
​debris​
 was not
 ​
​enclosed within the cell.​
However, the resolution settings of GS Yuasa’s CT equipment was such that many internal cell features, including individual winding layers, could not be identified during the NTSB’s visit to GS Yuasa’s facility. As a result, perturbations in the windings and small-sized FOD and burrs might not be recognized with the CT equipment used at the time of the NTSB’s visit. GS Yuasa stated that it was not aware that FOD might not be visible on the company’s CT scans.

GS Yuasa’s CT scans also did not detect features that the NTSB, TIAX, and UL identified during this investigation. For example, TIAX’s and UL’s DPAs of cells from the main battery on the incident airplane found cell windings with numerous wrinkles, folds, and creases.

The wrinkles and folds can modify the anode-to-cathode ratio locally, resulting in lithium deposits on the anode surface.

The NTSB also made this observation about YG Yuasa's quality control:
GS Yuasa performed most of its quality control inspections after a cell was fully manufactured and indicated that less than 1 percent of manufactured cells were rejected. This low rejection rate could be the result of few defects to detect; however, most of the evidence (that is, poor internal manufacturing steps along with the lack of a formal inspection for manufactured defects) indicated that GS Yuasa’s inspection process did not adequately screen for defects that developed during the manufacturing process. GS Yuasa’s checks during the manufacturing process relied on visual inspection rather than formal sampling processes that included rejection or acceptance criteria.

Vigilance decrements might also have played a role in the inadequate inspection process. GS Yuasa indicated that personnel involved with cell manufacturing for LVP-8-402 batteries worked 8-hour shifts. The visual inspection tasks performed by these personnel were vigilance tasks.121 Human factors research on inspection indicates that, as time on task increases, defects are more likely to be missed, especially if they seem to rarely occur (Fisher and others 2006, 997-1024)   

The greatest performance decrements are expected within the first 30 minutes of time on task; during that time, performance can be reduced as much as 30% but is typically reduced no more than 10% (Teichner 1974, 339-353). A 30% performance decrement is more likely for a highly detectable signal (defect) during a 3-hour period.




Tuesday, December 16, 2014

New Report on the Boeing 787 Dreamliner Battery Problems of 2013

I wrote (March 14, 2013) about the two battery incidents that occurred on the new Boeing 787 aircraft in early 2013 and noted that there was a lot of focus on fixes and not enough on causes of the problem.

At the time, the only cause identified was a short circuit in a cell.  There were some battery production process changes made as well as these design changes:
Boeing modified the 787 battery design and its installation configuration to include (1) additional insulation between the battery cells, (2) vents in the side of the battery case, (3) a stainless steel enclosure for the battery case, and (4) an ECS duct that vents from the interior of the stainless steel enclosure to the exterior of the airplane to prevent smoke from entering the occupiable space of the airplane.

On November 21 2014 the NTSB released an update to the original report:


The report notes the original two incidents, and an additional third incident in January 2014:
On January 14, 2014, an LVP65-8-403 main battery failed on a JAL 787 airplane that
was parked at a gate at NRT. (The airplane was being prepared for scheduled flight.) The Japan Civil Aviation Bureau is investigating this incident with assistance from the JTSB and the NTSB. Maintenance personnel reported seeing smoke outside the cockpit window. Preliminary information indicated that one cell had overheated and vented electrolyte and that the enclosure for the battery case contained the vented electrolyte.

The report is there to read.  Some of my takeaways are:

  • For complex assemblies where there are areas of the assembly hidden from observation it is important to do teardown inspections and/or x-ray inspections to look for problems.
  • A higher lever corollary to the teardown/x-ray comment above: when launching a new product, do your inspections as though you were doing a problem investigation. In other words, instead of simply confirming your components are "to print", your process is running according to the control plan, that end of line tests (EOL) are passed, and the assembly is to print; dig in and search for anomalies, differences between parts, and internal damage. With the battery, the NTSB really lifted the skirt and probed deep into the battery cells.  
  • I am reminded that designs are surprisingly robust and EOL tests are surprisingly weak. Part variation and defects still allow the assembly to work, for a time.  EOL tests fail to catch deviations. Within the battery cells the investigation