Online processing performance increases up to 1000% of previous levels

Flash storage deployed for critical systems used in over 20,000 branches throughout Japan

JAPAN POST BANK, a major Japanese financial institution, improved the storage used for its information system. As part of its aim to maintain stable system performance and to provide optimal service to its customers, the bank eliminated drops in processing performance caused by dramatic increases in the volume of data being processed. As its new storage medium, the bank adopted flash storage devices manufactured by Hitachi, Ltd. (hereinafter, Hitachi), which improved performance for online and batch processing. Furthermore, by utilizing Hitachi’s assessment and Proof of Concept (PoC) services, the bank was able to successfully construct a highly reliable, high-performance storage environment in a short amount of time.

Increasing amounts of data handled by the information system had led to storage performance issues

As the core financial institution in the JAPAN POST GROUP, JAPAN POST BANK provides a multitude of financial services to individuals and enterprises. In recent years, as one of the largest institutional investors in Japan, the bank has worked to strengthen the investment strategy.

The System Division supports the bank's duties from an IT perspective. JAPAN POST BANK’s Yukio Koguchi says, "Our System Development Department 1 is the main department in charge of areas related to infrastructure. For a financial institution, stable system operation is extremely important. Therefore, the System Development Department 1 advances a variety of initiatives aimed at creating the ultimate environment for business operations."

The recent strengthening of the performance of the Information System storage platform was one of the focus areas for these initiatives. The system houses a range of data from a multitude of sources; including individual and enterprise customer information and transaction histories, as well as sales and managerial information. This data is used for business operations by staff at the head office and at approximately 24,000 branches throughout Japan. As an example, the sales staff uses this data to run marketing and information analysis reports that will help determine the best solutions to offer their customers.

This system is also an integral part of the bank’s business operations, as it manages transaction histories and the issuing of account balances. These operations require exceedingly high performance and stability. However, rapid increases in the volume of operational data had led to insufficiencies in system processing performance.

Aiming to improve I/O performance by deploying Hitachi flash storage

When the bank’s IT infrastructure was originally constructed, it was estimated that the maximum volume of data processed per day would be approximately 60 million transactions. "But these days," Koguchi says, "due to online shopping, for example, there are numerous payment methods available. As a result, withdrawals from accounts are concentrated on specific days. Because of this, we began to experience peak days on which the volume of data being processed exceeded the maximum of 60 million events only about three short years after the system was constructed." This large increase in data volume began to also have a huge impact on daily system operations. JAPAN POST BANK’s Toshikazu Saotome says, "Although we used delayed processing to import data from the accounting systems, we became gradually unable to meet our own deadlines because of the increased volume of data. When the import processing was delayed, it would naturally affect subsequent batch processing, date change processing, and even the online launch the next morning. For that reason alone, we felt it was necessary to solve the problem immediately."

Various problems had also arisen that affected the usage of information by users. "For example, if we attempted to extract and analyze data in units of multiple years, processing would take so long that it would time out. We had to make up for this issue in our operations, by separating the data into yearly chunks to extract and analyze it," says Koguchi.

In order to solve these issues, the bank initiated the deployment of new storage products.

The main requirement was the dramatic improvement of batch processing performance in order to guarantee online availability on the subsequent morning. Fur t hermore, data processing performance would be improved so that the system could handle more than 80 million transactions per day.

The Hitachi Virtual Storage Platform G1000 (VSP G1000) disk array system for Hitachi high-end systems equipped with Hitachi Accelerated Flash high-performance, high-reliability flash modules (HAF), was adopted in order to meet these requirements (Figure 1).

JAPAN POST BANK made use of Hitachi flash storage in the informational system that house large volumes of data sent from systems such as the accounting system, and achieved marked performance increases for batch and online processing.

Possible methods of solving the system performance problems included adding fixes to applications. This approach was also considered during product selection. However, due to the risk of unforeseen problems, it was seen as likely to be costly and time-consuming. Because it was clear at that point that the current problem was an I/ O performance bottleneck, it was decided to deploy flash storage products with superior speed.

The thorough verifications showed the flash storage adoption was an effective way to solve these issues

However, for JAPAN POST BANK, the deployment of flash storage was the first time in their attempts to improve performance. Detailed tests were performed by using Hitachi's service Flash Solution for Oracle, to see if the desired performance levels could be obtained.

"We performed two levels of verifications, the assessment services and PoC test using the actual machines to perform. For the assessment services, we used database design reports to predict the effect of deploying flash storage devices. For the PoC tests, we performed processing by using test data which emulated the actual environment. Because the objective was to raise performance levels above those of the current system, we also ran tests on simulated cases in which system load was increased to 3 to 4 times the current load." says Saotome.

The tests provided the following results: in the desktop performance verification, the processing performance of the database system after the deployment of flash storage reached levels of up to 7.1 times the previous performance levels. In addition, in PoC service tests using actual machines, single unit update processing response times improved to about 33 to 46 times the previous speed, and simultaneous multiple processing times, based on batch processing at night during busy periods, improved up to 14 times the previous speed. (Figure 2)"Although we had an idea of what the performance levels would be like from the service proposals, there were many areas about which we were still concerned. We needed to try actual operations in order to alleviate our worries. This rigorous testing brought us peace of mind and allowed us to proceed with the initiative," says Saotome.

Test data was imported into the best-practice configuration to verify the effect that migration to flash storage would have on task- based processing. Single unit update processing response times improved to 33 to 46 times the previous speed. Even multiplex processing that simulated batch processing at night during busy periods showed an improvement of up to about 14 times the previous speed.

The reliability of Hitachi storage was also an important factor. "Over the years, our company has used a variety of storage vendors, and Hitachi has had the lowest failure rate among them. Among internal staff, trust in Hitachi storage is exceptionally high. This factor also contributed to our decision. Furthermore, during the actual deployment, everything from development work to data migration was completed smoothly. Since actual operations began in May 2016, there has not been any major incident, and in addition to the storage products, we have recognized Hitachi had advanced system integration techniques," says Koguchi.

Solving delays in batch processing, and speeding up the application of information

Through the new deployment of VSP G1000, the bank’s system performance problems were completely solved. "Batch processing times were drastically reduced," says Saotome, "just as the testing results had shown. In the past, processing would finish right before the deadline. However, now there are absolutely no delays. We have approximately four to five hours’ leeway, and there is no need to worry about the effect of batch processing times on subsequent processing or on online availability the next morning." Another aspect that impressed the bank was the rapid improvement in online processing. 

"Actually," says Koguchi, "since we started actual operations of VSP G1000, users who perform analysis have asked whether the system was operating correctly. When those users were asked the reason for their concerns, they said that processing results were returned so quickly that they were suspicious of whether or not the data was processed as requested." Depending on the application or the data table structure, online processing performance has increased to up to approximately 10 times the previous performance levels. 

In the future, the System Division will continue its initiatives to create a stable environment for business operations. Looking ahead, Koguchi says, "Although we have fully experienced the effects of flash storage, this is not the end. In the future, we would like to review a variety of systems, centered around flash storage." This could mean further expansion of the use of HAF and VSP G1000.