 |
|||||||||
|
|
|||||||||
        |
Feature Articles: Technologies for Promoting Use of Open Source Software that Contribute to Reducing TCO of IT Platform Vol. 12, No. 6, pp. 27–32, June 2014. https://doi.org/10.53829/ntr201406fa4 Faster System Building with
No Reworking!
|
||||||||
 |
 |
1. Introduction
In the NTT Group, we are consolidating the work of building and managing systems by building and migrating various internal systems that have been operated separately onto an integrated infrastructure. In doing this, as the number of servers comprising the integrated infrastructure increases, the need for ever more efficient operation in system building and management tasks increases.
Various commercial tools that support system building and management have been offered to satisfy such needs. However, there are barriers to using them such as being restricted to vendor-specific hardware, or a very high cost.
To solve these issues, the NTT Open Source Software (OSS) Center has been studying and developing the OSS Suites Verified Technically (OSSVERT) Smart Builder (O-SB) tool. Our objectives are focused on reducing the effort required and the human error involved in building systems by using open source software (OSS), and maximizing the use of our accumulated OSSVERT system-building know-how.
2. OSSVERT Smart Builder
2.1 Overview
System building and management work can be divided into three processes: design, installation and configuration, and testing (Fig. 1). Of these, the processes of installing (or upgrading) the operating system (OS) and middleware, configuration, and testing increase proportionately with the number of servers, and are prone to human error.
Fig. 1. System building and management process.
In developing the O-SB, we focused our study on the process of installing and configuring the OS and middleware, which consumes a particularly large amount of time, and we implemented functions to automate these processes. A concrete representation of O-SB operation is shown in Fig. 2. The O-SB server automatically installs and configures the target server over the network according to a predefined procedure. It is fully automated from the start through the completion of installation and configuration and requires no operations by the user. The O-SB server also has a function to maintain the target server in a correct state after the initial system building process is completed, in case an error occurs in the environment changes or in another area.
Fig. 2. O-SB basic operation.
2.2 Technical points
The sequence of operations between the O-SB server and the server being built is shown in Fig. 3. O-SB has two technical features, as follows.
Fig. 3. Sequence of operations between the O-SB server and the server being built.
(1) It uses OSS (Puppet*1, Kickstart*2) that has a proven track record of building and managing real systems used in a variety of fields such as finance, information, and SNS (social networking services).
(2) It uses OSSVERT system-building know-how. Puppet and OSSVERT are particularly important and are therefore explained in more detail.
Puppet is an OSS automatic system configuration management tool. The user describes the desired state of the system in a configuration file called a manifest. The puppet master reads the manifest and uses a puppet agent placed on the server to configure the system on the server being built according to the manifest (Fig. 4). This allows installation and configuration of middleware and other software, and various types of configuration for the OS. The description in the manifest is extremely flexible, so there are multiple ways to describe any given configuration. This could cause confusion about how to describe a configuration, so O-SB provides an outline of the description created in order to reduce the work required by users.
Fig. 4. Organization of Puppet.
O-SB also provides puppet manifest templates that allow various OS and middleware servers to be built immediately. These templates have been created by adding OSSVERT know-how to the aforementioned outline descriptions. OSSVERT is the OSS stack recommended by the NTT OSS Center when building a web-based three-tier system. OSSVERT validates the behavior (performance, long-term stability, maintainability, etc.) of an entire web-based three-tier system, including benchmark applications, and gathers technical materials including installation manuals, environment definition documents (e.g., a list of recommended settings), and test reports.
O-SB automatically builds systems that have been verified to operate correctly, according to the OSSVERT installation procedure and environment configuration documents and using the described manifest. However, some values (addresses etc.) need to be changed in real scenarios, so a document is provided that summarizes the set of parameters to be changed and where in the templates the changes can be made.
3. Application example and results
We measured the effectiveness of the tool when building a web-based three-tier system of an OSSVERT configuration with six physical servers (web, application, and database servers, each in a double or redundant configuration). The building procedure was done in two steps, starting with the six physical servers in their initially purchased state and physical connections in place: (1) installing and configuring the OS (including drivers), and (2) installing and configuring various types of middleware. We confirmed that this would require approximately 60 hours of manual work by a typical technician, but this was reduced to approximately 8 hours (one-seventh the time required to do it manually) by using O-SB. This 8-hour period includes the time to build the O-SB server itself and to code the definition file (manifest) for the servers to be built. To rebuild the same environment (second and subsequent times), the O-SB server can be reused, so the systems can be built in about 2 hours (in one-thirtieth the time) (Fig. 5). We also confirmed through simulation that if the number of servers increases further, O-SB can run in parallel, so for up to a certain number of servers, the building time will not increase, depending on the specifications of the network and servers.
Fig. 5. Potential time savings by using O-SB.
This confirms that O-SB is effective in reducing the time required for installation and configuration. We also confirmed other beneficial effects such as a reduction in verification work necessary during the design process by incorporating OSSVERT design know-how into the manifests, a reduction in human errors in the configuration process, and elimination of the need for operator intervention during the installation and configuration process.
4. Future plans
The software that can be built using O-SB includes typical items in OSSVERT configurations, as indicated in Table 1. For the software listed in the table, users can create manifests to install and configure the desired software in a relatively short time using templates, even if versions are somewhat different.
Table 1. Software that can be installed and configured using O-SB.
NTT OSS Center is preparing training materials for users to guide them in areas such as revising templates and creating a manifest from scratch. We are also providing support for inquiries regarding Puppet through a help desk.
As the scale of systems becomes even larger, we expect the number of scenarios in which O-SB can be used to increase even more, so we will continue our research and development while anticipating applications of O-SB in various environments in the future.
References
| [1] | Y. Hirakawa, “RHEL6 in Practice! Anyone can Build a Home Cloud (Pt. 4)—Building a Provisioning Server,” Nikkei Linux, July 2011 issue, 2011 (in Japanese). |
|---|---|
| [2] | S. Uchikawa, “The OSSVERT Initiative: For Using Open Source Skillfully,” NTT Technical Journal, Vol. 19, No. 3, pp. 60–62, 2007 (in Japanese). |
| [3] | H. Toyoda and M. Okumura, “OSSVERT Extensions Supporting Introduction of Open Source Software,” NTT Technical Journal, Vol. 23, No. 8, pp. 8–13, 2011 (in Japanese). |
