A shortage of artisans in the Navy aviation maintenance arena has created the demand for alternatives to classic training and documentation systems that suggests the need for a Maintainer's Electronic Performance Support System (MEPSS™). The P-3 Operational Advisory Group (OAG) Maintenance Working Group estimates that the provision of a MEPSS™ system and its "just-in-time" training and troubleshooting would reduce the mistaken removal of functioning Engine Driven Compressors (EDC) by 25 percent (a cost of $1,053,390 annually) (GTRI, 2000). Online manuals, decision-support systems, training, diagrams, communication systems, and real-time access to the maintenance information system provide alternatives to traditional maintenance procedures. The problem with the existing system and process is that it is cumbersome to use, fails to provide a friendly user interface, and does not provide "at the equipment" support.

MEPSS™ Design

MEPSS™ is a web-based Electronic Performance Support System that uses simple technology to improve worker performance. In analyzing the Navy maintenance arena, we identified the following areas as having the most potential to improve worker performance:

• preventing false removals of EDC,
• improving the accuracy in reporting actions taken,
• reducing the amount of time maintainers take to troubleshoot a problem.

After observing users in their environment, we identified several critical components that MEPSS™ needs to incorporate into single entry access points: troubleshooting, parts database, maintenance records, technical manuals, and training. Based on this information, we developed an open architecture, web-based system that communicates with other main computing systems in the maintainers' environment. Since the architecture consists of non-proprietary technologies, the system can be easily modified or updated as the newer technologies become available and as other maintenance information systems are developed for the US Navy. The modular design also ensures that the system can be easily updated to reflect changes in the community and/or organization. Therefore, the MEPSS™ open architecture design ensures that the system can evolve as the organization grows and as technology advances.

Major Design Elements

Portability

Since the majority of work in the Navy maintenance environment is completed in the field (in the hangar bay, on the runway, on the flight line), we had to design a system that was easily accessible and portable. Furthermore, military standards (MIL-STD) and certain environmental hazards required that the system meet specific physical characteristics concerning weight and size. Although MEPSS™ can be hosted on most types of hardware, this particular system is housed on a portable MIL-STD laptop that is both light and small enough to not interfere with mechanics tasks and workspace environment (Figure 1). We designed the interface so that the system can be used on devices that have small screens and added interface widgets to allow for a touch screen displays. These features provide maintainers with a portable, ruggedized system that allows them to attain "just-in-time" help while in their work environment.

Figure 1. P-3 Maintainer uses MEPSS™ on a MIL-STD laptop while on board a US Navy P-3 aircraft.

Usability

• Ease of navigation: The system provides three ways to navigate (search, main menu, and breadcrumbs) so that maintainers can easily and quickly retrieve the desired information.
• Learnability: We used terminology native to their environment to reduce the amount of time it takes for maintainers to become familiar with the system.
• Recoverability: Cancel and back buttons, breadcrumbs, confirmation windows, and alert boxes allow for error recoverability.
• Consistency: Page layout, color, style, elements and icons are consistent throughout each page to provide familiarity.

Addressing these four major usability issues ensured that our system could be easily inserted into the current work environment and adopted by the maintainers.

Context-Specific Content

Figure 3a. MEPSS™ login screen.

Figure 3b. MEPSS™ homepage with user name, current discrepancy and time.

Figure 3c. Pre-filled fields.

Interface Layout

Main Menu

The main menu is made up of five basic components that are accessible from every page: the navigation menu, a search engine, breadcrumbs, discrepancy description, and shortcuts (Figure 4).

Figure 4: MEPSS™ main menu

1. Navigation menu - One-click menu allows users to access each major module in the shortest possible time.

2. Search engine - To locate information quickly, users can enter keywords into the search field. Results are categorized by module type (training, technical manuals, parts, etc.) and by relevance (Figure 5).

   
   Figure 5: Sample Search Results

3. Breadcrumbs - Helps maintainers keep track of the pages they last visited, and allows them to quickly jump to any of those pages with one click.

4. Current discrepancy description - Posted at the top of every page so that maintainers can refer to it as they troubleshoot a problem.

5. Shortcuts menu - Provides one-click access to common actions. Shortcuts allows users to access the home page, send comments to QA, or logoff. The customized home page contains the three most recent passdowns associated with the current discrepancy, the current date and time, technical manual updates, helpful links, and important news and announcements.

Modules

1. Discrepancies - Discrepancies allow maintainers to view, edit, or submit their current discrepancy and all work center discrepancies. They are listed in a table that contains details about each problem, such as the JCN, BUNO, priority code or description. Discrepancies can be marked as complete or incomplete. In addition, we have included passdown capability. A passdown function allows maintainers to either report the repairs they have completed thus far so that they can pass on their discrepancy to another maintainer, or to pass on helpful tips and hints to other maintainers about repair procedures.

2. Troubleshooting - Troubleshooting references are designed to assist the maintainer in troubleshooting a specific problem in the quickest, easiest manner. Since maintainers learning styles vary greatly, the system provides two types of reference tools: a decision tree and common problems coupled with resolutions.
   • Decision tree: The decision tree contains a list of the most common problems associated with the EDC. To use the decision tree, maintainers click "Troubleshooting" on the main menu that opens a page that lists the highest order of problems associated with the EDC. Maintainers click the description that most accurately characterizes their problem. The system then displays the next level of the decision tree and the user continues in the same manner until he or she has reached some point of satisfaction or the end of that path (Figure 6). Once users have reached the end of the path, they can branch to the technical manual associated with that error to view the correct procedure.

   • Common problems: The Troubleshooting menu contains a link to a list of the most historically common EDC problems and their associated resolutions. Each procedure is cross-referenced with the fault tree and provides one-click access to the fault tree.

   
   Figure 6: MEPSS™ Fault Tree

3. Training: Multimedia training resources demonstrate repair procedures using photographs, technical drawings and animations (Figure 7). Since safety is of utmost importance, each training module contains consistently colored and placed hazard alerts to inform maintainers of hazards when performing specific procedures. Links to associated training documents and procedures are cross-referenced throughout the modules.

   
   Figure 7: Sample Training Module

4. Technical Manuals - This module provides a list of all the technical publications associated with the EDC and one-click access to commonly used publications (Figure 8). Publications are in pdf format and a left frame provides quick links to bookmarked procedures within the manual. If maintainers find errors within a tech pub, they can submit deficiency reports to quality assurance (QA) using a built-in form.

   
   Figure 8: Technical manual viewer.

5. Parts/Supply database - This module allows users to access the Parts/Supply database to order, locate, or obtain information on specific parts. Currently, the parts database is pre-populated with commonly ordered components. In the future, this component can be linked to the US Navy's maintenance information system and thus updated in real-time. The parts database includes the following subcomponents:

   • parts catalog: lists information about all EDC parts, including manufacturer, description, and technical manual references.

   • part ordering form: allows users to fill out electronic part order forms thus reducing the number of potential data entry errors and reducing the amount of information users must input into the system.

   • Illustrated Parts Breakdown (IPB) manuals: provide in-depth information about each EDC part.

   • maintenance records: manages maintainers' discrepancies. Each discrepancy has a unique Maintenance Action Form (MAF) which includes the problem encountered, the corrective action and other related information (Figure 9).

     
     Figure 9: MAF Print Form

     Maintainers are required to fill out the MAF with any information used to complete the discrepancy, such as ordered parts, how maintainers solved the problem, and man-hours. When the problem is resolved and the MAF is complete, the MAF is handed over to management who use the information for assessing and tracking maintainers' performance. To reduce the amount of information they enter, MEPSS™ automatically pre-fills maintainers' information (obtained at login) and the parts ordered (obtained in the parts order form). Extensive error checking is included on this form to enforce consistent data standards. The form used to edit the maintenance action form in Figure 5 only contains the fields the maintainers are responsible for editing and does not include extraneous fields.  This helps to streamline the process of completing forms.  Users can print the form, which has been designed to look like the paper-based form (Figure 9).

• Search function provides maintainers with the ability to quickly find information.
• Consistent layout, breadcrumbs and menus on every page allows for easy navigation and error recoverability.
• Automated error checking improves data consistency and completeness.
• Session data automatically pre-fills fields in required forms to reduce the time maintainers spend completing forms.
• Training layout encourages maintainers not to skip steps in training and technical manual procedures.
• Modular digital data means that technical manuals can be quickly and easily updated.

2. Establishes, or aids in establishing, goals.

• The main menu provides maintainers with idioms and visual clues that are specific to the maintenance arena, e.g. the menu buttons Discrepancies, Technical Publications, Training, Parts and Troubleshooting. Each buttons serves as a visual reminder to guide users through the troubleshooting process.
• The mechanic's current discrepancy is posted at the top of every page, thus providing a ubiquitous description of the problem at all times.

3. Minimizes terminology translation or interpretation.

• Labels, menus, and descriptions reflect the idioms and syntax specific to the maintenance arena.
• Digital forms use verbiage consistent with paper forms.
• Pre-fill sessions provide automatic input into fields where errors and redundancy may occur (for example, part numbers or JCN).

4. Provides access to supporting and learning resources.

• Access to training resources designed to demonstrate procedures and repairs with photographs, digital drawings and animations.  This module also alerts users of hazards involved in performing certain procedures.
• Access to a searchable database that includes previous problems and resolutions performed by others in the community.
• Access to electronic technical manuals.

5. Focuses on task(s), processes, and the natural flow of work.

• Reduces the paperwork burden, by automatically inserting information into forms. For example, if a user orders a part then the part information is automatically included in the MAF.
• Since the MAF is a key element in the maintenance environment, users have one-click access to the form associated with each discrepancy. Forms can be updated at any time during the repair process.
• System portability allows users to take the system into the field as they would a paper-based manual.

6.  Stretches the PCD/EPSS paradigm.

• The system simplifies the troubleshooting process. MEPSS™ provides two tools (common discrepancies and a fault tree) to guide users in identifying the correct procedure for correcting a problem and to using the appropriate technical manual associated with the discrepancy.
• MEPSS™ has the ability to learn from its users and becomes a more effective tool as more users repair discrepancies and report their corrective actions.
• Passdowns allows maintainers to communicate with each other and to relay important information about "repairs in progress" to other maintainers.
• Simple interface reduces the number of actions a maintainer has to take to perform common tasks.

The maintainers' previous environment was paper-based. All information related to repairs was first filled out on paper forms by the maintainers, and then handed over to managers who then entered the information into a database which was located and housed in another department. Parts order forms, MAFs, passdowns and tech pub errors were handwritten. As soon as the next generation maintenance information system is implemented, the intermediate step of entering information into two systems can be eliminated.

Maintainers were restricted to using one set of technical manuals per work unit and manuals had to remain in one location. Although maintainers are required to take manuals to the repairs site, they were extremely heavy and cumbersome to use.

Communications, like passdowns and completing discrepancies, were completed either by word-of-mouth or written in notebooks or written in small wheelbooks. Wheelbooks are similar to cheat sheets, and are considered "unauthorized" in many places. However, many maintainers considered wheelbooks to contain more accurate repair procedures than many of the technical manuals.

Initial training was provided in classrooms, with some classes on CD-ROM or in PowerPoint format. Classrooms were located away from the hangers where repairs took place. Other training was provided on-site at the repair site in a hands-on format. Many maintainers said that they learned new procedures either by following a Senior maintainer's lead, or by reading how to perform the repair in the tech manual.

Users expected to employ MEPSS™ are professionally trained Naval Aviation Maintainers. Maintainers have a wide range of backgrounds. Most range in age and experience from 18 and inexperienced, to 40 and fully experienced. There is a high percentage of men in the maintenance area, and less than 10% of all maintainers were women.

Since Navy training requires that users have some academic and computer experience, most users have a high school diploma, some basic computer knowledge and are familiar with the Windows environment. However, we discovered that some of our system users had no internet browsing experience.

Maintainers perform advanced troubleshooting and failure analysis in order to determine the cause of failure of all major systems on Navy airplanes, in addition to their related clerical and upkeep tasks. They also undergo constant skill upgrading and are well versed in policy and procedures.

After extensive research, we found that the current issues with inserting technology into legacy military systems fell into three categories: hardware, software or human-computer interaction issues. Maintainers wanted online, timely access to all technical publications, including parts ordering, work orders and training. Displays had to have high resolution, be portable, rugged, and lightweight and have a long battery life. Interfaces had to be easy to read, understand and navigate, databases had to be easy to search and update, and private information had to remain private. However, users sited a need for access to other maintainer's notes and tips concerning typical procedures.

Several areas of concern became apparent. When creating large-scale architectures on several different servers or networks, problems may arise concerning ownership of the information, system maintenance, information access and privacy of personal data. These questions allowed us to generate a list of constraints:

• Hardware and software environment: the US Navy MEPSS™ was developed and deployed using an MS Windows operating system for interoperability with the existing infrastructure.
• End-user environment: Concerns about effectively implementing a wireless network within the current structural infrastructure (certain building materials may interfere with different types wireless communication). Also, the potential for physical damage to the unit also constrains the types of materials it can be constructed from.
• Cost and size of the system will determine the number of on-site, portable systems.
• All interface and protocol requirements must fall under Military Standard (MIL-STD) guidelines as documented.
• Since the system provides for performance tracking, users must login requiring the system to be password protected.
• Verification and validation requirements follow rigid testing and evaluation procedures.
• Memory and other capacity limitations will depend on the user environment.
• Data repository and distribution must follow MIL-STDs .

Initial usability tests show that maintainers of all experience and levels were fairly comfortable with system navigation. In one particular instance, a user who had absolutely no internet browsing experience was able to complete a pre-determined list of tasks without any assistance. In fact, this maintainer was able to complete tasks that more experienced computer users were not able to complete.

MEPSS™ will be used in everyday activities beginning in late December 2001. We plan to collect metrics on several areas to determine the effectiveness of the system. Some of the metrics we will collect include:

• Number of P-3 EDC removals
• Number of maintenance man-hours for repair
• Number of false removals due to better troubleshooting
• Number of required tools
• Improved data integrity
• Cost
• Improved readiness
• Ease of use

Analysis to date suggests that better access to training, troubleshooting and tech pubs will assist in decreasing the number of false removals of the EDC.