PTLIB:
Review Policy for Parallel Debuggers
and Performance Analyzers

PTLIB is to be a repository of parallel systems software and tools. The following is a list of evaluation criteria and properties that PTLIB maintainers are considering in selecting debugging and performance analysis tools to be included in the repository. This list is drawn from various sources, such as the Baseline Development Environment, Ptools Consortium guidelines, and Ptools user survey, and from the parallel tools literature. Not all criteria and properties are applicable to all tools. A tool might not have all the properties listed below and might still be a good tool. At present we are not making any normative judgments about what properties or what level of compliance are required or desirable to achieve different levels of review award.

Evaluation criteria Evaluation properties
Capabilities Support for multiple threads per node
Presents different levels of abstraction, from global to individual threads, procedures, or data structures
Single point of control for parallel debugging ( BDE-2b)
Attach/detach to/from running program
Breakpoints and data watchpoints
Program state examination
Program state modification
Handles optimized code
Event tracing mechanism
Cache and memory reference tracking/display (Ptools User Survey)
Remote data access pattern analysis
Message tracing/display
Input/output characterization
Real-time monitoring
Post-mortem analysis
Profiling at level of procedures and coarse blocks
Utilization display
(communications/idle/IO/computation) (Ptools User Survey)
Statistical analysis
Performance prediction (Ptools User Survey)
Comparison between different runs
Race detection
Execution replay
Meets demonstrable need in the user communitry ( Ptools Project Guidelines)
Versatility Languages/programming models/communication libraries supported
Runs on currently popular platforms
Platform dependencies isolated
Support for heterogeneous environments
Interacts with current or soon-to-be standards
(e.g., PVM, MPI, HPF) ( Ptools Project Guidelines)
Uses SDDF
Change/customize/add new views easily
Performance Scalable data collection
Scalable data presentation
Acceptable monitoring overhead
Intrusion compensation
Acceptable response time
Memory/disk requirements
Maturity Runs without crashing monitored program
Reports and recovers from error conditions
Support
Ease of use Adequate documentation
Ease of installation
Command-line interface
Window-based interface
GUI common look-and-feel (OSF/Motif Style Guide) ( Ptools Project Guidelines)
Privilege-free installation
Reports information at source code level
Automated instrumentation
Usable without special linking
Other Commercial/research
Cost

Last updated October 11, 1996

ptlib_maintainers@nhse.org