HP NonStop Batch Scheduler - Overview
MultiBatch is a powerful, maintainable Batch Processing environment specifically designed to exploit the HP NonStop server platform architecture both for the Guardian and Open System Services (OSS) operating systems. It allows parallel batch processing with calendar and conditional logic to occur in a single processor or multi-processors concurrently. Batch processing can be invoked locally or across an entire NSK network.
MultiBatch is a maintained, future-proofed product compatible with all HP NonStop system ranges (including the HP NonStop SuperCluster, Itanium and latest Integrity series). The NonStop scheduling software employs strict fault-tolerant techniques and multi-tiered job dependency structuring for the robust and efficient execution of batch schedules.
MultiBatch is used by some of the largest financial and central banking organisations in the HP NonStop user community. It is reliably used in contingency situations to provide an ordered and optimised strategy for site switching during Disaster Recovery and for more general NSK closedown and start-up maintenance procedures.
MultiBatch can also be used to provide an accelerated method for the activation of XPNET ATM and POS networks. For example, making use of concurrent start-up approaches, local or networked XPNET environments can be initiated in parallel by PATHWAY, Logical Node, Class or Station Priority in order to safeguard agreed SLAs.
The MultiBatch product comes equipped with an intuitive GUI interface (PATHWAY screens version also available) for batch schedule configuration and status reporting. Optional extensions allow for certified integration with industry leading Enterprise Management solutions including HP Operations Center (formerly OpenView Operations), BMC Performance Manager (formerly Patrol) and TIVOLI as well as remote alerting using mobile SMS and email.
All job status data can be relayed to a Windows server for both graphical and tabular reporting for Management Reports and Real-Time graphical tracking.
Integrated monitoring, reporting and management of batch schedules are available, including the following features :
- Pathway and / or GUI interfaces
- Batch tracking
- Run Guardian and Open System Services (OSS) applications within the same batch schedule
- NetBatch conversion utility available
- Automated Calendaring and Scheduling
- High performance parallel batch
- Network control transparent
- Simultaneous job execution
- Supports object parameters to ease maintenance in global schedule updates for multiple jobs
- A migration tool can be used to convert batch schedules between nodes for easy replication in a multi-node environment
- Optional checkpoint / restart
- Dynamically amend the start time of job(s) e.g. /- 1 hour
- SNMP gateways available
- 3 levels of job hierarchy, 2 levels of dependency
- Secure execution shell
- Fully EMS compatible
Advantages :
Basically, there are a number of advantages to using MultiBatch in preference to NetBatch and these are summarised below :
Maintenance & Development
NetBatch is a mature product and will not be subject to further development. New facilities and functionality are still being developed for MultiBatch, driven by customer requirements.
Interface
NetBatch has either a command line or Pathway interface. MultiBatch has both of these and a GUI.
Scheduler
NetBatch controls the execution of a batch schedule and relies heavily on a JCL to control the actual jobs within a schedule. MultiBatch controls the execution of each element of a batch schedule. Individual jobs and their elements are more visible within MultiBatch. The attributes of each job within the schedule can be viewed and altered using MultiBatch application screens and all changes are subject to the appropriate security clearance and are recorded in an audit log. NetBatch jobs can be amended using an editor and are thus not subject to any security vetting or auditing.
Failures
NetBatch again relies on a complex JCL to detect failures in the elements of a schedule. Because MultiBatch controls each element of a batch schedule the detection of failures and the restarting of a schedule at the correct point is a simple matter.
Parallel Processing
Once again, NetBatch relies on the JCL for parallel processing and monitoring whilst it is an easy task to configure a MultiBatch schedule to handle this.
Reporting
NetBatch status information is written to a log file or spooler, making it difficult to integrate with a 3rd party enterprise manager. MultiBatch writes status information in the form of EMS events, allowing monitoring by 3rd party tools. Additionally, it is closely integrated with other products from ITL for more in-depth monitoring. In addition to the above, a migration tool is available that will allow a NetBatch schedule to be converted to a MultiBatch schedule.
Click here to view sample MultiBatch screenshots.
HP NonStop Batch Job Scheduling Comparison :
Both HP’s NetBatch and Insider Technologies Ltd’s MultiBatch products are designed to provide a Batch Scheduler for the HP NonStop server environment. Both products do accomplish this, however each uses a different methodology to achieve its aim, resulting in each product having quite different capabilities.
NetBatch and MultiBatch both consist of a PATHWAY interface to a scheduling engine, as well as a command line utility. The MultiBatch product also includes a GUI. NetBatch requires the user to develop and maintain a JCL, written in NBEXEC or TACL, to control the sequencing of jobs. This requires knowledge of both these products and will incur a maintenance overhead.
Although MultiBatch can use both TACL and NBEXEC as executors, natively there is no MultiBatch JCL. All job run parameters, sequencing and dependencies are maintained by the MultiBatch application. The user only requires knowledge of MultiBatch and the batch schedule.
Both products have a calendar facility, enabling automation to be controlled on a time/date basis. Whilst NetBatch applies this control to the batch executor, MultiBatch is the executor, and thus applies this control to the batch jobs themselves.
Although both products have been around for some time, MultiBatch is still being enhanced, with new facilities and features being developed. Insider Technologies Ltd. have no plans for stopping development or withdrawing support. HP classes NetBatch as a ‘Mature’ product, hence no further development or enhancement will be undertaken.
| Competitive Points |
Comments |
NetBatch |
MultiBatch |
| Single point of operations control across the EXPAND network |
|
N |
Y |
| Requires no command-line syntax to configure / initiate jobs |
|
N |
Y |
| GUI available for operational control and status tracking |
|
N |
Y |
| GUI available for batch job configuration |
|
N |
Y |
| GANTT style job status / progress GUI (MCP) |
|
N |
Y |
| GUI available for user configuration management and security control |
|
N |
Y |
| Calendaring facility, includes holidays, etc. specification |
|
Y |
Y |
| GUI available for calendar set-up and management |
|
N |
Y |
| Job control integrity check (pre-run) |
|
N |
Y |
| Application / Job API to start / stop control jobs by application |
|
N |
Y |
| Job dependency checks (job release control) |
|
Y |
Y |
| Device dependency checks |
|
Y |
N |
| Designed and coded for NonStop (Fault Tolerant running) |
|
Y |
Y |
| TMF transaction protected "safe auto-recovery from fail" facility |
Jobs may be safely auto-restarted after failure of a CPU |
N |
Y |
| Able to launch work to be managed by TACL control program / file |
|
Y |
Y |
| Job streams may be run in parallel |
|
Y |
Y |
| Job steps within streams maybe run in parallel |
|
N |
Y |
| Able to launch Open System Services (OSS) programs |
|
Y |
Y |
| Able to accept OSS filenames of full-length |
|
Y |
Y |
| Does not require Shell or Shell Knowledge to launch OSS jobs |
|
N |
Y |
| Security audit log of changes |
|
N |
Y |
| Optional block-mode green-screen supplied as standard for fast bulk entry |
|
N |
Y |
| Ability to prepare a virtual copy of the schedule |
|
N |
Y |
MultiBatch can be provided with an optional Mangement Information Reports module (MCP). This is installed on a nominated MS Windows Server platform with SQL Server 2005. All job related data is stored in the SQL database detailing all batch schedule metrics, e.g. job run times, key jobs.
Users can also script their own reports using the MS Reporting Services tool with SQL Server 2005. This provides a very extensible solution for MI reports for all HP NonStop platforms including Integrity and Blade technologies.
Examples:
HP NonStop Batch Scheduling - MultiBatch Job Run Times
This MultiBatch report provides a bar chart showing the job run times of the specified job id. Additional details are provided in the associated table.
The report allows a user to specify a Start & End Date/Time range; HP NonStop Node name; MultiBatch BMON Process Id; Job Id.
A user can also report on all jobs and/or all BMON processes by specifying 'All' (without quote marks) in the parameter fields.
The line graph and associated table contains the run time (in seconds) of how long the job has taken to 'run'.
Note: There are various reasons for the job 'completion', e.g. the job may have completed successfully, abended, aborted by operator or reached maxtime and aborted. The report is based on the various types of 'completion' events and not just on the MultiBatch 'job complete' event.
The 'Event Text' column contains the MultiBatch EMS event text which will show the reason for the job 'completion'.
HP NonStop Batch Scheduling - MultiBatch Report Key Jobs
This MultiBatch report provides a line graph to show the start & end points of a specified Unit and Job, respectively. These specified values are for key jobs which a user expects to have Started and Completed by a certain time.
The report allows a user to specify a Start & End Date/Time range; HP NonStop Node name; MultiBatch BMON Process Id and the id of the 'start' Job.Segment.Unit and the id of the 'end' Job.
Two additional parameters are required for completion: Batch Hour & Batch Min. A user must specify in these two fields, an expected (or usual) Start Time for the Job.Segment.Unit. For example, if the specified Unit of 'JOBA.AA.01' usually starts ar 15:30, then specify 15 in the Batch Hour field and 30 in the Batch Min field.
The line graph will display two lines, one for the 'start' job.segment.unit and one for the 'end' job. If the 'start' time of the specified Unit differs from the value entered, e.g. 15:30 has been specified but on a particular day the specified Unit actually started at 15:15, then the line graph and associated table will show the differences in start time.
The associated table will show JOB Id1 and its actual start time, with a column showing any + or - differences from the specified 'Usual Start Time'. JOB Id2 is displayed in the table, followd by the 'duration' in minutes, between the start of JOB Id1 and the end of JOB Id2.
Minimum Windows Hardware Requirements for
MultiBatch MS Management Information Reports
For optimum performance, it is recommended that the minimum specification of your hardware and software is as follows :
- Windows Server 2000 / 2003 / 2008 with the latest service packs
- Microsoft SQL Server 2005 / 2008 with the latest service packs
- Either; Tomcat, IBM WebSphere, JBoss (formerly BEA WebLogic) web application servers (ask ITL about MS Internet Information Services- IIS) - also requires Java SE version 6+ (1.6)
- Browsers; Internet Explorer (IE) ver. 7+, Google Chrome, Safari, Firefox
- Intel Mid to High-End, Multi Core processor
- 16+ GB RAM recommended
- SCSI interface (SCSI2 Ultra-Wide recommended)
- 10 GB Single Drive for operating system and SQL Server software
- *40 GB Single Drive for the SQL server database (RAID 0+1 recommended)
- 20 GB Single Drive for the SQL server log file (RAID 0+1 recommended)
- Graphics resolution 1024 x 768 recommended
- 17" or larger colour monitor is also recommended
The above specification is for guidance only. The specification of your Windows server will be dependent on your individual needs. Please contact the Insider Technologies Helpdesk for assistance in establishing the specification of your server.
SQL Server Versions Supported by Sentra (also install Microsoft SQL Reporting Services)
The Sentra database is compatible with the following variants of SQL Server :
- 2005 / 2008 Standard Edition
- 2005 / 2008 Enterprise Edition
- 2005 / 2008 Developer Edition*
- 2005 / 2008 Express – the default installation on the CD uses SQL Express with Advanced services, so that SQL Reporting Services is available
* Some SQL editions include a concurrent workload governor. Performance degrades when more than five queries are executed concurrently. Sentra will work with these versions of SQL Server but performance may be unacceptably slow and its installation is not recommended for high volume usage.
SQL Express editions support databases with a limited maximum size (4Gb for SQL Express 2005). Users who anticipate large database storage requirements should consider installing the Enterprise edition of Microsoft SQL Server, or contact Insider Technologies for advice.
To gain full access to the MultiBatch product demonstration files, please register with this website (click the 'Register' link - at top right of page).
NOTE: Please provide a company / organisation email address during the registration process.
Once your website registration is approved (usually within 24 hours), then 'Login' to the website and navigate to this MultiBatch web page and the demonstration files will be available for download at the bottom of the page.
If you are already a member of this website, 'Login' as normal.
Hardware and software pre-requisites for the MultiBatch product are detailed below:
• HP NonStop (ServerNet or Integrity platforms)
• Guardian D38 Operating System (or above)
• TMF
• Pathway (TS/MP)
• TCP/IP (if using the MultiBatch GUI interface only)
MultiBatch is installed using a menu driven TACL macro for easy deployment. The macro will perform checks and output the results of each stage of the MultiBatch product install so that if problems are encountered they can be resolved easily.
Windows Platform for GUI Install
(not required if using the MultiBatch Pathway Green Screen interface):
• Modern Pentium PC Specification
• 32Mb on-board RAM
• 20Mb free hard disk space
• Minimum 17” monitor recommended
• Graphics Resolution: 1027 x 768 x 16 minimum
• Windows XP, Vista, 7
For MultiBatch management information reports, a Windows Server will be required with Microsoft SQL Server installed. Click the MI Reports tab to view hardware and software requirements for the MultiBatch MI plug-in.
Latest News :
MultiBatch Version 9.4 released
1. Audit Log Housekeeping
From version 9.4 it is now possible to delete or archive information from the audit detail (AUDDET) and audit summary (AUDLOG) files.
The files hold details of PATHWAY based maintenance; insert, delete and amend of records.
Previously you had to stop the MultiBatch software, then COPY and PURGEDATA the files.
2. Error Log Housekeeping
From version 9.4 it is now possible to delete or archive information from the Error Log (ERRLOG) file.
The file holds details of MultiBatch run time errors issued by BMON and captured by Watcher.
Previously you had to stop the MultiBatch software, then COPY and PURGEDATA the file.
To list out the syntax key RUN ERRLOGHK
3. Calendar File Housekeeping
From version 9.4 it is now possible to delete or archive information from the Calendar file (PARAMCAL)
The file holds details of MultiBatch calendar entries; e.g. BANKHOL and LASTSUNDAY and can be used as part of the Include/Exclude scheduling functionality.
Previously you had to delete the entries manually.
To list out the syntax key RUN MBPARCHK
4. Closer Integration of MultiBatch and DS-EventTimer
At release 9.4 of MultiBatch the DS-EventTimer product has been altered to be able to access the MultiBatch database and vice-versa.
The following DS-EventTimer software components will now need to access the DATACONF file to determine the location of the DS-EventTimer files (SCHEDULE and HOLIDAY) rather than access then directly;
• ETIMSRV - The EVENT-TIMER PATHWAY Server.
• RESCHED - Recalculate the next run time of an Event Timer job
• PSCHED - Print out the Event Timer database.
• CALENDAR - Monitor Event Timer start times
• TRIGGER - Start MultiBatch jobs and calculate next run time on completion.
• INTEGSHD - Synchronise the Duplicate SCHEDULE database
• INTEGCAL - Synchronise the Duplicate CALENDAR database
• INTEGHOL - Synchronise the Duplicate HOLIDAY database
Support has been withdrawn for the following components;
• MIGETIM - Migrate the DS-EventTimer database. This is now supported by the Migrator.
• INTEGHOL - Synchronise the Duplicate MBOWNER database.
• Support for the MultiBatch Security check feature.
5. Use of Alternate Collectors by Trigger and Calendar
At release 9.4 the CALENDAR and TRIGGER programs have been extended to support the use of EMS alternate collectors.
A previous MultiBatch release saw the introduction of alternate collector support for MBSTREAM and WATCHER.
6. Wildcard facility for BCOM STATUS JOB command
It is now possible to issue a BCOM STATUS job command with the syntax.
STATUS JOB Prefix<*>
For example;
• STATUS JOB JOB*
• STATUS JOB JO*
• STATUS JOB J*
This command will list out jobs beginning with the designated letters.
The DETAIL and qualifier options, e.g. RUNNING or FAILED are not supported with the wild card facility.
7. Customisable BCOM Output
The output of the BCOM INFO JOB command is re-formatted.
8. MultiBatch GUI Encryption
At version 9.4, the MultiBatch GUI now supports client – server encryption.
There are no NonStop configuration changes required.
When you connect to the GUI, then select Blowfish encryption from the Preferences -> Encryption menu option.
Note: Do NOT select the DES option as there are known NonStop performance issues with this option.
A session key will be established between the MultiBatch client and the FASTPIPE-SERVER PATHWAY server class. The PATHWAY server can handle a mixed client population of encrypted and non-encrypted sessions. Multiple FASTPIPE-SERVER sessions are not required for this scenario.