11-21-2005 01:24 PM
We have a program written (largely) in COBOL. It's compiled & linked on an AlphaServer DS20E running OpenVMS V7.3-2 then copied to an AlphaServer 4100 (also running V7.3-2). The COBOL command line does not specify any architecture-specific compile qualifiers.
Using PCA we see that 75% or more of the program's CPU time and elapsed time is spent in AMAC$EMUL_CALL -- whatever that is. The program consumes about 30 minutes of CPU time.
Our problem is that the new version of the program -- which has had several enhancements -- runs 10 times slower than the previous version. We expected it might run twice as slow at worst.
The program reads a few million raw billing records and produces a "print file" containing invoices for each account & item. A very typical COBOL application!
So now we're trying to work out why it's so slow. It seems to be compute-bound most of the time which is crazy because all it should be doing is reading a record, adding up various values, and summarising by account code & line item code. It should be I/O bound if anything.
We have recompiled everything /DEBUG and linked /DEBUG then ran it using PCA (Performance & Coverage Analyzer).
I then ran PCA PLOT PROGRAM BY MODULE and the result was very odd: 75% of the CPU time (and elapsed time) is spent in something called AMAC$EMUL_CALL. The next biggest contributor is the main program PRINT_BILLS at 16% and then assorted other modules.
I thought maybe the problem was alignment faults but after identifying the worst offenders and fixing those, the program's run time *increased* slightly and the portion assigned to AMAC$EMUL_CALL increased slightly.
(I verified that the alignment faults had in fact reduced.)
I then identified an INITIALIZE statement in the COBOL which was responsible for about 1/4 of the CPU time in PRINT_BILLS (it was initialising an 8MB data structure) and we decided that wasn't necessary. Our next run took about the same elapsed time and CPU time as out previous runs, but now PCA tells me the amount of time spent in PRINT_BILLS has decreased (as I expected) and the amount of time spent in AMAC$EMUL_CALL has increased (as if it's trying to maintain a constant CPU consumption!).
Has anyone seen behaviour like this before?
11-21-2005 01:38 PM
Please log a case and send me some sample code.
COBOL has some rather baroque CALL semantics, I suspect you're unwittingly using something that requires emulation.
11-21-2005 02:02 PM
Call logged. Case ID: 2206642079
The large size of this program makes it difficult to identify an extract to send you. Can you give me some clues as to the sort of things I should be looking for?
11-21-2005 02:15 PM
I think AMAC$EMUL_CALL is a red herring. It's a routine for setting up a procedure call, there's very little to it, just move the arguments to the right place, and JSR to the correct place. It's the business end of (for one) LIB$CALLG.
From a PCA perspective, yes, it will appear that you're "inside" AMAC$EMUL_CALL when you're really inside whatever it is you've called through AMAC$EMUL_CALL - maybe that's PRINT_BILLS?
If you don't have any explicit LIB$CALLG type calls, maybe you're using COBOL's funky "call by name"?
MOVE "MY-ROUTINE" TO RTN.
CALL RTN USING ARG1,ARG2.
Ultimately this will become an emulated call. It also requires COBOL to look up the routine name to find the procedure descriptor. If it's a very large module, this might be CPU intensive.
Make sure you have /LIST/MACHINE listings for all your COBOL modules, and LINK/MAP/FULL/CROSS. It should be relatively easy to locate the source code calling AMAC$EMUL_CALL.
11-22-2005 02:59 PM
AMAC$EMUL_CALL is used to call the main program from DCOB$MAIN, so it appears to PCA that 100% of the time is spent inside that routine. It's like the joke about the guy in the balloon in the fog... the answer is 100% accurate, but completly useless for solving the problem at hand.
It looks like the problem with Jeremy's program is using the INITIALIZE verb to initialize several large arrays. Because of language semantics, for some field PIC definitions, INITIALIZE has to write individual fields within each array element, often via a call to DCOB$EDITPC to format the field according to the PICTURE clause. For a very large array, that gets very expensive.