Can Apache display logfiles of deatched processes (313 Views)
Reply
Regular Advisor
Toine_1
Posts: 106
Registered: ‎07-12-2009
Message 1 of 10 (313 Views)
Accepted Solution

Can Apache display logfiles of deatched processes

Hi,

I want to use Apache on OpenVMS to browse to the output files of detached processes.
I can browse the files but when I open them I only see a part of the output file.
Is there a way that Apache can handle these kind of files without to change the record format to streamlf.

The format of the file is:

Size: 6/16
Created: 24-JAN-2010 04:27:24.58
Revised: 24-JAN-2010 04:27:24.58 (0)
Expires:
Backup:
Effective:
Recording:
Accessed:
Attributes:
Modified:
Linkcount: 1
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 16, Extend: 0, Global buffer count: 0
No version limit
Record format: VFC, 2 byte header, maximum 0 bytes, longest 393 bytes
Record attributes: Print file carriage control
RMS attributes: None
Journaling enabled: None
File protection: System:RWED, Owner:RWED, Group:, World:
Access Cntrl List: None
Client attributes: None

I start my process with the command below.

RUN SYS$SYSTEM:LOGINOUT -
/INPUT=SYS_COM:RSYR300 -
/PROCESS_NAME=VOLVOPROC -
/DETACHED -
/OUTPUT=SYS_DAT:VOLVOPROC.LOG

/Toine
Honored Contributor
Hoff
Posts: 4,962
Registered: ‎01-29-2006
Message 2 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Apache appears somewhere between peripherally involved and easily removed from the environment.

Your detached process logging design looks weak.

Write real log files under real application control.

Set sharing when you open the files. You can then access these logs. Using LOGINOUT as part of your logging means you're depending on OpenVMS to set your sharing and such.

If Apache is your core requirement and you're not in a position to update your current design, then use CGI and bypass the locks with CONVERT/SHARE or analogous, or with your own tools that bypass RMS and monitor and compress your logging data. The CGI can be DCL, or it can be your own code. And your CGI can do data reduction, too, as some of these logs I've seen over the years can be huge, and often filled with mostly-noise uninteresting messages.

For a non-shared log, I'd not expect to see the last hunk of data written to the log; that tends to be cached.

Personally, I'd probably punt on the logging here save for dealing with critical failures (most log files these days don't get much I/O beyond some version numbers or such, until and unless there's a nasty failure) and use syslog or such to send the "routine" logging messages over to a syslog daemon. Certainly having a local log is nice for some (critical) things and for supporting the RSYR300 daemon, err, detached processes here, but having a distributed log can allow the servers hosting multiple Apache web sites and multiple host servers to be managed as more of a unit; it makes dealing with the more routine traffic easier.

Getting debugging better integrated makes this stuff easier to manage and maintain.
Honored Contributor
labadie_1
Posts: 1,221
Registered: ‎08-07-2003
Message 3 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Have a look at the trick of John Gillings in this thread

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=629558
Regular Advisor
Toine_1
Posts: 106
Registered: ‎07-12-2009
Message 4 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Thanks for the answers,

But Apache is not able to read normal VMS process logfiles.

When I use a cgi script via Apache it works OK:

$ type SYS_DAT:VOLVOPROC.LOG

My question is still valid.

Why can't Apache display the same information via browsing the files and opening the selected file.

/Toine
Honored Contributor
Hoff
Posts: 4,962
Registered: ‎01-29-2006
Message 5 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

My guess is that C I/O is slamming into RMS sharing and into RMS buffering; that's the usual trigger for these sorts of shared-access cases.

C by itself doesn't particularly do file sharing by default, and Apache likely isn't expecting to coordinate shared file access as that's simply not the norm on Unix platforms. The Apache port for OpenVMS has itself had various file I/O issues over the years, too. (I've posted up source examples over the years that show how to do file sharing within your own C code.)

In general, you can apportion your time on what seems to be a low-payback Apache debug and rework, or on uprating the debugging and diagnostics support inside this RSYR300 widget. I'd tend toward the latter; toward better diagnostics and logging. And if you're just looking to get this stuff out the door, then you already have a solution with the CGI.

If this research is of more than academic interest to you, then t would be best to go look at the Apache source code; to debug this. That's the arbiter here. Or if you have a support contract in place, then go ask HP to have a look at this stuff for you. (It's their Apache port, after all.)
Honored Contributor
John Gillings
Posts: 2,995
Registered: ‎07-31-2003
Message 6 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

/Toine

>Why can't Apache display the same
>information via browsing the files and
>opening the selected file.

In theory RMS could keep all attributes of a file up to date on every I/O. So, imediately after a WRITE operation, all other processes could see all the data out to the updated EOF, regardless. In practice, this is neither necessary or desirable, as it would incur significant performance degradation for a requirement which is comparatively rare.

The behaviour of reading logfiles while they're being written depends on the sharing attributes specified by the writer when the file was created, and how the reader opens the file. There are enough permutations in OpenVMS for all possible requirements, but the defaults are the most common usage and least costly in terms of performance.

In your case you have (implicit) control over the writer (your detached process), but no control at all over the reader (Apache). From your observations it's clear that Apache is taking the low impact option when opening files. It makes no attempt to force RMS to update the file. Most people would say this is correct.

As you've discovered, you can take control over the reader side with a CGI script which accesses the file the way you want it accessed.

You could request HP change the way Apache opens files, but, even if they accepted your request, it would take a long time before you get the updated version.

I doubt such a request would succeed, because it imposes an unnecessary performance cost on all users, and there's a viable workaround (implement a CGI script) for folk like you with non-standard requirements.
A crucible of informative mistakes
Regular Advisor
Toine_1
Posts: 106
Registered: ‎07-12-2009
Message 7 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Hi John,

Thanks for your answer.

The problem is not open files.

The problem is that Apache can't handle the file format properly. Even when these files are not open by any process.

When I have a file of the format below Apache doens't display the whole content.

Record format: VFC, 2 byte header, maximum 0 bytes, longest 393 bytes
Record attributes: Print file carriage control.

It looks like it detects some EOF but it is just data in the file.

/Toine
Honored Contributor
Bojan Nemec
Posts: 842
Registered: ‎06-02-2004
Message 8 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Toine

This is probably the Apache problem to display VMS files. You can instruct Apache to convert files with the EnableMMap option. See

http://forums13.itrc.hp.com/service/forums/questionanswer.do?threadId=1347832


Bojan
Regular Advisor
Toine_1
Posts: 106
Registered: ‎07-12-2009
Message 9 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Some feedback from HP.

HP has identified the problem and it is a CRTL problem.

We have to wait for the next CRTL update.

Thanks for all your help.

/Toine
Regular Advisor
Toine_1
Posts: 106
Registered: ‎07-12-2009
Message 10 of 10 (313 Views)

Re: Can Apache display logfiles of deatched processes

Hi,

I got a good solution from HP.

When you define this logical name then Apache is displaying the complete logfile.

$define/system DECC$STDIO_CTX_EOL "ENABLE"

With DECC$STDIO_CTX_EOL enabled, writing to stdout and stderr for stream access is deferred until a terminator is seen or the buffer is full.

With DECC$STDIO_CTX_EOL disabled, each fwrite generates a separate write, which for mailbox and record files generates a separate record.

Just for your info.

/Toine
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.