Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name (870 Views)
Reply
Regular Advisor
Peter Barkas
Posts: 157
Registered: ‎03-14-2005
Message 1 of 16 (1,147 Views)
Accepted Solution

HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

I suspect that this is a CIFS problem, but it is possible that I am missing a C patch or that upgrading to OpenVMS 8.4 may fix the problem.

 

I will log a call with HP but wondered if anyone is able to reproduce the problem or confirm that it is not a problem on 8. 4?

 

When an ODS2 filename is 39 characters the "." disappears and CIFS presents the file type as part of the file name.

 

For example:

 

123456789012345678901234567890123456789.txt

 

is presented as:

 

123456789012345678901234567890123456789txt

Esteemed Contributor
Jess Goodman
Posts: 390
Registered: ‎09-21-2004
Message 2 of 16 (1,098 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

I can confirm that this problem also occurs on my system:

 

HP I64VMS OPENVMS V8.4 (update 8)

HP I64VMS SAMBA V1.2-ECO1 (PS2_11)

 

--------------------

 

Directory command from DCL on VMS:

 

Directory DISK$ODS2TEST:[CIFSTEST]

12345678901234567890123456789012345.TXT;1        0.50KB  28-JAN-2014 22:03:31
123456789012345678901234567890123456.TXT;1          1KB  28-JAN-2014 22:03:23
1234567890123456789012345678901234567.TXT;1         1KB  28-JAN-2014 22:02:56
12345678901234567890123456789012345678.TXT;1        1KB  28-JAN-2014 22:02:49
123456789012345678901234567890123456789.TXT;1    0.50KB  28-JAN-2014 22:02:36

Total of 5 files, 4KB

 

-----------------------

 

LS command from samba cilent (SAMBA$EXE:SAMBA$SMBCLIENT) on I64 VMS 8.4:

 

Domain=[VMSCIFS] OS=[OpenVMS] Server=[Samba 3.0.28a]
  .                                   D        0  Tue Jan 28 21:55:16 2014
  ..                                  D        0  Tue Jan 28 21:52:26 2014
  12345678901234567890123456789012345.TXT             465  Tue Jan 28 22:03:31 2014
  123456789012345678901234567890123456.TXT             556  Tue Jan 28 22:03:23 2014
  1234567890123456789012345678901234567.TXT             536  Tue Jan 28 22:02:56 2014
  12345678901234567890123456789012345678.TXT             871  Tue Jan 28 22:02:49 2014
  123456789012345678901234567890123456789TXT             366  Tue Jan 28 22:02:36 2014


                9297 blocks of size 512. 64256 blocks available

 

--------------

 

DIR command from Command Prompt on Windows XP:

 

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\goodman>dir \\172.20.90.42\ODS2TEST\
 Volume in drive \\172.20.90.42\ODS2TEST is ods2test
 Volume Serial Number is 1188-2DDB

 Directory of \\172.20.90.42\ODS2TEST

01/28/2014  04:55 PM    <DIR>          .
01/28/2014  04:52 PM    <DIR>          ..
01/28/2014  05:03 PM               465 12345678901234567890123456789012345.TXT
01/28/2014  05:03 PM               556 123456789012345678901234567890123456.TXT
01/28/2014  05:02 PM               536 1234567890123456789012345678901234567.TXT
01/28/2014  05:02 PM               871 12345678901234567890123456789012345678.TXT
01/28/2014  05:02 PM               366 123456789012345678901234567890123456789TXT
               5 File(s)          2,794 bytes
               2 Dir(s)   1,106,640,896 bytes free

 

-----------------------

 

Luckily for us we do not use ODS2 operationally anymore.

 

Jess Goodman

AccuWeather, Inc.

http://www.accuweather.com

I have one, but it's personal.
Regular Advisor
Peter Barkas
Posts: 157
Registered: ‎03-14-2005
Message 3 of 16 (1,076 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Jess, thank you for taking the time to confirm that this is also a problem on Intel OpenVMS 8.4.

 

I would prefer to use ODS5, but I cannot implement this immediately.

 

HP Support have reproduced the problem on both CIFS and Advanced Server.

 

It has been suggested by HP that CIFS may not be changed because the behaviour is consistent with Advanced Server.

Honored Contributor
John Gillings
Posts: 2,994
Registered: ‎07-31-2003
Message 4 of 16 (1,044 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

 

>It has been suggested by HP that CIFS may not be changed because the behaviour is consistent with Advanced Server.

 

This is a classic example of what seems to be the new "engineering excellence" in HP OpenVMS engineering. Creative excuses to not fix manifest bugs.

 

By this logic no bug that ever gets into the wild will ever be fixed "because someone might depend on it".

A crucible of informative mistakes
Respected Contributor
Brad McCusker
Posts: 285
Registered: ‎05-01-2003
Message 5 of 16 (1,031 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

John -

 

I agree with your concern about not fixing bugs "becuase that's the way its always been".  But, I'm not so sure that is the case here.

 

PATHWORKS implemented the "float the dot" implementation that allowed file names to have greater than 39 characters before the extension.  I recall we broke this at some point, I think trying to fix this exact problem (I am not sure if our "break" ever got released).  I believe we then ended up documenting a limitation or restriction regarding certain file names on ODS2.  I don't recall and I can't find what that limitation was.

 

It is possible that the limitation is this exact situation.  I've searched through a lot of my old stuff and I can't find anything.  I have a feeling it was all in the internal notes conference (HYDRA notes if there are any HP folks reading who still have access to Notes).

 

Just out of curiosity, what happens when you have a 40 character file name?  Does it float the dot properly?

 

Brad McCusker

Software Concepts International

www.sciinc.com

Respected Contributor
Brad McCusker
Posts: 285
Registered: ‎05-01-2003
Message 6 of 16 (1,028 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

The more I think about this, I don't believe the behavior documented here matches the PATHWORKS behavior - does anyone still have a PATHWORKS V6 or Advanced Server machine that can test this? 

 

When testing, include filenames that exceed 39 characters, please. 

 

Brad McCusker

Software Concepts International

www.sciinc.com

Advisor
john Dite
Posts: 26
Registered: ‎06-15-2004
Message 7 of 16 (1,021 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Hi,

 

I can confirm that on my ALPHA OpenVMS V8.3 system using Advanced Server V7.3B  

on an ODS-2 disk a file named

123456789012345678901234567890123456789.txt;1

gets presented as

123456789012345678901234567890123456789TXT

 

John

Like a blind man in a dark room, looking for a black cat ...that isn't even there
Respected Contributor
Brad McCusker
Posts: 285
Registered: ‎05-01-2003
Message 8 of 16 (1,017 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Thanks John. 

 

As I think about it, I think that makes sense.  There is no way for us to know if that dot after the 40th character is a real dot or a pseuod dot for an eventual floated dot.  One might quickly think that we should know its real becuase there is no "__2E" floowing it (which is the "floated dot") and I do believe that is what we once tried to fix and ended up concluding that it couldn't be fixed (without breaking something else).  Or something like that. 

 

Oh what I would give to have access to those old notes conferences - I am pretty sure this was thoroughly hashed out at the time.  I also thought it was documented - probably in some release notes somewhere, but, again I couldn't find them anywhere.

 

Brad McCusker

Software Concepts International

www.sciinc.com

Trusted Contributor
Doug Phillips
Posts: 287
Registered: ‎02-25-2004
Message 9 of 16 (1,008 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Curiosity made me fire up the old MV3100 with OpenVMS V6.2 and:

 

$ admin show vers

PATHWORKS V6.1 for OpenVMS (Advanced Server)

 

! from Windows File Manager in Advanced Server share
! New -> Text File -- created 2 files,  39.3 and 40.3

 

! the dot disappeared on the 39.3 file.

! looking from CMD


K:>dir/b
1234567890123456789012345678901234567890.TXT
123456789012345678901234567890123456789TXT

 


! from vms

$ dir/col=1/nohead
DKA0:[USER.DOUG.TEST]123456789012345678901234567890123456789.0__2ETXT;1
DKA0:[USER.DOUG.TEST]123456789012345678901234567890123456789.TXT;1

! vms sees a dot

 

! Use Windows File Manager to rename the 39 file, inserting the dot before the TXT

 

K:\>dir/b
1234567890123456789012345678901234567890.TXT
123456789012345678901234567890123456789.TXT

 

! rename kept the dot


! from vms

 

$ dir/col=1/nohead

123456789012345678901234567890123456789.0__2ETXT;1
123456789012345678901234567890123456789.__2ETXT;1

! the dot is there and the extension is now escaped

 

So, rename treats the names differently. Interesting.

 

Respected Contributor
Brad McCusker
Posts: 285
Registered: ‎05-01-2003
Message 10 of 16 (975 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

From the PATHWORKS V6.1 release notes (not sure if this ever appeared in any Advanced Server release notes):

 

        7.3.1 OpenVMS File Names with a Dot at Position 40 Are Not

              Displayed Properly from the Client

 

              Problem:

 

              If a file created natively on an OpenVMS system has a dot

              at position 40 of its file name, the dot does not appear in

              the file name when viewed from a client.

 

              Solution:

 

              If you want a file that is created on an OpenVMS system to

              be viewed correctly from the client, give the file a name

              that does not have a dot at position 40.

 

              When performing an operation from the client on an OpenVMS

              file that has a dot at position 40 of its name, treat the

              file as if it has no dot. When you view a file created

              on the client, a dot at position 40 in the file's name is

              displayed, and you can perform any other operations on that

              file as you normally would, including the dot in the file

              name.

Regular Advisor
Peter Barkas
Posts: 157
Registered: ‎03-14-2005
Message 11 of 16 (971 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Response from CIFS engineering:

 

The behaviour observed in the case of 39 character filenames in ODS-2, is a limitation by design. While saving files in a share hosted on ODS-2 disks, CIFS uses certain conventions to encode the filename, in order to overcome the restrictions imposed by ODS-2 syntax to some extent. The issue that customer is facing (disappearing dot) is because of this mechanism.

 

For example, consider the below file created by Windows in a network share, hosted by CIFS on ODS-2. This filename has 40 characters, and doesn't have any extension (filename extension is optional in Windows):

 

 >

 

 > C:\>DIR  O:\DIR1

 

 >  Volume in drive O is ONC_ODS2_STREAM  >  Volume Serial Number is 50FD-0BAF  >  >  Directory of O:\DIR1  >

 

 > 01/31/2014  12:38 PM    <DIR>          .

 

 > 08/27/2013  05:05 PM    <DIR>          ..

 

 > 01/31/2014  01:26 PM               132 10-CHAR---10-CHAR---10-CHAR---10-CHAR---

 

 >                1 File(s)            132 bytes

 

 >                2 Dir(s)  36,213,514,240 bytes free

 

 >

 

 > C:\>

 

When CIFS saves the same file in ODS-2 disk, it'll append a dot after 39 characters. Basically, this is a "pseudo dot" i.e. it's not actually part of the original file name in Windows. Here CIFS is trying to support filenames more than 39 character length (which is the maximum allowed in ODS-2) by marking the 40th character onwards as part of extension instead of filename.

 

Hence the file created by Windows will appear in VMS as below:

 

 >

 

 > ONC59 $ DIR

 

 >

 

 > Directory $3$DKD0:[CIFS_ODS2.ONC_ODS2_STREAM.DIR1]

 

 >

 

 > 10-CHAR---10-CHAR---10-CHAR---10-CHAR--.-;1

 

 >

 

 > Total of 1 file.

 

 > ONC59 $      

 

Now, consider a scenario where Windows creates a file that actually has dot after 39 characters:

 

 >

 

 > C:\>DIR  O:\DIR1

 

 >  Volume in drive O is ONC_ODS2_STREAM  >  Volume Serial Number is 50FD-0BAF  >  >  Directory of O:\DIR1  >

 

 > 01/31/2014  12:38 PM    <DIR>          .

 

 > 08/27/2013  05:05 PM    <DIR>          ..

 

 > 01/31/2014  01:26 PM               132 10-CHAR---10-CHAR---10-CHAR---10-CHAR---

 

 > 01/31/2014  01:51 PM               144 10-CHAR---10-CHAR---10-CHAR---9-CHAR---.TXT

 

 >                2 File(s)            276 bytes

 

 >                2 Dir(s)  36,213,473,280 bytes free

 

 >

 

 > C:\>

       

Here also, CIFS will have to use pseudo dot after 39 characters; otherwise, it would be impossible to differentiate a pseudo dot and a real dot. Now, after the pseudo dot, CIFS cannot put the extension ".TXT" as-is, because ODS-2 doesn't allow more than one dot in filename. Hence, CIFS encodes the dot as "__2E" (the ASCII hexadecimal value of dot; same mechanism is used for other special characters restricted by ODS-2, but supported by Windows). So the filename will show up as below in VMS:

 

 >

 

 > ONC59 $ DIR /COL=1

 

 >

 

 > Directory $3$DKD0:[CIFS_ODS2.ONC_ODS2_STREAM.DIR1]

 

 >

 

 > 10-CHAR---10-CHAR---10-CHAR---10-CHAR--.-;1

 

 > 10-CHAR---10-CHAR---10-CHAR---9-CHAR---.__2ETXT;1

 

 >

 

 > Total of 2 files.

 

 > ONC59 $

 

The bottom line is that CIFS uses it's own file naming conventions while saving files on ODS-2 disks, in order to alleviate the gap between Windows file system and the limited syntax of ODS-2. This works smoothly for files are accessed only from Windows. However, when same files are accessed by applications running in Windows and VMS alike, in some cases, the filename might be interpreted differently by CIFS and other applications running in OpenVMS; which is what happens in the scenario of 39 character filename. A similar example is if an application creates file named "A__2CB.TXT;1" in the

 

ODS-2 directory, CIFS will interpret it as "A,B.TXT" because comma is encoded as "__2C" while creating files from Windows.

 

We won't be able to fix this in code without dropping support for filenames above 39 characters; and withdrawing support would be unacceptable for most users. As a work-around, the customer can use ".__2E" while creating filenames with 39 characters. But if that is not feasible, the only alternative is to use ODS-5 disks, which doesn't impose any of these naming restrictions.

Regular Advisor
Peter Barkas
Posts: 157
Registered: ‎03-14-2005
Message 12 of 16 (964 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

[ Edited ]

My understanding therefore is that CIFS does not fully support ODS-2 compliant filenames on an ODS-2 share because CIFS attempts to provide long filename support on ODS-2.

 

Apparently to support ODS-2 compliant filenames in CIFS requires an ODS-5 share.

Respected Contributor
Brad McCusker
Posts: 285
Registered: ‎05-01-2003
Message 13 of 16 (958 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Keep in mind that CIFS is a Windows or LanManager file server.  CIFS (and Advanced Server and PATHWORKS) are attempting to support Windows filenames on a VMS file system.  The result is that one specific file name scenario is not supported (refer to the release note above).

 

I acknowledge that the one specific file name scenario happens to be an ODS2 compliant file name, but, that really is irrelevant in the Windows world which was a significant consideration in the design decisions we made.

 

Brad McCusker

Software Concepts International

www.sciinc.com

Regular Advisor
Peter Barkas
Posts: 157
Registered: ‎03-14-2005
Message 14 of 16 (870 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Response from CIFS engineering.

 

      > Is there a way to prevent the ODS2 VFS module from automatically       

 

      > loading?   If not, is it feasible to implement the ability to  

 

      > negate the loading of the ODS2 module (i.e., vfs objects = no_ods2)?   

 

      > Obviously, that would come with the caveat that any attempt to (use    

 

      > the CIFS server to) create a filename that is not ODS-2 compliant      

 

      > will fail.             

 

       

 

As of now, there is no straight-forward method to facilitate this. But hypothetically, even if we introduce some tweaks to follow this approach, it still may be a risky option.

 

 

 

Internally, the routines in CIFS will treat a file/share as either ODS-2 or ODS-5. Taking the above approach would mean that CIFS has to treat a share as ODS-5 even though it is actually ODS-2. This could lead to unpredictable behaviour. Like you mentioned earlier, the caveat here is any attempt to create a filename that is not ODS-2 compliant will fail; however, we should also keep in mind that such failure need not be a graceful one. It could internally lead to process crash or file corruption.

 

       

 

One possible improvement we might be able to do to address this matter altogether, will be to avoid the encoding/decoding of special characters completely, based on a new flag in SMB config. If this flag is true, the filenames should be used as-is, and the usage of special characters or any filename larger than 39 characters should throw an error (i.e. fail gracefully).   

 

The challenge here is that in the current design, the encoding/decoding mechanism is implemented in several modules; it's not just one global routine that handles this logic. So the code changes required will be a bit more. The testing also will have to be extensive, to make sure backward compatibility is not compromised. Hence we can take this up as an enhancement request for a future release, but I am afraid we won't be able to provide a fix in the current implementation.

Regular Advisor
Peter Barkas
Posts: 157
Registered: ‎03-14-2005
Message 15 of 16 (869 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

To me it is a bug, to others, it seems, including CIFS engineering, it is a side effect of a feature.

Respected Contributor
Brad McCusker
Posts: 285
Registered: ‎05-01-2003
Message 16 of 16 (859 Views)

Re: HP AXPVMS SAMBA V1.2-ECO1 CIFS PS2_11 ALPHA OpenVMS 8.3 ODS2 39 character file name

Peter -

 

Your summary view of the situation is accurate.  If we were talking about PW/AS, however, there would be no ambiguity - it's a know limitation in the product.  Fortunately I no longer need to defend that position :) 

 

(It probably won't be any consolation, but I can assure you that there was indeed significant effort put into removing this limitation including a "fix" that made it at least as far as some builds.  It wasn't done on a whim)

 

Brad McCusker

Software Concepts International

www.sciinc.com

 

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.