Page 1 of 1

Split a file into n files, depending on register content.

Posted: Fri Feb 17, 2012 2:27 am
by filikindri
I think it should be able to do this, using icetool, but i am a totally beginner.

I have got a file, FB, LRECL = 4000. The file is something like this:

Code: Select all

HEADERFILE0200
THIS IS DATA AND THE LENGHT IS 200.REGISTER 1
THIS IS DATA AND THE LENGHT IS 200.REGISTER 2
THIS IS DATA AND THE LENGHT IS 200.REGISTER 3
...
...
...
THIS IS DATA AND THE LENGHT IS 200.REGISTER X
HEADERFILE0333
THIS IS DATA AND THE LENGHT IS 333.REGISTER 1
THIS IS DATA AND THE LENGHT IS 333.REGISTER 2
THIS IS DATA AND THE LENGHT IS 333.REGISTER 3
...
...
...
THIS IS DATA AND THE LENGHT IS 333.REGISTER y
Somebody is sending me many files, with different record lenghts and sizes.

I would like to split the file into as many files as 'HEADERFILE' ocurrences in position 1,10.
Each file should content its header and the data till next header; and it would be wonderful if output files have the lrecl as shown in the header record.

In my example i would like to obtain 2 output files (but it could be as many as 50-60, in real life), one of then with lrecl=200 and the other with lrecl=333,

outfile1:

Code: Select all

HEADERFILE0200
THIS IS DATA AND THE LENGHT IS 200.REGISTER 1
THIS IS DATA AND THE LENGHT IS 200.REGISTER 2
THIS IS DATA AND THE LENGHT IS 200.REGISTER 3
...
...
...
THIS IS DATA AND THE LENGHT IS 200.REGISTER X

outfile2:

Code: Select all

HEADERFILE0333
THIS IS DATA AND THE LENGHT IS 333.REGISTER 1
THIS IS DATA AND THE LENGHT IS 333.REGISTER 2
THIS IS DATA AND THE LENGHT IS 333.REGISTER 3
...
...
...
THIS IS DATA AND THE LENGHT IS 333,REGISTER y
In the "headerfile" record, lenght is not character format, it is packed, but I think this is a minor problem.

PLease, a little bit of help

Thanks in advance.

Posted: Fri Feb 17, 2012 4:59 am
by Frank Yaeger
You can use a DFSORT/ICETOOL job like the following to do what you asked for.
DO NOT specify the LRECL on the OUTnn DD statements - DFSORT will set the LRECL correctly for each output data set "automagically".

You will need to change:

11,4,ZD,TO=ZD,LENGTH=5

to correspond to your PD field in the header, e.g.

11,3,PD,TO=ZD,LENGTH=5

Code: Select all

//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=...  input file (FB/4000)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CTL2CNTL DD DSN=&&C2,UNIT=SYSDA,SPACE=(TRK,(5,5)),DISP=(,PASS)
//OUT01 DD DSN=...  output file 01 (FB/x)
...
//OUT60 DD DSN=...  output file 60 (FB/y)
//TOOLIN DD *
COPY FROM(IN) TO(T1) USING(CTL1)
COPY FROM(T1) USING(CTL2)
/*
//CTL1CNTL DD *
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,10,CH,EQ,C'HEADERFILE'),
    PUSH=(4001:ID=8))
  OUTFIL FNAMES=T1
  OUTFIL FNAMES=CTL2CNTL,INCLUDE=(1,10,CH,EQ,C'HEADERFILE'),
    BUILD=(C'  OUTFIL FNAMES=OUT',SEQNUM,2,ZD,
      C',INCLUDE=(4001,8,ZD,EQ,',SEQNUM,2,ZD,C'),BUILD=(1,',
      11,4,ZD,TO=ZD,LENGTH=5,C') ',80:X)
/*

Posted: Fri Feb 17, 2012 9:43 pm
by filikindri
It works!!!! Thanks a lot, really.

I had a little trouble, because the record length was packed decimal unsigned. x'0162' stands for lrecl=162, and as far I have read, DFSORT doesn't have a record format for this.

I have find a workaround, inrec a byte x'0C' , after x'0162'.

Finally I outrec PD,DIV,+10, to ZD, and ok. Several try and errors fixing records lenght, and it is working.

Now I've got my files separated, with correct lrecl, as I needed.
Next step, I have to submit a jcl proc, to process each file, and using some information in the HEADERFILE record.

I think I could use DFSORT again, using a template jcl file, and sortin out to the internal reader. But, firstly, i would have a glance at a REXX forum.

Thanks a lot again Frank. Your post reply have been really useful.

Edit: I did not notice: Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com . uuauuu!!!! You are behind the screen, every time i use DFSORT. It is exciting. Great, great product. I like mainframes, they are so faithful. I started working at mainframe. Then five years with sql, windows and so on, and now in a hurry, again with mainframes. After fight with distributed computation, I am really fall in love with mainframes, z/os and dfsort.

Posted: Fri Feb 17, 2012 11:27 pm
by Frank Yaeger
I had a little trouble, because the record length was packed decimal unsigned. x'0162' stands for lrecl=162, and as far I have read, DFSORT doesn't have a record format for this.


Actually, you can convert X'0162' to 0162 quite easily with:

p,2,HEX

So if your 2-byte value is in positions 11-12 of the header, you can just replace the last line of my control statements with:

Code: Select all

     11,2,HEX,C') ',80:X)    
I think I could use DFSORT again, using a template jcl file, and sortin out to the internal reader.
Yes, you could do that.

Thanks for the kind words about DFSORT and me. :D

Posted: Sat Feb 18, 2012 12:02 am
by filikindri
Actually, you can convert X'0162' to 0162 quite easily with:

p,2,HEX
It is "a little" more elegant than my workaround, no doubt :D

Posted: Sat Mar 03, 2012 4:15 pm
by filikindri
I need to fix a little bug. I don't need the header record in output files:

outfile1:


Code: Select all

THIS IS DATA AND THE LENGHT IS 200.REGISTER 1 
THIS IS DATA AND THE LENGHT IS 200.REGISTER 2 
THIS IS DATA AND THE LENGHT IS 200.REGISTER 3 
... 
... 
... 
THIS IS DATA AND THE LENGHT IS 200.REGISTER X


outfile2:

Code: Select all

THIS IS DATA AND THE LENGHT IS 333.REGISTER 1 
THIS IS DATA AND THE LENGHT IS 333.REGISTER 2 
THIS IS DATA AND THE LENGHT IS 333.REGISTER 3 
... 
... 
... 
THIS IS DATA AND THE LENGHT IS 333,REGISTER y 
So I think I shoud include an OMIT statement here, inside the build string

Code: Select all

 OUTFIL FNAMES=CTL2CNTL,INCLUDE=(1,10,CH,EQ,C'HEADERFILE'), 
    BUILD=(C'  OUTFIL FNAMES=OUT',SEQNUM,2,ZD, 
      C',INCLUDE=(4001,8,ZD,EQ,',SEQNUM,2,ZD,C'),BUILD=(1,', 
      11,4,ZD,TO=ZD,LENGTH=5,C') ',80:X) 
to omit records where

Code: Select all

1,10,ch,eq,c'HEADERFILE'
isn't it? somethig like that (sorry, at home, no TSO, so I can't verify statement syntax.)
I could use a third step, but outiles could be really big, and I would prefer no more intermediate files.

Code: Select all

 OUTFIL FNAMES=CTL2CNTL,INCLUDE=(1,10,CH,EQ,C'HEADERFILE'), 
    BUILD=(C'  OUTFIL FNAMES=OUT',SEQNUM,2,ZD, 
      C',INCLUDE=(4001,8,ZD,EQ,',SEQNUM,2,ZD,C'),BUILD=(1,', 
      11,4,ZD,TO=ZD,LENGTH=5,C') ',C'OMIT=(1,10,CH,EQ,
     C'HEADERFILE')', 80:X) 

Posted: Sat Mar 03, 2012 5:06 pm
by filikindri
I'm trying:

Code: Select all

   INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,12,CH,EQ,C'IDENTITAFICH'),   
   PUSH=(3881:ID=8)),                                             
         IFTHEN=(WHEN=(1,12,CH,EQ,C'IDENTITAFICH'),               
   BUILD=(1,18,C' ',20,3869))                                     
   OUTFIL FNAMES=T1                                               
***************************************************************** 
   OUTFIL FNAMES=T2,INCLUDE=(1,12,CH,EQ,C'IDENTITAFICH'),         
     BUILD=(3881,8,26,12,C' ')                                    
***************************************************************** 
   OUTFIL FNAMES=CTL2CNTL,INCLUDE=(1,12,CH,EQ,C'IDENTITAFICH'),   
     BUILD=(C'  OUTFIL FNAMES=OUT',SEQNUM,2,ZD,                   
     BUILD=(C'  OUTFIL FNAMES=OUT',SEQNUM,2,ZD,                       
       C',INCLUDE=(3881,8,ZD,EQ,',SEQNUM,2,ZD,C'),BUILD=(1,',         
       17,3,PD,DIV,+10,TO=ZD,LENGTH=4,C')',                           
       C',OMIT=(1,12,CH,EQ,C''IDENTITAFICH'')',80:X)                  
END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW    
DEBUG NOABEND,ESTAE                                                   
OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL1,SORTIN=IN,SO
               TOUT=T1,DYNALLOC                                       
SORT FIELDS=COPY                                                      
RECORD TYPE IS F - DATA STARTS IN POSITION 1                          
INCONSISTENT REFORMATTING FOR CTL2CNTL: REASON CODE 04, IFTHEN 0      
C5-K90025 C6-K90025 C7-K54603 C8-K62201 E9-K60823 E7-K62201           
END OF DFSORT                                                         
Where is the refformating problem ? I can't see it.

Edit. ok, i saw it. I've got a 80 bytes limit for the statement inside ctl2.

I changed 80 by 130, but, when next phase, sort statements from ctl2 are truncated, so

Code: Select all

0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R10 - 13:44 ON SAT M
    OUTFIL FNAMES=OUT01,INCLUDE=(3881,8,ZD,EQ,01,AND,1,12,CH,EQ,C'IDENT
    OUTFIL FNAMES=OUT02,INCLUDE=(3881,8,ZD,EQ,02,AND,1,12,CH,EQ,C'IDENT
                                                                  $    
E SYNTAX ERROR                                                         
    OUTFIL FNAMES=OUT03,INCLUDE=(3881,8,ZD,EQ,03,AND,1,12,CH,EQ,C'IDENT
No way to obtain a ctl2 file with two lines per file????

Posted: Sun Mar 04, 2012 1:25 pm
by filikindri
Okay. I think I've got it.

I will use

Code: Select all

160:X)
, then ICETOOL RESIZE TOLEN(80).

I just have to insert some blanks, to cut line in the correct place.

It should work... I'll try tomorrow-

Posted: Sun Mar 04, 2012 10:28 pm
by filikindri
It is working , again,

That's what I did. I suposse there are other ways more elegant, but this works.

Code: Select all

COPY FROM(IN) TO(T1) USING(CTL1)            
RESIZE FROM(CTL2CNTL) TO(CTL3CNTL) TOLEN(80)
COPY FROM(T1) USING(CTL3)                   

Code: Select all

   INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,12,CH,EQ,C'IDENTITAFICH'),  
   PUSH=(3881:ID=8)),                                            
         IFTHEN=(WHEN=(1,12,CH,EQ,C'IDENTITAFICH'),              
   BUILD=(1,18,C' ',20,3869))                                    
   OUTFIL FNAMES=T1                                              
*****************************************************************
   OUTFIL FNAMES=T2,INCLUDE=(1,12,CH,EQ,C'IDENTITAFICH'),        
     BUILD=(3881,8,26,12,C' ')                                   
*****************************************************************
   OUTFIL FNAMES=CTL2CNTL,INCLUDE=(1,12,CH,EQ,C'IDENTITAFICH'),  
     BUILD=(C'  OUTFIL FNAMES=OUT',SEQNUM,2,ZD,                  
       C',INCLUDE=(3881,8,ZD,EQ,',SEQNUM,2,ZD,                   
       C',AND,1,12,CH,                            ',             
       C'NE,C''IDENTITAFICH''),BUILD=(1,',                       
       17,3,PD,DIV,+10,TO=ZD,LENGTH=4,C') ',160:X)               

Posted: Sun Mar 04, 2012 11:20 pm
by Frank Yaeger
DFSORT allows continuation - see the doc. You just have to set up your control statement generation to take advantage of that.

Another piece of the puzzle you'll need: With OUTFIL BUILD, you can use / to start a new record.

So instead of generating:

Code: Select all

  OUTFIL FNAMES=OUT01,INCLUDE=(3881,8,ZD,EQ,01,AND,1,12,CH,EQ,C'IDENTITAFICH') 
You would use continuation - generate something like:

Code: Select all

  OUTFIL FNAMES=OUT01,
   INCLUDE=(3881,8,ZD,EQ,01,AND,1,12,CH,EQ,C'IDENTITAFICH') 
or this:

Code: Select all

  OUTFIL FNAMES=OUT01,
   INCLUDE=(3881,8,ZD,EQ,01,AND,
      1,12,CH,EQ,C'IDENTITAFICH')