sort the input file excluding the trailer record

In this Mainframe Forum - You can post your queries on DFSORT, ICETOOL , SyncSort & JCL Utilities

Moderators: Frank Yaeger, Moderator Group

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

sort the input file excluding the trailer record

Post by mothukuri » Tue Oct 06, 2009 3:44 pm

Hi all

Can anyone please help me with the SORT condition to sort the input file which has trailer record at the end of it.....I need to extract certain fields from the input file omitting the trailer record and then place the trailer record at the end of the output file.

Thanks in advance

Anuj Dhawan
Moderator
Posts: 1625
Joined: Sat Aug 09, 2008 9:02 am
Location: Mumbai, India

Post by Anuj Dhawan » Tue Oct 06, 2009 6:40 pm

Suggest you show some sample input records, expected output and rules to get that output.

What is the LRECL and RECFM of input/output?

What SORT product are you using?
Regards,
Anuj

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Tue Oct 06, 2009 7:57 pm

mothukuri,

I suspect you can use the DATASORT operator of DFSORT's ICETOOL to do what you want. For more details, see:

http://www.ibm.com/support/docview.wss? ... g3T7000085

If you need more specific help, supply the information Anuj requested.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

Post by mothukuri » Wed Oct 07, 2009 1:59 pm

Hi Frank

I tried to submit job with DATASORT but it fails with S0C4 abend. Below is the error msg:

-------------------------
IEF237I JES2 ALLOCATED TO TOOLIN
IEF237I JES2 ALLOCATED TO CTL1CNTL
IEA995I SYMPTOM DUMP OUTPUT
SYSTEM COMPLETION CODE=0C4 REASON CODE=00000004
TIME=09.37.51 SEQ=57888 CPU=0000 ASID=005E
PSW AT TIME OF ERROR 078D0000 A9700006 ILC 6 INTC 04
ACTIVE LOAD MODULE ADDRESS=29700000 OFFSET=00000006
NAME=ICETOOL
DATA AT PSW 29700000 - D3C9C3C5 D5E2C5C4 40D4C1E3
GR 0: FD000008 1: 00006FF8
2: 00000040 3: 00ACF9D4
4: 00ACF9B0 5: 00AFF358
6: 00AB5FE0 7: FD000000
8: 00AFCC70 9: 00AFF618
A: 00000000 B: 00AFF358
C: 8643EB4A D: 00006F60
END OF SYMPTOM DUMP
-------------------------

Anuj,
I have a file F1 with N records, N+1 is the trailer record which has the type of the file, date, time, no. of records (XXXX_09072009090520090907200905:39:310907200905:39:41YYYYY 1.1 00001141700000000002227912).
The Record format is FB and Record Length is 9249. Similarly I have Files F2 to F10 i.e. 10 files with same record format and length. My requirement is to merge the input file F1 to F10 into a file F11 and then extract 4 fields based on the positions from F11 to F12 (F12 should not extract the 10 trailer records present in F11). We need to add a trailer record to F12 with the message "End of file....record count is: X".

Thanks & Regards
MSS

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Wed Oct 07, 2009 7:47 pm

What you've posted about the 0C4 doesn't tell me anything. I don't even know which sort product you're using or what your JCL looks like. I would at least need to see the complete JES log. If you're using DFSORT (ICExxxs messages), you can send it to me offline (yaeger@us.ibm.com) and I'll take a look.

Your description of what you want to do is not at all clear to me. Please show an example of the records in each input file and what you expect for output. Explain the rules for getting from input to output. Give the RECFM and LRECL of each input file. Give the starting position, length and format of each relevant field.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

Post by mothukuri » Wed Oct 07, 2009 8:08 pm

Hey Frank

Can you pls provide me with an example where we can extract fields from multiple files (same format and rec len) using SORT in a single step.

Thanks
MSS

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Wed Oct 07, 2009 9:59 pm

Can you pls provide me with an example where we can extract fields from multiple files (same format and rec len) using SORT in a single step.
First, you have to give me the information I asked for previously:
Please show an example of the records in each input file and what you expect for output. Explain the rules for getting from input to output. Give the RECFM and LRECL of each input file. Give the starting position, length and format of each relevant field.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

Post by mothukuri » Thu Oct 08, 2009 7:14 pm

Frank,here are the contents in one of the files:

---------------
ROCKY HUNGARY 0122232111051
DEEPA INDIA 0977363233091
ARNAV CANADA 0113348743011
Trailer 00003
---------------


Field positions
name - 1 to 9
place - 10 to 19
phone number - 20 to 29
code - 30 to 32

Requirement: There are 10 files in total which have data similar to above (Input file RF is FB, LRECL: 80, DFSORT). I want to extract the Name,phone number and code from all these 10 files and the output file (RF: FB, LRECL: 50) should look like below:
-------------------
ROCKY 0122232111051
DEEPA 0977363233091
ARNAV 0113348743011
ROBERT 012323233022
RICHARD 0112323225011
Trailer 5
-------------------

The fields should be extracted omitting the 10 trailer records of 10 input files and only display one trailer record in the output file with the record count of all the records in 10 files (excluding the trailer records)

I was able to accomplish this :) .....forum helped me.

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Thu Oct 08, 2009 9:03 pm

I was able to accomplish this .....forum helped me.
Good.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

Post by mothukuri » Fri Oct 09, 2009 2:07 pm

Hi Frank,

I have 10 sequential files (LRECL=180 and RECFM=FB) which have about a million records in each. I want to merge these files into a single file. Can we accomplish this using SORT? If so can you show an example.

Using IEBGENER is not recommended as the CPU usage would be high as the files are sequentially read from 10 files and then written to output file. Please suggest a better option.

Thanks & Regards
MSS

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Fri Oct 09, 2009 8:42 pm

I want to merge these files into a single file.
Do you mean "MERGE" in the classic sense of merging files that have been previously sorted on a key? Or do you really mean copy the files to output one after the other (MOD).

If you mean "MERGE" in the classic sense, then the DFSORT job would be:

Code: Select all

//S1 EXEC  PGM=SORT
//SYSOUT    DD  SYSOUT=*
//SORTIN01 DD DSN=...  input file1
//SORTIN02 DD DSN=...  input file2
...
//SORTIN10 DD DSN=...  input file10
//SORTOUT DD DSN=...   output file
//SYSIN    DD    *
  MERGE FIELDS=(p,m,f,s)
/*
where p,m,f,s describes the key on which the files have been previously sorted.
p is the starting position, m is the length, f is the format and s is A for ascending order or D for descending order.

If you're not familiar with DFSORT and DFSORT's ICETOOL, I'd suggest reading through "z/OS DFSORT: Getting Started". It's an excellent tutorial, with lots of examples, that will show you how to use DFSORT, DFSORT's ICETOOL and DFSORT Symbols. You can access it online, along with all of the other DFSORT books, from:

http://www.ibm.com/support/docview.wss? ... g3T7000080
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

Post by mothukuri » Mon Oct 12, 2009 4:40 pm

Hello Frank

Actually I was looking for the other way, copying files to output one after the other. Thanks alot for the DFSORT link, it is very helpful.

Can we use INREC and OUTFIL in a single step? As of now I'm using INREC to extract required fields in step1 and OUTFIL to add a trailer record at the end of the file along with the record count in step2.

Following is the code currently used:

//STEP001 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=File1,DISP=SHR
// DD DSN=File2,DISP=SHR
// DD DSN=File3,DISP=SHR
//SORTOUT DD DSN=File4,UNIT=SYSDA,
// DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(100,100),RLSE),
// DCB=(LRECL=80,RECFM=FB,BLKSIZE=80)
//SYSIN DD *
OMIT COND=(1,4,CH,EQ,C'TEST')
SORT FIELDS=COPY
INREC FIELDS=(1:1,3,4:4,19,23:23,4,27:116,19)
/*
//STEP002 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=File4,DISP=SHR
//SORTOUT DD DSN=File4,DISP=MOD
//SYSIN DD *
OPTION COPY
OUTFIL REMOVECC,NODETAIL,
TRAILER1=(1:C'EOF',1X,
COUNT=(EDIT=(IIIIIIIT)))
/*
-------------------------------

Is there a way to accomplish the task in a single step?

Thanks
MSS

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Mon Oct 12, 2009 7:59 pm

Yes, you can use INREC and OUTFIL in a single pass:

Code: Select all

  OMIT COND=(1,4,CH,EQ,C'TEST')  
  INREC FIELDS=(1:1,3,4:4,19,23:23,4,27:116,19) 
  SORT FIELDS=COPY 
  OUTFIL REMOVECC,NODETAIL, 
     TRAILER1=(1:C'EOF',1X, 
       COUNT=(EDIT=(IIIIIIIT))) 
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

mothukuri
Member
Posts: 9
Joined: Tue Oct 06, 2009 3:38 pm

Post by mothukuri » Mon Oct 12, 2009 8:07 pm

I tried the same and job ended with RC=16. Following is the msg in SYSOUT,

--------------------
ICE143I 0 BLOCKSET COPY TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AN
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 16:02 ON MON OCT
OMIT COND=(1,4,CH,EQ,C'TEST')
INREC FIELDS=(1:1,3,4:4,19,23:23,4,27:116,19)
SORT FIELDS=COPY
OUTFIL REMOVECC,NODETAIL,
TRAILER1=('EOF ',COUNT=(M12,LENGTH=7)
ICE201I E RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE222A 0 45 BYTE FIXED RECORD LENGTH IS NOT EQUAL TO 80 BYTE LRECL FOR SORTOUT
ICE751I 0 C5-K26318 C6-K90007 C7-K90000 C8-K23476 E9-K90007 E7-K24705
ICE052I 3 END OF DFSORT
--------------------

Please help to fix this.

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Mon Oct 12, 2009 8:56 pm

That message indicates you have LRECL=80 for the SORTOUT file, but your INREC statement is only creating 45 byte records. (I didn't happen to notice that in your original post with your JCL.)

Remove LRECL=80 from the SORTOUT DD. If you want LRECL=45 for SORTOUT, that will do it. If you want LRECL=80 for SORTOUT, change your INREC statement to:

Code: Select all

   INREC FIELDS=(1:1,3,4:4,19,23:23,4,27:116,19,80:X)
Do you really want BLKSIZE=80 for SORTOUT ... that's a very inefficient BLKSIZE. If you remove BLKSIZE=80, DFSORT will select a more optimum BLKSIZE. You also don't need RECFM=FB for SORTOUT. DFSORT sets the attributes automatically if you let it.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

Post Reply

FREE TUTORIALS

Tutorials
Free tutorials from mainframegurukul
  • JCL Tutorial
    Covers all important JCL concepts.
  • Cobol Tutorial
    This tutorials covers all Cobol Topics from STRING to COMP-3.
  • DB2 Tutorial
    DB2 Tutorial focuses on DB2 COBOL Programming.
  • SORT Tutorial
    This Tutorial covers all important aspects of DFSORT with examples
  • CICS Tutorial
    This CICS tutorial covers CICS concepts and CICS Basics, CICS COBOL Programming.
Interview
Mainframe Interview questions



Other References
Mainframe Tools and others