Merged file to remove duplicate so order is not affected

In this Mainframe Forum - You can post your queries on JCL, OS/390 JCL, MVS JCL, z/OS JCL, JES2 & JES3

Moderators: Frank Yaeger, DikDude, Moderator Group

Post Reply
SUBARNA ROY
Member
Posts: 5
Joined: Wed Sep 26, 2007 12:47 pm

Merged file to remove duplicate so order is not affected

Post by SUBARNA ROY » Fri Feb 08, 2008 12:44 pm

Hi
I need to merge two file and I need to remove the duplicates based upon some specific field ( eg class number), but whenever i am doing so, the out put file is getting sorted in the order of the sort field.
But business need is the files should be merged , and the records in the merged file should be in order as they were in the individual file. And also the merged file shud not contain any duplicate records with same class number( the field name i had mentioned above) only.

Please help me.
Subarna Roy

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Sat Feb 09, 2008 1:46 am

Please show an example of the records in your input files (relevant fields only) and the expected output records. Use the example to explain the "rules" for going from input to output. Give the starting position, length and format of each relevant field. Give the RECFM and LRECL of the input files.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

SUBARNA ROY
Member
Posts: 5
Joined: Wed Sep 26, 2007 12:47 pm

Post by SUBARNA ROY » Mon Feb 11, 2008 12:46 pm

Hi Frank,

The record format of the input file1 and input file 2 is FB and record length is 742.

the records are like this input file 1
0144556788568897........
0289098877956987........

the records are like this in input file2
0144556788268897........
0289088784559872........
0544556786568897........

now the output file needed is

0144556788568897........
0289098877956987........
0144556788268897........
0289088784559872........

Regards

Subarna
The starting position for the sorting field is 11 and the length it is occupying is 6
Subarna Roy

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Mon Feb 11, 2008 9:51 pm

Assuming (as in your example), there are no duplicates within input file1 and no duplicates within input file2, you can use a DFSORT/ICETOOL job like the following to do what you asked for. Note that this is NOT a MERGE operation (which would be simpler) since in a MERGE operation each input file must be in sorted order and the output is in sorted order.

Code: Select all

//S1    EXEC  PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG  DD SYSOUT=*
//IN1 DD DSN=...  input file1 (FB/742)
//IN2 DD DSN=...  input file2 (FB/742)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(MOD,PASS)
//T2 DD DSN=&&T2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=...  output file (FB/742)
//TOOLIN DD *
COPY FROM(IN1) TO(T1) USING(CTL1)
COPY FROM(IN2) TO(T1) USING(CTL2)
SELECT FROM(T1) TO(T2) ON(11,6,CH) FIRST
SORT FROM(T2) TO(OUT) USING(CTL3)
/*
//CTL1CNTL DD *
  INREC OVERLAY=(743:C'1',744:SEQNUM,8,ZD)
/*
//CTL2CNTL DD *
  INREC OVERLAY=(743:C'2',744:SEQNUM,8,ZD)
/*
//CTL3CNTL DD *
  SORT FIELDS=(743,9,ZD,A)
  OUTREC BUILD=(1,742)
/*
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

SUBARNA ROY
Member
Posts: 5
Joined: Wed Sep 26, 2007 12:47 pm

Post by SUBARNA ROY » Tue Feb 12, 2008 12:26 pm

Hi

The above code mentioned by u was working fine, but as per business need we can't use ICETOOL, they r suggesting to use SYNC SORT. Can u pls tell me how I can do it with SYNC SORT or something else ( other than ICE TOOL)

Regards
Subarna
Subarna Roy

User avatar
Frank Yaeger
Moderator
Posts: 812
Joined: Sat Feb 18, 2006 5:45 am
Location: San Jose, CA
Contact:

Post by Frank Yaeger » Tue Feb 12, 2008 10:07 pm

I'm a DFSORT developer. DFSORT and Syncsort are competitive products. I'm happy to answer questions on DFSORT and DFSORT's ICETOOL, but I don't answer questions on Syncsort.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort

kaitki
Member
Posts: 4
Joined: Wed Mar 12, 2008 12:38 pm

Merging files without sorting

Post by kaitki » Wed Mar 19, 2008 1:34 pm

U could try the following -
Concatenate the datasets by using IEFBR14 and then in the next step omit duplicates using Omit cond = () of DFsort.

Post Reply

FREE TUTORIALS

Tutorials
Free tutorials from mainframegurukul
  • JCL Tutorial
    Covers all important JCL concepts.
  • Cobol Tutorial
    This tutorials covers all Cobol Topics from STRING to COMP-3.
  • DB2 Tutorial
    DB2 Tutorial focuses on DB2 COBOL Programming.
  • SORT Tutorial
    This Tutorial covers all important aspects of DFSORT with examples
  • CICS Tutorial
    This CICS tutorial covers CICS concepts and CICS Basics, CICS COBOL Programming.
Interview
Mainframe Interview questions



Other References
Mainframe Tools and others