Home      Mainframe Forum      Mainfarme Tutorials      IBM Manuals      Mainframe Interview Questions      Mainframe Books      IT News     SiteMap     Downloads


     
 
MAINFRAME - TIP OF THE DAY : When you specified V for RECFM parameter, LRECL value is largest record in the file plus 4 bytes. These four bytes contain the actual length of each variable length record in the file

Google
 
Web mainframegurukul.com

Eliminate duplicate records in file

 
Post new topic   Reply to topic    mainframegurukul.com Forum Index -> DFSORT , ICETOOL & Utilities
  View previous topic :: View next topic  
Author Message
banupriyab
Member


Joined: 17 Apr 2009
Posts: 16

Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
1 salutes

PostPosted: Tue Oct 06, 2009 10:55 am    Post subject: Eliminate duplicate records in file Reply with quote

Hi,
I have eliminated duplicate records in a file using SUM FIELDS=NONE in SORT.

//SYSIN DD *
SORT FIELDS=(1,23,CH,A)
SUM FIELDS=NONE

But, I DONT want the records to get aligned in ascending order and then get duplicates eliminated. Is there any other way to remove dups without getting sorted in asc or desc order?

Eg.,
Input records:

MOHANK 123456789012345 RAJES
ANTAKS 123456789012345 MIRAJ
MOHANK 123456789012345 NANAK

Output records:
ANTAKS 123456789012345 MIRAJ
MOHANK 123456789012345 RAJES

I want MOHANK in the first place followed by ANTAKS. Pls help.


Thanks,

BanuPriya B
Back to top
View user's profile Send private message
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.

Anuj Dhawan
Moderator


Joined: 09 Aug 2008
Posts: 1625
Location: Mumbai, India

Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Tue Oct 06, 2009 6:56 pm    Post subject: Reply with quote

Probably you want to say, "I would like to keep the order of the input records when they are copied to the output file"... if so, eliminating duplicates, either by using SUM FIELDS=NONE or SELECT, requires sorting the records so that the records with the same key are in order.

If you need to keep the records in their original order, then you can use the trick of adding a sequence number before you eliminate the duplicates, and then sorting on that sequence number to get the remaining records back in their original order.
_________________
Regards,
Anuj
Back to top
View user's profile Send private message
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
Anuj Dhawan
Moderator


Joined: 09 Aug 2008
Posts: 1625
Location: Mumbai, India

Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Tue Oct 06, 2009 7:03 pm    Post subject: Reply with quote

Other suggestion which comes to mind is to use EQUALS. EQUALS tells the process to preserve the original order of the data within the "sort keys". Your site default is probably EQUALS. DFSORT is shipped with NOEQUALS as the default, but the site can change that to EQUALS.

If you're using DFSORT (and I'm not sure you are), you can see the value for EQUALS in message ICE128I ... it will have EQUALS=N or EQUALS=Y.

You can try using:
Code:
   OPTION NOEQUALS
to turn off EQUALS and see what you get.
_________________
Regards,
Anuj
Back to top
View user's profile Send private message
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
Frank Yaeger
Moderator


Joined: 18 Feb 2006
Posts: 812
Location: San Jose, CA

Answer was approved by Question owner.
Useful Post
2 votes
Not an useful post
Greate Post!! Salute Author
1 salutes

PostPosted: Tue Oct 06, 2009 8:05 pm    Post subject: Reply with quote

BanuPriya,

Here's a DFSORT/ICETOOL job that will do what you asked for. I assumed your input file has RECFM=FB and LRECL=80, but the job can be changed appropriately for other attributes.

Code:

//S1   EXEC  PGM=ICETOOL
//TOOLMSG   DD  SYSOUT=*
//DFSMSG    DD  SYSOUT=*
//IN DD DSN=...  input file (FB/80)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=...  output file (FB/80)
//TOOLIN DD *
SORT FROM(IN) TO(T1) USING(CTL1)
SORT FROM(T1) TO(OUT) USING(CTL2)
/*
//CTL1CNTL DD *
  INREC OVERLAY=(81:SEQNUM,8,ZD)
  SORT FIELDS=(1,23,CH,A),EQUALS
  SUM FIELDS=NONE
/*
//CTL2CNTL DD *
  SORT FIELDS=(81,8,ZD,A)
  OUTREC BUILD=(1,80)
/*


Note that EQUALS will keep the duplicate records in their original order, but will not keep all of the records in their original order. For that, you need two passes over the data.
_________________
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
Anuj Dhawan
Moderator


Joined: 09 Aug 2008
Posts: 1625
Location: Mumbai, India

Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Tue Oct 06, 2009 9:40 pm    Post subject: Reply with quote

Quote:
Note that EQUALS will keep the duplicate records in their original order, but will not keep all of the records in their original order.
Thank you Frank -- I was little unsure about it.

Have a good one,

Regards,
Back to top
View user's profile Send private message
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
banupriyab
Member


Joined: 17 Apr 2009
Posts: 16

Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Tue Oct 13, 2009 1:35 pm    Post subject: Reply with quote

Thank u!! Very Happy

The ICETOOL code worked!!
But dont we have any other means with normal SORT/SYNCSORT, because none of the jobs in our system uses ICETOOL.
Back to top
View user's profile Send private message
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
Frank Yaeger
Moderator


Joined: 18 Feb 2006
Posts: 812
Location: San Jose, CA

Answer was approved by Question owner.
Useful Post
1 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Tue Oct 13, 2009 8:19 pm    Post subject: Reply with quote

I don't know what you're asking for. If you have DFSORT, then you have ICETOOL. ICETOOL has been part of DFSORT since 1991! You said ICETOOL worked, so what is the problem?

If you don't want to use ICETOOL for some reason, then you can just use two DFSORT steps instead.
_________________
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
Alissa Margulies
Member


Joined: 28 Apr 2009
Posts: 25
Location: USA

Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Mon Oct 26, 2009 9:53 pm    Post subject: Reply with quote

banupriyab wrote:
...dont we have any other means with normal SORT/SYNCSORT, because none of the jobs in our system uses ICETOOL.

SyncSort ships with ICETOOL as an alias to SYNCTOOL. If you prefer, as Frank suggested, you can code the following SyncSort job:
Code:
//STEP1 EXEC PGM=SORT
//SORTIN  DD DSN=input.file
//SORTOUT DD DSN=&&TEMP
//SYSOUT  DD SYSOUT=*
//SYSIN   DD *
  INREC OVERLAY=(81:SEQNUM,8,ZD)
  SORT FIELDS=(1,23,CH,A),EQUALS
  SUM FIELDS=NONE
/*
//STEP2 EXEC PGM=SORT
//SORTIN  DD DSN=&&TEMP
//SORTOUT DD DSN=output.file
//SYSOUT  DD SYSOUT=*
//SYSIN   DD *
  SORT FIELDS=(81,8,ZD,A)
  OUTREC BUILD=(1,80)
/*

_________________
Alissa Margulies
SyncSort Mainframe Product Services
zos_tech@syncsort.com
201-930-8260
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
academyindia4
Guest





Answer was approved by Question owner.
Useful Post
0 votes
Not an useful post
Greate Post!! Salute Author
0 salutes

PostPosted: Mon Jan 25, 2016 10:19 pm    Post subject: Topic deleted by Admin Reply with quote

<< Content deleted By Admin >>
Back to top
Click here, If this post answer your question. <-- Click on right mark icon. If this post answer your question.
Display posts from previous:   
Post new topic   Reply to topic    mainframegurukul.com Forum Index -> DFSORT , ICETOOL & Utilities All times are GMT + 5 Hours
Page 1 of 1



 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Cobol Tutorial
This cobol tutorial covers most of the important topics like STRING, UNSTRING, COMP, COMP-3.....
DB2 Tutorial
DB2 Tutorial focuses on DB2 COBOL Programming. Explains in simple language. Some Chapters are locked, Forum members have free access to these chapters
CICS Tutorial
This CICS tutorial covers CICS concepts and CICS Basics, CICS COBOL Programming...
JCL Tutorial
This is most popular JCL tutorial from mainframegurukul. It does contain important jcl ....
SORT Tutorial
This Tutorial covers all important aspects of DFSORT. Has more SORT examples
INTERVIEW Questions
This page covers important interview questions
Mainframe Jobs
Mainframe Jobs posted by members of mainframegurukul forum
FaceBook Page
MainframeGurukul FaceBook Page
LinkedIn Page
MainframeGurkul Linkedin Page

This widget requires Flash Player 9 or better







Go to top of the page
 

Online ABEND Reference ||  JCL References ||  COBOL References ||  VSAM References ||  Tutorials by Drona Series ||  SQL tutorial ||  BOOKS  ||  DB2 INTERVIEW QUESTIONS ||  COBOL INTERVIEW QUESTIONS  ||  JCL INTERVIEW QUESTIONS ||  JCL2 INTERVIEW QUESTIONS ||  VSAM INTERVIEW QUESTIONS ||  CICS INTERVIEW QUESTIONS  ||  Online tutorials ||  Online ABEND Reference ||  JCL References ||  COBOL References ||  VSAM References ||  Tutorials by Drona Series ||  SQL tutorial ||  BOOKS  ||  SiteMap  ||  Expeditor Tutorial  ||  FILE-AID Tutorial  ||  Changeman Tutorial  ||  COBOL   ||  DB2   ||  JCL  ||  CICS  ||  VSAM  ||  DB2 Interview Questions ( 110 )   || Simple JCL Tutorials  || JCL Tutorial from MainframeGurukul.com   || Simple JCL Tutorial - Chapter1 ;|| Mainframe Forum - Tutorials  || Mainframe Tutorials || Mainframe CICS tutorial|| Mainframe COBOL Tutorial

Drona Educational Forums - Mainframe Cobol DB2 CICS Board
Powered by phpBB

mainframetopic-4685.html