how to import only new data into PF

[vbp1@hotmail]

Hello,
I have a project that needs to import data from CSV files into AS/400 PF file

thanks to all users here who helped me get the project of the ground (http://www.code400.com/forum/showthr...ile-processing)


I have all the initial processing code I need to :
1. call ftp script and get the files on IFS
2. process the files using "CPYFRMIMPF"
3. clean up the ftp by executing rename script created during the initial stage of the processing

my issue now is that if I reprocess the files again (might need to do that on accasion) even though I use parameter *updadd
it stills adds the duplicate records to the PF instead of updating existing.
is there a way to process files without creating duplicates?

thanks...

[Brian Rusch]

Re: how to import only new data into PF

Does the physical file have a unique key on it? A unique key is needed for this to work properly.

[DAG0000]

Re: how to import only new data into PF

Its your job to check for duplicates, what are you doing when you process the files. Are you doing a CPYFRMIMPF file directly into the file(s) you need the data to reside?

[vbp1@hotmail]

Re: how to import only new data into PF

at the moment yes, I do the CPYFRMIMPF directly into file(s) I want the data in.
but if there is no way to control the duplicates during that process I might consider creating the tempfile first.

as such, I should not really have duplicates per say as I am removing the source files after process but
if I would need to reprocess the files for some reason (like if I thought that some file was missed or something ) and the reprocessing create the duplicates it can be an issue

[vbp1@hotmail]

Re: how to import only new data into PF

no the PF does not have index or key on it.
so if there is no way to control import using CPYFRMIMPF
I have no objections in having an intermediery files and than processing the data using second CL or RPG program after import.
it's a PITA but doable

[Izodius]

Re: how to import only new data into PF

Sounds to me like the best thing to do would be to add the data to a file in QTEMP, check for dupes against the PF, delete dupes, then add the data to the main PF you're supposed to be processing.

[vbp1@hotmail]

Re: how to import only new data into PF

test reply.
for some reason my posts are not getting through again...