Skip to content

XAware Community

Get the Flash Player to see this player.
Flash Image Rotator Module by Joomlashack.
XAware 5.6
Need Help
Webinars and Events
Advanced Tutorials
Webinars and Events

You are here: Home arrow Forums
XAware Community Forums
Welcome, Guest
Please Login or Register.    Lost Password?
Processing large Fixed Length Column format files (1 viewing) (1) Guest
Go to bottom Post Reply Favoured: 0
TOPIC: Processing large Fixed Length Column format files
#3885
dkokkeel (User)
XAware user
Posts: 25
User Offline Click here to see the profile of this user
Processing large Fixed Length Column format files 11 Years, 2 Months ago Karma: 2  
Hi,

I have a question what would be the best practice for reading large fixed length column files.

My usage scenario is that I want to read the file and push the data from the file directly into a 'similar' structured database table.

When I create file BizComponent to read the records from the file and for each record directly do the insert in that database table it performs not very good.
A Fixed Length Column file of 550Mb takes several hours to process and processor and memory usage is very low on my server.

Could you explain how I could improve performance (by leveraging my server resources) in this scenario?

I tried reading using 5 readers in parrallel, but this did not seem to increase performance with more than 25%
Would adding a command message queue be an option?

Any hints, samples would be greatly appreciated...

Thanks,

Dennis
 
Report to moderator   Logged Logged  
  The administrator has disabled public write access.
      Topics Author Date
    thread link
Processing large Fixed Length Column format files
dkokkeel 2009/03/10 16:36
    thread link
thread linkthread link Re:Processing large Fixed Length Column format fil
prichards 2009/03/11 15:40
    thread link
thread linkthread linkthread link Re:Processing large Fixed Length Column format fil
dkokkeel 2009/03/18 16:13
    thread link
thread linkthread linkthread linkthread link Re:Processing large Fixed Length Column format fil
prichards 2009/03/18 16:53
Go to top Post Reply
Powered by FireBoardget the latest posts directly to your desktop

Community Login

Get the Flash Player to see this player.
Flash Image Rotator Module by Joomlashack.
Commercial
Free Training
QuickStart Packages
Image 4 Title
Image 5 Title

Visit XAware.com