Data Warehousing Community Forum
February 09, 2012, 01:45:36 pm

Pages: [1]   Go Down
« previous next »
  Print  
Author Topic: Need solution for high volume SCD processing  (Read 859 times)
blackenry
DW Fresher
*

Reputation: +1/-0
Offline Offline

Posts: 1


View Profile
« on: June 23, 2010, 10:08:51 am »
Hi,

I have a dimension table which will contain 24 million records after backloading. It is a monthly run and every month 400,000 records will come from source.

I am currently using a dynamic lkp to do the SCD. But i really doubt if it will run as the lkp has to build a cache for 24 million records + 400000 every month going forward.

Can someone suggest a better way to do the SCD in this case??

Many Thanks in advance!!
Henry.M
Logged
Sipra
Founder
*****

Reputation: +59/-0
Offline Offline

Posts: 323


Am the King...


View Profile WWW
« Reply #1 on: July 20, 2010, 09:36:54 pm »
Henry - Good stuff..!

As a solution, why can't you take a help of session partitioning.

And as a second approach, you can split your lookup cache build into channels.
Creating a simple proc at db level will do this split work.

Let me know still you need any info..  Smiley
Logged
Sipra
Founder

www.dwforum.net
Pages: [1]   Go Up
  Print  
« previous next »
 
Jump to: