Monday, March 19, 2012
Database Discussion to change perpective
I have been doing relational database forever(or a long time) and have been intruduces to a team that uses a highly normalized database(propietary) to manage workflow.
We are capturing data in an AUDIT Trail EAV format.(500 million rows)
It is my task to build this into a data warehouse for reporting and I need to have with my team a relational database discussion. The relational database knowledge on this team is DB2 based, IDMS, and other past evolutions.
The common processes used are recieve a flat file and process this file sequentially using C# or VB doing lookups of other databse tables and writing out another flat file to be converted in XML for load to the propritary system.
My goal is to attempt to introduce new design concepts to my team and these are some talking points that I have come up with for a lunch and learn session.
can anyone else add to this list I don't want to get into a deep discussion about 3rd NF, Star Schemas vs Snowflake, etc.. I want to keep is informational and light to eliceit discussion and relat it back to older technologies.
some of the topics we can discuss are:
Why the data warehouse
Real-time tables what needs to stay in prod
What is going to happen to reporting database
Interaction between database on the same cluster/server
Interaction between databases on different servers (linked servers not allowed)
Set processing as opposed to cursor processing.
Table types
EAV
Type1
Type2
Fact
Dimensions
CodeYou could consider a demonstration of speed by getting together a flatfile of data, and then import it with the old way, then with BCP or BULKINSERT. I find people respond to demos better than theoretical discussion.
Sunday, February 19, 2012
database design - joins on which fields?
Hi,
I,m trying to develop a relational database but confused so much & need help to streamline the basics of the the database design error free.
Below is the case,
There r six databases with the joined fileds as below,
Buyer( filed used to join with style / po / production / shipped = Buyer Name )
Factory ( filed used to join with style / po / production / shipped = Factory Name )
Style ( one style have multiple pos ) ( filed used to join with po / production / shipped = Style # )
Po ( one po have multiple styles )( filed used to join with production / shipped = Po # )
Production
Shipped
There is nothing unique in all the 6 databases to make any filed primary key.
If in Buyer database, if we change the buyer name then all the records ralated to this buyer disturbed in all the databases & same is the case for factory name, style # & po # in Factory / Style & PO databases & any of these changes in first 4 databases ( i.e Buyer / Factory / Style / Po ) also badly effects the record linking & records in Production & Shipped databases.
Need advice how to join the databases in this case?
Is I use the auto serial # method in all the 6 databases or is there other ways to do so?
Will appreciate if someone let me know the way or guide me on this?
Thanks / Sulman
Sorry, but due to this unclear information given, I think it is hard to find a quick solution for you, perhaps you post some DDL and sample data.
HTH, Jens Suessmeyer.
http://www.sqlserver2005.de